Having prior knowledge of Soar syntax will make your life awfully easier.
This is the code for 4 differend PD and IPD agents, with and without reinforcment learning abilities. Specifically:
pd-standalone.soar
is a PD agent that plays against an opponent (opponents are: random, always coop, always defect)pd-standalone-rl.soar
is a PD adent that uses reinforcement learning to handle the above opponents (it always results to the dominant strategy, of course)ipd-rl-look-back.soar
in an IPD agent that uses reinforcement learning to win any one of the following 12 opponentsipd-multi-opponent-runs.soar
in an IPD agent that uses reinforcement learning and plays consequtively with all the following 12 opponents, straight. That is 2400 iterations.
The rules are Axelrod's Tournament rules, 200 iterations per game.
The 12 opponent strategies are the following:
- Cooperate
- Defect
- Interchange
- Random
- Tit for Tat (TFT)
- Suspicious Tit for Tat (SuspTFT)
- Tit for 2 Tats (TF2T)
- Grofman
- Grudger
- Pavlov (win-stay, lose-switch)
- Extort (ZD_Extort2)
- Joss
More information about the strategies can be found in the Axelrod Python library documentation
- Download Soar Suite here. (cerrent version is 9.6.0)
- Pull or DL the repository anywhere on your computer
- Launch
SoarJavaDebugger.bat
from the Suite folder. - Run the following 3 commands in the debugger's command line
rl --set learning on
indifferent-selection –-epsilon-greedy
indifferent-selection --epsilon 0.3
- Reduce the epsilon as training progresses to refine results. Do so by using the last command.
- Open the agent you want to run, e.g.
ipd-rl-look-back.soar
in an editor - Uncomment the single strategy you want to play against; RANDOM is the default uncommented one. Warning: You can have only ONE opponent strategy uncommented at any given time, otherwise the opponent will run in an infinite ellaboration error. If you want to play against a chain opponents, select the
ipd-multi-opponent-runs.soar
and move to the next step. - After having also completed the
Prepare to Run
part, click on theSource
button in the SoarJavaDebugger and load the selected agend. - Click
Initialize
follower byRun
- Watch the magic happen