0.1 Battle of the Sexes. 0.2 Chicken. 0.3 Coordination Game

Transcription

1 This is a record of most of the different games we have tested with RSRS. In all cases, the prediction algorithm used is fictitious play, and games are repeated times. In each figure the top graph shows performance against a random opponent, the center graph shows performance against an omniscient opponent, and the bottom graph shows performance against a worst-case opponent. Note that we do not include games with a single dominant strategy, such as Prisoner s Dilemma. This is because RSRS is not an algorithm for cooperation - it will always choose Defect, because Defect is always the best response.. Battle of the Sexes Player Player Opera Boxing Opera Boxing Battle of the Sexes is a frequently used example in game theory. Performance is shown in figure. RSRS is slightly slower to adapt to the random opponent, possibly because it is preparing for a best responding opponent. Against a best responding opponent all agents except RNR manage to find the advantageous equilibrium. In the worst case only RSRS and SPS are able to adopt the necessary randomized strategy to get some chance of a payoff.. Chicken Player Player Don t Dare Dare Don t Dare 9 9 Dare Chicken is another popular game from game theory. Performance is shown in figure. Most agents are quick to adapt to the worst case opponent, although there is some exploration. Against an omniscient opponent RNR, RSR, and RSRS all manage to find the favored equilibrium. RSRS is still a bit slow to find the best response to a random opponent.. Coordination Game Player Player A B C D a b c d This is a strict coordination game we have constructed, with equilibria, which both players rank identically. Performance is shown in figure. All players are able to eventually find the best response to a random opponent, and all except RNR are able to find the best equilibrium with an omniscient

2 opponent. Only RSRS and SPS are able to find the best response to the worst case opponent, and it takes them a while to find it. That could probably be improved by better parameter selection for calculating the risk factor to adopt.. Cuban Missile Crisis Soviets US Withdraw Maintain Blockade Air strike Cuban missile crisis is another example from game theory. It is the label given to a class of player games with that payoffstructure. The Soviets choose between withdrawing from Cuba and staying in Cuba, and the US chooses between a blockade of Cuba, and airstrikes on Cuba(which could lead to nuclear war - the, option). Performanceisshowninfiguresand5. WhenplayingastheUSRNR,RSR, and RSRS where all slightly worse against a random opponent, but significantly better against an omniscient or a worst-case opponent. When playing as the Soviets, all strategies performed well against a random opponent. Performance against an omniscient opponent varied widely, with RSRS and RSR having slightly higher peak performance. In the worst-case scenario all agents were able to figure out they should Withdraw..5 Pursuer/Evader Evader Pursuer Location Location Location Location Location Location Location Location This is a game we constructed for testing. The pursuer receives points for chosing the same location as the evader, otherwise the evader receives those points. Players also value ending up in particular locations, with location being most preferred. Performance is shown in figures and 7. As the pursuer RSRS and RSR are a bit slow to find the optimal response against a random strategy, but they perform significantly better against an omniscient opponent. In the worst-case scenario, only RSRS and SPS find a good strategy. As the evader RNR and SPS perform comparatively better against the omniscient opponent, but again only RSRS and SPS perform well in the worst case.

3 . Inspector/Worker Worker Inspector Don t Work Work Don t inspect Inspect This is another example from game theory. We have a worker who would prefer not to work, unless the inspector is watching, and an inspector who wants the workerto work, but would rather not have to inspect. Performanceis shown in figures and 9. As the inspector all agents do ok against the random opponent and the worst-case opponent. Against an omniscient opponent it s very noisy, due to fluctuations in the predictor. As the worker RSRS RSR, and RNR are slightly slower to find a response to the random opponent, but they all do better against an omniscient opponent. In the worst case only RSRS and SPS find the appropriate mix of working and slacking off..7 Hero Weak player Strong Player Weak option Strong option Weak option Strong option Hero is a variant of Battle of the Sexes in which both players prefer to play their own strategy, if they don t succeed in choosing the same strategy. Performance is shown in figure. In this game RNR is slightly slower to find the best response to a random strategy. All agents do well against an omniscient opponent, but only SPS and RSRS find a good response to the worst-case opponent.. Matching Pennies Player Player Heads Tails Heads Tails This is an asymetric version of matching pennies, in which both players would prefer to win while Player chooses Heads. Performance is shown in figures and. Because it is zero-sum, performance is the same against an omniscient opponent and in the worst case. Despite the asymmetry of the game performance is similar as both the first and second player. RSRS, RSR, and RNR are all slower to find a best-response to a random opponent, but all do better against the omniscient/worst-case opponent.

4 .9 Pursuit of the Israelites Pharoah Moses Pursue Don t Pursue Help Israelites Don t Help Pursuitofthe Israelitesisanothername foracategoryofgames. Thegeneral idea is that Pharoah would prefer to pursue the Israelites, but only if Moses doesn t pray to God to help them. Moses would prefer to pray, but only if the Pharoah is pursuing. Performance is shown in figures and. RSRS, RSR, and RNR don t perform quite as well against a random opponent, but do better against an omniscient opponent. In the worst case, only RSRS and SPS find the best strategy.. Stag Hunt Player Player Stag Rabbit Stag Rabbit Stag Hunt is another classic example from game theory. Both players choose whethertohuntstagorrabbits. Ifbothplayerschoosestag,theyreceiveapayoff of, but it s safer to choose rabbit, which doesn t require help from the other player. Performance is shown in figure 5. RSR and RNR perform slightly worse against random opponents, but all are able to hunt stag with the omniscient opponent. In the worst case, RSRS and SPS figure out that they need to hunt rabbit.. Travellers Dilemma In the Traveller s Dilemma both players choose a value between and. Each player receives the value chosen by the lower player, but the player chosing the lowest value gains a point from the other player (nothing happens in the case of a tie). Performance is shown in figure. All players perform roughly the same against random opponents. Against an omniscient opponent the Stackelberg based strategies are able to find the best strategy while other approaches move towards the worst outcome. In the worst case all agents other than RSR are able to find the best response.

5 Outcomes vsrandom Opponent inbos.. with Safety Outcomes vsomniscient Opponent inbos.. with Safety Outcomes vsworstcase Opponent inbos.. with Safety Figure : Performance in the Battle of the Sexes game. 5

6 9 Outcomes vsrandom Opponent inchicken with Safety Outcomes vsomniscient Opponent inchicken with Safety Outcomes vsworstcase Opponent inchicken with Safety Figure : Performance in Chicken.

7 .5 Outcomes vsrandom Opponent incoordination with Safety Outcomes vsomniscient Opponent incoordination with Safety Outcomes vsworstcase Opponent incoordination with Safety Figure : Performance in a coordination game. 7

8 .5 Outcomes vsrandom Opponent incubanmcus with Safety Outcomes vsomniscient Opponent incubanmcus with Safety Outcomes vsworstcase Opponent incubanmcus with Safety Figure : Performance in the Cuban Missile Crisis game as the Soviet player.

9 Outcomes vsrandom Opponent incubanmcsov.5 with Safety Outcomes vsomniscient Opponent incubanmcsov.5 with Safety Outcomes vsworstcase Opponent incubanmcsov.5 with Safety Figure 5: Performance in the Cuban Missile Crisis game as the US player. 9

10 Outcomes vsrandom Opponent inpursuer with Safety Outcomes vsomniscient Opponent inpursuer with Safety Outcomes vsworstcase Opponent inpursuer with Safety Figure : Performance in the Pursuer/Evader game as the evader.

11 Outcomes vsrandom Opponent inevader with Safety Outcomes vsomniscient Opponent inevader with Safety Outcomes vsworstcase Opponent inevader with Safety Figure 7: Performance in the Pursuer/Evader game as the pursuer.

12 Outcomes vsrandom Opponent ininspecterevader.5 with Safety Outcomes vsomniscient Opponent ininspecterevader.5 with Safety Outcomes vsworstcase Opponent ininspecterevader.5 with Safety Figure : Performance in the Inspector/Worker game as the worker.

13 Outcomes vsrandom Opponent inevaderinspecter.5 with Safety Outcomes vsomniscient Opponent inevaderinspecter.5 with Safety Outcomes vsworstcase Opponent inevaderinspecter.5 with Safety Figure 9: Performance in the Inspector/Worker game as the inspector.

14 Outcomes vsrandom Opponent inherostrong.5 with Safety Outcomes vsomniscient Opponent inherostrong.5 with Safety Outcomes vsworstcase Opponent inherostrong.5 with Safety Figure : Performance in the Hero game.

15 Outcomes vsrandom Opponent inmatchingpenniescyclea.5 with Safety Outcomes vsomniscient Opponent inmatchingpenniescyclea.5 with Safety Outcomes vsworstcase Opponent inmatchingpenniescyclea.5 with Safety Figure : Performance in the custom matching pennies game as player. 5

16 Outcomes vsrandom Opponent inmatchingpenniescycleb.5 with Safety Outcomes vsomniscient Opponent inmatchingpenniescycleb.5 with Safety Outcomes vsworstcase Opponent inmatchingpenniescycleb.5 with Safety Figure : Performance in the custom matching pennies game as player.

17 Outcomes vsrandom Opponent inpursuitisraelitesmoses.5 with Safety Outcomes vsomniscient Opponent inpursuitisraelitesmoses.5 with Safety Outcomes vsworstcase Opponent inpursuitisraelitesmoses.5 with Safety Figure : Performance in the Pursuit of the Israelites game as Pharoah. 7

18 Outcomes vsrandom Opponent inpursuitisraelitespharoah.5 with Safety Outcomes vsomniscient Opponent inpursuitisraelitespharoah.5 with Safety Outcomes vsworstcase Opponent inpursuitisraelitespharoah.5 with Safety Figure : Performance in the Pursuit of the Israelites game as Moses.

19 Outcomes vsrandom Opponent instag.. with Safety Outcomes vsomniscient Opponent instag.. with Safety Outcomes vsworstcase Opponent instag.. with Safety Figure 5: Performance in the Stag Hunt game. 9

20 9 7 Outcomes vsrandom Opponent intravellersdilemma with Safety Outcomes vsomniscient Opponent intravellersdilemma with Safety Outcomes vsworstcase Opponent intravellersdilemma with Safety Figure : Performance in the Travellers Dilemma game.