Section Notes 6. Game Theory. Applied Math 121. Week of March 22, understand the difference between pure and mixed strategies.

Transcription

1 Section Notes 6 Game Theory Applied Math 121 Week of March 22, 2010 Goals for the week be comfortable with the elements of game theory. understand the difference between pure and mixed strategies. be able to identify a player s best reponse. understand the concept of a Nash equilibrium and know how to find Nash equilibria. Contents 1 Let s play a game: a burglary gone bad Two player games, Payoff Matrices, Pure Strategies 3 3 Best Responses and Dominated Strategies 5 4 Pure Nash Equilibrium 7 5 MiniMax Technique for finding Mixed Nash Equilibria in Zero-Sum Games 8 1

2 1 Let s play a game: a burglary gone bad... Exercise 1 Imagine the following scenario: two suspects, A and B, are arrested by the police. The police believes suspects A and B have committed a burglary, but they have insufficient evidence for a conviction. Thus they will offer the prisoners the following deal: if one testifies for the prosecution against the other and the other remains silent, the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only one year in jail for a minor charge (e.g. trespassing). If each betrays the other, each receives a five-year sentence. Each prisoner must make the choice of whether to betray the other or to remain silent. 1. What should the two prisoners do when they are held in the same room and can communicate with each other and can make a decision on what to do together? 2. What should each prisoner individually do if they are kept separate, i.e. the prisoners cannot talk to each other? 3. Give an explanation for why this game is called The Prisoner s Dilemma (in the second case when the players are kept separate). End Exercise 1 By completing the above exercises, you should now: have some intuition about a non-zero sum game be able to identify how players would look at their rewards. 2 Two player games, Payoff Matrices, Pure Strategies So far in this course we have used optimization methods (linear programming up until now) to maximize the objective function of an individual assuming that all the decision making power is in the hands of that individual. However, the situation changes drastically when 2 or more players are competing with each other. Generally, different players have conflicting interests, i.e. they want to maximize different objective functions (imagine two car companies, imagine two presidential candidates,... ). In these cases, we can use game theory to analyze such situations, reason about 2

3 what will probably happen, and maximize our personal utility given the actions of all the other players around. 2.1 Review A two-player game is defined by the following elements: A set of player 1 and 2. A set of actions {a 1,..., a n }, {b 1,..., b m } for each player 1,2. These actions are also called pure strategies. A payoff matrix defining the utility for each combination of strategies played by each player. The entry a ij specifies a tuple (π 1 (i, j), π 2 (i, j)) for the case when player 1 chooses action i and player 2 chooses action j. The tuple (π 1 (i, j), π 2 (i, j)) denotes that player 1 gets utility π 1 (i, j) and player 2 gets utility π 2 (i, j). If all entries a ij of the payoff matrix is such that payoffs for players 1 and 2 add up to 0, that is, π 1 (i, j) + π 2 (i, j) = 0 i, j, then we call such a game a two-player zero sum game. Sometimes, we will then simply omit the payoffs for player 2 when writing down the payoff matrix for a zero sum game, because they are simply the negative of the payoffs for player Practice Exercise 2 Consider the following TF-Student Game: Every week, a TF of AM121 has to choose whether he puts a lot of effort, little effort, or no effort at all into preparing for the section. The students, on the other hand must decide whether they attend the section or not (for simplicity, let s model the students as one player here). Obviously, preparing the section is costly for the TF, but if a lot of students come and he is well-prepared, he gets a very high reward. On the other, if he is badly-prepared and his students attend section he will be embarrassed. The students payoff, of course, depends on how well the TF is prepared. And obviously, if the students don t attend section, they get 0 payoff. 1. List all pure strategies for both players of the game. 2. Write down a payoff matrix with payoffs that you think represent the TF-Student game most appropriately. 3. What do you think will be the outcome of the game when played? End Exercise 2 3

4 By completing the above exercises, you should now: understand when game theory is used be comfortable with the elements of game theory. 3 Best Responses and Dominated Strategies 3.1 Review We need a few more solution concepts: 1. Best Response: When the strategy of player 2 is given, player 1 has a best response strategy to the strategy by player 2. Remember, in the Prisoner s Dilemma game, the best response of player 1 was always to testify, independent of what player 2 s strategy was. In general, when player 2 s strategy j is fixed, the best response of player 1 is found by examining all payoff entries a 1j,..., a nj that correspond to player 2 s choice and selecting the strategy that maximizes player 1 s payoff among these choices. 2. Dominated Strategy: Strategy i for player 1 dominates strategy i if for all choices of j by player 2 the payoff for player 1 π 1 (i, j) π 1 (i, j) and π 1 (i, j) > π 1 (i, j) for some j {1,..., m}. We say that strategy i is dominated by strategy i for player Iterated Elimination of Dominated Strategies: No player will ever choose a dominated strategy. Thus, those can be eliminated from the set of pure strategies. This process can be iterated, i.e. after one strategy has been eliminated (by either player) any remaining dominated strategies can be eliminated (by either player) and so on. 4

5 3.2 Practice Exercise 3 For this exercise, consider the following game: Player 2 Left Center Right Up (3,0) (0,1) (0,2) Player 1 Middle (0,0) (2,2) (1,-1) Down (0,0) (3,1) (0,0) 1. What is the best response of player 1 when player 2 plays Right? What is the payoff he gets? 2. What is the best response of player 2 when player 1 plays Middle? What is the payoff he gets? 3. Apply iterated elimination of dominated strategies to this game. 4. What are the strategy profiles (tuples of strategies) that remain? What are the corresponding payoffs for both players? End Exercise 3 By completing the above exercises, you should now: be able to identify a player s best reponse and dominated strategies. know how to iteratively remove dominated strategies. 4 Pure Nash Equilibrium 4.1 Review Probably the most important definition in game theory: 1. In a two-player game, with players 1 and 2, strategies (x, y) are a Nash equilibrium (NE) if π 1 (x, y) π 1 (x, y) for all x x, and if π 2 (x, y) π 2 (x, y ) for all y y. In other words, (x, y) is a NE if x is a best response to y and if y is a best response to x. 5

6 2. We denote a NE where the players are only allowed to play pure strategies (i.e. they have to choose a single strategy from the finite set of strategies) as pure Nash equilibrium. 3. Mixed Strategies: Let x = (x 1,.., x n ), x i 0, n i=1 x i = 1 denote a mixed strategy of row player; the probability x i denotes the probability with which the row player plays each strategy i {1,..., n} 4. A Nash equilibrium where one or more players play a mixed strategy is called a mixed Nash equilibrium. 4.2 Practice Exercise 4 1. Find all pure Nash equilibria of the Prioner s Dilemma game. 2. In the previous exercise we solved the following game Player 2 Left Middle Right Up (3,0) (0,1) (0,2) Player 1 Middle (0,0) (2,2) (1,-1) Down (0,0) (3,1) (0,0) Find all pure NE s of this game. 3. Remember the TF-Student game: Students attend section don t attend section much effort (10,10) (-10,0) TF little effort (5,5) (-5,0) no effort (-5,-5) (0,0) Find all pure NE s of this game. What do you think will happen if this game will be played? End Exercise 4 6

7 By completing the above exercises, you should now: understand when a Nash equilibrium exists and how to identify it. understand the difference between pure and mixed strategies. 5 MiniMax Technique for finding Mixed Nash Equilibria in Zero- Sum Games 5.1 Mixed Nash Equilibria Pure Nash equilibria can be found by simply analyzing the payoff table and checking whether a pair of strategies constitutes best responses to each other. Finding mixed strategy Nash equilibria is somewhat harder. We will restrict ourselves to mixed Nash equilibria for two-player zero-sum games here, because only for them can we use particularly nice techniques. But let s first see an example of a mixed Nash equilibrium: Exercise 5 Consider the following zero-sum game, called Matching Pennies. Both players have a set of pennies. In each round, each player must put one penny on the table. If both pennies show heads or tails, player 1 gets both pennies. If both pennies show something different, player 2 gets both pennies. This is the corresponding payoff matrix: P2 Heads Tails Heads (1,-1) (-1,1) P1 Tails (-1,1) (1,-1) Obviously this is a zero-sum game. 1. Find all pure Nash equilibria of this game. 2. Find all mixed Nash equilibria of this game. End Exercise 5 7

8 5.2 Review: MiniMax Technique in zero-sum games Remember the special structure of two-player zero-sum games: The payoff for player 2 is simply the negative of player 1 s payoff. This is why, when writing down the payoff matrix, it is sufficient to only write down player 1 s payoff and assume that player 1 s goal is to maximize his payoff and player 2 s goal is to minimize his payoff. It now makes sense to introduce the following definitions: 1. Maximin Strategy: The maximin strategy for the row player (player 1) is to choose the row with the maximum worst case payoff, i.e. maximize assuming that the column player minimizes. More formally, player 1 will choose max i min j π 1 (i, j). 2. Minimax Strategy: The minimax strategy for the column player (player 2) is to choose the column with the minimum worst case payoff, i.e. minimize assuming that the row player maximizes. More formally, player 2 will choose min j max i π 1 (i, j). 3. Minimax Criterion: A strategy profile (x, y) satisfies the minimax criterion if x is player 1 s maximin strategy and y is player 2 s minimax strategy and if the same entry of the payoff matrix yields the maximin value for player 1 and the minimax value for player 2, i.e. if max i min j π 1 (i, j) = min j max i π 1 (i, j). 4. Theorem: For every zero sum, two-player game the pair of strategies (x, y ), optimal according to the minimax criterion, is a Nash equilibrium, and if v is the maximin value and w is the minimax value then v = w = v is the value of the game. 8

9 5.3 Practice Exercise 6 1. Consider the following zero-sum game: Player 2 Left Middle Right Up Player 1 Middle Down Find a NE of this game via the minimax criterion. 2. Now consider the zero-sum game Matching Pennies again: P2 Heads Tails Heads (1,-1) (-1,1) P1 Tails (-1,1) (1,-1) Can you easily find a NE of this game via the minimax criterion? End Exercise 6 By completing the above exercises, you should now: be able to identify when there is a pure or mixed Nash equilibrium. understand how to identify what strategy a player will pick using the minimax technique (for zero-sum games) 5.4 Review: Solving two-player zero-sum games via Linear Programming Note that we can easily compute the maximin strategy via an LP: max min x j n a ij x i i=1 9

10 n s.t. x i = 1 x i 0 i {1,..., n} where we can easily remove the non-linearity in the objective function via: i=1 max v s.t. v n i=1 a ijx i j {1,..., m} n i=1 x i = 1 x i 0 i {1,..., n} We can do the analogous process for the minimax strategy: min max y j s.t. n a ij y j i=1 m y j = 1 j=1 y j 0 j {1,..., m} where we can easily remove the non-linearity in the objective function via: min w s.t. w m j=1 a ijy j i {1,..., n} n j=1 y j = 1 y j 0 j {1,..., m} Now note that the LP for the minimax strategy is the dual of the LP for the maximin strategy. From duality we know that if the primal has optimal value v then the dual has the same value, i.e. w = v. Thus, to find a Nash equilibrium in a two-player zero-sum game, we effectively only have to solve one LP. 5.5 Practice Exercise 7 Formulate an LP to solve for the NE of the matching pennies problem. End Exercise 7 10

11 By completing the above exercises, you should now: be comfortable writing an LP to find a Nash equilibrium. 11

12 6 Solutions Solution 1 1. We can represent the game the two players a playing via the following payoff matrix: P2 silent P2 testify P1 silent (-1,-1) (-10,0) P1 testify (0,-10) (-5,-5) Because the players can talk to each other and make a decision together it is reasonable to assume that they are cooperative, i.e. their goal is to maximize their joint utility (minimize the sum of the years they spend in prison). From the matrix we see that the best thing the players can agree on is to both remain silent such that they both get a 1-year sentence. 2. This time, because the players cannot talk to each other and can t make a decision together, they will maximize their individual payoff (we assume they are selfish). Again, looking at the payoff matrix, we can see that independent of what player 2 does, it is better for player 1 to testify. Thus, he will testify. The same reasoning applies to player 2 (by symmetry), thus player 2 will also testify. Thus, the outcome will be 5 years in prison for both players. 3. This game is called Prisoner s Dilemma because it is a true dilemma for the players. They both know that if they follow their selfish strategies, they will end up with 5 years in prison while cooperating would have resulted in a better outcome of only 1 year in prison. However, cooperation is not an equilibrium in this game and thus the best an individual player can do is to testify against the other player. Both players testifying is the only equilibrium of this game. End Solution 1 Solution 2 1. Pure strategies for the TF: a) much effort, b) little effort, c) no effort. Pure strategies for the students: a) attend section, b) don t attend section. 2. The payoff matrix for this game should look similarly (at least structurally) to the one below. Note that we have just made up some numbers that we think represent the TF-Student game well. Of course your specific payoff numbers may deviate from ours. Students attend section don t attend section much effort (10,10) (-10,0) TF little effort (5,5) (-5,0) no effort (-5,-5) (0,0) 3. From our specific payoff matrix we can immediately see that (much effort, attend section) is the best outcome for both players. Thus, we would hope that this is in fact what happens when the game is played. End Solution 2 12

13 Solution 3 1. The best response of player 1 is to play Middle which gives him a payoff of 1 (vs. 0 when he would play Up or Down). 2. The best response of player 2 is to play Center which gives him a payoff of 2 (vs. 0 or -1 when he plays Left or Right). 3. First we eliminate Left which is strictly dominated by Center. Then we eliminate Up which is strictly dominated by Middle. Then we eliminate Right which is now strictly dominated by Center. Then we eliminate Middle which is now strictly dominated by Down. 4. We are left with the strategy profile (Down, Center) which gives player one a payoff of 3 and player 2 a payoff of 1. End Solution 3 Solution 4 1. The only NE of the game is (testify, testify). As discussed before, testifying is always a best response for both players independent of what the other players choose. 2. The only NE of this game is (Down, Middle). This is easy to see because we had eliminated all other strategies via iterated elimination of dominated strategies in the previous exercise. As we know that a player will never play a dominated strategy, if there is only one strategy profile remaining, it must be a NE. 3. There are two Nash equilibria in this game: (much effort, attend section) is one because much effort is the TF s best response to attend section, and attend section is the students best response to much effort. But (unfortunately), there is also a second NE in this game, namely (no effort, don t attend section) due to analogous reasoning. If this game is played, it is unclear what will happen because there is not a unique NE. However, because the payoffs for the NE (much effort, attend section) are strictly higher for the TF AND for the students than for the other NE, it is likely that this NE will be chosen by both players. In particular, if this game is played repeatedly over time, some sort of coordination for the preferred equilibrium is likely to emerge ;-) End Solution 4 Solution 5 1. There are no pure NE in this game. 2. The only mixed NE is (1/2H + 1/2T, 1/2H + 1/2T ), i.e. both players play Heads and Tails with 50% probability. That this is a NE is easy to see because given player 1 s strategy, any strategy for player 2 is a best response and vice versa. End Solution 5 13

14 Solution 6 1. Player 1, following the maximin strategy, chooses row Middle to get a maximal worst-case payoff of 1. Player 2, following the minimax strategy, chooses column Middle, for a minimum worst-case payoff of 1. Note that the minimax and the maximin values are the same. Thus, we found a NE, namely (Middle, Middle). Note that we could not have found this NE via iterated elimination of dominated strategies. 2. No, applying the minimax criterion to the matching pennies game is not that easy because (as we have already seen before) we need to find mixed strategies for the Nash equilibrium. End Solution 6 Solution 7 max v s.t. v x x 2 ( 1) v x 1 ( 1) + x 2 1 x 1 + x 2 = 1 x 1, x 2 0 which we bring into the following form: max v s.t. v x 1 + x 2 0 v + x 1 x 2 0 x 1 + x 2 = 1 x 1, x 2 0 We can the solve this LP in AMPL. And, the solution vector will be x = (0.5, 0.5) and the value of the game is 0. expected, right? End Solution 7 All as 14