Microeconomics II Lecture 2: Backward induction and subgame perfection Karl Wärneryd Stockholm School of Economics November 2016

Transcription

1 Microeconomics II Lecture 2: Backward induction and subgame perfection Karl Wärneryd Stockholm School of Economics November

2 Games in extensive form So far, we have only considered games where players make their decisions simultaneously and independently of each other More generally, however, games may have a time order of decisions Example (coordination game): Player 1 L R Player 2 ` r ` r 1, 1 0, 0 0, 0 2, 2 An extensive form (or game tree) consists of nodes, which represent previous histories of the game and points at which some player makes a decision, branches, which represent actions by a player or by Nature, information sets, which contain all the nodes at which a player believes he may be at that point in the game, and finally payo s for each player at each terminal node In the example, all information sets consist of a single node This means we assume Player 2 observes Player 1 s initial move with certainty 2

3 A strategy of an extensive form game is a complete contingent plan of action, which for each information set a player could be at specifies what action he takes there In the example, Player 2 observes Player 1 s action, so a strategy for Player 2 in this game specifies one of Player 2 s actions ` and r for each of Player 1 s actions L and R Hence Player 2 s strategy set in this game is S 2 = {``, `r, r`, rr} (if we write what Player 2 does if Player 1 played L in the first position and what Player 2 does if Player 1 played R in the second position) Since Player 1 starts ( and therefore cannot make his action contingent on anything), a strategy for Player 1 is simply one of his actions L and R A Nash equilibrium is, of course, just as before, a strategy profile such that each player plays a best reply We may also write the game in its normal form, ie, listing all the possible strategy profiles and their corresponding payo s Player 2 `` `r r` rr Player 1 L 1, 1 1, 1 0, 0 0, 0 R 0, 0 2, 2 0, 0 2, 2 3

4 Now instead assume that Player 2 cannot observe Player 1 s initial move, or, equivalently, that they make their moves simultaneously The extensive form then looks as follows Player 1 L R ` r ` r Player 2 1, 1 0, 0 0, 0 2, 2 The dashed line connects the two nodes that form Player 2 s single information set This game has the following normal form Player 2 ` r Player 1 L 1, 1 0, 0 R 0, 0 2, 2 We can always find all the Nash equilibria of a game by studying its normal form But, as we shall see, some equilibria of extensive form games may involve behavior that seems implausible in a dynamic perspective, which is not captured by the normal form 4

5 Sequential games and backward induction Consider the following game tree Player 1 L 1 R 1 0,2 Player 2 L 2 R 2-1,-1 1,1 Player 1 first chooses between L 1 and R 1 If Player 1 chooses L 1, that is the end of the game Otherwise, Player 2, having observed Player 1 s move, gets to choose between L 2 and R 2 Example applications: Entry deterrence Wage bargaining The game has the following normal form Player 2 L 2 R 2 Player 1 L 1 0, 2 0, 2 R 1 1, 1 1, 1 We see that the game has two pure-strategy equilibria, (L 1,L 2 ) and (R 1,R 2 ) Player 2 would prefer the former, Player 1 the latter 5

6 But the normal form does not take into account the sequential structure of the game, ie, that decisions are not made at the same time Suppose that a (not very well educated) game theorist has recommended (L 1,L 2 ) as a solution to the game, but Player 1 gets a strange impulse, deviates, and instead plays R 1 Would Player 2 then actually play L 2? Not if he is rational, since he gets a higher payo by playing R 2 given that Player 1 has played R 1 The equilibrium (L 1,L 2 ) may therefore be said to rely on a non-credible threat, which would never actually be carried out Since Nash equilibria may imply dynamically irrational behavior of this type, we need to supplement the equilibrium concept with some additional, stronger criterion 6

7 In this case, we were able to solve the game using backward induction, ie, by starting by contemplating the last stage of the game and requiring that all earlier decisions be rational given the later ones Backward induction is an extension of the notion of iterated elimination of dominated strategies to games with a time dimension We observe that backward induction also relies on mutual knowledge of rationality (Up to what level?) We can now return to our first extensive form game and conclude that there is something fishy about the equilibrium that induces the payo pair (1, 1) All backward induction solutions are Nash equilibria, but the converse is not true Backward induction solutions are special cases of the more powerful concept of subgame perfection, which we shall deal with later on 7

8 Stackelberg oligopoly (1934) Consider the same model as in our earliest Cournot example, but assume one firm makes its quantity decision before the other one does Time order: 1 Firm 1 chooses q 1 2 Firm 2 observes q 1 and chooses q 2 3 Firm i gets payo i (q i,q j )=q i (p(q 1,q 2 ) c) =q i (a q 1 q 2 c) We start at the end with Firm 2 s decision, given q 1 Firm 2 s optimal choice depends on q 1 in the same manner as in the Cournot model, ie, its best reply given any q 1 it observes from Firm 1 is given by q 2(q? 1 )= a q 1 c 2 Now Firm 1 has to take into account that Firm 2 will adapt optimally later That is, Firm 1 wishes to maximize 1 (q 1,q 2(q? 1 )) = q 1 (a q 1 q 2(q? a q 1 c 1 ) c) =q 1 2 We di erentiate with respect to q 1, set the derivative equal to zero, and solve for Firm 1 s optimal output q? 1 = a 8 c 2

9 It follows that q? 2(q? 1)= a c 4 We may draw the firms iso-profit curves in the Cournot diagram a c q 2 (a c)/2 BR 1 (q 2 ) (a c)/2 a c q 1 BR 2 (q 1 ) We now see that the solution of the Stackelberg model involves letting Firm 1 pick its favorite point on Firm 2 s best-reply curve We note that Firm 1 s profit is higher and Firm 2 s lower than under Cournot competition with simultaneous decisions Firm 1 benefits from going first it has a first-mover advantage Equivalently, Firm 1 gains from being able to commit itself, ie, from, in contrast with Firm 2, not being able to react to its opponent s decision This is also an example of how better information may hurt a player in a game, something that cannot happen in a single-person decision problem 9

10 Bargaining with alternating o ers Two players take turns suggesting how to split $ 1 Both have the discount factor 2 (0, 1), ie, $ x in the next period is only worth $ x now Time order: Period 1 a Player 1 proposes s 1 for himself, 1 s 1 for Player 2 b Player 2 accepts or rejects the proposal If he rejects, the game continues to period 2 Period 2 a Player 2 proposes s 2 for Player 1, 1 s 2 for himself b Player 1 accepts or rejects the proposal If he rejects, the game continues to period 3 Period 3 Player 1 gets s, Player 2 gets 1 exogenously given s, where s is We begin backward inducting by assuming play has reached period 2 Period 2 Player 1 can get s in period 3 if he rejects Player 2 s proposal Hence he only accepts if s 2 s Therefore the most Player 2 can get in period 2 is 1 s Otherwise he gets 1 s in period 3, which is worth (1 s) now Since we have (1 s) < 1 s, he must propose s? 2 = s Period 1 Player 1 knows that Player 2 is guaranteed 1 s? 2 in period 2, which is worth (1 s? 2) now It follows that s? 1 =1 (1 s? 2)=1 (1 s) 10

11 Rubinstein (1982) discusses the case of an infinite time horizon* Here is a (possibly somewhat suspect) way of handling this Suppose we start by adding 2 more periods of alternating bidding along the same lines Note that in period 3 a game then begins that is identical to the 2-period game, and therefore has the same solution In the game starting in period 3 Player 1 is therefore guaranteed 1 (1 s) Following similar logic as before, the solution of the fourperiod problem is then s? 1 =1 (1 (1 (1 s))) If we continue adding 2 periods at a time, Player 1 s equilibrium payo approaches = 1X i=0 ( ) i = 1 1+, since we know that P 1 i=0 xi =1/(1 x) for 0 apple x<1 (a fact we shall shortly use again) * Ståhl (1972) pioneered the study of the finite time horizon problem 11

12 Subgame perfection So far, we have studied extensive form games with perfect information in the sense that each player can observe all actions taken previously in the game Unless this is the case, we cannot use backward induction Example: Player 1 L 1 R 1 2,5 Player 2 L 2 R 2 `1 r 1 `1 r 1 1,0 0,0 0,2 5,2 Player 1 The dashed line signifies that Player 1 does not know which of the connected nodes he is at This could be, eg, because both players move simultaneously after Player 1 has played R 1 We say that both of the connected nodes are in Player 1 s information set if he gets to act again That is, a player s information set at some point in a game is the set of nodes at which he could be at that point If the player knows for certain that he is at a particular node, then the information set consists of that node only 12

13 Our example game has the following normal form Player 2 L 2 R 2 Player 1 L 1`1 2, 5 2, 5 L 1 r 1 2, 5 2, 5 R 1`1 1, 0 0, 2 R 1 r 1 0, 0 5, 2 13

14 We also see that we cannot use backward induction in the extensive form, since Player 1, if it became his turn again, would like to play `1 at the left node of his information set and r 1 at the right node There is no one action which is best in both cases But note that the part of the game that starts with Player 2 acting may be considered a game in its own right, with the following normal form Player 2 L 2 R 2 Player 1 `1 1, 0 0, 2 r 1 0, 0 5, 2 This subgame has the unique pure Nash equilibrium (r 1,R 2 ), with payo s (5, 2) Hence Player 1 should play R 1 and get 5, rather than play L 1 and get 2 14

15 Selten (1965) introduced the notion of a subgame perfect equilibrium, a generalization of the idea of backward induction to games with imperfect information Subgame perfect equilibria are Nash equilibria that are not based on non-credible threats We now also note that two di erent types of randomization are possible in extensive form games A mixed strategy is, as before, a probability distribution over pure strategies of the normal form But since a player now may have more than one information set, we can also allow a player to assign local, independent probabilities over the actions available in a particular information set A set of such probability distributions, one for each information set, is called (for obscure historical reasons) a behavior strategy Since these two notions are equivalent in games with perfect recall, the only type we shall study, in the following we shall consider behavior strategies This allows us to define subgame perfection also with randomization 15

16 The Absent-Minded Driver Here is a single-person decision problem with imperfect information (due to Piccione and Rubinstein 1997) A man is sitting in a bar late at night He knows that once he is on the road and comes to an exit, he will be unable to recall whether he has passed an exit already Bar E Bad place (0) C E Home (4) C Hotel (1) What is an optimal pure strategy for this problem? Since the driver should realize he will never be able to get home, he should decide to (C)ontinue at an exit But then when he actually gets to an exit, it may be either one with equal probability Then it is now optimal to (E)xit Hence optimal plans may become inconsistent over time, without any new information, change of preferences, or the like There is also an optimal nontrivial behavior strategy, which you will be asked to find Note that in standard games against Nature, randomizing can never be optimal 16

17 Definition (Subgame) A subgame consists of a node that is the only one in its information set and is not a terminal node, and all descendant nodes of this node, and contains all nodes of the information sets that are part of the subgame It follows that the entire game itself is always a subgame Definition (Subgame perfect equilibrium) A Nash equilibrium is subgame perfect if it prescribes a Nash equilibrium for every subgame Since every backward induction solution is a subgame perfect equilibrium, we shall use the latter, more general term in the following 17

18 Example: Player 1 L 1 R 1 0,3 Player 2 L 2 R 2 1,-1 `1 r 1-2,-2 0,2 The game has the following normal form Player 1 Player 2 L 2 R 2 Player 1 L 1`1 0, 3 0, 3 L 1 r 1 0, 3 0, 3 R 1`1 1, 1 2, 2 R 1 r 1 1, 1 0, 2 There are four equilibria in pure strategies, (L 1`1,R 2 ), (L 1 r 1,R 2 ), (R 1`1,L 2 ), and (R 1 r 1,R 2 ) 18

19 There are three subgames: The game itself, the subgame starting at Player 2 s node, and the subgame starting at Player 1 s last node In the last subgame, Player 1 has to play r 1 The subgame starting at Player 2 s node has the following normal form Player 2 L 2 R 2 Player 1 `1 1, 1 2, 2 r 1 1, 1 0, 2 This subgame has two equilibria in pure strategies, (`1,L 2 ) and (r 1,R 2 ) Since subgame perfection implies that Player 1 must not play `1, only (r 1,R 2 ) is subgame perfect in the subgame starting at Player 2 s node It follows that only (L 1 r 1,R 2 ) and (R 1 r 1,R 2 ) are subgame perfect equilibria of the game as a whole 19

20 Bertrand competition with entry costs We study a model with two firms and two time periods In the first period, the firms simultaneously and independently decide whether they each want to make an investment 0 < k<((a c)/2) 2, which is necessary in order to produce in the market In the second period, a firm that has decided to enter sets its price and meets demand given by q(p) =a p If both firms have entered, everything is like in the Bertrand example with homogenous goods and common constant average cost c A firm that has not entered gets zero profit We now look for a subgame perfect equilibrium of this game We start by considering the four possible period 2 subgames that start when both firms have entered, Firm 1 has entered but not Firm 2, Firm 2 has entered but not Firm 1, and neither has entered, respectively 20

21 Case 1: Both firms have entered In the unique equilibrium of this subgame, we know from before that both firms must set price equal to marginal cost c Case 2: Firm i has invested, but Firm j 6= i has not invested Firm i then has the profit function which is maximized when i (p i )=(p i c)(a p i i = a 2p i + c =0, ie, when Firm i sets its monopoly price (a + c)/2 and hence gets profit ((a c)/2) 2 (Note that the fixed cost k is sunk at this stage) Case 3: Neither of the firms has invested Since they are not operating in the market, neither one gets any profit 21

22 Viewed from the perspective of the start of the game, given rational behavior later (in the sense that equilibria are played), the game can now be reduced to the following normal form Firm 2 Invest Firm 1 Invest k, k a c Don t invest 0, a c 2 2 Don t invest 2 2 k 0, 0 k, 0 Hence there is only one type of subgame perfect equilibrium in pure strategies in this game, namely that one firm stays out and the other one enters and sets the monopoly price Even for k arbitrarily small, the basic Bertrand result has disappeared completely Instead of price equal to marginal cost, we get monopoly 22

23 Since the game is symmetric, having two asymmetric solutions may be considered unsatisfying, since there is no method for determining which of the firms should enter However, there is also a symmetric equilibrium in behavior strategies Let r be a firm s probability of investing in period 1 In a subgame perfect equilibrium in behavior strategies, we must have that 2 a c r( k) + (1 r) k! = r 0 + (1 r) 0, 2 ie, that r = ((a c)/2)2 k ((a c)/2) 2, and of course that the firms play the equilibria in period 2 23

24 Does this game have any other Nash equilibria? Yes For example, the subgame perfection criterion has rejected equilibria of the following type: Firm 1 invests, and sets the monopoly price if Firm 2 does not invest, but if Firm 2 invests, Firm 1 sets its price lower than c Firm 2 does not invest This is a best reply for Firm 2, since k is the best payo it can get if it invests Firm 1 s strategy is a best reply since Firm 2 does not invest, which gives Firm 1 the monopoly profit minus the fixed cost But this equilibrium is based on the non-credible threat on the part of Firm 1 to set price less than average cost, something a rational Firm 1 would never actually do if Firm 2 entered after all 24

25 A finitely repeated game The following game is played twice without discounting, ie, payo s for the whole game are simply the sum of payo s from each period The players can observe what happened in the first period before the second period starts Is it possible to get the payo profile (4, 4) in the first period in a subgame perfect equilibrium in pure strategies? Player 2 L C R Player 1 T 3, 1 0, 0 5, 0 M 2, 1 1, 2 3, 1 B 1, 2 0, 1 4, 4 Yes, that is indeed possible Note that the complicating factor is that B is not a best reply for Player 1 against R in the one-period game Player 2 would not mind (B,R) Hence the equilibrium must be such that Player 1 is given an incentive to play B in the first period, eg, by a threat of punishment in period 2 We know that subgame perfection requires that a oneperiod equilibrium is played in period 2 Hence (T,L) or (M,C) has to be played in period 2 Player 1 would prefer (T,L) Could one, perhaps, let him have (T,L) if he behaves nicely in the first period and (M,C) otherwise? 25

26 Consider the following strategy pair: Player 2: Play R in period 1 If (B,R) was played in period 1, play L in period 2 If something other than (B,R) was played in period 1, play C in period 2 Player 1: Play B in period 1 If (B,R) was played in period 1, play T in period 2 If something other than (B,R) was played in period 1, play M in period 2 Suppose Player 2 plays the proposed strategy, but Player 1 deviates optimally The best thing he can do in period 1 is then to play T and get the payo 5 Then Player 2 will play C in period 2, so Player 1 has to play his best reply M, and in total gets 5+1=6 If instead he plays the proposed strategy, he gets 4+3=7 So the proposed strategy is indeed a best reply for Player 1 It is easy to see than the proposed strategy for Player 2 is also a best reply Hence we have found an equilibrium It is subgame perfect, since it always prescribes play of a one-period equilibrium in period 2 26

27 The Centipede (Rosenthal 1982) Although backward induction is an a priori appealing principle of rationality, it can lead to conclusions that are intuitively unsatisfying Here is a popular example 1 C 2 C 1 C 2 C 1 C 2 C S S S S S S 1, 0 0, 2 3, 1 2, 4 5, 3 4, 6 6, 5 Players 1 and 2 participate in a mysterious process where the potential gain for both keeps increasing if no player stops The players take turns choosing between stopping (S) or continuing (C) Each player would prefer stopping when it is his turn over his opponent stopping the next time The unique backward induction solution of this game is for Player 1 to stop immediately, even though both would have been better o if the game had continued This seems particularly unreasonable if the game is even bigger 27

28 The backward induction solution is based on the idea that a player always expects his opponent to stop the next time it is his turn even if he has continued many times before This reveals a paradox in the application of the notion of common knowledge of rationality The argument depends on, among other things, the idea that Player 2 would act rational at his last decision node But this node is never reached unless Player 2 has acted irrationally earlier! This paradox is still a controversial issue at the frontiers of game theory research Feel free to contribute 28

29 Ultimatum bargaining Actual human beings often act in ways that appear to contradict game-theoretical rationality, eg, as it is embodied in the notion of backward induction A game that really brings this out is the ultimatum bargaining game of Werner Güth This 2-player game concerns the division of a sum of money M > 2 Player 1 proposes a split that gives him M 1 apple M 1 cents and Player 2 the rest Player 2 then either accepts or rejects the proposal In case Player 2 rejects, nobody gets anything Any value of M 1 could result from some Nash equilibrium of this game For instance, consider only strategies on the part of Player 2 of the form accept if M M 1 M M? 1, reject otherwise, for some M? 1 Given that Player 2 plays such a strategy, it is a best reply for Player 1 to suggest M 1 = M? 1 Given that Player 1 suggests M 1 = M? 1, Player 2 s strategy is a best reply (since it is always best for Player 2 to accept) But the unique backward induction solution is the one where M 1 = M 1 and Player 2 accepts, since a rational Player 2 must accept all proposals 29

30 In experiments carried out by Güth and others, where anonymous subjects play ultimatum games, Player 2 typically rejects some proposals that give him more than one cent (or the lowest specified payo ) Many explanations have been o ered for this type of behavior Güth thinks this shows that people are simply not rational in the game-theoretical sense, but are instead guided by social norms concerning what is a fair division in situations like this Others think di erently 30

31 Problem Find the optimal behavior strategy for the absent-minded driver problem Problem Firm 1 and Firm 2 produce imperfect substitutes Demand is given by q 1 =1 2p 1 + p 2 and q 2 =1 2p 2 + p 1, where q i and p i are Firm i s quantity and price, respectively Each firm has a large unsold stock of its good, and hence no production cost Now consider a game with two stages In stage one, the firms simultaneously and independently decide whether to be price-setters or quantity-setters In stage two, they play the market game, controlling the decision variables they selected in the first stage Find a subgame perfect equilibrium outcome of this game 31

32 Problem Consider the following game Player 2 S 2 C 2 Player 1 S 1 5, 2 3, 1 C 1 6, 3 4, 4 a) Suppose the players make their choices simultaneously Find the unique equilibrium b) Suppose Player 1 makes his decision before Player 2 makes his, and Player 2 observes Player 1 s decision Find the normal form and the extensive form of this game Find the backward induction solution c) Suppose Player 1 makes his decision first, but Player 2 does not directly observe Player 1 s decision Instead, Player 2 gets a signal, which with some very small probability " gives the wrong indication about Player 1 s choice That is, we have P ( = S 1 S 1 )=1 " and P ( = C 1 C 1 )=1 " Find the normal form of this game Find the unique equilibrium (Note that we cannot apply backward induction in this case) 32

33 Problem Find subgame perfect equilibria in the following five games Player 1 L R Player 2 ` r u d 10, 2 2, 1 1, 1 1, 1 Player 1 L R ` r ` r Player 2 0, 0 0, 0 0, 0 1, 1 33

34 Player 1 L R Player 2 ` r u d 10, 10 10, 5 5, 10 5, 5 Player 1 L 1 R 1 2,1 Player 2 L 2 R 2 `1 r 1 `1 r 1 1,2 0,2-1,-1 4,2 34 Player 1

35 Player 1 `1 r 1 1, 2, 4, 4 `2 r 2 Player 2 2, 1, 3, 3 `3 r 3 Player 3 Player 4 `4 r 4 `4 r 4 0, 0, 2, 1 2, 2, 0, 0 2, 2, 0, 0 0, 0, 1, 2 35

36

37

38

39

40

41