Strategies and Game Theory

Transcription

1 Strategies and Game Theory Prof. Hongbin Cai Department of Applied Economics Guanghua School of Management Peking University March 31, 2009

2 Lecture 7: Repeated Game 1 Introduction 2 Finite Repeated Game 3 Infinitely Repeated Games

3 1 Introduction 2 Finite Repeated Game 3 Infinitely Repeated Games

4 Short term and Long Term Relationships Many interactions are not one-shot, but last for a while: e.g., family, friends, teachers-students, firm-firm relationships, politicians and voters,... People in long term relationship seem to behave differently from when they are in one-shot interactions: customers versus waiters/waitress: local or highway restaurants. friends versus strangers trader associations versus anonymous trading firms and suppliers: the American versus Japanese models

5 Why do people behave differently in long term relationships Self-selections: only if you like someone will you deal with him/her again in the future Psychological: the longer you interact with someone, the more you like them. Rational: people have different incentives in long term relationships

6 1 Introduction 2 Finite Repeated Game 3 Infinitely Repeated Games

7 Example 1: Prisoner s Dilemma 2 C NC 1 C 1, 1 5, 0 NC 0, 5 4, 4 If this game is played once, (C, C) is played. Can we get the Pareto dominance outcome (NC, NC) if the two players play this game repeatedly?

8 Example 1: Prisoner s Dilemma If we play the prisoner s dilemma twice, what is the subgame perfect equilibrium? Note that there are five subgames in this finite repeated game. There is one subgame in stage 1, but four in stage 2, each after one possible history of stage 1. In fact, the number of subgames increase dramatically as the game is repeated as shown below. This is because there is a subgame after every history. Therefore, do not attempt to draw the entire game tree unless necessary. stage number of subgames total = = = = 64 85

9 Example 1: Prisoner s Dilemma We may solve the two stage prisoner s dilemma by backward induction: 1 Stage 2: This is just the one-shot prisoner s dilemma game. Hence, there is a unique equilibrium, that is, (C, C) will be played no matter what happened in the first stage.

10 Example 1: Prisoner s Dilemma 2 Stage 1: Plugging in the second stage payoffs, the reduced game is (in normal form): 2 C NC 1 C 2, 2 6, 1 NC 1, 6 5, 5 Since C > 1 NC and C > 2 NC, (C, C) is the unique Nash equilibrium. Note that this reduced game has payoffs almost identical to that of the stage game. The only difference is the reduce game payoffs are all shifted up by 1 since the second stage payoff for (C, C) is 1.

11 Example 1: Prisoner s Dilemma 3 Hence, the only subgame perfect equilibrium is stage1 stage2 stage1 {}}{ stage2 {}}{{}}{{}}{ ( C CCCC, C CCCC). In other words, no cooperation can be supported.

12 No-Cooperation of the Finite Prisoner s Dilemma By the same logic, even if we repeated the prisoner s dilemma 1,000 times, it can easily be shown that they will play (C, C) at stage 1000, moving back to stage 999, still play (C, C), stage 998 is still (C, C), etc. Hence, the unique subgame perfect equilibrium is playing (C, C) forever, i.e., finitely repeated PD games cannot induce cooperation!

13 No-Cooperation of the Finite Prisoner s Dilemma The reason is that the backward induction logic always holds. In fact, there are two critical points for this result: 1 Finiteness: As long as there is an end, backward induction always goes through. 2 The stage game has only one Nash equilibrium: A player won t be punished if he cheated" before. Neither can he be rewarded if he cooperated" before.

14 Example 2: Modified Prisoner s Dilemma Let try to change the second point described above. Consider using a modified prisoner s dilemma with more than one Nash equilibrium as our stage game: 2 C NC D C 1, 1 5, 0 0, 0 1 NC 0, 5 4, 4 0, 0 D 0, 0 0, 0 3, 3 The Nash equilibria are (C, C) and (D, D). By IEDS, we still have weak dominance: C 1,2 NC, but now have one more equilibrium (D, D).

15 Example 2: Modified Prisoner s Dilemma Consider the following strategy: Play NC in stage 1, s1 = s2 = Play C in stage 2 if stage 1 s outcome (NC, NC), Play D in stage 2 if stage 1 s outcome = (NC, NC).

16 Example 2: Modified Prisoner s Dilemma Let s see if this a subgame perfect equilibrium: 1 For stage 2: (C, C) and (D, D) are both Nash equilibrium. 2 For stage 1: According to the strategy profile, by adding all of the second stage payoffs into the first stage, we may construct the normal form of the reduced game. Here, (NC, NC) becomes a Nash equilibrium! 2 C NC D C 2, 2 6, 1 1, 1 1 NC 1, 6 7, 7 1, 1 D 1, 1 1, 1 4, 4 3 Hence, (s 1, s 2 ) is indeed a subgame perfect equilibrium, and the outcome is (NC, NC) in stage 1 and (D, D) is stage 2. The payoffs are (7, 7).

17 Example 2: Modified Prisoner s Dilemma Note that by using the punishment (C, C) and the reward (D, D), we can support cooperation to achieve the efficient outcome (NC, NC) in stage 1, even though (NC, NC) is not even a Nash equilibrium in the stage game! Also note that this is not the unique subgame perfect equilibrium. For example, playing (C, C) twice is a subgame perfect equilibrium. In fact, repeating a stage game Nash equilibrium is always a subgame perfect equilibrium. Hence, playing (D, D) twice is also a subgame perfect equilibrium. There are many subgame perfect equilibria. However, we mainly care about (s1, s 2 ) because it supports cooperation.

18 Renegotiation Proof There is a potential problem in (s1, s 2 ): If player 1 cheated and played C in the first stage, can she convince player 2 not to punish her in the second stage? Since 3 > 1, renegotiation is possible if they can communicate with each other. Hence, we need a further modification to solve this.

19 Example 3: Further Modified Prisoner s Dilemma Suppose the following stage game is played twice: 2 A B C D A 3, 1 0, 0 0, 0 5, 0 1 B 0, 0 1, 3 0, 0 0, 0 C 0, 0 0, 0 2, 2 0, 0 D 0, 0 0, 5 0, 0 4, 4 (D, D) is the efficient outcome but not a Nash equilibrium. The Nash equilibria for the stage game are (A, A), (B, B), and (C, C).

20 Example 3: Further Modified Prisoner s Dilemma Consider the following strategy: s1 = s2 = Play D in stage 1, Play C in stage 2 if stage 1 s outcome is neither (D, B) nor (A, D), Play A in stage 2 if stage 1 s outcome = (D, B).(Punish2andreward1) Play B in stage 2 if stage 1 s outcome = (A, D).(Punish1andreward2) This strategy profile cannot be renegotiated! Hence, (s is a renegotiation-proof subgame perfect equilibrium. 1, s 2 )

21 Finitely Repeated Games: summary If the stage game has a unique NE, then the finitely repeated game has a unique SPE that plays the NE in every period. In a repeated game, repeatedly playing a NE in every period is always a SPE. To support cooperation (that is not a NE of the stage game), there must be credible punishments and rewards in the future. If the players can renegotiate, then there needs strong reward for the punisher in order to make the punishment credible.

22 1 Introduction 2 Finite Repeated Game 3 Infinitely Repeated Games

23 Infinitely Repeated Games Instead of adding additional punishments, as we did in the previous lecture, we now consider alternating the other assumption finite repetition. Now, suppose the players repeatedly play the stage game infinitely, and they discount the later periods with a discount factor δ. That is, if a player gets a payoff stream of 1, 1, 1,, then his utility is u 1 = 1 + δ + δ 2 + δ 3 + = 1 1 δ Note that if u 1 = 1 + δ + δ 2 + δ 3 +, then δu 1 = δ + δ 2 + δ 3 + δ 4 +, and hence, u 1 δu 1 = 1 = (1 δ)u 1.

24 Example 4: Infinitely Repeated Prisoner s Dilemma 2 A B 1 A r, r D, d B d, D R, R Assumptions: d < r < R < D. Since now the game has no end, we cannot use backward induction. However, one notable subgame perfect equilibrium is s 1 = s 2 = {play A always}. Since (A, A) is a Nash equilibrium in the stage game, playing A repeatedly is also a Nash equilibrium for all of the subgames.

25 Example 4: Infinitely Repeated Prisoner s Dilemma The interesting question is: Can we support (B, B) in a subgame perfect equilibrium? Consider the following trigger strategy Play B in stage 1, s 1 = s 2 = If last stage s outcome is (B, B), play B in this stage, If last stage s outcome is not (B, B), play A forever. To check this trigger strategy profile forms a subgame perfect equilibrium, it is suffice to check if any player wants to deviate at any stage (One Stage Deviation Principle).

26 Example 4: Infinitely Repeated Prisoner s Dilemma 1 Does player 1 want to deviate from playing B in the starting stage? Cooperation yields: Deviation yields: u 1 (s1, s 2 ) = R + Rδ + Rδ2 + = R 1 δ u 1 (s 1, s 2 ) = D + rδ + rδ2 + = D + rδ 1 δ

27 Example 4: Infinitely Repeated Prisoner s Dilemma 1 Hence, player 1 would not deviate if R 1 δ D + rδ 1 δ R D(1 δ) + rδ δ D R D r Note that if (B, B) is played last stage, the game starting from this stage is the same as the original game! Hence, we have showed that under the above condition, player 1 would not deviate from playing B if (B, B) was played previously.

28 Example 4: Infinitely Repeated Prisoner s Dilemma 2 Does player 1 want to deviate from playing A in the punishment stage? Since playing (A, A) forever is a subgame perfect equilibrium, player 1 would not want to deviate. 3 By symmetry, player 2 would not deviate in any stage either. Thus, we showed that under the above condition, the trigger strategy profile constitutes a SPE of the infinitely repeated game. In this equilibrium, the two players start with cooperation, and continue cooperation forever.

29 Lesson In the infinite repeated game, when players value future payoffs high enough, i.e. (δ D R D r ), cooperation can be supported even if the stage game has only one Nash equilibrium. The idea is the same as before: future interactions provide credible punishments for deviations ( non-cooperation forever") and credible rewards for cooperation ( continuing cooperation"). When will cooperation be more likely sustained? R is higher: greater benefit from cooperation increases incentives to cooperate r is smaller: worse consequence from non-cooperation increases incentives to cooperate D is smaller: smaller deviation gain increases incentives to cooperate

30 Group Reputation and Community Enforcement In some situations, one person does not interact with another person repeatedly, but interacts with someone from a group repeated: e.g., a group of buyers and a group of sellers. In such situations, it can be shown that group reputation can help sustain cooperation. Conditions: past group behavior must be observable (even though individual behavior may not be perfectly observable) bad group behavior must be punished, and the punishers should be rewarded (otherwise they would be punished in the future) within groups, punishment for bad behavior can greatly enhance cooperation.

31 Group Reputation and Community Enforcement If players below to a community, who can reward good behavior and punish bad behavior, then cooperation is more likely to emerge. E.g., long distance trader groups in the middle age (Greif).

32 Multiple Equilibria and the Folk Theorem We see that in infinitely repeated games, there can be many SPEs, some support cooperation, some do not. So the analysis shows the possibility of supporting cooperation in such situations, how factors that affect cooperation. The issue of multiple equilibria is actually more serious. See the Folk Theorem below. Theorem When the discount factor goes to one, any individual rational payoff vector can be supported in SPE in an infinitely repeated game.

33 Applications of Repeated Games Firm Collusion: Remember the Bertrand puzzle? One explanation is that in repeated interactions, the duopoly firms will not compete all the way to marginal cost pricing. Anti-trust Analysis: Central Bank Reputation: (see Gibbons). Rotation of government officials and marketing managers.

34 Repeated Games: Some Conceptual Issues Question: Why can t we do this kind of trigger strategy in the finite repeated game? This is because at the last stage, players cannot cooperate, and hence, cannot cooperate in the next to last stage, and so on. Question: Are infinitely repeated games reasonable given that in the long run we are all dead? The key feature here is not infinite time, but the possibility we will play again. Hence, as long as we don t know when the game will end, it is as if it is an infinitely repeated game.

35 Repeated Games: Some Conceptual Issues Question: When do people value future payoffs more? Frequency of interactions. People are more patient. Alternatively, you can view the discount factor as the probability of playing again, and risk neutral players discount the future accordingly. Experimental Evidence: Dal Bo (AER, 2005).

36 Food for Thoughts: The Role of Religions Most Religions teach people to be moral, nice, socially responsible, etc. In a finite life (at the births of the main religions, life expectance was very short), when deviation gains are substantial (in the early days, life was tough and being selfish probably could increase survival rates significantly), how could people be convinced to behave morally"? All religions promised after-life: making life an infinite game! In the after-life, there would be punishments (e.g., Hell) and rewards (e.g., Haven). Different actions were associated with different levels of punishments and rewards, extreme sacrifice (e.g., life) must be rewarded sufficiently, and extreme crimes must be punished sufficiently.