3-2 Lecture 3: January Repeated Games A repeated game is a standard game which isplayed repeatedly. The utility of each player is the sum of

Transcription

1 S294-1 Algorithmic Aspects of Game Theory Spring 2001 Lecturer: hristos Papadimitriou Lecture 3: January 30 Scribes: Kris Hildrum, ror Weitz 3.1 Overview This lecture expands the concept of a game by introducing variations such as extensive form games, repeated games, and games of incomplete information. We also introduce new notions of euilibrium, such as maxminimizer, dominance, and subgame perfect euilibrium. Lectures one and two focused on two-person games, and we willcontinue to do so here, but most results and definitions can be extended to games with more than two players. 3.2 Game Variations Extensive Games An extensive game is a game where the players take turns. The possible actions a player can take on his turn depend on the previous actions taken by both players. The utility of each player at the end of the game also depends on the whole history of actions taken. An extensive game can be represented by a decision tree as in Figure 3.1. The game starts at the root. Each internal node is labeled with the name of the player whose turn it is to take an action. The possible branches of the node represent the possible actions that this player can take atthatpoint. Each leaf represent a certain end result of the game with certain utilities for each player. s 1»» T s Ts fi 2 fi s 1 s» 1» T» T» T s s s 2 Ts» 2» T» T» T s T»» T s Ts 1 s s s Figure 3.1: Extensive game An extensive game can be translated to strategic (standard) form in the following way. A strategy for a given player is a description of this player's actions at all the nodes labeled with her name. Each pair of strategies (one for each player) has a uniue end result which can be resolved bytraversing the tree according the chosen actions until a leaf is reached. The utilities of the players from a given pair of strategies are the utilities associated with reaching that leaf. 3-1

2 3-2 Lecture 3: January Repeated Games A repeated game is a standard game which isplayed repeatedly. The utility of each player is the sum of the utilities she gets in each of the rounds. Repeated games can be represented as extensive games where each node is labeled with the names of both players (they both choose an action at that point). The possible branches from each node are all the possible pairs of actions the players can take. For example, let's lookat a repeated prisoner dilemma game. Recall that the possible strategies and utilities in a single round prisoner dilemma is as described in Table 3.1. The repeated version of this game can be described asatreeas in Figure ,3 0,4 4,0 1,1 Table 3.1: Prisoner's dilemma " s 1,2 "» Tb bbb " (,) "» TT "» b(,) " " (,)» (,) T bbb s"" s 1,2» T s 1,2 1,2 bs 1,2 L L L L LL LL LL LL fi Ξ fi Ξ fi Ξ fi Ξ fi Ξ L fi Ξ L fi Ξ L fi Ξ L fi Ξ L fi Ξ L fi Ξ L fi Ξ L Figure 3.2: Repeated prisoner's dilemma Notice that in the above example, when the number of rounds is finite, then backwards induction leads the players to play every round in the same manner they would play a single round game (both defect). When the players have only bounded rationality, different euilibria mayemerge,aswe shall see later in the lecture Signaling, Bargaining A signaling game includes states where the players can signal each other and share some of their intended play, orprivate information (e.g., in the hicken game a strategy of the form first signal that you will dare and, if the other player does not signal, then dare, otherwise chicken" becomes legitimate). In a Bargaining game, there are states for making binding offers which the other player can reject or accept. These games also reduce to standard ones with more states Games with Incomplete Information These games model situations where the players have to take actions without having full information on the different factors that influence their utility. The formalism goes as follows:

3 Lecture 3: January ffl Ω is the set of possible states of the world. ffl The players have their own beliefs as to the state of the world which manifest in probability distributions (p for player 1, for player 2) on Ω. ffl In addition, each player gets some signal as to the real state of the world. S and T are the sets of possible signals. Each state of the world is associated with a uniue signal for each player, i.e., there are functions f :Ω! S and g : Ω! T such that if the state of the world is! than player 1 gets the signal f(!) andplayer 2 the signal g(!). Without loss of generality, we can assume S and T are disjoint. ffl As usual, X and Y are the strategy spaces for each player respectively. ffl The utility ofeach player depends on the actions of both players as wellasthestateoftheworld, i.e., U i : X Y Ω! R. A game as described above can be translated into a standard game in the following way. A strategy of a player in the original game is specifying an action for each possible signal he gets, i.e., a : S! X, b : T! Y. In the resulting standard game, the set of players is S [T. Thus, a strategy of player 1 in the original game is translated into a combination of strategies, one for each of the S players, in the standard game. The utility of each of the S players is: U s2s (a; b) = X f (!)=s p(!) p(f 1 (s)) U 1(a(s);b(g(!));!) This is because, given the signal a player gets, he updates his view on the possible state of the world. Using this conditional probability, he can now calculate his (subjective) expected utility given the strategies. Note: when p = and U i (x; y;!) =U i (x; y) the game is called a correlation game. The state of the world does not affect the utilities of the players. However, it can be used, through the signals, to correlate actions. 3.3 Rational Play? In this section we shortly discuss whether a randomized (mixed) strategy can be rational. learly, there are cases where players use randomization. The IRS policy on audits, as well as bluffing in poker are examples. However, in these cases randomization is used to create uncertainty, which, combined with risk aversion of the other players, works for the advantage of the player using the randomization. It is not clear that any mixed strategy which is part of a Nash euilibrium is necessarily rational. The following proposition exemplify some of the problems with justifying a mixed strategy as a concept of rationality. Proposition 3.1 If ß 1 and ß 2 are probability distributions over strategies for player 1and player 2 respectively such that (ß 1 ;ß 2 ) is a Nash Euilibrium, then any x with ß 1 (x) > 0 is a best response to ß 2. Proof: U 1 (ß 1 ;ß 2 )= X x2x ß 1 (x)u 1 (x; ß 2 ) Suppose by contradiction that there exists an x such thatu 1 (x; ß 2 ) is sub-optimal and ß 1 (x) > 0. Then, ß 1 is not a best response to ß 2 because player 1 can improve his utility by shifting some weight from x to one of the optimal actions. This contradicts the assumption that (ß 1 ;ß 2 ) is a Nash euilibrium. Probabilities make more sense in the following situations:

4 3-4 Lecture 3: January In populations, where the probabilities represent the ratio of the individuals predisposed to each strategy. 2. If the probabilities represent observed freuencies of actions taken. 3. Believed perturbations (a famous theorem by Harsanyi states that Nash euilibria can be thought of as uniuely optimal responses, in the limit, in slightly perturbed games). 3.4 Zero Sum Games As mentioned in the first lecture, a zero sum game is a special case of a standard game where U 1 (x; y)u 2 (x; y) = 0 for any pair of strategies (x; y). In this special class of games the Nash euilibrium has another characterization which makes it easy" to find. Before we continue we make a note on the notation we use in the rest of the lecture: X and Y are the sets of all possible mixed strategies of player 1 and player 2 respectively. efinition 3.2 We say that x Λ is a max-minimizer for player 1iff min y2y (U 1 (x Λ ;y)) min(u 1 (x; y)) 8x 2 X y2y y Λ is defined as a max-minimizer for player 2 in a similar way. A max-minimizer captures the idea of precedence. If player 2 k nows which mixed strategy player 1 chose before choosing his (player 2) own, then the best player 1 can do (assuming player 2 will optimize his utility) is to choose a max-minimizer for player 1. Theorem 3.3 In any zero sum game, any mixed Nash euilibrium is a max-minimizer for both players. Proof: Let (x Λ ;y Λ ) be a (mixed) Nash euilibrium. Since, by definition, y Λ is a best response to x Λ, then: U 2 (x Λ ;y Λ ) U 2 (x Λ ;y) 8y or euivalently, In a similar way, U 1 (x Λ ;yλ) = min y U 1 (x Λ ;y Λ )» U 1 (X Λ ;y) 8y U 1 (x Λ ;y)» max min U 1(x; y) x y U 1 (x Λ ;y Λ ) U 1 (x; y Λ ) 8x U 1 (x Λ ;y Λ ) min U 1 (x; y) 8x y U 1 (x Λ ;y Λ ) max min U 1(x; y) x y

5 Lecture 3: January U 1 (x Λ ;y Λ ) = max min U 1(x; y) x y We can derive the same for the max-minimizer of player 2, and get: U 1 (x Λ ;y Λ ) = max x min y U 1(x; y) = min max U 1(x; y) =Z y x Notice that this theorem entails that all Nash euilibriums give the same utility, Z, toplayer 1 (and Z to player 2). Finding a max-minimizer for player 1 can be formulated as the following linear program: max Z X x s:t: ß x U 1 (x; y) Z X x ß x =1 8y Linear programs can be solved efficiently, and thus, a Nash euilibrium can be found efficiently in this class of games. Notice that the duality theorem for linear programming also entails the euality of the two max-minimizers (the problem of finding the max-minimizer of player 2 is the dual to the above program). We do not know of any efficient algorithm to find a Nash euilibrium in general games, even if restricted to just two players. However, for games restricted to two players, a pivoting techniue (as used in solving linear programs) can be used to show that all Nash euilibria are always rational numbers. 3.5 ominance and Rationality A pure strategy x dominates x 0 (written x > x 0 ) if u 1 (x; y) > u 1 (x 0 ;y); 8y 2 μ Y. For example, in the prisoner's dilemma, dominates, since whatever player 2 chooses, is a better option for player one. Suppose we perform the following operation: List all pure strategies for both players. While either player has a dominated strategy, eliminate it. We will call this algorithm iterated dominance. Note that eliminating one of player 1's strategies may cause one of player 2's strategies to be dominated, and vice-versa. Iterated dominance can take O(n 5 ) time, where n is the number of strategies. Theorem 3.4 The result does not depend on the order of elimination. A strategy ^x is rationalizable if there is a subset ^X of player 1's pure strategies, and ^Y a subset of player 2's pure strategies, and ^x 2 ^X is the best response to some mixed strategy of ^Y. Theorem 3.5 In two players games, a strategy is rationalizable if and only if it survives iterated dominance.

6 3-6 Lecture 3: January 30 Tit for Tat Punish Once, Punish Forever Switch on, Figure 3.3: Four automata for prisoner's dilemma A Player 1 B (4, 2) Player 2 (0,0) (5,1) Figure 3.4: A game with multiple Nash euilibria Weak ominance ominance is sometimes an overly strict definition, so we say that x weakly dominates x 0 if 8y u 1 (x; y) u 1 (x 0 ;y)andu 1 (x; y) >u 1 (x 0 ;y) for some y. Performing the iterated dominance algorithm using weakdominance instead of dominance is not deterministic as the results can depend on the order in which actions are eliminated. If you create all possible two-state automata and havethemplay an infinite repeated version of the prisoner's dilemma against each other, in the resulting game strong dominance does not eliminate very many, however, weakdominance reduced the 32 down to four shown in Figure Subgame perfect Euilibria In certain cases where the strategic game is an extensive game, some Nash euilibria are intuitively wrong because they support non-credible threats. For an example, see Figure 3.4. In this case A is a Nash Euilibrium. Player 2 never gets to move ifone play A,so it does not matter whether player 2's strategy is or, and if player 2 is going to play,thenplayer 1 should certainly not play B.

7 Lecture 3: January However, while correct, this does not make sense. If player 2 ever did get to play, no rational player would chose. So player 2's threat to play is not credible, so player 1 should never play A. For this reason, we introduce the concept of a subgame perfect euilibria. An euilibrium is subgame perfect if when played from any point in the game is a Nash euilibrium. A subgame is a subset of nodes that still form a game. In the above example, A would not be a subgame perfect euilibrium because in the subgame starting at player 2's move the restriction of A to this subgame would not itself be an euilibrium. But there are further objections to this definition, so several other variations have been introduced, including trembling hand euilibrium and seuential euilibria (for extensive games with incomplete information).