Backward Induction. ISCI 330 Lecture 14. March 1, Backward Induction ISCI 330 Lecture 14, Slide 1

Transcription

1 ISCI 330 Lecture 4 March, 007 ISCI 330 Lecture 4, Slide

2 Lecture Overview Recap ISCI 330 Lecture 4, Slide

3 Subgame Perfection Notice that the definition contains a subtlety. n agent s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward player has three pure strategies, and player has eight (why?). But now consider the game shown in Figure 5.. B C E F (3,8) (8,3) (5,5) G H Figure 5. (,0) (,0) perfect-information game in extensive form. There s something intuitively wrong with the equilibrium (B, H), (C, E) In order to define a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. Why S = would {(,G),(,H),(B,G),(B,H)} player ever choose to play H if he got to the second S = {(C,E),(C,F),(,E),(,F)} choice node? fter all, G dominates H for him It is important to note that we have to include the strategies (,G) and (,H), even though once is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5. can be converted into the normal form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are Multi gent Systems, draft of September 9, 006 ISCI 330 Lecture 4, Slide 3

4 Subgame Perfection Notice that the definition contains a subtlety. n agent s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward player has three pure strategies, and player has eight (why?). But now consider the game shown in Figure 5.. B C E F (3,8) (8,3) (5,5) G H (,0) (,0) Figure 5. perfect-information game in extensive form. There s something intuitively wrong with the equilibrium (B, H), (C, E) In order to define a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. Why S = would {(,G),(,H),(B,G),(B,H)} player ever choose to play H if he got to the second S = {(C,E),(C,F),(,E),(,F)} choice node? fter all, G dominates H for him He does it to threaten player, to prevent him from choosing F, and so gets 5 It is important to note that we have to include the strategies (,G) and (,H), even though once is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information However, this game of seems Figure 5. like canabe non-credible converted into the normal threat form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are If player reached his second decision node, would he really follow through and play H? Multi gent Systems, draft of September 9, 006 ISCI 330 Lecture 4, Slide 3

5 Formal efinition efine subgame of G rooted at h: the restriction of G to the descendents of H. efine set of subgames of G: subgames of G rooted at nodes in G s is a subgame perfect equilibrium of G iff for any subgame G of G, the restriction of s to G is a Nash equilibrium of G Notes: since G is its own subgame, every SPE is a NE. this definition rules out non-credible threats ISCI 330 Lecture 4, Slide 4

6 Back to the Example (,0) Figure 5. (,) The Sharing game. (0,) Notice that the definition contains a subtlety. n agent s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward player has three pure strategies, and player has eight (why?). But now consider the game shown in Figure 5.. B C E F (3,8) (8,3) (5,5) G H (,0) (,0) Figure 5. perfect-information game in extensive form. In order to define a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. Which equilibria from the example are subgame perfect? S = {(,G),(,H),(B,G),(B,H)} S = {(C,E),(C,F),(,E),(,F)} It is important to note that we have to include the strategies (,G) and (,H), even though once is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5. can be converted into the normal form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are ISCI 330 Lecture 4, Slide 5

7 Back to the Example (,0) Figure 5. (,) The Sharing game. (0,) Notice that the definition contains a subtlety. n agent s strategy requires a decision at each choice node, regardless of whether or not it is possible to reach that node given the other choice nodes. In the Sharing game above the situation is straightforward player has three pure strategies, and player has eight (why?). But now consider the game shown in Figure 5.. B C E F (3,8) (8,3) (5,5) G H (,0) (,0) Figure 5. perfect-information game in extensive form. In order to define a complete strategy for this game, each of the players must choose an action at each of his two choice nodes. Thus we can enumerate the pure strategies of the players as follows. Which equilibria from the example are subgame perfect? (, G), (C, F ) is subgame perfect S = {(,G),(,H),(B,G),(B,H)} (B, S H) is an non-credible threat, so (B, H), (C, E) is not = {(C,E),(C,F),(,E),(,F)} subgame perfect (, H) is also non-credible, even though H is off-path It is important to note that we have to include the strategies (,G) and (,H), even though once is chosen the G-versus-H choice is moot. The definition of best response and Nash equilibria in this game are exactly as they are in for normal form games. Indeed, this example illustrates how every perfectinformation game can be converted to an equivalent normal form game. For example, the perfect-information game of Figure 5. can be converted into the normal form image of the game, shown in Figure 5.3. Clearly, the strategy spaces of the two games are ISCI 330 Lecture 4, Slide 5

8 Lecture Overview Recap ISCI 330 Lecture 4, Slide 6

9 Centipede Game 5 Reasoning and Computing with the Extensive Form (3,5) (,0) (0,) (3,) (,4) (4,3) Figure 5.9 Play this as a fun game... The centipede game place. In other words, you have reached a state to which your analysis has given a probability of zero. How should you amend your beliefs and course of action based on this measure-zero event? It turns out this seemingly small inconvenience actually raises a fundamental problem in game theory. We will not develop the subject further here, but let us only mention that there exist different accounts of this situation, and they depend on the probabilistic assumptions made, on what is common knowledge (in ISCI 330 Lecture 4, Slide 7

10 Computing Subgame Perfect Equilibria Idea: Identify the equilibria in the bottom-most trees, and adopt these as one moves up the tree Procedure starting at each terminal node, move up to its parent node and label it with the utility values of the terminal node that would be the best response for the player who gets to choose at this parent node repeat this procedure, treating the highest already labeled nodes as terminal nodes, until the root is reached Stop when the root of the tree is labeled Note: Before performing this procedure at any given node, it must be performed at all subnodes first the procedure doesn t return an equilibrium strategy, but rather labels each node with a vector of real numbers. This labeling can be seen as an extension of the game s utility function to the non-terminal nodes The equilibrium strategies: take the best action at each node. ISCI 330 Lecture 4, Slide 8

11 5 Reasoning and Computing with the Extensive Form (3,5) (,0) (0,) (3,) (,4) (4,3) Figure 5.9 The centipede game What happens when we use this procedure on Centipede? In the only equilibrium, player goes down in the first move. place. In other words, you have reached a state to which your analysis has given a probability However, of zero. How thisshould outcome you amend is Pareto-dominated your beliefs and courseby ofall action but based one on this measure-zero other outcome. event? It turns out this seemingly small inconvenience actually raises Twoa fundamental considerations: problem in game theory. We will not develop the subject further here, but let us only mention that there exist different accounts of this situation, and they depend practical: on the probabilistic human subjects assumptions don t made, on go what down is common right knowledge away (in particular, theoretical: whether there what is common should knowledge you of do rationality), as player and on if exactly player how one doesn t revises one s go down? beliefs in the face of measure zero events. The last question is intimately related to the subject SPE of analysis belief revision says to discussed go down. in Chapter However,. that same analysis says that P would already have gone down. How do you 5. Imperfect-information update yourextensive-form beliefs upon observation games of a measure zero event? Up to this point, but in our if player discussion knows of extensive-form that you ll games dowe something have allowedelse, players it is to specify the action rational that they for would himtake notat to every gochoice downnode anymore... of the game. a paradox This implies that players know there s node a whole they are literature in, and recalling on thisthat question in such games we equate nodes with the histories that led to them all the prior choices, including those ISCI of330 other Lecture 4, Slide 9