CMU-Q Lecture 20:

Transcription

1 CMU-Q Lecture 20: Game Theory I Teacher: Gianni A. Di Caro

2 ICE-CREAM WARS 2

3 GAME THEORY Game theory is the formal study of conflict and cooperation in (rational) multi-agent systems Decision-making where several players must make choices that potentially affect the interests of other players: the effect of the actions of several agents are interdependent (and agents are aware of it) Example: Auctioning! Psychology: Theory of social situations 3

4 ELEMENTS OF A GAME The players: how many players are there? Does nature/chance play a role? Players are assumed to be rational A complete description of what the players can do: the set of all possible actions. 4

5 ELEMENTS OF A GAME A description of the payoff / consequences for each player for every possible combination of actions chosen by all players playing the game. A description of all players preferences over payoffs Utility function for each player 5

6 AGENT DESIGN VS. MECHANISM DESIGN Agent strategy design: Game theory can be used to compute the expected utility for each decision, and use this to determine the best strategy (and its expected return) against a rational player Strategy Policy System-level mechanism design: Define the rules of the game, such that the collective utility of the agents is maximized when each agent strategy is designed to maximize its own utility according to ASD 6

7 MAKING DECISIONS: BASIC DEFINITIONS Decision-making can involve: one action or a sequence of actions Action outcomes can be certain or subject to uncertainty A set A of alternative actions to choose from is given, it can be either discrete (finite or numerable) or continuous (infinite) A = {a ',a ),, a + } A = a a 0,10 } Strategy (=Policy): tells a player what to do for every possible situation (state) throughout the game (complete algorithm for playing the game). It can be deterministic or stochastic 1 Strategy set S: set of all strategies available for a ' a the players to play. Set S can be finite or infinite ) 2 3 b ' b ) c ' c ) Sequential game, one player States: {1,2,3, T} A ' = a ', a ), A ) = b ', b ), A 1 = c ', c ), A 3 = S = {a ' b ', a ' b ),a ) c ', a ) c ) } E.g. strategy: s = {a ' b ' } 7

8 MAKING DECISIONS: BASIC DEFINITIONS One-action (static) games a ' a ) b ' b ) c ' c ) States: {1,2,3, T} A ' = a ', a ), A ) = b ', b ), A 1 = c ', c ), A 3 = S = (1, a ', (1, a ) ), (2, b ' ), (2, b ) ), (3, c ' ), (3, c ) )} E.g. strategy: s = {(1, a ' ), (2, b ) ), (3, c ' )} The strategy defines the behavior of an agent The observed behavior of an agent following a given strategy is the outcome of the strategy Pure strategy: a strategy in which there is no randomization, one specific action from the set A is selected with certainty at each state / decision node The strategy set S is also indicated as the pure strategy set 8

9 How do we choose the strategy? PAYOFFS AND UTILITIES Rational agents: Principle of Maximum Expected Utility Payoffs ~ Rewards in MDPs: what results from taking an action Payoff (for a single agent): function that associates a numerical value with every action in A π: A R Payoff (for a multi-agent scenario): The payoff of the action a for agent i depends on the actions of the other players! π: A A A R Utility: it can be any convenient additive function u of the payoffs In the following the payoffs will coincide with the utility of the agents (it fully makes sense for the static games that we will consider) Notation: we will use π B and u B quite interchangeably 9

10 INFORMATION AND TYPES OF GAMES Complete information game: Utility functions, payoffs, strategies and types of players are common knowledge Incomplete information game: Players may not possess full information about their opponents (e.g., in auctions, each player knows its utility but not that of the other players). Parameters of the game are not fully known Perfect information game: Each player, when making any decision, is perfectly informed of all the events that have previously occurred (e.g., chess) [Full observability] Imperfect information game: Not all information is accessible to the player (e.g., poker, prisoner s dilemma) [Partial observability] 10

11 TURN-TAKING VS. SIMULTANEOUS MOVES Static games All players take actions simultaneously Imperfect information games Morra Complete information Single-move games Dynamic games o Turn-taking games max o Fully observable Perfect Information Games o Complete Information o Repeated moves min 11

12 (STRATEGIC-) NORMAL-FORM GAME Let s focus on static games There is a strategic interaction among players Strategy profile: a set of strategies for all players which fully specifies all actions in a game. It must include one and only one strategy for every player A game in normal form consists of: o Set of players N = {1,, n} o Set of actions available to each player, that defines the strategy set S = {s ', s ),, s G } o For each i N, a utility function u B defined over the set of all possible strategy profiles u B S + R Payoff matrix Payoff matrix in a 2-player game If each player j N plays the strategy s J S, the utility of player i is u B s ',, s + that is the same as player i s payoff when strategy profile (s ',, s + ) is chosen 12

13 THE ICE CREAM WARS N = 1,2 S = [0,1] s i is the fraction of beach.. u B s B, s J = K L MK N ) 1 K LMK N ', s B < s J ), s B > s J ), s B = s J 13

14 THE PRISONER S DILEMMA (1962) Two men are charged with a crime. Police suspects they are the authors of the crime but doesn t have enough evidence They are taken into custody and can t communicate with each other They are told that: o If one rats out and the other does not, the rat will be freed, other jailed for 9 years o If both rat out, both will be jailed for 6 years They also know that if neither rats out, both will be jailed for 1 year 9 N = 1,2 S = {, Don Y t confess} Strategy profiles: { C, C, C, D, D, C, D, D } u [ C, C = 6, u [ C, D = 0, u [ D, C = 9, u [ D, D = 1 Symmetric for u^ 14

15 THE PRISONER S DILEMMA (1962) 15

16 PRISONER S DILEMMA: PAYOFF MATRIX Don t confess = Don t rat out Cooperate with each other = Rat out Don t cooperate to each other, act selfishly! A Don t Don t B -1,-1-9,0 0,-9-6,-6 What would you do? 16

17 PRISONER S DILEMMA: PAYOFF MATRIX Don t Don t B -1,-1-9,0 B Don t confess: If A don t confess, B gets -1 If A confess, B gets -9 B : If A don t confess, B gets 0 If A confess, B gets -6 A 0,-9-6,-6 Rational agent B opts to 17

18 PRISONER S DILEMMA (Defection = Acting selfishly) is a dominant strategy for B: no matters what A plays, the best reply strategy is always to confess (Strictly) dominant strategy: yields a player strictly higher payoff, regardless of which decision(s) the other player(s) choose Weakly dominant strategy: ties in some cases Because of symmetry, is a dominant strategy also for A A will reason as follows: B s dominant strategy is to, therefore, given that we are both rational agents, B will also and we will both get 6 years. 18

19 PRISONER S DILEMMA But, is the dominant strategy (C,C) the best strategy? Don t B A Don t -1,-1-9,0 0,-9-6,-6 19

20 PARETO OPTIMALITY VS. EQUILIBRIA Being selfish is a dominant strategy, but the players can do much better by cooperating: (-1,-1), which is the Pareto-optimal outcome Pareto optimality: an outcome such that there is no other outcome that makes any player better off without making at least another one player worse off Outcome (Don t, Don t confess): (-1,-1) A strategy profile forms an equilibrium if no player can benefit by switching strategies, given that every other player sticks with the same strategy, which is the case of (, ) An equilibrium is a local optimum in the space of the strategies 20

21 UNDERSTANDING THE DILEMMA (Self-interested & Rational) agents would choose a strategy that does not bring the maximal reward The dilemma is that the equilibrium, outcome, that derives from the dominant strategy, is worse for both players than the outcome they would get if both refuse to confess Related to the tragedy of the commons: Situation in a shared-resource system where individual users acting independently according to their own self-interest behave contrary to the common good of all users by depleting or spoiling that resource through their collective action CO2 emissions / climate, oceans, water, energy, welfare,. 21

22 ON TV: GOLDEN BALLS If both choose Split, they each receive half the jackpot. If one chooses Steal and the other chooses Split, the Steal contestant wins the entire jackpot. If both choose Steal, neither contestant wins any money Watch the video! 22