Logic in Classical and Evolutionary Games

Transcription

1 Logic in Classical and Evolutionary Games MSc Thesis (Afstudeerscriptie) written by Stefanie Kooistra (born April 24th, 1986 in Drachten, The Netherlands) under the supervision of Prof. dr. Johan van Benthem, andsubmittedto the Board of Examiners in partial fulfillment of the requirements for the degree of MSc in Logic at the Universiteit van Amsterdam. Date of the public defense: September 5th, 2012 Members of the Thesis Committee: Prof. dr. Johan van Benthem Dr. Alexandru Baltag Prof. dr. Benedikt Löwe

2 Contents 1 Introduction 4 2 Classical Game Theory Strategic Form Games Terminology Game Matrix Solution Concepts Iterated Elimination of Dominated Strategies Conclusion Extensive Form Games Terminology Game Tree Equilibria Normalisation Backward Induction Imperfect Information Conclusion Evolutionary Game Theory Background Evolutionary Interpretation of Game Theoretic Terms Evolutionary Stability A Note on the Relationship Between ESS, NE and SNE The Replicator Dynamics Graphing Replicator Dynamics Conclusion Logic in Classical Game Theory Logic for Strategic Form Games Games in Static Modal Logic Games in Dynamic Modal Logic Relations in Strategic Games Games in Hybrid Logic Conclusion

3 4.2 Logic for Extensive Form Games Bonanno s Account of Backward Induction The Theory of Play and Extensive Form Games Freedom in Extensive Form Games Freedom, Forcing and the Process Model Conclusion Logic in Evolutionary Game Theory The Problem of Choice and Rationality Alternative Views of Rationality Rationality in Game Theoretic Pragmatics An Acceptable Way to Abandon Rationality Logic A Hybrid Logic for Evolutionary Stable Strategies Logical Reinterpretation of Classical Terms Conclusion Conclusion 80 2

4 Acknowledgements Iwouldliketoexpressmyimmensegratitudetomysupervisor,JohanvanBenthem, whose patient but persistent guidance gave me the motivation, support and know-how I needed to learn how to write a thesis. The competences I have gained from writing my thesis with Johan are invaluable and will certainly stick with me. I would also like to thank the members of the thesis committee, Alexandru Baltag and Benedikt Löwe, for taking the time to read and evaluate my thesis. Last, I am grateful to Davide Grossi for his technical lessons, which were vital to the results in this thesis, my mentor Dick de Jongh, who in many ways guided and encouraged me throughout the process of the MoL programme, and Henk Zeevat for his time when I started my thesis. This thesis is dedicated to my parents, Henk and Sieneke Kooistra, who in every way made it possible for me to complete this work. Thank you for your tremendous support, friendship, patience and love. Also thank you to my sister, Lydia, for being an always supportive, loving and helpful friend, never afraid to be honest and always reminding me to just deal with it. Bart J. Buter, I can t imagine having gone through this process without you; we took classes together, we wrote our theses in each other s company, and we faced all the related challenges and successes together. Thank you for your support, your faith in my ambitions, and patience for my sometimes eccentric style of working. Rob Uhlhorn, thank you for always relativising everything good and bad with brilliant absurdity. Our regular debauchery kept me on track, totally straight-edge. And also thanks to Bill Mikesell, for your support, good advice, good cake, and for joining and helping us all keep things fun. Many many thanks to my friends, family and fellow students for the many forms of support; having coffees and dinners, ranting over skype, proof reading my thesis and more; so, thank you to Danielle, Maaike, Pietro, Ines, Robbert- Jan, Tjerk, Patrick, Sara, Hendrike, Silvia, Piet, Nynke, Kasper, Abel and Jelle ( finished first, after all). Last, this thesis will always remind me of Cody and Jimmie, my little companions, who passed away during the completion of this thesis. For the majority of my life, they were my role models for absolute contentment; never judgemental, always loving, always pondering but never thinking anything. May we all be reincarnated as cats. 3

5 Chapter 1 Introduction Game theory is the study of a fundamental and persistent aspect of human behaviour: the strategic interaction of agents. It is a characteristic ingredient of human culture [4]; it can be demonstrated in many kinds of social scenarios such as bartering over the price of a squash, figuring out what film to watch when all your friends have different tastes and choosing the best word a tourist who is looking for the bathroom is most likely to understand. Despite its application to what seems like a familiar everyday occurrence, game theory is a formal mathematical framework for social interaction; that is, game theory uses mathematics to express its ideas formally... however, [they] are not inherently mathematical [35]. Mathematics simply gives us a formal and reliable way to define key elements of strategic interaction in game-like scenarios. In game theory, a game is not always a game in the recreational sense 1,buteverystrategicscenario is considered to have their key elements in common with fun games: players, actions, outcomes and the value of the outcomes for each player. Because game theory delivers a formal and elegant model describing a very basic feature about human behaviour, game theory is deeply rooted in many social sciences. Game theory is prominent in economics, for instance. The emergence of game theory is often attributed to the 1944 publication of Theory of Games and Economic Behaviour by John von Neumann and Oskar Morgenstern [34], in which they represent economic and other social behaviour with many of the formal concepts game theorists use today. Other fields that use game theory are political science, linguistics, logic, artificial intelligence and psychology among others. This thesis is devoted to the study of logic and game theory. The connection between logic and game theory is itself intricate. Economics and other fields are generally considered to be applications of game theory 2 but 1 Chess, checkers, poker, Candyland... 2 Osborne and Rubinstein observe that the boundary between pure and applied game theory is vague; some developments in the pure theory were motivated by issues that arose in 4

6 logic can be seen as something to apply to game theory or as a way to describe game theory. Van Benthem, in his upcoming book Logic in Games, points out that logic in games is an ambiguous phrase; it seemstoreflectthedifferencebetweenlogic as an application and logic as a description. Van Benthem claims that we have game logics [which] capture essential aspects of reasoning about, or inside, games and we have logic games capturing basic reasoning activities and suggesting new ways of understanding what logic is [4]. The former uses logic to describe game theory, and the latter applies game theory to logic. Put simply, the aim of Logic in Games is to describe both perspectives and the intricate connections between them. For this thesis, in general, one point van Benthem makes is salient: Some students... [prefer] one direction while ignoring the other [4]. To some degree, this preference likely occurs for some students by taste or inclination, and, accordingly, this thesis adheres to the preference for game logics. The goal of this thesis is to take initial but key steps towards the inclusion of evolutionary game theory in that game logic debate. That is, it explores how we can describe evolutionary game theory by means of logic. Evolutionary game theory is an expansion of game theory into biology, where it is fitted to observe the stability of behaviour of populations of players in the animal kingdom over time. Moreover, it can also be used to understand the dynamics of our human behaviour where classical game theory would otherwise fall short. Although the step from classical to evolutionary game theory looks small (it is simply thinking of games in terms of populations instead of individuals, and it is still mathematically described by all the same components), it revolves around a dramatic change of perspective on the traditional mathematical components of game theory. But given that the study of classical game logics is already firmly grounded 3, one may conclude that game logics should be easily extended to evolutionary game theory since the theories seem so closely related. John Maynard Smith, one of the earliest authorities on evolutionary game theory, claimed the following in his article Evolutionary Game Theory [41]: applications. Nevertheless, we believe that such a line can be drawn... [This book]... stay[s] almost entirely in the territory of pure theory [35] 3 The Institute for Logic, Language, and Computation (ILLC) at the Universiteit van Amsterdam is one of the few institutions with a program specifically about logic and games. It is (or has once been) home to many of the academics who have given shape to the study of game logics: Johan van Benthem with many innovating publications on the subject as well as peripheral topics, Eric Pacuit, Alexandru Baltag, Benedikt Löwe, Olivier Roy, Boudewijn de Bruin, Wiebe van der Hoek, Marc Pauly, Robert van Rooij and more. One could, of course, go on to mention many more distinguished contributors to this field at other institutions around the world. 5

7 There are two main differences between classical and evolutionary game theory. 1. The replacement of utility by fitness The replacement of rationality by natural selection. This thesis will address and elaborate upon Maynard Smith s points. The following two issues will therefore be major themes throughout this thesis, and will be the main obstacles to overcome in establishing an evolutionary game logic. 1. Many established game logics are not sufficient to simply extend to evolutionary game theory. Whereas classical game theory is elegantly described by modal logics with modalities for knowledge and preference, evolutionary game theory raises two classes of issues. First, what the logic expresses and what evolutionary concepts mean do not always coincide. Second, evolutionary game theoretic concepts are formally different to such a degree that it implies finding a more expressive language. In order to figure out how to remedy these issues, we take a close look at the essential differences between classical game theory and evolutionary game theory, how those differences influence the logical structure of classical and evolutionary game theory, and then search in our logical toolbox for a solution. 2. Rationality, which underpins the enterprise of game theory, is radically questioned in the evolutionary game setting. The nature of evolutionary game theory is such that the players are programmed to play strategies instead of burdened to rationalize and deliberate over what they should choose. It is therefore prudent to ask, what is game theory without rationality and choice? This thesis will accomplish the above goals by describing the basic background and details of classical and evolutionary game theory, exploring some appealing logics for classical game theory and then exploring how those fit with evolutionary game theory. Chapter 2 introduces the basic mathematical concepts and definitions of classical game theory. As mentioned above, game theory is mathematical and is therefore the tool used to elegantly and consistently describe the components of a game; players, actions, outcomes, and preferences. Classical game theory has two forms strategic games and extensive games. A strategic form game describes a one-time simultaneous strategic event between to or more players, and an extensive game describes a turn-taking sequence of events. This chapter will introduce the formal terminology of classical game theory as well as what we can say about game with them. This includes the Nash equilibrium and various other solution concepts. This chapter will also emphasize the rationality 6

8 assumptions that are foundational to the solution concepts. Chapter 3 introduces evolutionary game theory. I will motivate the evolutionary perspective by explaining its unique background with its historical origins in Darwin s expedition on the Beagle. Understanding the origins of evolutionary game theory will foreshadow not only why it is very interesting to look at in logic, but also the challenges we will face given what it stands for. This chapter will describe two basic formal approaches to evolutionary game theory: evolutionary stability and replicator dynamics. It will also take the opportunity to comment on the role that rationality plays in evolutionary game theory. The crucial chapter 4 takes a detailed look at how classical game theory in strategic and extensive form has been and can be described by existing logics. An in depth analysis of classical game logic is paramount to motivating the evolutionary game logic in the following chapter. Game theory and logic are akin, for they share many formal components such as possible worlds or states and reasoning agents with preferences. This chapter will focus primarily on examples of static and dynamic modal logics as evidence showing how game theory resembles logic. A result of this analysis is a set of relational modalities freedom, knowledge, andpreference, whichexpresskeyrelationshipsbetweenthestates in a strategic form game. In combination with hybrid logic, a logic that brings to modal logic the classical concepts of identity and reference [12], it is possible to elegantly redefine some notions in the above modal logic for both strategic and extensive form games; this new perspective on those notions consequently sheds some light on their meaning. Chapter 5 finally approaches the question of whether the logic game theory interface of chapter 4 is applicable given what we know about evolutionary game theory from chapter 3. I will argue that the difference in interpretation of classical game theoretic terms in evolutionary game theory is responsible for the issues arising in inventing a logic for evolutionary games. My approach to this investigation is two fold: the reinterpretation of terms has effected the role that rationality plays in evolutionary games, and the reinterpretation of terms has also effected how we must think of the logical components of game theory such as players, strategies and preferences: First, there is a significant disparity between what rationality means for classical game theory and evolutionary game theory, and that has consequences for the logic of classical and evolutionary game theory. I consider three ways we can think of rationality that may fit evolutionary game theory and logic of evolutionary game theory without compromising the evolutionary perspective. Second, because the game theoretic components players, strategies and preferences, have been reinterpreted under the evolutionary perspective, we should also reinterpret them logically for a correct evolutionary game logic. I propose two ways of doing this; first by means of the hybrid logic with freedom, knowledge and preference as binders, and second by means of an introduction of a new relation simply based defining strategies as players. Defining evolutionary game 7

9 theoretic terms in the latter logic will prove to be the most intuitive alternative, for it best expresses what evolutionary game theory stands for. In conclusion, chapter 6 summarizes everything that has been discussed in this thesis towards including evolutionary game theory in the game logics debate. It describes the novel results appearing in this thesis and identifies possible next steps and areas of development. The topics and results described are complex and highly connected within the fields of logic and game theory, so there are many facets of the intersection that remain to be explored. I suggest some future work towards this end as well. 8

10 Chapter 2 Classical Game Theory Classical game theory establishes the framework for studying and analysing strategic interaction. Strategic interaction incorporates the decision-making behaviour of rational intelligent agents with preferences over the possible outcomes, which are determined by the joint actions of all agents. In general, the study of classical game theory specifies terminology, the solution concepts and the reasoning behind the processes of strategic interaction. Classical game theory provides the theoretical background and structure on which many practical studies of strategic interactions base their investigations. Classical game theory is very popular and informative to researchers in various fields in which strategy is involved. There are two forms of classical games: the strategic form game and the extensive form game. The former describes games in which players act once and simultaneously. This form is straightforward; players can only deliberate beforehand, act, and then accept what follows from his and his opponents actions. An instance of a game that takes this form is rocks, paper, scissors. Neither player is to observe his opponents action before taking his own. Therefore, a player could only choose what to play based on what he thinks his opponent might play. The latter, the extensive form game, is more complex. It describe strategic interaction that takes a sequential, or turn taking, form. A game such as chess is an instance of an extensive form game. The players take turns one after the other, until one player reaches a last move which concludes the game at an outcome in which he wins or loses. This chapter will describe the terminology and results of both forms. This chapter will focus on information about strategic and extensive form games that will be relevant to the upcoming discussions in this thesis. The interaction of players, actions and preferences are crucial to eventually coining a logic that can describe classical games, so that will be emphasized in this chapter. Moreover, this chapter is to equip the reader with knowledge of the features of 9

11 classical game theory that will, in chapter 3, be adapted under the development on classical game theory, evolutionary game theory. Evolutionary game theory has the same basic machinery as classical game theory, but the meaning of the terminology is changed. Because these new meanings imply that, among other things, rationality is no longer a factor, I will describe the influence that rationality has on classical game theory. By being aware of that influence, we will be able to evaluate the consequences of abandoning rationality in theories based on classical game theory, such as evolutionary game theory. 2.1 Strategic Form Games A strategic form game is the framework by which game theorists express oneshot strategic interaction. The main factors that compose this framework are players, actions, and preferences over the outcomes that result from the joint actions of the players. This section will first describe the basic terminology, a tool that visualizes the main idea of the theory and some crucial solution concepts and results in classical game theory Terminology A player is a decision maker who is to choose an action in a game. Agent is a more specific term referring to a player who we assume to be intelligent and rational. A strategic form game is defined 1 as: Definition A strategic form game Γ is a tuple N,(A i ) i N, ( i ) i N where: N is the set of players. (A i ) i N is a non-empty set of actions for each player i N ( i ) i N is the preference relation for player i N on the set of action profiles A = j N A j This is the basic definition of a strategic form game. An action profile a A represents the list of actions (a 1,..., a n ) played by each player i N, wherea i is the i th projection. An a A is also an outcome of the game, for it denotes the unique situation resulting from each player s action. The preference component of a strategic form game can be further specified by the strict and weak variety: a i a if i prefers a more than or equally to a a i a if a i a and a i a a i a if a i a and a i a 1 Components of this definition originate in [35] and [10] 10

12 Preference over outcomes can also be expressed in terms of utility. Utility expresses that an outcome has a particular value to a player, which is determined by a numerical consequence of that outcome. Preference and utility achieve the same goal, but by different means. The choice to express a game by utility or preference is usually a question of convenience, i.e. sometimes it is just more appropriate to talk about preference than utility or vice versa. So, in this discussion, as well as others, both concepts will be used where appropriate. The formal guide to connecting utility with preference is 2 : Definition U i : A R defines a preference relation i by the condition that a i a if and only if U i (a) U i (a ). A strategic form game may also representscenarioswhereplayerstakemixed actions. For instance, player i chooses to play action a i p percent of the time and a i p percent of the time 3,where a i A i p(a i ) = 1 and for all a i A i, p(a i ) 0. An alternative possible interpretation of a mixed strategy is as the probability by which a player i may play an action in A i. The definition of a strategic form game that accounts for mixed actions is: Definition The mixed extension of the strategic form game Γ is a tuple N, (A i ) i N, (U i ) i N where: (A i ) i N is the set of probability distributions over A i. (U i ) i N is a function from mixed action profiles to real numbers, (U i ) i N : j N (A j ) R. Strategy is the default term that describes actions of players, in general, whether they are playing a pure action or mixed set of actions. From now on, actions of a player i will be denoted as strategies σ i even though it is usually apurestrategywherea i would be sufficient 4. We thus identify outcomes with the set S = i N A i of strategy profiles [10]. Definition A strategy profile σ S denotes the list of pure or mixed actions played by each player i N, σ =(a 1,..., a n ), where σ i denotes the ith projection, i.e. σ i = a i and σ i denotes the choices of all players except i, σ i =(a 1,..., a i 1,a i+1,..., a n ).[10] This concludes the basic terms that compose strategic form games. In the following section, concepts such as the game matrix, solution concepts and procedures will be introduced. 2 Based on [35]. 3 This interpretation of a mixed strategy alludes to multiple plays of a game. 4 Strategic form games do not exhibit the complexity seen in other game forms, and therefore the difference between the terms actions and strategy are of little consequence for now. Nevertheless, we wish to distinguish between actions and strategies in preparation for other models relevant to this thesis 11

13 2.1.2 Game Matrix A game matrix is a visual tool that lists one player s actions in a row and the other s in a column, which results in a grid or table. The box where a column and row converge represents a strategy profile and is labelled by the utilities for each player assigned to that strategy profile. Although a game matrix is limited to representing only games with two players, it is a useful tool that can efficiently describe and visualize many fundamental concepts. The following figure is an arbitrary example of a game matrix, where player i s actions a i and b i are listed in the left column, player j s actions a j and b j are listed in the top row, and the cells represents the four possible combinations of i s and j s strategies: a j a i u i (a i,a j ), u j (a i,a j ) u i (a i,b j ), u j (a i,b j ) b i u i (b i,a j ), u j (b i,a j ) u i (b i,b j ), u j (b i,b j ) Each cell in the game matrix represents the outcome of the game from the strategy profile of the players strategies at the cell s respective row and column. It is prudent to note that one often identifies the outcomes with the set of strategy profiles... [4], and in the chapters about game theory and logic, we will see that it seems natural to use the strategy profiles themselves as possible worlds [4]. b j σ 1 σ σ 2 3 σ 4 Figure 2.1: This picture represents the four states and a few preference relations. Because games are often described in terms of preference instead of utility, it is also possible to construct a corresponding preference matrix, which is a pointed preference model. Instead of outcomes represented by cells, the outcomes are represented by points (possible worlds or states), and the preference is indicated by an arrow pointing towards a preferred world (away from every world that is less preferred). Figure 2.1 depicts an example of a preference model with some of the players preferences between worlds marked. 12

14 Types of Games By observing relationships between the outcomesofagame,wecanalsomake specific claims about properties of a game; that is, we can distinguish between types of games. These include symmetric, zero-sum, and coordination games. Symmetric Game A symmetric game implies that in the game matrix, the payoffs and corresponding utilities of one player are the transpose of the pay-offs/utilities of the other, and each player has the same strategies available to him. a i a i a i (n, n) (k, l) a i (l, k) (m, m) Notice that the rule holds that a 2-player game Γ is symmetric if ab : u 1 (a, b) = u 2 (b, a) =u(a, b). Because it does not matter which player is at the row or column position, it is sufficient to talk about utility in terms of just the row player u(a, b). Coordination Game A coordination game is a type of game where it is the best for both players to choose the same strategy. In general, a coordination game has the following game matrix composition: A B A N,n L, l B M,m K, k where N>M, K>L,andn>l, k>m. Thus, either case of choosing the same strategy has a higher utility than either case of choosing differing strategies. Zero-Sum Game A zero-sum game is strictly competitive; the gain of one player is the loss of another. An example of a zero-sum game is the Matching Pennies game: each player flips a coin, and the row player wants the pennies to match (both heads of both tails), and the column player wants the pennies to be opposites (one heads, one tails). The matrix is composed as: Head Tail Head 1, -1-1, 1 Tail -1,1 1, -1 We see that what one player s positive utility, is the other s negative utility. This is characterised by the rule for zero-sum games that for any strategies A, B in agame,u 1 (A, B)+u 2 (A, B) =0. Therefore,itiscalledazero-sum game. 13

15 The interaction between players in a strategic scenario is described by the definition and visualised by game matrix, but it does not yet address the process involved between the player and his action; that is, what motivates players to choose certain actions? Essentially, a player will choose an action that is in his best interest. Analyses of how players may reason towards his most preferred outcome given his opponents moves have resulted in concepts such as the Nash equilibrium (NE), which is one out of many possible solution concepts in game theory. The following section describes the some of most central results and solution concepts to classical game theory Solution Concepts A game theorist can reason about a game and decide which strategies rational and intelligent players in a game will play. The resulting strategy profiles can be given names and defined on the basis of the reasoning for why a game will result in that strategy profile. John Nash s Ph.D. thesis published in 1950 [33], which followed von Neumann and Morgenstern s The Theory of Games and Economic Behaviour [34] by only six years, introduced what is arguably the most important solution concept in game theory, the Nash Equilibrium. Definition Astrategyprofileσ is a Nash Equilibrium if for every player i N (σ i,σ i ) i (σ i,σ i ) for all σ i S i Nash equilibrium is also measurable by another means; the best response function. This function determines which action is the best response to an opponent s strategy, and is defined [35] as: Definition Astrategyσ i is a best response to i s strategy, B i (σ i ), if (σ i,σ i ) i (σ i,σ i) for all σ i S i Note that a the best response function is not always one-to-one. If there is a second (or more) strategy with the same utility as a best response, then it is also a best response. Therefore, the best response function results in a set of strategies. The set may be a singleton, but when it is not, it follows that there exist multiple equally preferred best responses. If both players are playing a best response to their opponent s best response, the result is a Nash equilibrium. The Nash equilibrium in terms of best response is thus defined as [35]: 14

16 Definition Astrategyprofileσ is a Nash Equilibrium if for every player i N. B i (σ i )=σ i Nash proves in [33] that every finite game has an equilibrium point. He demonstrates that if a game does not have a pure strategy Nash equilibrium, then there exists a Nash equilibrium in mixed strategies 5. The following game is a famous and authoritative example of a strategic game and demonstrates the Nash equilibrium solution concept. Example: The Prisoner s Dilemma This game describes a scenario where two individuals are arrested as suspects to a crime. Because they have insufficient evidence to convict either suspect, the arresting policemen offer the suspects the same deal: if the suspect cooperates and thus tattles on the other and the other defects and remains silent, the suspect who tattled goes free. If they both tattle on each other, both go to jail for 5 years. If they both remain silent, they each get jailed for 1 year. They must each therefore choose to cooperate or defect. This is represented in the following matrix: Cooperate Defect Cooperate 3, 3 0, 6 Defect 6, 0 1, 1 In the Prisoner s dilemma the outcome (Defect, Defect) is a Nash equilibrium. If the row player changes his action, he will receive a lesser utility of 0, and if the column player changes his strategy, he will also receive a lesser utility of 0. Nash equilibrium, as well as other solution concepts, have refinements. The Nash equilibrium described in definition 2.1.5, also referred to as a weak Nash equilibrium, has a refinement where the players strictly prefer one outcome over the other. This is called the strict Nash equilibrium: Definition An outcome σ is a Strict Nash Equilibrium if for all σ i S i, (σ i,σ i) i (σ i,σ i) and (σ i,σ i) i (σ i,σ i) The following example demonstrates some additional refinements of Nash equilibrium. 5 To prove this, Nash relies on Brouwer s fixed point theorem. The proof consists of a mapping that satisfies the conditions of Brouwer s fixed point theorem; that it is compact, convex, and closed. This therefore requires that the function has fixed points. Nash proves that these fixed points are exactly the equilibrium points [32]. 15

17 Example: Stag Hunt Game There are many more refinements of the Nash equilibrium that reveal interesting additional information about a game when there are multiple Nash equilibria. Consider the following example of the Stag Hunt game, which has multiple Nash equilibria 6 : Stag Hare Stag 2,2 0,1 Hare 1,0 1,1 (Stag, Stag) and and(hare, Hare) are the Nash equilibria in this coordination game. Two refinements of Nash equilibrium are exemplified in this game: pay off dominant and risk dominant equilibria. (Stag, Stag) is the pay off dominant equilibrium, because it yields the highest utilities, and (Hare, Hare) is the risk dominant equilibrium, because it has the lowest risk of a low utility should one player deviate for some reason Iterated Elimination of Dominated Strategies A special procedure of reasoning about a game is by means of iterated elimination of dominated strategies (IEDS), which is the repeated application of the notion dominated strategy to a game. With each application of IEDS a dominated strategy is removed from the game. It operates under the assumption that a rational player will never choose to play strategies that give him unilaterally (for any strategy the opponent plays) strictly lower utilities than at least one other strategy. Following this reasoning, one may repeat this until a smaller game or only one outcome remains. This procedure depends on the solution concept dominated strategy: Definition Astrategyσ i is a dominated strategy for player i if σ i σ i (σ i,σ i ) i (σ i,σ i ) In other words, if there is a better strategy σ than the one in question σ for every possible move of i s opponent, then σ is a dominated strategy. A rational player j will reason that his opponent i will never choose to play a dominated strategy, so j may rule out the outcomes where i plays a dominated strategy. Player i will reason that because player j has ruled out one of his own dominated strategies, one of j s strategies becomes dominated, and can be eliminated as well. This is repeated as many times as it takes for no more strategies to be deleted. The process will result in a strategy profile (one or more) that survive after the sequence of deletions. Consider the game in the following example and the single outcome, (D, A) that the IEDS procedure selects. 6 The story of the Stag Hunt game is as follows: two hunters can choose each to hunt for hare or stag. Hunting hare is safer but less rewarding than hunting a stag. The stag is far more rewarding, but if a hunter hunts a stag alone, he is likely to get injured. Thus, if the two hunters work together to catch a stag, they will split the large reward, which is better than each catching and not sharing a hare. 16

18 A B C D 2,3 2,2 1,1 E 0,2 4,0 1,0 F 0,1 1,4 2,0 Figure 2.2: Matrix 1 borrowed from [2] The IEDS procedure in this example is described in the following figure 2.3: A B D 2,3 2,2 E 0,2 4,0 F 0,1 1,4 A B D 2,3 2,2 E 0,2 4,0 A D 2,3 A D 2,3 E 0,2 Figure 2.3: Matrices (2), (3), (4), ( ) respectively In matrix (1), theinitialgame,thestrategyc for the column player is dominated resulting in matrix (2). HerewecanseethatthestrategyF is now dominated for the row player, which, after deletion, results in matrix (3). Following this procedure, we can delete strategy B resulting in matrix (4) and then E resulting in the solution matrix where (D, A) is the solution that survives the process of IEDS Conclusion It is prudent to note that we are bound to strict assumptions of rationality and reasoning ability in order to make solution concepts like Nash equilibrium, best response and IEDS realistic. If we did not assume that the players consistently reasoned towards and chose the strategies that would result in the most preferred outcome possible, then Nash equilibrium, IEDS and best response would be hard to justify. These solution concepts all require rationality in some form 7. The strategic form is nevertheless an intuitive and robust way to describe strategic scenarios, but this form only partly composes classical game theory. The extensive form game is, similarly to the strategic form game, a model by which we can understand the strategic interaction and adheres to the stringent rationality requirements as well. An initial definition of an extensive form game by von Neumann in 1928 was further developed in 1953 by Harold Kuhn, a long-time colleague and friend of Nash, into what is now the default definition of an extensive form game. As we will see in the next section, the extensive form game is expressive like the 7 See [37] for an in-depth account of the rational requirements for rationality in (extensive) games. 17

19 strategic form game, but it models features of games that the strategic form is not equipped to handle, namely sequential turn-taking games. 2.2 Extensive Form Games Whereas a strategic form game expresses a unique one-time, one-move scenario, an extensive form game can express a game that involves sequences of actions and turn taking. As with strategic form games, an extensive form game is defined by players, actions and utilities of outcomes, but there are some differences that clearly sets it apart from a strategic form game. For instance, an extensive form game is represented not by a matrix but by a game tree, it introduces a new notion history, and it uses an alternative perspective on the notion strategy. With these concepts, an extensive form game represents all possible sequences of the players choices, which lead to unique outcomes that are associated with one particular sequence. Those outcomes are also assigned utilities for each player in the game. In this section, I will first describe the basic terms and definitions of extensive form games. Second, I will describe the game tree. Also, I describe solution concepts for extensive form games. This includes the Nash equilibria which exist for extensive form games, but because they ignore the sequential nature of the game, another solution concept, subgame perfect equilibrium is used to more accurately describe a solution intuitive to the game. Generally, extensive form games operate under the assumption that the players observe each other s moves and know what state they are in; under these circumstances the game is a perfect information extensive form game. However, there are extensive form games with imperfect information meaning that the players are not always aware of the other player s move. I will describe these games as well Terminology The following terms and definitions are used to describe extensive form games 8. We formally define an extensive form game as Definition An extensive form game is a tuple G = N,H,P,( i ) i N where: Players As with strategic form games, there is a set of players N, withi N as some individual player. Histories A set of histories H describes all possible sequences of actions taken by the players, where h = is the start of the game (i.e. the empty sequence). If amemberofh (a sequence of actions) is h K =(a 1,..., a K ), then for every L< K, it holds that h L =(a 1,..., a L ) is a member of H. A history h K =(a 1,..., a K ) 8 Based on [35] 18

20 is a member of the set of terminal histories 9 Z H if there is no a K+1 such that h K+1 =(a 1,..., a K+1 ) is in H. The game ends at the terminal history, but after a non-terminal history h, the set of actions available are denoted by A(h) ={a :(h, a) H}. Turns Each non-terminal history h H\Z is assigned to a player in N by the player function P.ThusP(h) =i indicates that after h, it is player i turn. Preferences For each player i there is a preference relation ( i )overtheset of terminal histories Z. We denote (z,z ) ( i )asmeaningthatiprefers z over z or prefers them equally. In an extensive form game, a strategy has a more involved notion of strategy than in strategic form games: Astrategy,sofar,hasbeenunderstoodtobetheactionorprobabilitydistribution of actions that a player can take in a game. In extensive form games, however, players take multiple actions progressing through the game, so we take an alternative approach to account for this. A strategy in extensive form games specifies for each history which action will be taken by the player whose turn it is. That is, a player has a planned response to every action that his opponents may take. Formally, a strategy for i associates with each h for which P (h) =i an action a A(h). Because a strategy is a sequence of actions, an outcome cannot be described directly by strategy profiles. Instead we can define an outcome for every strategy profile σ as O(σ) istheterminalhistorythatresultswheneachplayeri N follows the precepts of σ i [35]. This only holds for perfect information games Game Tree A game tree is a non-cyclical pointed graph that is a visual tool representing extensive form games as described above. Like the matrix for strategic form games, the game tree is useful to understand the concepts described by the formal terms. The extensive form game, and thus the game tree, differs from the game matrix, because the game matrix...describes only a situation where each player makes a single choice, in ignorance of the choices made by the other players, and the game is then over. The tree thus appears to be a more general description, allowing players to move more than once and also to observe what other players do [14]. The game tree is composed of nodes, which represent histories, and edges which represent possible actions a player can take given the preceding node. The terminal nodes are also labelled by utilities, which correspond to the preference ordering over outcomes for each player 10 These concepts are illustrated by the tree in figure Iexcludethepossibilityofinfinitehistories, for it is not currently relevant. 10 The utilities can alternatively be represented as preferences. See section

21 1 L R 2 2 l r l r z 1 z 2 z 3 z 4 Figure 2.4: This is an extensive form game with the terminal nodes marked by zs Equilibria Extensive form games have as a solution concept Nash equilibrium, but it also introduces the solution concept subgame perfect equilibrium. Nash equilibrium [35] in extensive form games essentially claims the same thing as in strategic form games. No player can do better by defecting from his current strategy. Definition A Nash equilibrium of an extensive game with perfect information N,H,P( i ) is a strategy profile σ such that for every player i N we have O(σ i,σ i) i O(σ i,σ i) for every strategy σ i of player i However, Nash equilibrium does not reflect the sequential nature of extensive form games, and the following example, borrowed from [35], verifies this and consequentially motivates a solution concept reflecting the sequential nature of extensive form games, the subgame perfect equilibrium. The following extensive form game, borrowed from [35], exhibits how the solution concept Nash equilibrium can lead to unintended conclusions. The game in figure 2.5 has two Nash equilibria: (A, r) and(b,l). However, the equilibrium (B,l) is motivated by the explanation that player 1 will play B because of player 2 s threat of playing l. Playing l still amounts to the same payoff, after all. But at player 2 s choice node it would never be optimal to play l because playing r affords a greater payoff, so the threat of playing l is incredible. Therefore, it is better to base an equilibrium for extensive form games on what is optimal for an acting player at every node h. This relies on the notion subgame. The definition of a subgame of an extensive form game Γ with perfect information is [35]: 20

22 1 A B l 2 r 1,2 0, 0 2, 1 Figure 2.5 Definition A subgame of the extensive game with perfect information Γ= N,H,P,( i ) that follows history h is the extensive game Γ(h) = N,H h,p h, ( i h ) where H h is the set of sequences h of actions for which (h, h ) H, P h is defined by P h (h )=P (h, h ) for each h H h,and i h is defined by h i h h if and only if (h, h ) i (h, h ). The subgame perfect equilibrium is a solution where after each history h, each player s strategy is the optimal one given his opponent s move. By basing an equilibrium concept on subgames, we preserve the sequential nature of the scenario. Thus, a subgame perfect equilibrium is based on the concept of a subgame: Definition A subgame perfect equilibrium of an extensive game with perfect information Γ= N,H,P,( i ) is a strategy profile σ such that for every player i N and every non-terminal history h H\Z for which P (h) =i we have: O h (σ h,σ i h ) i h O h (σ i,σ i h ) for every strategy σ i of player i in the subgame Γ(h) Where O h is the outcome function of Γ(h). It is also possible to view extensive form games as a strategic form game by means of normalisation, and subsequently we can examine solutions in strategic form Normalisation Because we can list all the strategies available to each player, it follows that we can convert any extensive form game into a strategic form game by putting the strategies into the matrix and assigning the corresponding utilities. Formally [35], 21

23 Definition The strategic form of the extensive game with perfect information Γ= N,H,P,( i ) is the strategic game N,(S i ), ( i ) in which for each player i N S i is the set of strategies of player i in Γ i is defined by σ i σ if and only if O(σ) i O(σ ) for every σ i N S i. 1 L R 2 2 l r l r 4, 1 1, 3 3, 2 2, 4 (a) Extensive form game tree. L R ll (4,1) (3,2) lr (4,1) (2,4) rl (1,3) (3,2) rr (1,3) (2,4) (b) Strategic form game matrix. Figure 2.6: (b) is the strategic form game matrix that results from the extensive form game tree in (a) Notice in figure 2.6 that expressing an extensive form games in this manner is inefficient, for the conversion can lead to very large matrices. This process leads to redundantly listing outcomes. Notice in the matrix above that the outcome with utilities (4,1) is listed twice (as are all the outcomes) for the strategy profiles (L, ll) and(l, lr) arethesame. Given player 1 plays L, both of the strategies ll and lr express that he plays l when 1 plays L and l when 1 plays R. Because the strategy profile already specifies that 1 plays L, thefactthat2mayplayl or 1 after 1 plays R has no bearing on the outcome. Therefore, expressing extensive form games in matrices leads to inefficient redundancies. The comparison does reflect an interesting fact about matrices: The reason is that the matrix can, in fact, be thought of as modelling any interaction, even ones in which the players move more than once. The key idea here is that strategies of the matrix can be thought of, 22

24 not as single moves, but rather as complete plans of action for the tree. [14] The fact that this is possible will be of interest in the upcoming discussion on logic in classical game theory Backward Induction The procedure of backward induction is the process that proves Kuhn s Theorem that every finite extensive game with perfect information has a subgame perfect equilibrium [35]. The idea is that there is a best terminal node branching from anodeh for the player whose turn it is. If that player is rational, he will choose the action corresponding to that terminal node. For this reason, we can back that value up the tree to that node h. If the player whose turn it is at the node leading to this newly valued node h, and that value is preferred to his other actions, then he will take that action, allowing us to move that value further up the tree. Continuing in this fashion, we can determine a path in the tree that selects an outcome called the backwards induction solution. For an in-depth discussion of backwards induction see sources such as [5], [9] and [26]. 1 L R 2 2 l r l r 4, 1 1, 3 3, 2 2, 4 Figure 2.7 The bold lines in the game in 2.7 represent the path in the game that results in a backward induction solution. Every game has a unique backward induction solution if the preference over terminal histories is strict for each player. Otherwise, the backward induction path would split at any point h where the values of the possible actions are the same. Both outcomes resulting from h would be backwards induction solution Imperfect Information An imperfect information extensive form game is an extension of extensive games where the players at some point in the game do not have knowledge of past moves. This implies that a player, at some point in the game, is uncertain of which state he is in. This occurs when his opponent s move is not public 23

25 to him, or if moves are made by chance. These games have added structure that represent what knowledge the acting players have about the state of the game. Furthermore, the special player chance is added. The definition accounts for these factors with two additional concepts in the tuple: Definition An imperfect information extensive form game is a tuple G = N,H,P,f c, (I i ) i N ( i ), where: N, H, P, and i These are as described in definition above. Probability Distribution The actions of player chance are determined by the probability distribution f c ( h) ona(h). That is, the probability that player chance plays a at history h is f c (a h). Because of the chance component, the utility function is defined as lotteries over terminal histories, since chance induces a non-deterministic component over terminal histories. Information Partition The structures that represent the knowledge of the acting players are information sets. Information sets partition the set of all histories where a player acts i.e. I i is a partition on {h H : P (h) =i}, and an information set is a I i I i. These structures are interpreted as follows, the acting player i cannot distinguish between h and h when h, h I i. Because players cannot distinguish between histories in the same information set, they have the same set of actions to choose from if the histories are members of the same information set. Hence A(h) =A(h )wheneverh, h I i. For if A(h) A(h ) players could deduce in what history they are by the actions available to them. Furthermore, because players cannot distinguish between histories in the same information set, they can only decide upon one (stochastic) action per information set. Therefore, histories are no longer the primitive of the game over which is reasoned, but rather information sets take its place. In general, backward induction is not possible under imperfect information [26], because in imperfect information games a player chooses an action per information set. In backward induction, you assign a value to every history. Moreover, a player takes an action per history. However, in imperfect information games, aplayerchoosesanactionperinformationset. Therefore,itisunclearhowto assign values to actions in the information set. Backward induction is generally not possible for imperfect information games Conclusion With these basic concepts that compose classical game theory, we can move on to describe evolutionary game theory. There are multiple ways one can choose 11 Some new literature suggests that for specific imperfect information games backward induction-like algorithms are possible [15]. 24