OPIM 319, Spring 2007: Advanced Decision Systems: Agents, Games & Evolution Steven Kimbrough OPIM 319, Agents, Games & Evolution, explores applications and fundamentals of strategic behavior. Strategic, or game-theoretic, topics arise throughout the social sciences. The topics include and we shall study trust, cooperation, market-related phenomena (including price equilibria and distribution of wealth), norms, conventions, commitment, coalition formation, and negotiation. They also include such applied matters as design of logistics systems, auctions, and markets generally (for example, markets for electric power generation). In addressing these topics we focus on the practical problem of finding effective strategies for agents in strategic situations (or games). Our method of exploration will be experimental: we review and discuss experiments on the behavior of agents in strategic (or game-theoretic) situations. In focusing on the design and behavior of artificial agents in strategic (or game-theoretic) situations, we will be especially concerned with strategic contexts of commercial import, such as markets, bargaining, and repeated play. We shall dwell on effective agent learning techniques, including evolutionary methods and reinforcement learning. A main theme in the course is the inherent difficulty, even unknowability, of the problem of strategy acquisition. We will rely mainly on computational experiments (or simulations), in distinction to analytic mathematical methods, for studying strategy formation and strategic behavior (either by individuals or by groups). Much of the class work will be devoted to discussing and interpreting computational experiments that have been reported in the literature, or that can be undertaken with tools provided in class. In doing so, we draw upon the rapidly growing literature in agent-based modeling and agent-based simulation. Agent-Based Computational Economics (for example, http://www.econ.iastate.edu/tesfatsi/ace.htm) and Agent-Based Social Science (for example, http://www.brookings.edu/es/dynamics/papers/ csed wp41.htm) have come to denote active communities of research and application. We shall draw upon them. Computer programming is neither required nor discouraged for the course. The instructor invites, and will support, projects using NetLogo (as well as other envirnments). Many of the computational demonstrations and experiments we will examine are available as NetLogo programs (http://ccl. northwestern.edu/netlogo/). Students are not, however, at all required to undertake programming exercises, in NetLogo or in any other environment. Students completing the course can expect to come away with: Solid understanding of what is known and what is not known about the problem of designing procedures for strategic behavior, Familiarity with the principal methods, and results of applying those methods, for the modeling of human agents and design of artificial agents in strategic contexts, and Deepened appreciation for contexts of strategic interaction. 1
Grading is based on class participation, assigned short essays undertaken during the term, a midterm quiz, and a term project. For further information, contact the principal instructor for the course, Professor Steven O. Kimbrough (kimbrough@wharton.upenn.edu). 1 Required Texts The Evolution of Cooperation, Robert Axelrod, Basic Books, 1984. The Stag Hunt and the Evolution of Social Structure, Brian Skyrms, Cambridge University Press, 2004. Growing Artificial Societies, Joshua Epstein and Robert Axtell, MIT Press, 1996. In addition, various other readings will be assigned. These will generally be handed out or made available online. In particular, we will read a number of chapters from Professor Kimbrough s draft manuscript, Agents, Games & Evolution, which is referred to below as the AGEbook. 2 Class Schedule 1. January 9, 2007. Introduction and overview of the course. Strategic interaction. Illustrations of games in the wild. Read: Contexts of Strategic Interaction, chapter 1 of the AGEbook (handed out in class and made available beforehand on Web Café). Recommended reading: chapter 2 of the AGEbook, Four Themes. Movie clips: Gutman interrogating Sam Spade in The Maltese Falcon. Opening scenes from The Rounders. 2. January 11, 2007. Emergence: Surprises in strategic interaction. Plus more on games in the wild. For that read chapter 3 of the AGEbook, Games in the Wild. Read: Micromotives and Macrobehavior, pp. 11-43 of Micromotives and Macrobehavior, Thomas C. Schelling, W.W. Norton & Co., New York, 1978. Note: Schelling recently (10 October 2005) won a Nobel Prize for doing this sort of work. http: //nobelprize.org/economics/laureates/2005/press.html. Recommended reading: Vehicles: Experiments in Synthetic Psychology by Valentino Braitenberg (1984), chapters 1 8 (42 pages). Demonstrations of emergence with NetLogo. Introducing agent-based models. 3. January 16, 2007. Elements of strategic analysis. Games in strategic form, games in extensive form. Focus on 2 2 games. Equilibrium. Mixed equilibrium. Solution concepts. Folk Theorem. Read: Games in the Abstract chapter of AGE. Recommended reading: Wikipedia article on game theory: http://en.wikipedia.org/wiki/ Game theory. Recommended reading: Ross, Don, Game Theory, The Stanford Encyclopedia of Philosophy (Spring 2006 Edition), Edward N. Zalta (ed.), URL = http://plato.stanford. edu/archives/spr2006/entries/game-theory/. Assignment 1 handed out. 2
4. January 18, 2007. Cooperation and its evolution. Read: The Evolution of Cooperation, by Robert Axelrod, Basic Books, 1984, chapters 1-3, appendix B. 5. January 23, 2007. Cooperation and its evolution. Read: The Evolution of Cooperation, by Robert Axelrod, Basic Books, 1984, chapters 4-5. 6. January 25, 2007. Cooperation and its evolution. Read: The Evolution of Cooperation, by Robert Axelrod, Basic Books, 1984, chapters 6-9 (skim). Assignment 1 due. 7. January 30, 2007. ESS: Evolutionarily Stable Strategy. Strategies that will be favored by evolution in repeated play of a game. Read: Wikipedia article on ESS: http://en.wikipedia.org/wiki/evolutionarily stable strategy. Recommended reading: John Maynard Smith, The Basic Model, chapter 2 (pp. 10-27) of Evolution and the Theory of Games, Cambridge University Press, 1982. Recommended reading: Alexander, J. McKenzie, Evolutionary Game Theory, The Stanford Encyclopedia of Philosophy (Summer 2003 Edition), Edward N. Zalta (ed.), URL = http://plato.stanford.edu/archives/ sum2003/entries/game-evolutionary/. Movie clip: The Chickie Run from Rebel without a Cause. Assignment 2 (short essay) handed out. 8. February 1, 2007. Fairness and ultimatum games; commitment; reciprocity. Read: Brian Skyrms, Evolution of the Social Contract, chapters 1-2 ( Sex and Justice, pp. 1-21; Commitment, pp. 22-44), Cambridge University Press, 1996. Movie clip: Doomsday Machine sequence from Dr. Strangelove. 9. February 6, 2007. Beyond Prisoner s Dilemma: The Stag Hunt and other illuminating games. Read: Brian Skyrms, The Stag Hunt and the Evolution of Social Structure, Preface (pp. xi-xiv) and chapter 1, The Stag Hunt (pp. 1-14), Cambridge University Press, 2004. Assignment 3 (short essay) handed out. 10. February 8, 2007. Beyond Prisoner s Dilemma: The Stag Hunt and other illuminating games. Read: Brian Skyrms, The Stag Hunt and the Evolution of Social Structure, chapter 2, Bargaining with Neighbors (pp. 17-30) and chapter 3, Stag Hunt with Neighbors (pp. 31-44), Cambridge University Press, 2004. 11. February 13, 2007. Beyond Prisoner s Dilemma: The Stag Hunt and other illuminating games. Read: Brian Skyrms, The Stag Hunt and the Evolution of Social Structure, chapter 4, Evolution of Inference (pp. 45-64) and chapter 5, Cheap Talk (pp. 65-82), Cambridge University Press, 2004. 12. February 15, 2007. Beyond Prisoner s Dilemma: The Stag Hunt and other illuminating games. Read: Brian Skyrms, The Stag Hunt and the Evolution of Social Structure, chapter 6, Choosing Partners (pp. 87-104) and chapter 7, Coevolution of Structure and Strategy (pp. 105-124), Cambridge University Press, 2004. 3
13. February 20, 2007. Trust. Read: Garrett Hardin, The Tragedy of the Commons, Science 162:1243-8, 1968. Read: Steven O. Kimbrough, Foraging for Trust: Exploring Rationality and the Stag Hunt Game, in Trust Management: Third International Conference, itrust 2005, Paris, France, May 23-26, 2005. Proceedings, P. Hermann, Valérie Issarny and Simon Shiu, eds., Springer-Verlag GmbH, Berlin, Germany, LNCS: Lecture Notes in Computer Science, 3477 / 2005, pp. 1-16, 2005. Read: Jon Elster, Introduction: the two problems of social order, chapter 1 in The Cement of Society: A study of social order, Cambridge University Press, 1989, pp. 1-16. 14. February 22, 2007. Trust (continued). Experimental explorations of trust. 15. February 27, 2007. Trust (continued). Computational (and analytic) explorations of trust. 16. March 1, 2006, Thursday. Mid-term quiz. 17. March 13, 2007. Cellular automata models. * * * Spring break week of March 5, 2007 *** The Game of Life, among others. Read: What Is Life? in Winning Ways for Your Mathematical Plays, volume 2: Games in Particular, by Elwyn R. Berlekamp, John H. Conway and Richard K. Guy, Academic Press, 1982. 18. March 15, 2007. Growing artificial societies. Axtell, MIT Press, 1996, chapters 1-2. 19. March 20, 2007. Growing artificial societies. Axtell, MIT Press, 1996, chapters 3-4. 20. March 22, 2007. Growing artificial societies. Axtell, MIT Press, 1996, chapters 5-6. 21. March 27, 2007. Further explorations on the gridscape. Computational demonstrations and experiments. Recommended readings: chapters from the AGEbook. 22. March 29, 200. Games, complexity, computability. Instructor handouts. Recommended reading: Patrick Grim, Undecidability in the Spatialized Prisoner s Dilemma: Some Philosophical Implications at http://www.sunysb.edu/philosophy/ faculty/pgrim/spatialp.htm. 23. April 3, 2007. Applications: Stock markets. Read: Dhananjay K. Gode and Shyam Sunder, Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality, Journal of Political Economy, 101, no. 1, pp. 119 137, 1993. 4
24. April 5, 2007. Applications: Matching problems. Read: D. Gale and L. S. Shapley, 1962. College Admissions and the Stability of Marriage, The American Mathematical Monthly, 69, no. 1, pp. 9 15. Lawrence Bodin and Aaron Panken, 2003. High Tech for a Higher Authority: The Place of Graduating Rabbis from Hebrew Union College Jewish Institute of Religion, Interfaces, 33, no. 3, May June, pp. 1-11. 25. April 10, 2007. Applications: Classical markets (monopoly, oligopoly, perfect competition) and markets for electric power generation. Instructor handouts. Cournot duopoly models. Read: Steven O. Kimbrough, Ming Lu, and Frederic Murphy, 2004. Learning and Tacit Collusion by Artificial Agents in Cournot Duopoly Games, in Steven O. Kimbrough and D. J. Wu, eds., Formal Modelling in Electronic Commerce, Springer, pp. 477-492. 26. April 12, 2007. Evolutionary algorithms. Evolutionary computing. Genetic algorithms. Replicator dynamics. Genetic programming. 27. April 17, 2007. Evolutionary alogorithms. Evolutionary computing. Learning classifier systems. Models of individual learning in strategic contexts. 28. April 19, 2007. Last class. Summing up. Rationality redux. $Id: syllabus-s07-age.tex,v 1.4 2007/01/08 16:22:24 sok Exp $ 5