The Evolution of Knowledge and Search in Game-Playing Systems

Transcription

1 The Evolution of Knowledge and Search in Game-Playing Systems Jonathan Schaeffer Abstract. The field of artificial intelligence (AI) is all about creating systems that exhibit intelligent behavior. Computer games have been one of the most visible application areas for innovative AI research and, in part, this has led to numerous major success stories. However, fifty years of AI research has seen the recipe for building strong programs evolve from the early knowledge-intensive approaches to the current searchintensive approaches. Each step forward seems to move us further away from the human example. This paper talks about this evolution and its impact on building high-performance game-playing programs. 1 Introduction The evolution of how researchers think about artificial intelligence (AI) is fascinating, and the insights profound. In only half a century of AI research, we have made the fundamental leap from the simplistic model of emulating human thought processes to realizing that there are alternative computational models that are more effective. It took the AI community a long time to realize that the well-known how to fly insight was relevant: one does not need to copy nature s solution to having a bird fly; we can do things more effectively by using the human-created jet airplane architecture. AI continues to be a moving target. It is a truism, and a sad reflection on perceptions in the AI community, that once a problem gets solved it ceases to be AI. No one questions the utility of a spell checker or an auto-pilot on an airplane. Yet if you turned back the clock fifty years, people would have viewed such programs as being excellent examples of important applications for AI research. We understand these problems well enough that, in the appropriate circle, they are routine. The bizarre consequence is that they are now no longer considered AI. Why? The magic is gone. These problems don t need artificial intelligence to achieve intelligent behavior. As we understand applications (and AI) better, the mystery surrounding intelligent behavior disappears. As the research challenges of building high-performance game-playing programs are successfully addressed, we as a research community run the danger of losing relevance. This research artificial intelligence applied to games we belovedly consider to be fundamentally part of AI. Yet our successes may eventually result in a move from the research consciousness for this type of research Professor Jonathan Schaeffer is head of the department of Computing Science at the University of Alberta in Edmonton, Canada. jonathan@cs.ualberta.ca.

2 34 Jonathan Schaeffer to out of the AI realm altogether. Along the way, however, our community will continue to make important contributions that will have long-term impact. This paper provides a brief discussion of the role of human knowledge and its evolution to the construction of strong game-playing programs. In the early years of AI research, the emphasis was on knowledge-based systems. Today, fifty years later, knowledge is known to have its share of problems, and researchers have been moving steadily towards building systems with a minimal dependence on knowledge. The extreme case involves no knowledge (perhaps not much more than the rules of the game). It seems like a paradox that artificial intelligence is moving in a direction which involves no human-inspired intelligence in the decision-making process (but may still require intelligence in the pre-processing needed for the decision-making process). 2 The Evolution of (Computer) Thought The course of artificial intelligence research applied to classic board games has seen an evolution in the relationship between search and knowledge: 1. Knowledge-based systems: In the fledgling days of AI, the most common approach to building an AI system was to try and emulate the human example. This concept was easily understood and seen as an obvious recipe for success. Unfortunately, researchers eventually realized this approach was fragile. The reliance on explicit application-dependent knowledge required the use of domain experts, a resource that was not easy (and often costly) to obtain. Researchers had to overcome challenging issues such as knowledge acquisition, knowledge representation, knowledge manipulation, exception handling, and performance trade-offs. 2. Search-based systems: Search is knowledge or, rather, search can implicitly uncover properties (knowledge) of a domain. A milestone in the development of game-playing programs and, indeed, a profound insight for all of artificial intelligence research, was the realization that deep search with less dependence on knowledge was a viable solving technique. In the not-sodistant past, the term brute-force search was used derogatorily, as if the researchers using such a technique were in some way doing something disreputable. Brute-force search overcame its stigma and has emerged as a powerful AI technique, in part because of its simplicity and in part because it reduces the need for application-dependent knowledge. It also has the tremendous advantage that search is algorithmic (and inexpensive to implement), lessening our dependence on knowledge which is heuristic (and may be expensive to acquire). 3. Knowledge-free systems: With advancing technology, the ultimate solution technique has become increasingly popular: solved games. The solutions are pre-computed and are search intensive. The deployed run-time decisionmaking system, however, is devoid of application-dependent knowledge (typically, it just does a table lookup). Clearly, application-dependent knowledge

3 The Evolution of Knowledge and Search in Game-Playing Systems 35 may be needed to come up with an effective scheme for solving these domains (application pre-processing), however this is minor in comparison to the knowledge requirements of the previous solution techniques. The envelope keeps getting pushed here, and with the recent solving of checkers (search space is ) it has been pushed forward by many orders of magnitude [10]. Of note is that we are seeing more of the knowledge requirements moving from the application execution (the playing of a game; the decision-making process) to the pre-processing stage (building the data needed for the application execution). 3 Towards Knowledge-Free Game-Playing Programs The era of building knowledge-free game-playing programs is just dawning. It is now not confined to only two-player perfect-information games, where there have been numerous successes (e.g., chess endgame databases, checkers [10], Qubic and Go-Moku [3], Connect-Four [1, 2], awari [9]). In the area of games with imperfect information and/or stochasticity, the concept of a solved game remains, albeit with different performance expectations. Here we cite two radically different domains where the move is towards lessening (eliminating?) the role of knowledge. Consider the game of poker and the role of game theory. Poker is a game of imperfect information (hidden cards) and stochasticity (random dealing of cards). Building a complete Nash equilibrium strategy would allow for gametheoretic optimal play. Two-player limit Texas Hold em poker has a search space of size 10 18, so it is currently impractical to properly solve this game. Nevertheless, pseudo-optimal solutions (close approximations to an optimal solution) have been built using linear programming [4]. This recent development was a major advance in the state of the art, and represents the best programs in the world today (recent advances in this area are narrowing the gap between pseudo-optimal and optimal play [12, 6]). Again, all the magic happens in the pre-processing stage; the actual game play comes from a table. However, optimal play is not maximal play. In poker the performance metric is winning the most money, not the most hands. Hence, opponent modelling is essential to being able to maximize the money won. There is still much work to be done to turn this technology into a program that will consistently beat the best humans in the world (especially in games with more than two players). As another example, consider the use of statistical sampling (Monte Carlo search) as a means of gathering data to aid in the decision-making process. Sampling was used with some success in bridge [7], poker [5] and Scrabble [11]. However, in the game of Go, sampling seems to have found a home. The program plays random sequences of moves, evaluates the resulting position, and uses this information to collect statistics on the goodness of each child move of the root. The best 9 9 Go-playing programs in the world are using the Upper Confidence bounds applied to Trees (UCT) sampling algorithm [8], and the results for Go are beginning to look impressive. UCT easily handles the

4 36 Jonathan Schaeffer issues of exploration versus exploitation in the search, by carefully managing the decisions of where to acquire more knowledge about the search tree. Go-specific knowledge, the crucial ingredient in Go programs to date, has had its role reduced (but still important). Further improvements to the sampling methods will likely further decrease the importance of Go knowledge. These examples illustrate that new algorithmic ideas are creating innovative ways to look at the design of a high-performance game-playing program. With faster processors, more parallelism (especially with multi-core architectures), larger memories, and inexpensive disk, this trend will continue. 4 Jaap van den Herik Jaap van den Herik has spent a large portion of his life being fascinated by games, from the recreational, competitive and scientific points of view. From early on in his research career, Jaap has been infatuated with solved games: perfection. Perhaps this is because of the beauty of perfection, or the knowledge that one can reveal the secrets of a game that no one has ever been privileged to see before. A psychologist might argue it is because of a desire to see his creations aspire to more than their maker. For most of his career Jaap has actively participated in the construction of chess endgame databases (solved subsets of the game) and been a fastidious chronicler of the progress in this area. Every advance in solving games even obscure games is dutifully recorded in the pages of the International Computer Games Association Journal, which Jaap has edited for almost a quarter of a century. In addition, Jaap supervised the Ph.D. thesis of Victor Allis, who produced arguably the most important thesis in this research area [3]. Not only did the thesis turn the relatively unknown (and little-used) Conspiracy Numbers algorithm into the popular solving procedure Proof-Numbers Search, but it also presented the results for solving the games of Qubic and Go-Moku. With new games and new challenges constantly arising, there is no time to slow down. Indeed, it is inconceivable to think of Jaap s frenetic pace changing any time soon. The artificial intelligence and computer games communities are richer because of this. 5 Conclusions The words intelligence and knowledge seem to be inextricably entwined. Yet the research using games has shown that intelligent behaviour does not necessarily stem from knowledge. Games have been at the forefront of AI research since the inception of the field. Passionate AI-using-games researchers have made numerous important contributions to the AI literature. The move from knowledgebased, to search-based, to knowledge-free solutions may be another major result. It does not end there. We are in the midst of a renaissance in human game playing. The past decade has seen a wealth of innovative new and intellectually stimulating games being created. This new generation of games, especially those

5 The Evolution of Knowledge and Search in Game-Playing Systems 37 coming out of Germany, pose a wealth of interesting challenges for artificial intelligence research. The games span many fun ideas, including cooperative play, auctions, tile-based play, and map-based planning. For most of these games, the research is just beginning and, hence, is still in the knowledge-based-solution era. The community who uses games as their artificial intelligence experimental test bed are seeing numerous research opportunities presented to them. As with the classic games, the community will seek to better understand these domains and develop new solving techniques. History suggests they will be successful, and the AI community will be better because of it. Acknowledgments This work was sponsored by the Natural Sciences and Engineering Research Council of Canada (NSERC) and Alberta s Informatics Circle of Research Excellence (icore). References 1. J. Allen. A note on the computer solution of Connect-Four. In Heuristic Programming in Artificial Intelligence: The First Computer Olympiad, pages Ellis Horwood, V. Allis. A knowledge-based approach to Connect-Four. The game is solved: White wins. Master s thesis, Vrije Universiteit, Amsterdam, V. Allis. Searching for Solutions in Games and Artificial Intelligence. PhD thesis, Department of Computer Science, University of Limburg, D. Billings, N. Burch, A. Davidson, R. Holte, J. Schaeffer, T. Schauenberg, and D. Szafron. Approximating Game-Theoretic Optimal Strategies for Full-scale Poker. In proceedings IJCAI, pages , D. Billings, L. Peña, J. Schaeffer, and D. Szafron. Using Probabilistic Knowledge and Simulation to Play Poker. In proceedings AAAI, pages , A. Gilpin, T. Sandholm, and T. Sorensen. Potential-aware automated abstraction of sequential games, and holistic equilibrium analysis of Texas Hold em poker. In proceedings AAAI, pages 50 57, M. Ginsberg. GIB: Imperfect information in a computationally challenging game. Journal of Artificial Intelligence Research, 14: , L. Kocsis and C. Szepesvari. Bandit based Monte-Carlo planning. In European Conference on Machine Learning, pages , J. Romein and H. Bal. Solving the game of awari using parallel retrograde analysis. IEEE Computer, 36(10):26 33, J. Schaeffer, N. Burch, Y. Björnsson, A. Kishimoto, M. Müller, R. Lake, P. Lu, and S. Sutphen. Checkers is Solved. Science, Published online July 19, To appear in print. 11. B. Sheppard. Towards Perfect Play of Scrabble. PhD thesis, Universiteit Maastricht, M. Zinkevich, M. Bowling, and N. Burch. A new algorithm for generating equilibria in massive zero-sum games. In proceedings AAAI, pages , 2007.