Opponent Models and Knowledge Symmetry in Game-Tree Search
|
|
- Lorena Butler
- 5 years ago
- Views:
Transcription
1 Opponent Models and Knowledge Symmetry in Game-Tree Search Jeroen Donkers Institute for Knowlegde and Agent Technology Universiteit Maastricht, The Netherlands Abstract In this paper we investigate the role of symmetry in knowledge of the opponent during gameplaying by computers. The games under consideration are board games like Chess: large-scale two-player zero-sum games of perfect information. The solution of such games is easily defined but since these board games are large and complex, it is not feasible to obtain an actual solution. Research in Artificial Intelligence has resulted in efficient heuristic search methods that approximate solutions. The standard search methods assume pure competitive preferences with inherent knowledge symmetry. With the development of Opponent-Model search, a knowledge asymmetry is introduced: only one of the players is assumed to have a precise model of the opponent. We propose to add knowledge symmetry to opponent modelling and allow both players to have a model of each other and commonly know this. This leads to a switch from zero-sum to nonzerosum games. New search methods will have to be defined and investigated. We provide some suggestions for future research. 1 Introduction Within Artificial Intelligence, playing board games like Chess by computers has always been a topic of interest. Although some games with imperfect information are treated (e.g., Backgammon and Poker), most of the board games studied in AI are two-player zero-sum games of perfect information in extensive form. From a game-theoretic point of view, these games are trivial since their solution is easily defined [11, 12]. Moreover, backward induction produces a subgame-perfect equilibrium (SPE) in these games [8]. Unfortunately, the size of the Chess game-tree is too large to perform backward induction within a reasonable amount of time. In the more than fifty years of research that has been done on computer game-playing, the emphasis was on developing heuristic methods that estimate the solution as good as possible, within the time allowed by tournament rules. The normal practice of playing Chess and other board games by computers is to solve a series of reduced games during a game instead of one large game, one for every turn in the game [4]. (This procedure is called search in computer game-playing.) The reduced games are constructed using three mechanisms: (1) Histories are truncated at a certain length (depending on the available search time). The true (but unknown) game-values of the truncated histories are replaced by static heuristic evaluation values. (The length of the resulting histories is called the search depth in computer gameplaying.) (2) Some histories are removed from the reduced game. This is called forward pruning in computer game-playing. (3) All histories in the reduced games start with the moves that already 1
2 have been played in the game (the game history). Figure 1 illustrates the mechanism. The original game (in extensive form) is represented by the dashed triangle and the subsequent reduced games are represented by the grey triangles A, B, and C. The history of actual moves played is indicated by the bold zigzag line. At the first move, player 1 solves reduced game A, then player 2 solves reduced game B, and so on. Figure 1: Playing a game (dashed triangle) by solving a series of reduced games (triangles A, B, C). The resulting reduced games can be solved by backward induction (called Minimax search in computer game-playing). In order to find the best move to play next, in the most cases it is not necessary to obtain a complete solution since only the game-theoretic value of the root node of the reduced game needs to be determined. Often, large portions of the game tree do not influence the game-theoretic value of the root node, and can be disregarded (which is called pruning). Pruning is the basis of the well-known ff-fi search procedure [7]. Many enhancements and improvements have been developed for ff-fi search, resulting in more pruning and therefore in an increment of the size of the reduced games that can be handled in the available search time. Although this approach has proven to be successful in Chess (e.g., Deep Blue), in other games (e.g. Go) the approach is less profitable. Furthermore, it is not clear whether solving a sequence of these reduced games is the best approach to win a game. In Section 2 we indicate and discuss the knowledge symmetry in Minimax search. Section 3 introduces Opponent-Model search and Section 4 describes a recursive extension of this search method. The combination of knowledge symmetry and opponent modelling in nonzero-sum games is proposed in Section 5. Finding solutions for large nonzero-sum games is subject of Section 6. Using opponent models introduces risk. This is briefly discussed in Section 7. The paper ends with conclusions in Section 8. 2 Knowledge symmetry in Minimax search Board games like Chess are modelled as zero-sum games, since they are mostly considered completely competitive. In a tournament setting, however, it can sometimes be advantageous not to win a certain game in order to obtain better chances in a next round. Some computer programs have a provision (called the contempt factor) that adjusts the value of a draw, but the game is still being treated as a zero-sum game. In the normal practice of Minimax search, the reduced games are assumed to be completely competitive too. A single static heuristic evaluation function is used to produce a pay-off value for all histories for both players at once. As we will see below, it is not necessary for the reduced games to be completely competitive. 2
3 The competitive approach to reduced games assumes knowledge symmetry: both players use the same evaluation function and both players commonly know that the other uses this evaluation function in a rational way too [9]. In a real tournament game, it is highly unlikely that the two players share the same heuristic evaluation function. Still, most programs use the Minimax approach. An explanation is that Minimax acts as a kind of worst-case defence: a program is defending against the worst possible opponent it can imagine. 3 Opponent-Model search In 1993, Opponent-Model search was proposed [1, 6]. The motivation of this approach was that the difference in evaluation function should be taken into consideration. When the opponent is using a weaker evaluation function (whatever this means), and a player knows the opponent s function, then this knowledge can be exploited. OM search is based on knowledge asymmetry. Assume that player 1 uses function v 1 to evaluate histories, and that player 2 uses another function v 2. OM search assumes that player 2 is unaware of player 1 using v 1 and that player 2 thinks that player 1 uses Minimax search with the same evaluation function, v 2. Player 1 (using OM search) knows both the own evaluation function v 1 and the evaluation function v 2 of player 2, and knows the opponent s state of knowledge. The OM-search procedure is a variant of backward induction and works as follows (a more detailed description can be found in [4]). For histories h where player 2 is to move, first standard backward induction (e.g. ff-fi search), is used to solve the subgame at h with evaluation function v 2. The value v(h) for player 1 of the subgame at h is then taken as the value 1 v(h + m Λ ) of the subgame at h + m Λ, where m Λ is the move selected by player 2. For histories h where player 1 is to move, the value v(h) of the subgame is defined as the maximum v(h + m i ) over all moves m i available at h. Figure 2 gives a small example of Opponent-Model search. The squares indicate player 1, the circles player 2. Inside the squares and circles are the payoffs and subgame values in the case that no opponent model is used: the Minimax approach. The values V 1 and V 2 indicate the payoff for player 1 and the payoffs that player 1 assumes that player 2 uses. The values v 2 give the subgame values for player 2, who is using Minimax. The values v 1 are the values that player 1 gets when using the opponent model. In the Minimax approach, player 1 would select the left branch, leading to a value of 7. When the opponent model is used, player 1 selects the right branch. This happens because player 1 We use notation h + m to indicate the extension of history h with move m. Figure 2: An example comparing Opponent Model search and Minimax search. 3
4 2 selects the left branch at the right node on the basis of payoffs V 2, leading to a value of 8 for player 1. The solution is indicated by the dashed line. The procedure means that the player using OM search is in fact aware of the opponent s strategy. Obviously, OM search is risky when the opponent model as described in the previous paragraph is not correct. 4 Recursive Opponent-Model search In OM search, one of the player has an explicit opponent model of the other player. M* search, also developed in 1993 [1, 2], is a recursive extension of OM search. In this approach, both players can have an opponent model of each other, but the knowledge is asymmetric. M* assumes that player 1 can have a model of player 2, like in OM search. However, unlike OM search, player 2 can be aware of this fact and can have a model of player 1 that includes the model that player 1 has of player 2. Moreover, player 1 can even have a more general opponent model of player 2 that includes player 2 s complete model. The opponent models can be nested to any depth but one of the players is always one step ahead of the other. 5 Knowlegde Symmetry and Opponent Modelling Both OM search and M* search have distributed the knowledge asymmetry among the players since player 1 knows more about player 2 than vise versa. Therefore, a natural next step after OM search and M* search is to add knowledge symmetry to opponent models. We propose an approach to opponent modelling in which both players are aware of their opponent s evaluation function that may differ from their own. Like in Minimax search, both players are aware of each other s knowledge and rational behaviour. It means that we assume the reduced games to be nonzero-sum games of perfect information. Why is it justified to assume that two players have a different evaluation function and at the same time know each others function? Shouldn t both players select the strongest function of the two and use Minimax? One answer would be the common interest of the players during certain stages of the game. This can lead to nonzero-sum game, even if both players interpret the positions similarly. A Chess example may clarify this [3, 4]. It is well known that computer Chess programs play relatively better in open positions (i.e., positions having many open lines) than in closed positions, see figure 3. If two computer programs are matched, it is in their common interest to reach an open position. Suppose that the evaluation function v 1 for player 1 assigns a value X + Y to a given Chess position P. X represents the openness of a position and Y represents all other factors, including material count, king safety, et cetera. The higher the value of X, the more player 1 prefers the position. In the Minimax approach, the evaluation function v 2 for player 2 is the opposite of v 1, so it assigns the opposite value to the same position: (X + Y ). It means that the higher the value of X for a position, the less player 2 prefers the position. In other words, Minimax assumes that player 2 tries to avoid an open position. To model the true preference of player 2, it seems more natural that evaluation function v 2 assigns value X Y to the position. Then player 2 prefers positions with high values for X as well as player 1. As a consequence, the sum of the two payoffs (X + Y ) + (X Y ) = 2X is not zero anymore, hence the game is nonzero-sum. Both players interpret the position in the same way, since they assign 4
5 Figure 3: An open Chess position (left) and a closed Chess position (right). an equal value to X for the openness of the position, which is their common interest, and an equal value to Y for the competitive part of the evaluation. When players interpret and evaluate the positions in different ways, the nonzero-sum property of the game is obvious. However, even in that case, common interest can be defined implicitly. Suppose that player 1 assigns value A to a position and player 2 assigns value B to the same position. The common-interest part of the evaluation (C) will then be equal to C = (A + B)=2 and the competitive part (S) will be equal to S = (A B)=2. The value for player 1 is C + S = A and the value for player 2 is C S = B. 6 Solutions for large nonzero-sum games Switching from zero-sum reduced games to nonzero-sum reduced games means that a solution becomes more difficult to define and to obtain. It is known from game theory that every finite game of perfect information has at least one subgame perfect equilibrium (SPE) and that backward induction can be used to find the SPEs [8]. In contrast with zero-sum games in which all SPEs have the same value, in nonzero-sum games, SPEs can co-exist with different pairs of values for the players. This occurs when one player can select between two or more optimal moves that each have a different value for the other player. Which move a player will select in such a case depends on the willingness of the player to co-operate. Figure 4 shows an example of how two SPEs in one game co-exist, asking for a method to select between them. The meaning of the squares and circles is equal to those in figure 2. The number inside the squares and circles indicate the Minimax approach. Below the leaf nodes are the payoffs expressed in a competitive part (S) and a common-interest part (C). The values for V 1 and V 2 are C + S and C S, respectively. Note that player 2 is maximizing payoffs in this approach, like player 1. The values next to the other nodes give the subgame values for the SPE in which player 2 selects the left branch at the right node. This SPE is a co-operative equilibrium: player 1 receives a higher payoff (8) than in the Minimax approach (7). Would player 2 select the left branch, then player 1 selects the left branch at the top node, leading to a payoff of 7 for player 1 and a payoff of 1 for player 2. So, although player 2 is locally indifferent in selecting left or right branch at the right node, globally player 2 is better of selecting the right branch. For small nonzero-sum games, it is not difficult to obtain all SPEs and select one of them, but for reduced games of the size that for instance Chess programs solve during play, the task is non-trivial. Many techniques that programs use to enhance the search efficiency depend on the zero-sum property of a game (e.g., ff-fi pruning). Similar techniques, applied to nonzero-sum games, are expected 5
6 Figure 4: An example comparing a nonzero-sum approach and Minimax search. to be less effective for two reasons: (1) subgames have two values instead of one and the relation between the two values is in general not known, (2) in case of multiple SPEs, information on all SPEs has to be taken into account. With an additional demand upon the payoff functions, however, matters may improve. We expect that if the sum of the payoffs for two players is bounded (i.e., value C is bounded), the values of different SPEs will be bounded too. Such a bound could be used in an ff-fi like method to prune the search tree. We expect that the smaller the bound on the payoff sums, the more pruning can be applied. If the bound is zero, the game is a zero-sum game. Bounds on the payoff sum have been applied to pruning in n-player games in the Max n procedure [10], but it is likely that in the case of two players, higher efficiency can be achieved. Selection between multiple SPEs during game-tree search is a new and challenging issue. Some SPEs require co-operation between the players, others are more competitive. It is an open question how willingness of an opponent to co-operate could be measured and applied in a search method. 7 Risk in opponent modelling Previous research indicated that Opponent-Model search gives rise to two sorts of risk [4, 5, 6]. The first sort of risk is caused by an incorrect model of the opponent. Such an incorrect model will possibly lead to an incorrect prediction of the opponent s strategy and cause suboptimal behaviour. This can cause results that are worse than the results reached by not using an opponent model. The second sort of risk is caused by an estimation error in the own payoff function. If a player estimates a position too advantageous, and the opponent has a correct estimation, OM search will be attracted to such a position, again leading to a suboptimal result. The larger the error, the stronger the attraction. The two sorts of risk are also expected to occur when using nonzero-sum games for opponent modelling. The first sort is obvious, since an erroneous opponent model will lead to wrong SPEs. The second sort is more delicate. When the common-interest part (value C in section 5) of the payoff functions is estimated too high for a certain position, the SPEs will be attracted to this position. Also in this case holds: the larger the error, the stronger the attraction. Estimation errors in the competitive part (value S in section 5) will not lead to this kind of attraction because they are filtered out by a Minimax-like mechanism. 6
7 8 Conclusion Introducing opponent models together with knowledge symmetry enhances game-playing by computers since common interest between players can be modelled. It requires, however, the development of new search techniques and the adaptation of existing search enhancements. Future research must indicate whether the advantage of the enhanced model is larger than the expected loss in search efficiency and the effect of the expected risks. References [1] D. Carmel and S. Markovitch. Learning models of opponent s strategies in game playing. In Proceedings AAAI Fall Symposion on Games: Planning and Learning, pages , Raleigh, NC, [2] D. Carmel and S. Markovitch. The M Λ algorithm: Incorporating opponent models in adversary search. Technical Report CIS9402, Technion, Haifa, Israel, [3] D. Carmel and S. Markovitch. Pruning algorithms for multi-model adversary search. Artificial Intelligence, 99(2): , [4] H.H.L.M. Donkers. Nosce Hostem Searching with opponent models. PhD thesis, Universiteit Maastricht, Maastricht, The Netherlands, [5] H.H.L.M. Donkers, J.W.H.M. Uiterwijk, and H.J. van den Herik. Admissibility in opponentmodel search. Information Sciences, 154(3-4): , [6] H. Iida, J.W.H.M. Uiterwijk, H.J. van den Herik, and I.S. Herschberg. Potential applications of opponent-model search. Part 1: the domain of applicability. ICCA Journal, 16(4): , [7] D.E. Knuth and R.W. Moore. An analysis of alpha-beta pruning. Artificial Intelligence, 6(4): , [8] H.W. Kuhn. Extensive games and the problem of information. In H.W. Kuhn and A.W. Tucker, editors, Contributions to Game Theory, Volume II, Annals of Mathematics Studies, 28, pages , Princeton, NJ, Princeton University Press. [9] A. Rubinstein. Comments on the interpretation of game theory. Econometrica, 59: , [10] N. Sturtevant and R.E. Korf. On pruning methods for multi-player games. In Proceedings AAAI-00, pages , Austin, TX, [11] J. von Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, Princeton, NJ, [12] E. Zermelo. Über eine Anwendung der Mengenlehre auf die Theorie des Schachspiels. In Proceedings of the Fifth Congress of Mathematicians (Cambridge 1912), pages Cambridge University Press,
CS 1571 Introduction to AI Lecture 12. Adversarial search. CS 1571 Intro to AI. Announcements
CS 171 Introduction to AI Lecture 1 Adversarial search Milos Hauskrecht milos@cs.pitt.edu 39 Sennott Square Announcements Homework assignment is out Programming and experiments Simulated annealing + Genetic
More informationCS 2710 Foundations of AI. Lecture 9. Adversarial search. CS 2710 Foundations of AI. Game search
CS 2710 Foundations of AI Lecture 9 Adversarial search Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square CS 2710 Foundations of AI Game search Game-playing programs developed by AI researchers since
More informationARTIFICIAL INTELLIGENCE (CS 370D)
Princess Nora University Faculty of Computer & Information Systems ARTIFICIAL INTELLIGENCE (CS 370D) (CHAPTER-5) ADVERSARIAL SEARCH ADVERSARIAL SEARCH Optimal decisions Min algorithm α-β pruning Imperfect,
More informationArtificial Intelligence. 4. Game Playing. Prof. Bojana Dalbelo Bašić Assoc. Prof. Jan Šnajder
Artificial Intelligence 4. Game Playing Prof. Bojana Dalbelo Bašić Assoc. Prof. Jan Šnajder University of Zagreb Faculty of Electrical Engineering and Computing Academic Year 2017/2018 Creative Commons
More informationGames (adversarial search problems)
Mustafa Jarrar: Lecture Notes on Games, Birzeit University, Palestine Fall Semester, 204 Artificial Intelligence Chapter 6 Games (adversarial search problems) Dr. Mustafa Jarrar Sina Institute, University
More informationCPS 570: Artificial Intelligence Two-player, zero-sum, perfect-information Games
CPS 57: Artificial Intelligence Two-player, zero-sum, perfect-information Games Instructor: Vincent Conitzer Game playing Rich tradition of creating game-playing programs in AI Many similarities to search
More information4. Games and search. Lecture Artificial Intelligence (4ov / 8op)
4. Games and search 4.1 Search problems State space search find a (shortest) path from the initial state to the goal state. Constraint satisfaction find a value assignment to a set of variables so that
More informationArtificial Intelligence. Minimax and alpha-beta pruning
Artificial Intelligence Minimax and alpha-beta pruning In which we examine the problems that arise when we try to plan ahead to get the best result in a world that includes a hostile agent (other agent
More informationCS 380: ARTIFICIAL INTELLIGENCE ADVERSARIAL SEARCH. Santiago Ontañón
CS 380: ARTIFICIAL INTELLIGENCE ADVERSARIAL SEARCH Santiago Ontañón so367@drexel.edu Recall: Problem Solving Idea: represent the problem we want to solve as: State space Actions Goal check Cost function
More informationAlgorithms for Data Structures: Search for Games. Phillip Smith 27/11/13
Algorithms for Data Structures: Search for Games Phillip Smith 27/11/13 Search for Games Following this lecture you should be able to: Understand the search process in games How an AI decides on the best
More informationArtificial Intelligence
Artificial Intelligence CS482, CS682, MW 1 2:15, SEM 201, MS 227 Prerequisites: 302, 365 Instructor: Sushil Louis, sushil@cse.unr.edu, http://www.cse.unr.edu/~sushil Games and game trees Multi-agent systems
More informationCS 4700: Foundations of Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence selman@cs.cornell.edu Module: Adversarial Search R&N: Chapter 5 1 Outline Adversarial Search Optimal decisions Minimax α-β pruning Case study: Deep Blue
More informationLast update: March 9, Game playing. CMSC 421, Chapter 6. CMSC 421, Chapter 6 1
Last update: March 9, 2010 Game playing CMSC 421, Chapter 6 CMSC 421, Chapter 6 1 Finite perfect-information zero-sum games Finite: finitely many agents, actions, states Perfect information: every agent
More informationCOMP219: COMP219: Artificial Intelligence Artificial Intelligence Dr. Annabel Latham Lecture 12: Game Playing Overview Games and Search
COMP19: Artificial Intelligence COMP19: Artificial Intelligence Dr. Annabel Latham Room.05 Ashton Building Department of Computer Science University of Liverpool Lecture 1: Game Playing 1 Overview Last
More informationFictitious Play applied on a simplified poker game
Fictitious Play applied on a simplified poker game Ioannis Papadopoulos June 26, 2015 Abstract This paper investigates the application of fictitious play on a simplified 2-player poker game with the goal
More informationModule 3. Problem Solving using Search- (Two agent) Version 2 CSE IIT, Kharagpur
Module 3 Problem Solving using Search- (Two agent) 3.1 Instructional Objective The students should understand the formulation of multi-agent search and in detail two-agent search. Students should b familiar
More informationAdversarial Search Aka Games
Adversarial Search Aka Games Chapter 5 Some material adopted from notes by Charles R. Dyer, U of Wisconsin-Madison Overview Game playing State of the art and resources Framework Game trees Minimax Alpha-beta
More informationGeneralized Game Trees
Generalized Game Trees Richard E. Korf Computer Science Department University of California, Los Angeles Los Angeles, Ca. 90024 Abstract We consider two generalizations of the standard two-player game
More informationComputer Science and Software Engineering University of Wisconsin - Platteville. 4. Game Play. CS 3030 Lecture Notes Yan Shi UW-Platteville
Computer Science and Software Engineering University of Wisconsin - Platteville 4. Game Play CS 3030 Lecture Notes Yan Shi UW-Platteville Read: Textbook Chapter 6 What kind of games? 2-player games Zero-sum
More informationADVERSARIAL SEARCH. Chapter 5
ADVERSARIAL SEARCH Chapter 5... every game of skill is susceptible of being played by an automaton. from Charles Babbage, The Life of a Philosopher, 1832. Outline Games Perfect play minimax decisions α
More informationGame playing. Chapter 6. Chapter 6 1
Game playing Chapter 6 Chapter 6 1 Outline Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Chapter 6 2 Games vs.
More informationGame playing. Outline
Game playing Chapter 6, Sections 1 8 CS 480 Outline Perfect play Resource limits α β pruning Games of chance Games of imperfect information Games vs. search problems Unpredictable opponent solution is
More informationGame Playing. Philipp Koehn. 29 September 2015
Game Playing Philipp Koehn 29 September 2015 Outline 1 Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information 2 games
More informationCOMP219: Artificial Intelligence. Lecture 13: Game Playing
CMP219: Artificial Intelligence Lecture 13: Game Playing 1 verview Last time Search with partial/no observations Belief states Incremental belief state search Determinism vs non-determinism Today We will
More informationAnnouncements. Homework 1 solutions posted. Test in 2 weeks (27 th ) -Covers up to and including HW2 (informed search)
Minimax (Ch. 5-5.3) Announcements Homework 1 solutions posted Test in 2 weeks (27 th ) -Covers up to and including HW2 (informed search) Single-agent So far we have look at how a single agent can search
More informationProgramming Project 1: Pacman (Due )
Programming Project 1: Pacman (Due 8.2.18) Registration to the exams 521495A: Artificial Intelligence Adversarial Search (Min-Max) Lectured by Abdenour Hadid Adjunct Professor, CMVS, University of Oulu
More informationAdversarial Search. CS 486/686: Introduction to Artificial Intelligence
Adversarial Search CS 486/686: Introduction to Artificial Intelligence 1 Introduction So far we have only been concerned with a single agent Today, we introduce an adversary! 2 Outline Games Minimax search
More informationCS188 Spring 2014 Section 3: Games
CS188 Spring 2014 Section 3: Games 1 Nearly Zero Sum Games The standard Minimax algorithm calculates worst-case values in a zero-sum two player game, i.e. a game in which for all terminal states s, the
More informationProb-Max n : Playing N-Player Games with Opponent Models
Prob-Max n : Playing N-Player Games with Opponent Models Nathan Sturtevant and Martin Zinkevich and Michael Bowling Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada T6G
More informationCS 380: ARTIFICIAL INTELLIGENCE
CS 380: ARTIFICIAL INTELLIGENCE ADVERSARIAL SEARCH 10/23/2013 Santiago Ontañón santi@cs.drexel.edu https://www.cs.drexel.edu/~santi/teaching/2013/cs380/intro.html Recall: Problem Solving Idea: represent
More informationCS510 \ Lecture Ariel Stolerman
CS510 \ Lecture04 2012-10-15 1 Ariel Stolerman Administration Assignment 2: just a programming assignment. Midterm: posted by next week (5), will cover: o Lectures o Readings A midterm review sheet will
More informationGame Playing: Adversarial Search. Chapter 5
Game Playing: Adversarial Search Chapter 5 Outline Games Perfect play minimax search α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Games vs. Search
More informationLecture 5: Game Playing (Adversarial Search)
Lecture 5: Game Playing (Adversarial Search) CS 580 (001) - Spring 2018 Amarda Shehu Department of Computer Science George Mason University, Fairfax, VA, USA February 21, 2018 Amarda Shehu (580) 1 1 Outline
More informationGame-Playing & Adversarial Search
Game-Playing & Adversarial Search This lecture topic: Game-Playing & Adversarial Search (two lectures) Chapter 5.1-5.5 Next lecture topic: Constraint Satisfaction Problems (two lectures) Chapter 6.1-6.4,
More informationV. Adamchik Data Structures. Game Trees. Lecture 1. Apr. 05, Plan: 1. Introduction. 2. Game of NIM. 3. Minimax
Game Trees Lecture 1 Apr. 05, 2005 Plan: 1. Introduction 2. Game of NIM 3. Minimax V. Adamchik 2 ü Introduction The search problems we have studied so far assume that the situation is not going to change.
More informationRobust Game Play Against Unknown Opponents
Robust Game Play Against Unknown Opponents Nathan Sturtevant Department of Computing Science University of Alberta Edmonton, Alberta, Canada T6G 2E8 nathanst@cs.ualberta.ca Michael Bowling Department of
More informationGames vs. search problems. Game playing Chapter 6. Outline. Game tree (2-player, deterministic, turns) Types of games. Minimax
Game playing Chapter 6 perfect information imperfect information Types of games deterministic chess, checkers, go, othello battleships, blind tictactoe chance backgammon monopoly bridge, poker, scrabble
More informationCS188 Spring 2010 Section 3: Game Trees
CS188 Spring 2010 Section 3: Game Trees 1 Warm-Up: Column-Row You have a 3x3 matrix of values like the one below. In a somewhat boring game, player A first selects a row, and then player B selects a column.
More informationGame playing. Chapter 6. Chapter 6 1
Game playing Chapter 6 Chapter 6 1 Outline Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Chapter 6 2 Games vs.
More informationPlayout Search for Monte-Carlo Tree Search in Multi-Player Games
Playout Search for Monte-Carlo Tree Search in Multi-Player Games J. (Pim) A.M. Nijssen and Mark H.M. Winands Games and AI Group, Department of Knowledge Engineering, Faculty of Humanities and Sciences,
More informationLeaf-Value Tables for Pruning Non-Zero-Sum Games
Leaf-Value Tables for Pruning Non-Zero-Sum Games Nathan Sturtevant University of Alberta Department of Computing Science Edmonton, AB Canada T6G 2E8 nathanst@cs.ualberta.ca Abstract Algorithms for pruning
More informationMULTI-PLAYER SEARCH IN THE GAME OF BILLABONG. Michael Gras. Master Thesis 12-04
MULTI-PLAYER SEARCH IN THE GAME OF BILLABONG Michael Gras Master Thesis 12-04 Thesis submitted in partial fulfilment of the requirements for the degree of Master of Science of Artificial Intelligence at
More informationMultiple Agents. Why can t we all just get along? (Rodney King)
Multiple Agents Why can t we all just get along? (Rodney King) Nash Equilibriums........................................ 25 Multiple Nash Equilibriums................................. 26 Prisoners Dilemma.......................................
More informationArtificial Intelligence
Artificial Intelligence CS482, CS682, MW 1 2:15, SEM 201, MS 227 Prerequisites: 302, 365 Instructor: Sushil Louis, sushil@cse.unr.edu, http://www.cse.unr.edu/~sushil Non-classical search - Path does not
More informationCS885 Reinforcement Learning Lecture 13c: June 13, Adversarial Search [RusNor] Sec
CS885 Reinforcement Learning Lecture 13c: June 13, 2018 Adversarial Search [RusNor] Sec. 5.1-5.4 CS885 Spring 2018 Pascal Poupart 1 Outline Minimax search Evaluation functions Alpha-beta pruning CS885
More informationFoundations of AI. 6. Adversarial Search. Search Strategies for Games, Games with Chance, State of the Art. Wolfram Burgard & Bernhard Nebel
Foundations of AI 6. Adversarial Search Search Strategies for Games, Games with Chance, State of the Art Wolfram Burgard & Bernhard Nebel Contents Game Theory Board Games Minimax Search Alpha-Beta Search
More informationAdversarial Search and Game Playing. Russell and Norvig: Chapter 5
Adversarial Search and Game Playing Russell and Norvig: Chapter 5 Typical case 2-person game Players alternate moves Zero-sum: one player s loss is the other s gain Perfect information: both players have
More informationAdversarial Search. CS 486/686: Introduction to Artificial Intelligence
Adversarial Search CS 486/686: Introduction to Artificial Intelligence 1 AccessAbility Services Volunteer Notetaker Required Interested? Complete an online application using your WATIAM: https://york.accessiblelearning.com/uwaterloo/
More informationOutline. Game playing. Types of games. Games vs. search problems. Minimax. Game tree (2-player, deterministic, turns) Games
utline Games Game playing Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Chapter 6 Games of chance Games of imperfect information Chapter 6 Chapter 6 Games vs. search
More informationToday. Types of Game. Games and Search 1/18/2010. COMP210: Artificial Intelligence. Lecture 10. Game playing
COMP10: Artificial Intelligence Lecture 10. Game playing Trevor Bench-Capon Room 15, Ashton Building Today We will look at how search can be applied to playing games Types of Games Perfect play minimax
More informationCritical Position Identification in Application to Speculative Play. Khalid, Mohd Nor Akmal; Yusof, Umi K Author(s) Hiroyuki; Ishitobi, Taichi
JAIST Reposi https://dspace.j Title Critical Position Identification in Application to Speculative Play Khalid, Mohd Nor Akmal; Yusof, Umi K Author(s) Hiroyuki; Ishitobi, Taichi Citation Proceedings of
More informationgame tree complete all possible moves
Game Trees Game Tree A game tree is a tree the nodes of which are positions in a game and edges are moves. The complete game tree for a game is the game tree starting at the initial position and containing
More informationGenerating and Solving Imperfect Information Games
Generating and Solving Imperfect Information Games Daphne Koller University of California Berkeley, CA 9472 daphne@cs.berkeley.edu Avi Pfeffer University of California Berkeley, CA 9472 ap@cs.berkeley.edu
More informationArtificial Intelligence Adversarial Search
Artificial Intelligence Adversarial Search Adversarial Search Adversarial search problems games They occur in multiagent competitive environments There is an opponent we can t control planning again us!
More informationGame-playing AIs: Games and Adversarial Search FINAL SET (w/ pruning study examples) AIMA
Game-playing AIs: Games and Adversarial Search FINAL SET (w/ pruning study examples) AIMA 5.1-5.2 Games: Outline of Unit Part I: Games as Search Motivation Game-playing AI successes Game Trees Evaluation
More informationGame Playing. Dr. Richard J. Povinelli. Page 1. rev 1.1, 9/14/2003
Game Playing Dr. Richard J. Povinelli rev 1.1, 9/14/2003 Page 1 Objectives You should be able to provide a definition of a game. be able to evaluate, compare, and implement the minmax and alpha-beta algorithms,
More informationmywbut.com Two agent games : alpha beta pruning
Two agent games : alpha beta pruning 1 3.5 Alpha-Beta Pruning ALPHA-BETA pruning is a method that reduces the number of nodes explored in Minimax strategy. It reduces the time required for the search and
More informationInstability of Scoring Heuristic In games with value exchange, the heuristics are very bumpy Make smoothing assumptions search for "quiesence"
More on games Gaming Complications Instability of Scoring Heuristic In games with value exchange, the heuristics are very bumpy Make smoothing assumptions search for "quiesence" The Horizon Effect No matter
More informationAdversarial Search 1
Adversarial Search 1 Adversarial Search The ghosts trying to make pacman loose Can not come up with a giant program that plans to the end, because of the ghosts and their actions Goal: Eat lots of dots
More informationPlaying Games. Henry Z. Lo. June 23, We consider writing AI to play games with the following properties:
Playing Games Henry Z. Lo June 23, 2014 1 Games We consider writing AI to play games with the following properties: Two players. Determinism: no chance is involved; game state based purely on decisions
More informationCS 771 Artificial Intelligence. Adversarial Search
CS 771 Artificial Intelligence Adversarial Search Typical assumptions Two agents whose actions alternate Utility values for each agent are the opposite of the other This creates the adversarial situation
More informationAdversary Search. Ref: Chapter 5
Adversary Search Ref: Chapter 5 1 Games & A.I. Easy to measure success Easy to represent states Small number of operators Comparison against humans is possible. Many games can be modeled very easily, although
More informationAdversarial Search Lecture 7
Lecture 7 How can we use search to plan ahead when other agents are planning against us? 1 Agenda Games: context, history Searching via Minimax Scaling α β pruning Depth-limiting Evaluation functions Handling
More informationGame-playing: DeepBlue and AlphaGo
Game-playing: DeepBlue and AlphaGo Brief history of gameplaying frontiers 1990s: Othello world champions refuse to play computers 1994: Chinook defeats Checkers world champion 1997: DeepBlue defeats world
More informationSolving Problems by Searching: Adversarial Search
Course 440 : Introduction To rtificial Intelligence Lecture 5 Solving Problems by Searching: dversarial Search bdeslam Boularias Friday, October 7, 2016 1 / 24 Outline We examine the problems that arise
More informationA Quoridor-playing Agent
A Quoridor-playing Agent P.J.C. Mertens June 21, 2006 Abstract This paper deals with the construction of a Quoridor-playing software agent. Because Quoridor is a rather new game, research about the game
More informationCS188 Spring 2010 Section 3: Game Trees
CS188 Spring 2010 Section 3: Game Trees 1 Warm-Up: Column-Row You have a 3x3 matrix of values like the one below. In a somewhat boring game, player A first selects a row, and then player B selects a column.
More informationTilburg University. Opponent modelling and commercial games van den Herik, Jaap; Donkers, H.H.L.M.; Spronck, Pieter
Tilburg University Opponent modelling and commercial games van den Herik, Jaap; Donkers, H.H.L.M.; Spronck, Pieter Published in: Proceedings of the IEEE 2005 symposium on computational intelligence and
More informationSet 4: Game-Playing. ICS 271 Fall 2017 Kalev Kask
Set 4: Game-Playing ICS 271 Fall 2017 Kalev Kask Overview Computer programs that play 2-player games game-playing as search with the complication of an opponent General principles of game-playing and search
More informationExperiments on Alternatives to Minimax
Experiments on Alternatives to Minimax Dana Nau University of Maryland Paul Purdom Indiana University April 23, 1993 Chun-Hung Tzeng Ball State University Abstract In the field of Artificial Intelligence,
More informationLast-Branch and Speculative Pruning Algorithms for Max"
Last-Branch and Speculative Pruning Algorithms for Max" Nathan Sturtevant UCLA, Computer Science Department Los Angeles, CA 90024 nathanst@cs.ucla.edu Abstract Previous work in pruning algorithms for max"
More informationUNIVERSITY of PENNSYLVANIA CIS 391/521: Fundamentals of AI Midterm 1, Spring 2010
UNIVERSITY of PENNSYLVANIA CIS 391/521: Fundamentals of AI Midterm 1, Spring 2010 Question Points 1 Environments /2 2 Python /18 3 Local and Heuristic Search /35 4 Adversarial Search /20 5 Constraint Satisfaction
More informationGame Playing State-of-the-Art CSE 473: Artificial Intelligence Fall Deterministic Games. Zero-Sum Games 10/13/17. Adversarial Search
CSE 473: Artificial Intelligence Fall 2017 Adversarial Search Mini, pruning, Expecti Dieter Fox Based on slides adapted Luke Zettlemoyer, Dan Klein, Pieter Abbeel, Dan Weld, Stuart Russell or Andrew Moore
More informationOutline. Game Playing. Game Problems. Game Problems. Types of games Playing a perfect game. Playing an imperfect game
Outline Game Playing ECE457 Applied Artificial Intelligence Fall 2007 Lecture #5 Types of games Playing a perfect game Minimax search Alpha-beta pruning Playing an imperfect game Real-time Imperfect information
More informationRetrograde Analysis of Woodpush
Retrograde Analysis of Woodpush Tristan Cazenave 1 and Richard J. Nowakowski 2 1 LAMSADE Université Paris-Dauphine Paris France cazenave@lamsade.dauphine.fr 2 Dept. of Mathematics and Statistics Dalhousie
More informationGame Tree Search. CSC384: Introduction to Artificial Intelligence. Generalizing Search Problem. General Games. What makes something a game?
CSC384: Introduction to Artificial Intelligence Generalizing Search Problem Game Tree Search Chapter 5.1, 5.2, 5.3, 5.6 cover some of the material we cover here. Section 5.6 has an interesting overview
More informationCS 440 / ECE 448 Introduction to Artificial Intelligence Spring 2010 Lecture #5
CS 440 / ECE 448 Introduction to Artificial Intelligence Spring 2010 Lecture #5 Instructor: Eyal Amir Grad TAs: Wen Pu, Yonatan Bisk Undergrad TAs: Sam Johnson, Nikhil Johri Topics Game playing Game trees
More informationCS440/ECE448 Lecture 9: Minimax Search. Slides by Svetlana Lazebnik 9/2016 Modified by Mark Hasegawa-Johnson 9/2017
CS440/ECE448 Lecture 9: Minimax Search Slides by Svetlana Lazebnik 9/2016 Modified by Mark Hasegawa-Johnson 9/2017 Why study games? Games are a traditional hallmark of intelligence Games are easy to formalize
More informationEvaluation-Function Based Proof-Number Search
Evaluation-Function Based Proof-Number Search Mark H.M. Winands and Maarten P.D. Schadd Games and AI Group, Department of Knowledge Engineering, Faculty of Humanities and Sciences, Maastricht University,
More information1. Introduction to Game Theory
1. Introduction to Game Theory What is game theory? Important branch of applied mathematics / economics Eight game theorists have won the Nobel prize, most notably John Nash (subject of Beautiful mind
More informationAr#ficial)Intelligence!!
Introduc*on! Ar#ficial)Intelligence!! Roman Barták Department of Theoretical Computer Science and Mathematical Logic So far we assumed a single-agent environment, but what if there are more agents and
More informationGame Engineering CS F-24 Board / Strategy Games
Game Engineering CS420-2014F-24 Board / Strategy Games David Galles Department of Computer Science University of San Francisco 24-0: Overview Example games (board splitting, chess, Othello) /Max trees
More informationAdversarial Search: Game Playing. Reading: Chapter
Adversarial Search: Game Playing Reading: Chapter 6.5-6.8 1 Games and AI Easy to represent, abstract, precise rules One of the first tasks undertaken by AI (since 1950) Better than humans in Othello and
More informationADVERSARIAL SEARCH 5.1 GAMES
5 DVERSRIL SERCH In which we examine the problems that arise when we try to plan ahead in a world where other agents are planning against us. 5.1 GMES GME ZERO-SUM GMES PERFECT INFORMTION Chapter 2 introduced
More informationMonte Carlo Tree Search and AlphaGo. Suraj Nair, Peter Kundzicz, Kevin An, Vansh Kumar
Monte Carlo Tree Search and AlphaGo Suraj Nair, Peter Kundzicz, Kevin An, Vansh Kumar Zero-Sum Games and AI A player s utility gain or loss is exactly balanced by the combined gain or loss of opponents:
More information2 person perfect information
Why Study Games? Games offer: Intellectual Engagement Abstraction Representability Performance Measure Not all games are suitable for AI research. We will restrict ourselves to 2 person perfect information
More information2. The Extensive Form of a Game
2. The Extensive Form of a Game In the extensive form, games are sequential, interactive processes which moves from one position to another in response to the wills of the players or the whims of chance.
More informationGame-playing AIs: Games and Adversarial Search I AIMA
Game-playing AIs: Games and Adversarial Search I AIMA 5.1-5.2 Games: Outline of Unit Part I: Games as Search Motivation Game-playing AI successes Game Trees Evaluation Functions Part II: Adversarial Search
More informationAdversarial search (game playing)
Adversarial search (game playing) References Russell and Norvig, Artificial Intelligence: A modern approach, 2nd ed. Prentice Hall, 2003 Nilsson, Artificial intelligence: A New synthesis. McGraw Hill,
More informationNOTE 6 6 LOA IS SOLVED
234 ICGA Journal December 2008 NOTE 6 6 LOA IS SOLVED Mark H.M. Winands 1 Maastricht, The Netherlands ABSTRACT Lines of Action (LOA) is a two-person zero-sum game with perfect information; it is a chess-like
More informationLecture 14. Questions? Friday, February 10 CS 430 Artificial Intelligence - Lecture 14 1
Lecture 14 Questions? Friday, February 10 CS 430 Artificial Intelligence - Lecture 14 1 Outline Chapter 5 - Adversarial Search Alpha-Beta Pruning Imperfect Real-Time Decisions Stochastic Games Friday,
More informationA Move Generating Algorithm for Hex Solvers
A Move Generating Algorithm for Hex Solvers Rune Rasmussen, Frederic Maire, and Ross Hayward Faculty of Information Technology, Queensland University of Technology, Gardens Point Campus, GPO Box 2434,
More informationThe extensive form representation of a game
The extensive form representation of a game Nodes, information sets Perfect and imperfect information Addition of random moves of nature (to model uncertainty not related with decisions of other players).
More informationAdversarial Search (Game Playing)
Artificial Intelligence Adversarial Search (Game Playing) Chapter 5 Adapted from materials by Tim Finin, Marie desjardins, and Charles R. Dyer Outline Game playing State of the art and resources Framework
More information5.4 Imperfect, Real-Time Decisions
5.4 Imperfect, Real-Time Decisions Searching through the whole (pruned) game tree is too inefficient for any realistic game Moves must be made in a reasonable amount of time One has to cut off the generation
More informationNOSCE HOSTEM SEARCHINGWITH OPPONENT MODELS HHLM DONKERS
NOSCE HOSTEM SEARCHINGWITH OPPONENT MODELS HHLM DONKERS Searching with Opponent Models PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Universiteit Maastricht, op gezag van de Rector Magnificus,
More informationAdversarial Search. Human-aware Robotics. 2018/01/25 Chapter 5 in R&N 3rd Ø Announcement: Slides for this lecture are here:
Adversarial Search 2018/01/25 Chapter 5 in R&N 3rd Ø Announcement: q Slides for this lecture are here: http://www.public.asu.edu/~yzhan442/teaching/cse471/lectures/adversarial.pdf Slides are largely based
More informationUMBC 671 Midterm Exam 19 October 2009
Name: 0 1 2 3 4 5 6 total 0 20 25 30 30 25 20 150 UMBC 671 Midterm Exam 19 October 2009 Write all of your answers on this exam, which is closed book and consists of six problems, summing to 160 points.
More informationImproving Best-Reply Search
Improving Best-Reply Search Markus Esser, Michael Gras, Mark H.M. Winands, Maarten P.D. Schadd and Marc Lanctot Games and AI Group, Department of Knowledge Engineering, Maastricht University, The Netherlands
More informationToday. Nondeterministic games: backgammon. Algorithm for nondeterministic games. Nondeterministic games in general. See Russell and Norvig, chapter 6
Today See Russell and Norvig, chapter Game playing Nondeterministic games Games with imperfect information Nondeterministic games: backgammon 5 8 9 5 9 8 5 Nondeterministic games in general In nondeterministic
More information