Intelligent Gaming Techniques for Poker: An Imperfect Information Game
|
|
- Daisy Evans
- 5 years ago
- Views:
Transcription
1 Intelligent Gaming Techniques for Poker: An Imperfect Information Game Samisa Abeysinghe and Ajantha S. Atukorale University of Colombo School of Computing, 35, Reid Avenue, Colombo 07, Sri Lanka Tel: [+94 77] , samisa_abeysinghe@yahoo.com Abstract This paper is a review paper on research done on imperfect information games. It is focused on the card game poker, on which much research has been done over the years. The paper reviews the solutions researchers have proposed for various problems of the game of poker. The main challenges in this imperfect information game are searching, learning, opponent modeling and deception. Machine learning, pattern recognition, adaptive evolutionary techniques, statistics and simulation are some of the technologies explored in this highly complex problem domain. This paper introduces the rationale for research on imperfect information games, describes the techniques used in solving the problems, review some successful programs built to solve such problems and finally discuss research opportunities in this area. Key words: Bayesian learning, Imperfect information, Intelligent gamming, Machine learning, Pattern recognition 1. Introduction Research on imperfect information has come to light in the recent past. With the advent of the Internet and the new forms of business opportunities made possible by the ICT technologies, there is elevated interest and activity in research on dealing with imperfect information. There are many situations that imperfect information could come in handy in the Internet era. Ecommerce applications such as online auctions, online gaming systems, and online business applications requiring negotiation /bargaining are some of the few applications that would benefit from the automated capability of dealing with imperfect information. While humans are so skilled in dealing with imperfect information, it will take some more time before the computers are made capable of dealing with imperfect information. This very reason has drawn many researchers to explore the possibilities in this field. Research on imperfect information often focuses on a particular case rather than taking up the challenge in general. The main reason for this is that dealing with imperfect information effectively is often problem specific. It does not make sense to build a generic model to deal with context specific situations for various problems. Poker has been one of the most researched problems to explore the techniques to deal with imperfect information. Poker is a card game that involves probability (some call it luck chance of getting a good card hand), imperfect information (hidden cards) and deception (bluffing). One s success depends not only on the cards that (s)he gets, but also on the ability to predict the opponents actions (often called opponent modeling)[3]. There have been numerous research efforts, many of them between 1998 and to date, on how to make computer programs play effective, world class poker. Various researchers have adopted different approaches to deal with the problem. 1.1 The Poker Game There are several variants of the Poker game. Various researchers have taken up different variants of the game for their research. Texas Hold'em and Five Card Stud are two variants of the Poker game that researchers often pick. In Texas Hold'em, multiple players can play the game and each player is dealt two hidden cards. Then five open cards are dealt face up, that are shared among the players. In Stud game each player gets his/her own cards, and there are no shared cards. Some of the cards are open and some are hidden. Billings et al, 2001 [3], provide
2 comprehensive details on the Texas Hold'em game. It also details the rules of the game (dealing cards and betting rounds). Korb et al, 1999 [12], provide details on the Poker Five Card Stud game and the hand evaluation rules. As the aforementioned papers provide ample details on these forms of the game, we do not wish to detail those in this paper. 2. Research The Department of Computer Science of University of Alberta, Canada, has a Poker research group. They have done much research in this area and have produced several valuable research papers on the subject. Their famous poker program Poki [1] uses learning techniques to build statistical models of each opponent and is capable of dynamically adopting itself to exploit observed patterns and techniques. They have used Texas Hold'em as their test bed. There has been some research at the School of Computer Science and Software Engineering, Monash University, Australia. Their poker program named Bayesian Poker Program (BPP) [12] uses a Bayesian network to model a poker player. BPP is built to play Five Card Stud game. Researchers from many other institutes have done considerable amount of research on Poker. However, none of the programs developed so far are yet capable of challenging best human players. The programs play effectively with humans, but there are still areas to be explored. 3. Techniques Billings et al, 2001 [3], in their paper titled The Challenge of Poker explains why, card games like poker are harder to solve than the games like chess. Traditional methods like deep search could be used to solve problems faced in games such as chess, because those are perfect information games. In other words, one could generate and evaluate the combinations of possibilities from one state to the next state of the game, with reasonable computing power and within acceptable time limits. However, in games like poker, where some or all of the opponents cards are hidden, it is not possible to do a deep search. There are several approaches to research on artificial intelligence with Poker. One is to study simplified variants. The other is to look at a subset of the game, and address sub problems in isolation [3]. Game theory and mathematical techniques have been used with simplified variants of Poker [11]. There have been some efforts to apply game theory for full scale poker [2]. Machine learning techniques have been used with a particular subset of Poker [16]. Many studies look into two player games. The simplifications, while reducing the complexity, may also destroy the most challenging aspects of the problem. However, given that the problem is too complex, it is often acceptable to focus on a simpler version of the game. As research becomes mature, solving the initial problems the real game would be taken up by researchers without simplifications. Poki program by Billings et al, 2001 [3], uses an approach that tackles the entire problem, choosing a real variant of poker and addressing all the considerations necessary. This minimizes the risk of loosing vital challenging aspects of the problem. Koller and Pfeffer, 1997 [11], have researched on the possibility of investigating Poker from a game-theoretic point of view. The algorithm they used in their system, named Gala, avoids the exponential blow up when converting the problem into normal form. However, the translated problem that they generate is still proportional to the size of the game tree. This makes it prohibitively large for most common variants of Poker. In his partition search, Ginsberg, 1997 [9], defined equivalence classes. The rationale was to use abstraction, collecting many instances of similar problems into a single class. He used this technique to solve the Bridge game. There are many states in Poker game that are isomorphic to each other, and hence the same concept could be used effectively. Another approach is to construct a shallower game tree. However unlike in perfect information games, the states of an imperfect information game tree are not independent. Often it is possible to take a move that looks good at shallow depth, but that could be revealed as bad at greater depth [13]. Alpha beta framework is effective for two player deterministic games. The search is done in full breadth but limited depth. For imperfect information games, full depth but limited breadth search is more appropriate, as it is not possible to examine full game tree. Even though this technique does not consider each and every possibility in the game tree, for non-deterministic games, this technique helps to reach a decision with sufficient confidence level. Repeated trials on the search tree have diminishing returns beyond a certain point [14]. Billings et al, 1998 [5], have defined five requirements for a world class Poker player. They are hand strength, hand potential, bluffing, unpredictability and opponent modeling. Unpredictability is to make sure that the other players could not use the imperfect information available to them to do successful predictions. modeling is the reverse of unpredictability where a player tries to utilize the available imperfect information to the maximum to predict the opponents successfully.
3 Korb et al, 1999 [12], have researched on the use of Bayesian network for solving the problems related to Poker. The rationale is that if the accurate probability of winning could be computed, then decisions could be taken on the next action to be taken with confidence. However, probability alone cannot ensure the success in the game, as it is dependent on opponent actions and their hand strength. 4. Poki The Poki program [3] has a dealer and multiple players, either human or computer. In the design of Poki, the betting strategy is divided into two parts, pre-flop strategy and post-flop strategy. Flop is the three shared cards that are dealt in the second round in Texas Hold'em. 4.1 Betting Strategy In pre-flop betting, there is little information as all the cards are hidden. A technique called roll-out simulation [3] could be used here. As the information available is less, a formula based expert system that uses information from roll-out simulations could be used in pre-flop stage. A future enhancement to pre-flop betting is to make the program autonomous, adopting to the observed game conditions and making context sensitive decisions on its own. A refinement to the roll-out simulation is to use repeated iterations of the technique, where the previous results govern the betting decision. After flop, there is more information available. In Poki, post-flop betting decision is taken using three steps. Compute effective hand strength (EHS) Translate EHS into action probability Pick an action randomly from probability distribution (unpredictability) In order to make the betting decisions based on the current game context and historical information, simulation based betting strategy is used. Here an estimate is established for each betting action. In order to solve the problem of not being able to carry out exhaustive game tree search, a selective sampling technique is used [6]. 4.2 modeling modeling is based on opponent's actions and is useful in two ways, first, deduce the strength of opponent's hand and second, predict opponent's actions in a given situation Generic opponent modeling (GOM) is a technique where one assumes the opponents would use a betting strategy similar to his/her own. Another technique is to assume that an opponent's actions are to be similar to the actions in the past. This is called specific opponent modeling (SOM) [15]. However when multiple players are involved in a game, blended with many complexities of many possible contexts, opponent modeling becomes one of the most difficult problems of machine learning [3]. Billings et al, 1998 [4], describes a statistical model for opponent modeling. They estimated some initial weights based on the behavior at the start of the game play and used a reweighing mechanism using EHS. Poki uses a standard feed forward neural network for opponent modeling. This has been trained on contextual data. The input layer takes in values corresponding to the properties of game context. The network consists of three outputs corresponding to the three actions fold, call and raise. Outputs of the network is cross validated against data collected from past games with each opponent [8]. Decision trees and expert systems could also be used for predicting opponents. However empirically it has proven that neural network based solution outperforms those techniques [7]. There have been some efforts to use evolutionary strategies for opponent modeling. Kendall and Willdig, 2001 [10], have looked into simple adaptation strategies to test out the use of adaptation with poker. They have used four playing styles: loose passive, loose aggressive, tight passive and tight aggressive, and tested the adoptive player against those. The evolution algorithm they have used is very simple; if player wins the learning value is increased and if player looses the learning value is decreased. Barone and While, 1999 [1], used a more advanced evolutionary strategy. Their poker player consists of a hypercube of populations of candidates (i.e. possible actions). The hypercube is segmented into two dimensions, one representing the position of play (early, middle or late) and the other representing risk management (possible actions taken by player representing conservative or aggressive play). Each element of the hypercube consists of a population of N candidates representing the functions corresponding to the likelihood of an action. Player's success on a given hand is used as feedback, based on which N/2 candidates are kept as parents of the next generation and rest discarded. Discarded ones are replaced with mutated copies of the selected parents.
4 5. BPP The Bayesian Poker Program (BPP) by Korb et al, 1999 [12], details a Bayesian Poker Network that is based on Bayesian network theory. This program is limited to two players. It incorporates the opponent's actions and known cards to model the opponent. There are some more assumptions in BPP that simplifies the problem to be solved: final hand types are independent (this makes sure that the network structure is a simple poly tree); opponent's action depends only on hand type (this reduces the number of nodes in the tree). The nodes representing hand types should consider not only the main hand categories, but also need to consider sub categories within main hand categories to be successful. For this BPP subdivides main hand categories into several sub categories. As an example, a hand with a pair can fall into one of 9 or lower, 10 to J and Q to A. Similarly other hand categories could be sub divided. BPP uses action matrices to represent rounds of betting. The matrices use conditional probabilities to represent possible actions per round given the opponent s current hand type. The matrices are adjusted over time using relative frequencies of opponent behavior. Belief updating is done by standard Bayesian network propagation rules. BPP uses a concept called betting curves to reach an action decision. These curves plot probability of wining versus probability of a given action. Each action has its own probability curve for each round of play. By moving a probability curve relative to the other curves, one could manage the nature of play, either aggressive or conservative [12]. Bluffing too is incorporated with a probability parameter. Once an action is selected using probability curve, BPP tries to over represent its action with some probability. This way if the suggested action by probability curves is to pass, the actual action taken would be to bet. 6. Discussion Searching, opponent modeling, deception strategies and machine learning are the challenging issues to be addressed in building programs that can play effective poker. There are numerous alternative solutions for searching problem. Full depth but limited breadth search has been used successfully. Despite the numerous researches already done, understanding the opponent is still challenging. Programs become slower to adjust as more information is collected on an opponent. This phenomenon is called build up of inertia [3]. It is the general understanding that opponent modeling is impossible to solve perfectly. Better players always change gears. A program playing poker must be deceptive. At the same time the program must be able to understand if and when its opponents are deceptive. This is difficult to deal with past data and pattern recognition. A historical decay function is proposed to deal with this situation [3]. Identifying useful features to focus on from gathered data is challenging. Learning in noisy domains is proposed [3]. Neural networks have been tried effectively for learning. It would be interesting to experiment with the combination of a neural network and an evolutionary strategy (genetic algorithm). The BPP by Korb et al, 1999 [12], can only deal with a two player game. The Bayesian network could be extended to cater for multiple players. Even for two players, there are numerous combinations of improvements that could be tried out. Especially improvements could be made to the technique used to decide on when to be aggressive or conservative. Rather than being purely random, this decision could be made based on learning outcomes. Poki program described earlier in the paper uses a neural network for learning. Combining Bayesian network with learning capabilities of a neural network is a possible research area. When using a Bayesian network, it will be useful to model the conservativeness or aggressiveness of the opponent. An opponent bluffing node could be used for this purpose [12]. BPP uses a distinct Bayesian network for each round of play. A dynamic Bayesian network could be used to model the interrelation on rounds of play. The two main programs that are the outcomes on research on Poker have used very different approaches. Poki tries to solve the real problem with minimal simplifications. BPP uses many simplifications. Thus it would be unfair to make comparisons between them. Poki has never tried out Bayesian techniques, however it has incorporated many useful techniques for opponent modeling, searching, bluffing and machine learning. It would be nice to combine those concepts with Bayesian approach and evaluate the outcomes. 7. Experiments Research was done using a Bayesian network with Texas Hold'em poker game (BPP [12] used Five Card Stud poker). Experiments were done using Bayesian Poker network by Korb et al, 1999 [12], as the basis. Few improvements were done to the network as discussed in the earlier section of this paper.
5 The improved Bayesian network used for experiments is shown in Figure 1. The probabilities required for the conditional probability tables of the Bayesian network were estimated by playing 1,000,000 games using two always-call players (players who play the game till the end of the game). The network was used in one of the players in a two player game, with the other player using only hand evaluation logic for selecting an action. 6,000 hands were played out and the results indicate a noticeable improvement over using mere hand evaluation logic. Program Win ability to adjust the nature of play to the current game conditions. Agregate Amount Won Logical Player Games Played Bayesian Player Final Hand Program Final Hand Figure 2: Results of play between Bayesian player and the logical player 8. Further Work Up Cards Current Hand Bluff Action Program Current Hand Figure 1: Proposed Bayesian network for poker Figure 2 shows the aggregate results for the games played. After 6,000 hands of play, Bayesian network based player had earned nearly 25,000 points more than its opponent. (Lower bet limit was 10 points. Upper bet limit was 20 points. Hence according to rules of play, the least amount a player could win was 20 points.) This clearly indicates the effectiveness of the Bayesian player. The success could be attributed to the ability to predict the opponent using the Bayesian network as well as the The network could be improved to relate opponent s bluffing to the current hand of the program. This could be done by having a link from the Program Current Hand node to the Bluff node. The program now initializes the network for each round. This means that for each round of play in a single game, it uses a new Bayesian network. Sensitiveness to the game context could be improved by using the same network and updating it through the rounds of play, based on the results of the previous rounds. However this would require the use of a dynamic Bayesian network. The experiment was done only with a logical player. There are plans to experiment with Bayesian networks with varying structures to identify their effectiveness. Especially it would be interesting to explore the memory usage and execution time of different networks and the tradeoffs between performance and resource utilization. The Bayesian network could be improved to learn the game context and update the conditional probability tables with experience. An open research area is to experiment how to use a neural network with the Bayesian network to improve learning capabilities. It is expected to explore the possibilities of extending this network for games with multiple players. References [1] Barone Luigi and While Lyndon, 1999, An Adaptive Learning Model for Simplified Poker Using Evolutionary Algorithms, Proceedings of the Congress on Evolutionary Computation
6 [2] Billings Darse, Burch Neil, Davidson Aaron, Holte Robert, Schaeffer Jonathan, Schauenberg Terence and Szafron Duane, 2003, Approximating Game-Theoretic Optimal Strategies for Full-scale Poker, Proceedings of IJCAI-03 (Eighteenth International Joint Conference on Artificial Intelligence) [3] Billings Darse, Davidson Aaron, Schaeffer Jonathan, and Szafron Duane, June 2001, The Challenge of Poker, Artificial Intelligence Journal, vol 134(1-2), pages [4] Billings Darse, Papp Denis, Schaeffer Jonathan, and Szafron Duane, 1998, Modeling in Poker, Proceedings of AAAI-98 (15th National AAAI Conference) [5] Billings Darse, Papp Denis, Schaeffer Jonathan, and Szafron Duane, 1998, Poker as a Testbed for Machine Intelligence Research, Proceedings of AI'98 (Canadian Society for Computational Studies in Intelligence) [6] Billings Darse, Peña Lourdes, Schaeffer Jonathan, and Szafron Duane, 1999, Using Probabilistic Knowledge and Simulation to Play Poker, Proceedings of AAAI-99 (Sixteenth National Conference of the American Association for Artificial Intelligence) [7] Davidson Aaron, 2002, Modeling in Poker: Learning and Acting in a Hostile Environment, M.Sc. thesis, Department of Computing Science, University of Alberta [8] Davidson Aaron, Billings Darse, Schaeffer Jonathan, and Szafron Duane, 2000, Improved Modeling in Poker, Proceedings of the 2000 International Conference on Artificial Intelligence (ICAI'2000), Las Vegas, Nevada, pages [9] Ginsberg Matthew L., 1997, Partition Search, AAAI National conference, pages [10] Kendall Graham and Willdig Mark, 2001, An Investigation of an Adaptive Poker Player, Australian Joint Conference on Artificial Intelligence [11] Koller Daphne and Pfeffer Avi, July 1997, Representations and Solutions for Game-Theoretic Problems, Artificial Intelligence, 94(1), pages [12] Korb B. Kevin, Nicholson E. Ann and Jitnah Nathalie, 1999 Bayesian Poker, Uncertainty in Artificial Intelligence, pages [13] Lincke Thomas, 1994, Perfect Play using Nine Men's Morris as an example [14] Peña Lourdes, 1999, Probabilities and Simulations in Poker, M.Sc. Thesis, School of Computer Science and Software Engineering, Monash University [15] Schaeffer Jonathan, Billings Darse, Peña Lourdes, and Szafron Duane, 1999, Learning to Play Strong Poker, ICML-99 Proceedings of the Sixteenth International Conference on Machine Learning [16] Smith Stephen F., August 1983, Flexible learning of problem solving heuristics through adaptive search, IJCAI- 83, pages
CASPER: a Case-Based Poker-Bot
CASPER: a Case-Based Poker-Bot Ian Watson and Jonathan Rubin Department of Computer Science University of Auckland, New Zealand ian@cs.auckland.ac.nz Abstract. This paper investigates the use of the case-based
More informationA Heuristic Based Approach for a Betting Strategy. in Texas Hold em Poker
DEPARTMENT OF COMPUTER SCIENCE SERIES OF PUBLICATIONS C REPORT C-2008-41 A Heuristic Based Approach for a Betting Strategy in Texas Hold em Poker Teemu Saukonoja and Tomi A. Pasanen UNIVERSITY OF HELSINKI
More informationOptimal Rhode Island Hold em Poker
Optimal Rhode Island Hold em Poker Andrew Gilpin and Tuomas Sandholm Computer Science Department Carnegie Mellon University Pittsburgh, PA 15213 {gilpin,sandholm}@cs.cmu.edu Abstract Rhode Island Hold
More informationPlayer Profiling in Texas Holdem
Player Profiling in Texas Holdem Karl S. Brandt CMPS 24, Spring 24 kbrandt@cs.ucsc.edu 1 Introduction Poker is a challenging game to play by computer. Unlike many games that have traditionally caught the
More informationUsing Selective-Sampling Simulations in Poker
Using Selective-Sampling Simulations in Poker Darse Billings, Denis Papp, Lourdes Peña, Jonathan Schaeffer, Duane Szafron Department of Computing Science University of Alberta Edmonton, Alberta Canada
More informationModels of Strategic Deficiency and Poker
Models of Strategic Deficiency and Poker Gabe Chaddock, Marc Pickett, Tom Armstrong, and Tim Oates University of Maryland, Baltimore County (UMBC) Computer Science and Electrical Engineering Department
More informationCan Opponent Models Aid Poker Player Evolution?
Can Opponent Models Aid Poker Player Evolution? R.J.S.Baker, Member, IEEE, P.I.Cowling, Member, IEEE, T.W.G.Randall, Member, IEEE, and P.Jiang, Member, IEEE, Abstract We investigate the impact of Bayesian
More informationAn Adaptive Learning Model for Simplified Poker Using Evolutionary Algorithms
An Adaptive Learning Model for Simplified Poker Using Evolutionary Algorithms Luigi Barone Department of Computer Science, The University of Western Australia, Western Australia, 697 luigi@cs.uwa.edu.au
More informationLearning to Play Strong Poker
Learning to Play Strong Poker Jonathan Schaeffer, Darse Billings, Lourdes Peña, Duane Szafron Department of Computing Science University of Alberta Edmonton, Alberta Canada T6G 2H1 {jonathan, darse, pena,
More informationStack Epoch
Adaptive Learning for Poker Luigi Barone and Lyndon While Department of Computer Science, The University of Western Australia, Western Australia, 697 fluigi, lyndong@cs.uwa.edu.au Abstract Evolutionary
More informationFrom: AAAI-99 Proceedings. Copyright 1999, AAAI (www.aaai.org). All rights reserved. Using Probabilistic Knowledge and Simulation to Play Poker
From: AAAI-99 Proceedings. Copyright 1999, AAAI (www.aaai.org). All rights reserved. Using Probabilistic Knowledge and Simulation to Play Poker Darse Billings, Lourdes Peña, Jonathan Schaeffer, Duane Szafron
More information4. Games and search. Lecture Artificial Intelligence (4ov / 8op)
4. Games and search 4.1 Search problems State space search find a (shortest) path from the initial state to the goal state. Constraint satisfaction find a value assignment to a set of variables so that
More informationThe first topic I would like to explore is probabilistic reasoning with Bayesian
Michael Terry 16.412J/6.834J 2/16/05 Problem Set 1 A. Topics of Fascination The first topic I would like to explore is probabilistic reasoning with Bayesian nets. I see that reasoning under situations
More informationUsing Fictitious Play to Find Pseudo-Optimal Solutions for Full-Scale Poker
Using Fictitious Play to Find Pseudo-Optimal Solutions for Full-Scale Poker William Dudziak Department of Computer Science, University of Akron Akron, Ohio 44325-4003 Abstract A pseudo-optimal solution
More informationAn Introduction to Poker Opponent Modeling
An Introduction to Poker Opponent Modeling Peter Chapman Brielin Brown University of Virginia 1 March 2011 It is not my aim to surprise or shock you-but the simplest way I can summarize is to say that
More informationBLUFF WITH AI. CS297 Report. Presented to. Dr. Chris Pollett. Department of Computer Science. San Jose State University. In Partial Fulfillment
BLUFF WITH AI CS297 Report Presented to Dr. Chris Pollett Department of Computer Science San Jose State University In Partial Fulfillment Of the Requirements for the Class CS 297 By Tina Philip May 2017
More informationEvolving Opponent Models for Texas Hold Em
Evolving Opponent Models for Texas Hold Em Alan J. Lockett and Risto Miikkulainen Abstract Opponent models allow software agents to assess a multi-agent environment more accurately and therefore improve
More informationPoker as a Testbed for Machine Intelligence Research
Poker as a Testbed for Machine Intelligence Research Darse Billings, Denis Papp, Jonathan Schaeffer, Duane Szafron {darse, dpapp, jonathan, duane}@cs.ualberta.ca Department of Computing Science University
More informationTexas Hold em Inference Bot Proposal. By: Brian Mihok & Michael Terry Date Due: Monday, April 11, 2005
Texas Hold em Inference Bot Proposal By: Brian Mihok & Michael Terry Date Due: Monday, April 11, 2005 1 Introduction One of the key goals in Artificial Intelligence is to create cognitive systems that
More informationImproving a Case-Based Texas Hold em Poker Bot
Improving a Case-Based Texas Hold em Poker Bot Ian Watson, Song Lee, Jonathan Rubin & Stefan Wender Abstract - This paper describes recent research that aims to improve upon our use of case-based reasoning
More informationOpponent Modeling in Poker
Opponent Modeling in Poker Darse Billings, Denis Papp, Jonathan Schaeffer, Duane Szafron Department of Computing Science University of Alberta Edmonton, Alberta Canada T6G 2H1 {darse, dpapp, jonathan,
More informationHeads-up Limit Texas Hold em Poker Agent
Heads-up Limit Texas Hold em Poker Agent Nattapoom Asavareongchai and Pin Pin Tea-mangkornpan CS221 Final Project Report Abstract Our project aims to create an agent that is able to play heads-up limit
More informationProbabilistic State Translation in Extensive Games with Large Action Sets
Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-09) Probabilistic State Translation in Extensive Games with Large Action Sets David Schnizlein Michael Bowling
More informationStrategy Evaluation in Extensive Games with Importance Sampling
Michael Bowling BOWLING@CS.UALBERTA.CA Michael Johanson JOHANSON@CS.UALBERTA.CA Neil Burch BURCH@CS.UALBERTA.CA Duane Szafron DUANE@CS.UALBERTA.CA Department of Computing Science, University of Alberta,
More informationLearning Strategies for Opponent Modeling in Poker
Computer Poker and Imperfect Information: Papers from the AAAI 2013 Workshop Learning Strategies for Opponent Modeling in Poker Ömer Ekmekci Department of Computer Engineering Middle East Technical University
More informationUnit-III Chap-II Adversarial Search. Created by: Ashish Shah 1
Unit-III Chap-II Adversarial Search Created by: Ashish Shah 1 Alpha beta Pruning In case of standard ALPHA BETA PRUNING minimax tree, it returns the same move as minimax would, but prunes away branches
More informationA Competitive Texas Hold em Poker Player Via Automated Abstraction and Real-time Equilibrium Computation
A Competitive Texas Hold em Poker Player Via Automated Abstraction and Real-time Equilibrium Computation Andrew Gilpin and Tuomas Sandholm Computer Science Department Carnegie Mellon University {gilpin,sandholm}@cs.cmu.edu
More informationarxiv: v1 [cs.ai] 20 Dec 2016
AIVAT: A New Variance Reduction Technique for Agent Evaluation in Imperfect Information Games Neil Burch, Martin Schmid, Matej Moravčík, Michael Bowling Department of Computing Science University of Alberta
More informationFictitious Play applied on a simplified poker game
Fictitious Play applied on a simplified poker game Ioannis Papadopoulos June 26, 2015 Abstract This paper investigates the application of fictitious play on a simplified 2-player poker game with the goal
More informationRegret Minimization in Games with Incomplete Information
Regret Minimization in Games with Incomplete Information Martin Zinkevich maz@cs.ualberta.ca Michael Bowling Computing Science Department University of Alberta Edmonton, AB Canada T6G2E8 bowling@cs.ualberta.ca
More informationPoker Opponent Modeling
Poker Opponent Modeling Michel Salim and Paul Rohwer Computer Science Department Indiana University Abstract Utilizing resources and research from the University of Alberta Poker research group, we are
More informationDerive Poker Winning Probability by Statistical JAVA Simulation
Proceedings of the 2 nd European Conference on Industrial Engineering and Operations Management (IEOM) Paris, France, July 26-27, 2018 Derive Poker Winning Probability by Statistical JAVA Simulation Mason
More informationCreating a Poker Playing Program Using Evolutionary Computation
Creating a Poker Playing Program Using Evolutionary Computation Simon Olsen and Rob LeGrand, Ph.D. Abstract Artificial intelligence is a rapidly expanding technology. We are surrounded by technology that
More informationVirtual Global Search: Application to 9x9 Go
Virtual Global Search: Application to 9x9 Go Tristan Cazenave LIASD Dept. Informatique Université Paris 8, 93526, Saint-Denis, France cazenave@ai.univ-paris8.fr Abstract. Monte-Carlo simulations can be
More informationDeepStack: Expert-Level AI in Heads-Up No-Limit Poker. Surya Prakash Chembrolu
DeepStack: Expert-Level AI in Heads-Up No-Limit Poker Surya Prakash Chembrolu AI and Games AlphaGo Go Watson Jeopardy! DeepBlue -Chess Chinook -Checkers TD-Gammon -Backgammon Perfect Information Games
More informationGame Playing. Philipp Koehn. 29 September 2015
Game Playing Philipp Koehn 29 September 2015 Outline 1 Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information 2 games
More informationStrategy Grafting in Extensive Games
Strategy Grafting in Extensive Games Kevin Waugh waugh@cs.cmu.edu Department of Computer Science Carnegie Mellon University Nolan Bard, Michael Bowling {nolan,bowling}@cs.ualberta.ca Department of Computing
More informationCS221 Final Project Report Learn to Play Texas hold em
CS221 Final Project Report Learn to Play Texas hold em Yixin Tang(yixint), Ruoyu Wang(rwang28), Chang Yue(changyue) 1 Introduction Texas hold em, one of the most popular poker games in casinos, is a variation
More informationPOKER AGENTS LD Miller & Adam Eck April 14 & 19, 2011
POKER AGENTS LD Miller & Adam Eck April 14 & 19, 2011 Motivation Classic environment properties of MAS Stochastic behavior (agents and environment) Incomplete information Uncertainty Application Examples
More informationComp 3211 Final Project - Poker AI
Comp 3211 Final Project - Poker AI Introduction Poker is a game played with a standard 52 card deck, usually with 4 to 8 players per game. During each hand of poker, players are dealt two cards and must
More informationAdversarial Search (Game Playing)
Artificial Intelligence Adversarial Search (Game Playing) Chapter 5 Adapted from materials by Tim Finin, Marie desjardins, and Charles R. Dyer Outline Game playing State of the art and resources Framework
More informationThe Evolution of Blackjack Strategies
The Evolution of Blackjack Strategies Graham Kendall University of Nottingham School of Computer Science & IT Jubilee Campus, Nottingham, NG8 BB, UK gxk@cs.nott.ac.uk Craig Smith University of Nottingham
More informationFoundations of AI. 6. Adversarial Search. Search Strategies for Games, Games with Chance, State of the Art. Wolfram Burgard & Bernhard Nebel
Foundations of AI 6. Adversarial Search Search Strategies for Games, Games with Chance, State of the Art Wolfram Burgard & Bernhard Nebel Contents Game Theory Board Games Minimax Search Alpha-Beta Search
More informationAchieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters
Achieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters Scott Watson, Andrew Vardy, Wolfgang Banzhaf Department of Computer Science Memorial University of Newfoundland St John s.
More informationAlgorithms for Data Structures: Search for Games. Phillip Smith 27/11/13
Algorithms for Data Structures: Search for Games Phillip Smith 27/11/13 Search for Games Following this lecture you should be able to: Understand the search process in games How an AI decides on the best
More informationAn evaluation of how Dynamic Programming and Game Theory are applied to Liar s Dice
An evaluation of how Dynamic Programming and Game Theory are applied to Liar s Dice Submitted in partial fulfilment of the requirements of the degree Bachelor of Science Honours in Computer Science at
More informationGame Playing: Adversarial Search. Chapter 5
Game Playing: Adversarial Search Chapter 5 Outline Games Perfect play minimax search α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Games vs. Search
More informationUsing Sliding Windows to Generate Action Abstractions in Extensive-Form Games
Using Sliding Windows to Generate Action Abstractions in Extensive-Form Games John Hawkin and Robert C. Holte and Duane Szafron {hawkin, holte}@cs.ualberta.ca, dszafron@ualberta.ca Department of Computing
More informationSpeeding-Up Poker Game Abstraction Computation: Average Rank Strength
Computer Poker and Imperfect Information: Papers from the AAAI 2013 Workshop Speeding-Up Poker Game Abstraction Computation: Average Rank Strength Luís Filipe Teófilo, Luís Paulo Reis, Henrique Lopes Cardoso
More informationCS 440 / ECE 448 Introduction to Artificial Intelligence Spring 2010 Lecture #5
CS 440 / ECE 448 Introduction to Artificial Intelligence Spring 2010 Lecture #5 Instructor: Eyal Amir Grad TAs: Wen Pu, Yonatan Bisk Undergrad TAs: Sam Johnson, Nikhil Johri Topics Game playing Game trees
More informationUnderstanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search
Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search Jeffrey Long and Nathan R. Sturtevant and Michael Buro and Timothy Furtak Department of Computing Science, University
More informationBetaPoker: Reinforcement Learning for Heads-Up Limit Poker Albert Tung, Eric Xu, and Jeffrey Zhang
Introduction BetaPoker: Reinforcement Learning for Heads-Up Limit Poker Albert Tung, Eric Xu, and Jeffrey Zhang Texas Hold em Poker is considered the most popular variation of poker that is played widely
More informationApplying Equivalence Class Methods in Contract Bridge
Applying Equivalence Class Methods in Contract Bridge Sean Sutherland Department of Computer Science The University of British Columbia Abstract One of the challenges in analyzing the strategies in contract
More informationCreating a New Angry Birds Competition Track
Proceedings of the Twenty-Ninth International Florida Artificial Intelligence Research Society Conference Creating a New Angry Birds Competition Track Rohan Verma, Xiaoyu Ge, Jochen Renz Research School
More informationCS 1571 Introduction to AI Lecture 12. Adversarial search. CS 1571 Intro to AI. Announcements
CS 171 Introduction to AI Lecture 1 Adversarial search Milos Hauskrecht milos@cs.pitt.edu 39 Sennott Square Announcements Homework assignment is out Programming and experiments Simulated annealing + Genetic
More informationMyPawns OppPawns MyKings OppKings MyThreatened OppThreatened MyWins OppWins Draws
The Role of Opponent Skill Level in Automated Game Learning Ying Ge and Michael Hash Advisor: Dr. Mark Burge Armstrong Atlantic State University Savannah, Geogia USA 31419-1997 geying@drake.armstrong.edu
More informationThe Evolution of Knowledge and Search in Game-Playing Systems
The Evolution of Knowledge and Search in Game-Playing Systems Jonathan Schaeffer Abstract. The field of artificial intelligence (AI) is all about creating systems that exhibit intelligent behavior. Computer
More informationTowards Strategic Kriegspiel Play with Opponent Modeling
Towards Strategic Kriegspiel Play with Opponent Modeling Antonio Del Giudice and Piotr Gmytrasiewicz Department of Computer Science, University of Illinois at Chicago Chicago, IL, 60607-7053, USA E-mail:
More informationV. Adamchik Data Structures. Game Trees. Lecture 1. Apr. 05, Plan: 1. Introduction. 2. Game of NIM. 3. Minimax
Game Trees Lecture 1 Apr. 05, 2005 Plan: 1. Introduction 2. Game of NIM 3. Minimax V. Adamchik 2 ü Introduction The search problems we have studied so far assume that the situation is not going to change.
More informationAdversarial Search and Game Playing. Russell and Norvig: Chapter 5
Adversarial Search and Game Playing Russell and Norvig: Chapter 5 Typical case 2-person game Players alternate moves Zero-sum: one player s loss is the other s gain Perfect information: both players have
More informationGame Mechanics Minesweeper is a game in which the player must correctly deduce the positions of
Table of Contents Game Mechanics...2 Game Play...3 Game Strategy...4 Truth...4 Contrapositive... 5 Exhaustion...6 Burnout...8 Game Difficulty... 10 Experiment One... 12 Experiment Two...14 Experiment Three...16
More informationOpponent Modeling in Texas Holdem with Cognitive Constraints
Carnegie Mellon University Research Showcase @ CMU Dietrich College Honors Theses Dietrich College of Humanities and Social Sciences 4-23-2009 Opponent Modeling in Texas Holdem with Cognitive Constraints
More informationFive-In-Row with Local Evaluation and Beam Search
Five-In-Row with Local Evaluation and Beam Search Jiun-Hung Chen and Adrienne X. Wang jhchen@cs axwang@cs Abstract This report provides a brief overview of the game of five-in-row, also known as Go-Moku,
More informationAutomatic Public State Space Abstraction in Imperfect Information Games
Computer Poker and Imperfect Information: Papers from the 2015 AAAI Workshop Automatic Public State Space Abstraction in Imperfect Information Games Martin Schmid, Matej Moravcik, Milan Hladik Charles
More informationComputer Poker Research at LIACC
Computer Poker Research at LIACC Luís Filipe Teófilo, Luís Paulo Reis, Henrique Lopes Cardoso, Dinis Félix, Rui Sêca, João Ferreira, Pedro Mendes, Nuno Cruz, Vitor Pereira, Nuno Passos LIACC Artificial
More informationA Reinforcement Learning Algorithm Applied to Simplified Two-Player Texas Hold em Poker
A Reinforcement Learning Algorithm Applied to Simplified Two-Player Texas Hold em Poker Fredrik A. Dahl Norwegian Defence Research Establishment (FFI) P.O. Box 25, NO-2027 Kjeller, Norway Fredrik-A.Dahl@ffi.no
More informationCreating a Dominion AI Using Genetic Algorithms
Creating a Dominion AI Using Genetic Algorithms Abstract Mok Ming Foong Dominion is a deck-building card game. It allows for complex strategies, has an aspect of randomness in card drawing, and no obvious
More informationSelecting Robust Strategies Based on Abstracted Game Models
Chapter 1 Selecting Robust Strategies Based on Abstracted Game Models Oscar Veliz and Christopher Kiekintveld Abstract Game theory is a tool for modeling multi-agent decision problems and has been used
More informationATHABASCA UNIVERSITY CAN TEST DRIVEN DEVELOPMENT IMPROVE POKER ROBOT PERFORMANCE? EDWARD SAN PEDRO. An essay submitted in partial fulfillment
ATHABASCA UNIVERSITY CAN TEST DRIVEN DEVELOPMENT IMPROVE POKER ROBOT PERFORMANCE? BY EDWARD SAN PEDRO An essay submitted in partial fulfillment Of the requirements for the degree of MASTER OF SCIENCE in
More informationarxiv: v1 [cs.gt] 23 May 2018
On self-play computation of equilibrium in poker Mikhail Goykhman Racah Institute of Physics, Hebrew University of Jerusalem, Jerusalem, 91904, Israel E-mail: michael.goykhman@mail.huji.ac.il arxiv:1805.09282v1
More informationLearning a Value Analysis Tool For Agent Evaluation
Learning a Value Analysis Tool For Agent Evaluation Martha White Michael Bowling Department of Computer Science University of Alberta International Joint Conference on Artificial Intelligence, 2009 Motivation:
More informationReflections on the First Man vs. Machine No-Limit Texas Hold 'em Competition
Reflections on the First Man vs. Machine No-Limit Texas Hold 'em Competition Sam Ganzfried Assistant Professor, Computer Science, Florida International University, Miami FL PhD, Computer Science Department,
More informationGame theory and AI: a unified approach to poker games
Game theory and AI: a unified approach to poker games Thesis for graduation as Master of Artificial Intelligence University of Amsterdam Frans Oliehoek 2 September 2005 Abstract This thesis focuses on
More informationAnalysis For Hold'em 3 Bonus April 9, 2014
Analysis For Hold'em 3 Bonus April 9, 2014 Prepared For John Feola New Vision Gaming 5 Samuel Phelps Way North Reading, MA 01864 Office: 978 664-1515 Fax: 978-664 - 5117 www.newvisiongaming.com Prepared
More informationCSC 396 : Introduction to Artificial Intelligence
CSC 396 : Introduction to Artificial Intelligence Exam 1 March 11th - 13th, 2008 Name Signature - Honor Code This is a take-home exam. You may use your book and lecture notes from class. You many not use
More informationCS 380: ARTIFICIAL INTELLIGENCE ADVERSARIAL SEARCH. Santiago Ontañón
CS 380: ARTIFICIAL INTELLIGENCE ADVERSARIAL SEARCH Santiago Ontañón so367@drexel.edu Recall: Problem Solving Idea: represent the problem we want to solve as: State space Actions Goal check Cost function
More informationProgramming Project 1: Pacman (Due )
Programming Project 1: Pacman (Due 8.2.18) Registration to the exams 521495A: Artificial Intelligence Adversarial Search (Min-Max) Lectured by Abdenour Hadid Adjunct Professor, CMVS, University of Oulu
More informationTraining a Back-Propagation Network with Temporal Difference Learning and a database for the board game Pente
Training a Back-Propagation Network with Temporal Difference Learning and a database for the board game Pente Valentijn Muijrers 3275183 Valentijn.Muijrers@phil.uu.nl Supervisor: Gerard Vreeswijk 7,5 ECTS
More informationARTIFICIAL INTELLIGENCE (CS 370D)
Princess Nora University Faculty of Computer & Information Systems ARTIFICIAL INTELLIGENCE (CS 370D) (CHAPTER-5) ADVERSARIAL SEARCH ADVERSARIAL SEARCH Optimal decisions Min algorithm α-β pruning Imperfect,
More informationA Brief Introduction to Game Theory
A Brief Introduction to Game Theory Jesse Crawford Department of Mathematics Tarleton State University April 27, 2011 (Tarleton State University) Brief Intro to Game Theory April 27, 2011 1 / 35 Outline
More informationCS 229 Final Project: Using Reinforcement Learning to Play Othello
CS 229 Final Project: Using Reinforcement Learning to Play Othello Kevin Fry Frank Zheng Xianming Li ID: kfry ID: fzheng ID: xmli 16 December 2016 Abstract We built an AI that learned to play Othello.
More informationCS 2710 Foundations of AI. Lecture 9. Adversarial search. CS 2710 Foundations of AI. Game search
CS 2710 Foundations of AI Lecture 9 Adversarial search Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square CS 2710 Foundations of AI Game search Game-playing programs developed by AI researchers since
More informationSet 4: Game-Playing. ICS 271 Fall 2017 Kalev Kask
Set 4: Game-Playing ICS 271 Fall 2017 Kalev Kask Overview Computer programs that play 2-player games game-playing as search with the complication of an opponent General principles of game-playing and search
More informationLecture 14. Questions? Friday, February 10 CS 430 Artificial Intelligence - Lecture 14 1
Lecture 14 Questions? Friday, February 10 CS 430 Artificial Intelligence - Lecture 14 1 Outline Chapter 5 - Adversarial Search Alpha-Beta Pruning Imperfect Real-Time Decisions Stochastic Games Friday,
More informationLearning to Play like an Othello Master CS 229 Project Report. Shir Aharon, Amanda Chang, Kent Koyanagi
Learning to Play like an Othello Master CS 229 Project Report December 13, 213 1 Abstract This project aims to train a machine to strategically play the game of Othello using machine learning. Prior to
More informationAr#ficial)Intelligence!!
Introduc*on! Ar#ficial)Intelligence!! Roman Barták Department of Theoretical Computer Science and Mathematical Logic So far we assumed a single-agent environment, but what if there are more agents and
More informationOpponent Modeling in Texas Hold em
Opponent Modeling in Texas Hold em Nadia Boudewijn, student number 3700607, Bachelor thesis Artificial Intelligence 7.5 ECTS, Utrecht University, January 2014, supervisor: dr. G. A. W. Vreeswijk ABSTRACT
More informationA PROGRAM FOR PLAYING TAROK
190 ICGA Journal September 2003 A PROGRAM FOR PLAYING TAROK Mitja Luštrek 1, Matjaž Gams 1 and Ivan Bratko 1 Ljubljana, Slovenia ABSTRACT A program for playing the three-player tarok card game is presented
More informationArtificial Intelligence Search III
Artificial Intelligence Search III Lecture 5 Content: Search III Quick Review on Lecture 4 Why Study Games? Game Playing as Search Special Characteristics of Game Playing Search Ingredients of 2-Person
More informationSimple Poker Game Design, Simulation, and Probability
Simple Poker Game Design, Simulation, and Probability Nanxiang Wang Foothill High School Pleasanton, CA 94588 nanxiang.wang309@gmail.com Mason Chen Stanford Online High School Stanford, CA, 94301, USA
More informationArtificial Intelligence. Minimax and alpha-beta pruning
Artificial Intelligence Minimax and alpha-beta pruning In which we examine the problems that arise when we try to plan ahead to get the best result in a world that includes a hostile agent (other agent
More informationOutline. Game Playing. Game Problems. Game Problems. Types of games Playing a perfect game. Playing an imperfect game
Outline Game Playing ECE457 Applied Artificial Intelligence Fall 2007 Lecture #5 Types of games Playing a perfect game Minimax search Alpha-beta pruning Playing an imperfect game Real-time Imperfect information
More informationComparison of Monte Carlo Tree Search Methods in the Imperfect Information Card Game Cribbage
Comparison of Monte Carlo Tree Search Methods in the Imperfect Information Card Game Cribbage Richard Kelly and David Churchill Computer Science Faculty of Science Memorial University {richard.kelly, dchurchill}@mun.ca
More informationGame playing. Chapter 6. Chapter 6 1
Game playing Chapter 6 Chapter 6 1 Outline Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Chapter 6 2 Games vs.
More informationBayesChess: A computer chess program based on Bayesian networks
BayesChess: A computer chess program based on Bayesian networks Antonio Fernández and Antonio Salmerón Department of Statistics and Applied Mathematics University of Almería Abstract In this paper we introduce
More informationCS 380: ARTIFICIAL INTELLIGENCE MONTE CARLO SEARCH. Santiago Ontañón
CS 380: ARTIFICIAL INTELLIGENCE MONTE CARLO SEARCH Santiago Ontañón so367@drexel.edu Recall: Adversarial Search Idea: When there is only one agent in the world, we can solve problems using DFS, BFS, ID,
More informationEvolution of Counter-Strategies: Application of Co-evolution to Texas Hold em Poker
Evolution of Counter-Strategies: Application of Co-evolution to Texas Hold em Poker Thomas Thompson, John Levine and Russell Wotherspoon Abstract Texas Hold em Poker is similar to other poker variants
More informationGame Playing AI Class 8 Ch , 5.4.1, 5.5
Game Playing AI Class Ch. 5.-5., 5.4., 5.5 Bookkeeping HW Due 0/, :59pm Remaining CSP questions? Cynthia Matuszek CMSC 6 Based on slides by Marie desjardin, Francisco Iacobelli Today s Class Clear criteria
More informationOptimal Unbiased Estimators for Evaluating Agent Performance
Optimal Unbiased Estimators for Evaluating Agent Performance Martin Zinkevich and Michael Bowling and Nolan Bard and Morgan Kan and Darse Billings Department of Computing Science University of Alberta
More informationMonte Carlo Tree Search. Simon M. Lucas
Monte Carlo Tree Search Simon M. Lucas Outline MCTS: The Excitement! A tutorial: how it works Important heuristics: RAVE / AMAF Applications to video games and real-time control The Excitement Game playing
More information