Project 2: Searching and Learning in Pac-Man

Size: px
Start display at page:

Download "Project 2: Searching and Learning in Pac-Man"

Transcription

1 Project 2: Searching and Learning in Pac-Man December 3, Quick Facts In this project you have to code A* and Q-learning in the game of Pac-Man and answer some questions about your implementation. You will have to work in groups of 2 people. You have to turn in source code and documentation of your project. The documentation you turn in should answer the questions asked in each of the different parts of the project. The project is designed to be coded in Java, but you can use other languages 1. For any questions, me at: santi@iiia.csic.es 2 Pac-Man Pac-Man is a Japanese arcade game developed by Namco (now Namco Bandai) and licensed for distribution in the U.S. by Midway, first released in Japan on May 22, Immensely popular in the United States from its original release to the present day, Pac-Man is universally considered as one of the classics of the medium, virtually synonymous with video games, and an icon of 1980s popular culture. Figure 1 shows a screenshot of the original Pac-Man. In the game, the player controls Pac-Man through a maze, eating dots. When all dots are eaten, Pac- Man is taken to the next stage. Four ghosts (known to most gamers as Blinky, Pinky, Inky and Clyde) roam the maze, trying to catch Pac-Man. If a ghost touches Pac-Man, a life is lost. When all lives have been lost, the game ends. Near the corners of the maze are four larger, flashing dots known as energizers or power pellets that provide Pac-Man with the temporary ability to eat the ghosts. The ghosts turn deep blue, reverse direction, and usually move more 1 if you are willing to code your own implementation of Pac-Man (or look for an alternative one) you can use any other programming language. Another option is to write code to connect the Java version of Pac-Man we provide with your favorite language. 1

2 Figure 1: A screenshot of the original PACMAN. slowly when Pac-Man eats an energizer. When a ghost is eaten, its eyes return to the ghost pen where it is regenerated in its normal color. Blue ghosts flash white before they become dangerous again and the amount of time the ghosts remain vulnerable varies from one board to the next, but the time period generally becomes shorter as the game progresses. In later stages, the ghosts do not change colors at all, but still reverse direction when an energizer is eaten. In addition to dots and energizers, bonus items, usually referred to as fruits (though not all items are fruits) appear near the center of the maze twice per level. These items score extra bonus points when eaten. The items change and bonus values increase throughout the game. The AI of the original Pac-Man was limited to the movement of the ghosts, and was indeed very well thought of. However, it used very basic techniques for two main reasons. First of all, the hardware in which it ran wouldn t support any computational intensive algorithm, and second, if the AI of the ghosts is improved, Pac-Man would become impossible to win. In this project you will actually implement more advanced AI techniques for Pac-Man, but from the reverse point of view: you will code AI for Pac-Man himself instead of for the ghosts. In this project you will have to implement and experiment with a collection of search and learning algorithms. The project is divided up into two major parts. A first one where you will experiment with search algorithms, and a second where you will do so with reinforcement learning algorithms. 2

3 3 Preliminaries: Setting Up your Environment The file PacManSrc.zip provided to you contains the source code of a Java open source implementation of Pac-Man designed specifically to test AI algorithms. Using your favorite Java development environment (Eclipse or Netbeans are recommended) create a project containing the source code we provided you. The main class is pacman.game, which you can run to run the game. Make sure you can run the game before proceeding with the project. By default, a very simple Pac-Man AI is coded into the game. You can experiment with the values of the static variables on the top of the pacman.game file to change the speed of the game (movetime), the number of ghosts (defaultnumberghosts), and other parameters that will come in handy for testing your algorithms later in the project. 3.1 Creating a Pac-Man AI Before starting with the main parts of this projects, let us see how to create a simple Pac-Man AI. So simple, that will just ask Pac-Man to move in a random direction. 1. Create a new class in the package player (call it as you prefer, I will call it RandomPacManPlayer) and make it implement the PacManPlayer interface. 2. Define the method public Move choosemove(game game) to look like this: p u b l i c Move choosemove (Game game ) { Random r = new Random ( ) ; Move [ ] moves = {Move.LEFT, Move.RIGHT, Move.UP, Move.DOWN} ; r eturn moves [ r. nextint ( 4 ) ] ; } 3. Now, go to the main method in the pacman.game class. And change the line: PacManPlayer pacman = new SimplePacManPlayer ( ) ; by PacManPlayer pacman = new RandomPacManPlayer ( ) ; 4. Run the game, your random Pac-Man should be controlling the game right now. 3

4 4 Part A: Pac-Man Searches Shortest Paths In this first part, we want you to implement an A* algorithm which will be used by Pac-Man to find the optimal path in which the dots can be eaten. Moreover, in order to do so, we will initially remove the ghosts from the game. Implement a first version of the PacManPlayer, which at each step uses A* to find the shortest path to a dot in the map, and starts moving in that direction. Design an appropriate heuristic, and make sure it is admissible. Implement a second version, which at each step uses A* to find the shortest path to eat all the dots in the map (notice that the search space is much, much larger this time). Design an appropriate heuristic, and make sure it is admissible. For each PacManPlayer, evaluate the number of nodes it explores during the search (min, max, and average). As well as comparing it to when no heuristic is used (just run your PamManPlayer with a heuristic which always returns 0). Notice that the second implementation might take too long to be usable in a real game. Optional improvements: why did we decide not to use ghosts? What happens when we add ghosts to the game? Can you describe or even implement a strategy which would still use A*, and that will take into account that there are ghosts in the game? 5 Part B: Pac-Man Learns to Play the Complete Game In this second part, we want you to implement a Q-learning algorithm, to help Pac-Man learn how to play the complete game. This time, we will add ghosts to the game again. In order to implement Q-learning, the very first thing you need is to decide on a suitable state space. Notice that if you try to have one state for each possible configuration of the game state you would have an enormous number of states and Q-learning would never converge (assuming you can hold the state table in memory). A possible way to reduce it can be to consider just a window around Pac-Man. For example, let s say that we only consider the cells that are immediately north, south, east or west of Pac-Man. And that we assume each cell can be either: empty, a wall, a dot or a ghost (there might be a ghost over a dot, but let s ignore that for now). With that representation we would only have 4 4 = 256 states. This is better, but might be to little and not lead to any interesting strategy. Can you come up with a state representation which captures the useful information in the game state for playing a good game of Pac-Man, but that generates a small number of states? (try to stay in the thousands range). Try to define a representation, where each state is identified by a unique integer. 4

5 Create a new PacManPlayer, which will use reinforcement learning to learn how to play the game. Once you have decided a suitable state representation, you need to define a function which given an instance of the State class, returns the integer representing the state it corresponds to. Decide a proper reinforcement signal. For instance +1 for eating a dot, -1 for being killed by a ghost (design your own, we are sure you can do better than this). Implement the learning operations of reinforcement learning over your state representation using Q-learning. Notice that initially, Pac-Man will play really bad (since it has not learned). Modify the game so that it lets your new PacManPlayer play multiple games in a row. Now, make sure that the Q-table learned from one game is transferred to the next game (so that Pac-Man learns more and more with every game it plays). Make Pac-Man play a sequence of games, and record how many points does he manage to score in each of them before killed. Notice that Pac-Man might need hundreds (or even thousands) or games to learn (depending on how good is the state representation you selected). Optional improvements: do you think the ghosts are smart? why don t you apply Q-learning also to the control of the ghosts? What happens when both Pac-Man and the ghosts use reinforcement learning at the same time? do they converge to a stable behavior? does Pac-Man manage to complete the game, or do the ghosts manage to always eat Pac-Man? 6 Part C: Final Questions Compare the A* approach with the reinforcement learning approach. Which are the benefits and drawbacks of each? Pac-Man is an apparently simple domain, do you see these techniques as applicable to larger domains (either computer games or real life applications)? 7 Bibliography Concerning the A* and Q-learning algorithms, you can use Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig. It is the reference book in artificial intelligence. Concerning Java, I d recommend Effective Java by Joshua Bloch, but most Java books will do. Additionally, feel free to use any language you use (although you ll have to code your own Pac-Man game!). 5

ADVANCED TOOLS AND TECHNIQUES: PAC-MAN GAME

ADVANCED TOOLS AND TECHNIQUES: PAC-MAN GAME ADVANCED TOOLS AND TECHNIQUES: PAC-MAN GAME For your next assignment you are going to create Pac-Man, the classic arcade game. The game play should be similar to the original game whereby the player controls

More information

Design task: Pacman. Software engineering Szoftvertechnológia. Dr. Balázs Simon BME, IIT

Design task: Pacman. Software engineering Szoftvertechnológia. Dr. Balázs Simon BME, IIT Design task: Pacman Software engineering Szoftvertechnológia Dr. Balázs Simon BME, IIT Outline CRC cards Requirements for Pacman CRC cards for Pacman Class diagram Dr. Balázs Simon, BME, IIT 2 CRC cards

More information

Project NMCGJ : Pac-Man Game

Project NMCGJ : Pac-Man Game Project NMCGJ 2017-2018: Pac-Man Game The aim of the project is to design and implement a variation of the video game Pac-Man. This game is among the most iconic video (arcade) games of all time; it is

More information

Clever Pac-man. Sistemi Intelligenti Reinforcement Learning: Fuzzy Reinforcement Learning

Clever Pac-man. Sistemi Intelligenti Reinforcement Learning: Fuzzy Reinforcement Learning Clever Pac-man Sistemi Intelligenti Reinforcement Learning: Fuzzy Reinforcement Learning Alberto Borghese Università degli Studi di Milano Laboratorio di Sistemi Intelligenti Applicati (AIS-Lab) Dipartimento

More information

The Kapman Handbook. Thomas Gallinari

The Kapman Handbook. Thomas Gallinari Thomas Gallinari 2 Contents 1 Introduction 6 2 How to Play 7 3 Game Rules, Strategies and Tips 8 3.1 Rules............................................. 8 3.2 Strategies and Tips.....................................

More information

Free Cell Solver. Copyright 2001 Kevin Atkinson Shari Holstege December 11, 2001

Free Cell Solver. Copyright 2001 Kevin Atkinson Shari Holstege December 11, 2001 Free Cell Solver Copyright 2001 Kevin Atkinson Shari Holstege December 11, 2001 Abstract We created an agent that plays the Free Cell version of Solitaire by searching through the space of possible sequences

More information

Artificial Intelligence Lecture 3

Artificial Intelligence Lecture 3 Artificial Intelligence Lecture 3 The problem Depth first Not optimal Uses O(n) space Optimal Uses O(B n ) space Can we combine the advantages of both approaches? 2 Iterative deepening (IDA) Let M be a

More information

Using Artificial intelligent to solve the game of 2048

Using Artificial intelligent to solve the game of 2048 Using Artificial intelligent to solve the game of 2048 Ho Shing Hin (20343288) WONG, Ngo Yin (20355097) Lam Ka Wing (20280151) Abstract The report presents the solver of the game 2048 base on artificial

More information

All theory, no practice

All theory, no practice RSS Feed Archive GameInternals All theory, no practice GameInternals aims to spread knowledge of interesting game mechanics beyond the game-specific enthusiast communities. Each post focuses on a specific

More information

Sokoban: Reversed Solving

Sokoban: Reversed Solving Sokoban: Reversed Solving Frank Takes (ftakes@liacs.nl) Leiden Institute of Advanced Computer Science (LIACS), Leiden University June 20, 2008 Abstract This article describes a new method for attempting

More information

Master Thesis. Enhancing Monte Carlo Tree Search by Using Deep Learning Techniques in Video Games

Master Thesis. Enhancing Monte Carlo Tree Search by Using Deep Learning Techniques in Video Games Master Thesis Enhancing Monte Carlo Tree Search by Using Deep Learning Techniques in Video Games M. Dienstknecht Master Thesis DKE 18-13 Thesis submitted in partial fulfillment of the requirements for

More information

Announcements. Homework 1. Project 1. Due tonight at 11:59pm. Due Friday 2/8 at 4:00pm. Electronic HW1 Written HW1

Announcements. Homework 1. Project 1. Due tonight at 11:59pm. Due Friday 2/8 at 4:00pm. Electronic HW1 Written HW1 Announcements Homework 1 Due tonight at 11:59pm Project 1 Electronic HW1 Written HW1 Due Friday 2/8 at 4:00pm CS 188: Artificial Intelligence Adversarial Search and Game Trees Instructors: Sergey Levine

More information

CSE 473 Midterm Exam Feb 8, 2018

CSE 473 Midterm Exam Feb 8, 2018 CSE 473 Midterm Exam Feb 8, 2018 Name: This exam is take home and is due on Wed Feb 14 at 1:30 pm. You can submit it online (see the message board for instructions) or hand it in at the beginning of class.

More information

COMP3211 Project. Artificial Intelligence for Tron game. Group 7. Chiu Ka Wa ( ) Chun Wai Wong ( ) Ku Chun Kit ( )

COMP3211 Project. Artificial Intelligence for Tron game. Group 7. Chiu Ka Wa ( ) Chun Wai Wong ( ) Ku Chun Kit ( ) COMP3211 Project Artificial Intelligence for Tron game Group 7 Chiu Ka Wa (20369737) Chun Wai Wong (20265022) Ku Chun Kit (20123470) Abstract Tron is an old and popular game based on a movie of the same

More information

22c:145 Artificial Intelligence

22c:145 Artificial Intelligence 22c:145 Artificial Intelligence Fall 2005 Informed Search and Exploration II Cesare Tinelli The University of Iowa Copyright 2001-05 Cesare Tinelli and Hantao Zhang. a a These notes are copyrighted material

More information

Computer Science. Using neural networks and genetic algorithms in a Pac-man game

Computer Science. Using neural networks and genetic algorithms in a Pac-man game Computer Science Using neural networks and genetic algorithms in a Pac-man game Jaroslav Klíma Candidate D 0771 008 Gymnázium Jura Hronca 2003 Word count: 3959 Jaroslav Klíma D 0771 008 Page 1 Abstract:

More information

Local Search. Hill Climbing. Hill Climbing Diagram. Simulated Annealing. Simulated Annealing. Introduction to Artificial Intelligence

Local Search. Hill Climbing. Hill Climbing Diagram. Simulated Annealing. Simulated Annealing. Introduction to Artificial Intelligence Introduction to Artificial Intelligence V22.0472-001 Fall 2009 Lecture 6: Adversarial Search Local Search Queue-based algorithms keep fallback options (backtracking) Local search: improve what you have

More information

Announcements. CS 188: Artificial Intelligence Fall Local Search. Hill Climbing. Simulated Annealing. Hill Climbing Diagram

Announcements. CS 188: Artificial Intelligence Fall Local Search. Hill Climbing. Simulated Annealing. Hill Climbing Diagram CS 188: Artificial Intelligence Fall 2008 Lecture 6: Adversarial Search 9/16/2008 Dan Klein UC Berkeley Many slides over the course adapted from either Stuart Russell or Andrew Moore 1 Announcements Project

More information

Adversarial Search. Human-aware Robotics. 2018/01/25 Chapter 5 in R&N 3rd Ø Announcement: Slides for this lecture are here:

Adversarial Search. Human-aware Robotics. 2018/01/25 Chapter 5 in R&N 3rd Ø Announcement: Slides for this lecture are here: Adversarial Search 2018/01/25 Chapter 5 in R&N 3rd Ø Announcement: q Slides for this lecture are here: http://www.public.asu.edu/~yzhan442/teaching/cse471/lectures/adversarial.pdf Slides are largely based

More information

AI Approaches to Ultimate Tic-Tac-Toe

AI Approaches to Ultimate Tic-Tac-Toe AI Approaches to Ultimate Tic-Tac-Toe Eytan Lifshitz CS Department Hebrew University of Jerusalem, Israel David Tsurel CS Department Hebrew University of Jerusalem, Israel I. INTRODUCTION This report is

More information

A Hybrid Method of Dijkstra Algorithm and Evolutionary Neural Network for Optimal Ms. Pac-Man Agent

A Hybrid Method of Dijkstra Algorithm and Evolutionary Neural Network for Optimal Ms. Pac-Man Agent A Hybrid Method of Dijkstra Algorithm and Evolutionary Neural Network for Optimal Ms. Pac-Man Agent Keunhyun Oh Sung-Bae Cho Department of Computer Science Yonsei University Seoul, Republic of Korea ocworld@sclab.yonsei.ac.kr

More information

Developing Frogger Player Intelligence Using NEAT and a Score Driven Fitness Function

Developing Frogger Player Intelligence Using NEAT and a Score Driven Fitness Function Developing Frogger Player Intelligence Using NEAT and a Score Driven Fitness Function Davis Ancona and Jake Weiner Abstract In this report, we examine the plausibility of implementing a NEAT-based solution

More information

CS 5522: Artificial Intelligence II

CS 5522: Artificial Intelligence II CS 5522: Artificial Intelligence II Adversarial Search Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials available at http://ai.berkeley.edu.]

More information

CMSC 372: Artificial Intelligence Lab#1: Designing Pac-Man Agents

CMSC 372: Artificial Intelligence Lab#1: Designing Pac-Man Agents CMSC 372: Artificial Intelligence Lab#1: Designing Pac-Man Agents Figure 1: The Pac-Man World Introduction In this project, you will familiarize yourself with the Pac-Man World. Over the next few assignments

More information

Reinforcement Learning to Train Ms. Pac-Man Using Higher-order Action-relative Inputs

Reinforcement Learning to Train Ms. Pac-Man Using Higher-order Action-relative Inputs Reinforcement Learning to Train Ms. Pac-Man Using Higher-order Action-relative Inputs Luuk Bom, Ruud Henken and Marco Wiering (IEEE Member) Institute of Artificial Intelligence and Cognitive Engineering

More information

Artificial Intelligence

Artificial Intelligence Artificial Intelligence Adversarial Search Instructors: David Suter and Qince Li Course Delivered @ Harbin Institute of Technology [Many slides adapted from those created by Dan Klein and Pieter Abbeel

More information

An Influence Map Model for Playing Ms. Pac-Man

An Influence Map Model for Playing Ms. Pac-Man An Influence Map Model for Playing Ms. Pac-Man Nathan Wirth and Marcus Gallagher, Member, IEEE Abstract In this paper we develop a Ms. Pac-Man playing agent based on an influence map model. The proposed

More information

a b c d e f g h 1 a b c d e f g h C A B B A C C X X C C X X C C A B B A C Diagram 1-2 Square names

a b c d e f g h 1 a b c d e f g h C A B B A C C X X C C X X C C A B B A C Diagram 1-2 Square names Chapter Rules and notation Diagram - shows the standard notation for Othello. The columns are labeled a through h from left to right, and the rows are labeled through from top to bottom. In this book,

More information

CSE 573: Artificial Intelligence Autumn 2010

CSE 573: Artificial Intelligence Autumn 2010 CSE 573: Artificial Intelligence Autumn 2010 Lecture 4: Adversarial Search 10/12/2009 Luke Zettlemoyer Based on slides from Dan Klein Many slides over the course adapted from either Stuart Russell or Andrew

More information

Influence Map-based Controllers for Ms. PacMan and the Ghosts

Influence Map-based Controllers for Ms. PacMan and the Ghosts Influence Map-based Controllers for Ms. PacMan and the Ghosts Johan Svensson Student member, IEEE and Stefan J. Johansson, Member, IEEE Abstract Ms. Pac-Man, one of the classic arcade games has recently

More information

University of Amsterdam. Encyclopedia of AI project. Tic-Tac-Toe. Authors: Andreas van Cranenburgh Ricus Smid. Supervisor: Maarten van Someren

University of Amsterdam. Encyclopedia of AI project. Tic-Tac-Toe. Authors: Andreas van Cranenburgh Ricus Smid. Supervisor: Maarten van Someren University of Amsterdam Encyclopedia of AI project Tic-Tac-Toe Authors: Andreas van Cranenburgh Ricus Smid Supervisor: Maarten van Someren January 27, 2007 Encyclopedia of AI, assignment 5 Tic-tac-toe

More information

CS 229 Final Project: Using Reinforcement Learning to Play Othello

CS 229 Final Project: Using Reinforcement Learning to Play Othello CS 229 Final Project: Using Reinforcement Learning to Play Othello Kevin Fry Frank Zheng Xianming Li ID: kfry ID: fzheng ID: xmli 16 December 2016 Abstract We built an AI that learned to play Othello.

More information

Second Annual University of Oregon Programming Contest, 1998

Second Annual University of Oregon Programming Contest, 1998 A Magic Magic Squares A magic square of order n is an arrangement of the n natural numbers 1,...,n in a square array such that the sums of the entries in each row, column, and each of the two diagonals

More information

Mittwoch, 14. September The Pelita contest (a brief introduction)

Mittwoch, 14. September The Pelita contest (a brief introduction) The Pelita contest (a brief introduction) Overview Overview Each Team owns two Bots Bots for team 0 Bots for team 1 Overview Each Team owns two Bots Each Bot is controlled by a Player Bots for team 0 Player

More information

The Implementation of Artificial Intelligence and Machine Learning in a Computerized Chess Program

The Implementation of Artificial Intelligence and Machine Learning in a Computerized Chess Program The Implementation of Artificial Intelligence and Machine Learning in a Computerized Chess Program by James The Godfather Mannion Computer Systems, 2008-2009 Period 3 Abstract Computers have developed

More information

CS188: Artificial Intelligence, Fall 2011 Written 2: Games and MDP s

CS188: Artificial Intelligence, Fall 2011 Written 2: Games and MDP s CS88: Artificial Intelligence, Fall 20 Written 2: Games and MDP s Due: 0/5 submitted electronically by :59pm (no slip days) Policy: Can be solved in groups (acknowledge collaborators) but must be written

More information

πgrammatical Evolution Genotype-Phenotype Map to

πgrammatical Evolution Genotype-Phenotype Map to Comparing the Performance of the Evolvable πgrammatical Evolution Genotype-Phenotype Map to Grammatical Evolution in the Dynamic Ms. Pac-Man Environment Edgar Galván-López, David Fagan, Eoin Murphy, John

More information

Monte-Carlo Tree Search in Ms. Pac-Man

Monte-Carlo Tree Search in Ms. Pac-Man Monte-Carlo Tree Search in Ms. Pac-Man Nozomu Ikehata and Takeshi Ito Abstract This paper proposes a method for solving the problem of avoiding pincer moves of the ghosts in the game of Ms. Pac-Man to

More information

CMSC 671 Project Report- Google AI Challenge: Planet Wars

CMSC 671 Project Report- Google AI Challenge: Planet Wars 1. Introduction Purpose The purpose of the project is to apply relevant AI techniques learned during the course with a view to develop an intelligent game playing bot for the game of Planet Wars. Planet

More information

Object-oriented Approach of Search Algorithms for Two-Player Games

Object-oriented Approach of Search Algorithms for Two-Player Games Proceedings of the 8 th International Conference on Applied Informatics Eger, Hungary, January 27 30, 2010. Vol. 2. pp. 29 34. Object-oriented Approach of Search Algorithms for Two-Player Games Márk Kósa,

More information

For slightly more detailed instructions on how to play, visit:

For slightly more detailed instructions on how to play, visit: Introduction to Artificial Intelligence CS 151 Programming Assignment 2 Mancala!! The purpose of this assignment is to program some of the search algorithms and game playing strategies that we have learned

More information

Reactive Control of Ms. Pac Man using Information Retrieval based on Genetic Programming

Reactive Control of Ms. Pac Man using Information Retrieval based on Genetic Programming Reactive Control of Ms. Pac Man using Information Retrieval based on Genetic Programming Matthias F. Brandstetter Centre for Computational Intelligence De Montfort University United Kingdom, Leicester

More information

Heuristics, and what to do if you don t know what to do. Carl Hultquist

Heuristics, and what to do if you don t know what to do. Carl Hultquist Heuristics, and what to do if you don t know what to do Carl Hultquist What is a heuristic? Relating to or using a problem-solving technique in which the most appropriate solution of several found by alternative

More information

AI Plays Yun Nie (yunn), Wenqi Hou (wenqihou), Yicheng An (yicheng)

AI Plays Yun Nie (yunn), Wenqi Hou (wenqihou), Yicheng An (yicheng) AI Plays 2048 Yun Nie (yunn), Wenqi Hou (wenqihou), Yicheng An (yicheng) Abstract The strategy game 2048 gained great popularity quickly. Although it is easy to play, people cannot win the game easily,

More information

HUJI AI Course 2012/2013. Bomberman. Eli Karasik, Arthur Hemed

HUJI AI Course 2012/2013. Bomberman. Eli Karasik, Arthur Hemed HUJI AI Course 2012/2013 Bomberman Eli Karasik, Arthur Hemed Table of Contents Game Description...3 The Original Game...3 Our version of Bomberman...5 Game Settings screen...5 The Game Screen...6 The Progress

More information

A Model-based Approach to Optimizing Ms. Pac-Man Game Strategies in Real Time

A Model-based Approach to Optimizing Ms. Pac-Man Game Strategies in Real Time - JANUARY 27, 2016 1 A Model-based Approach to Optimizing Ms. Pac-Man Game Strategies in Real Time Greg Foderaro, Member, IEEE, Ashleigh Swingler, Member, IEEE, and Silvia Ferrari, Senior Member, IEEE

More information

Learning from Hints: AI for Playing Threes

Learning from Hints: AI for Playing Threes Learning from Hints: AI for Playing Threes Hao Sheng (haosheng), Chen Guo (cguo2) December 17, 2016 1 Introduction The highly addictive stochastic puzzle game Threes by Sirvo LLC. is Apple Game of the

More information

Using Neural Network and Monte-Carlo Tree Search to Play the Game TEN

Using Neural Network and Monte-Carlo Tree Search to Play the Game TEN Using Neural Network and Monte-Carlo Tree Search to Play the Game TEN Weijie Chen Fall 2017 Weijie Chen Page 1 of 7 1. INTRODUCTION Game TEN The traditional game Tic-Tac-Toe enjoys people s favor. Moreover,

More information

CS 188 Introduction to Fall 2014 Artificial Intelligence Midterm

CS 188 Introduction to Fall 2014 Artificial Intelligence Midterm CS 88 Introduction to Fall Artificial Intelligence Midterm INSTRUCTIONS You have 8 minutes. The exam is closed book, closed notes except a one-page crib sheet. Please use non-programmable calculators only.

More information

UMBC 671 Midterm Exam 19 October 2009

UMBC 671 Midterm Exam 19 October 2009 Name: 0 1 2 3 4 5 6 total 0 20 25 30 30 25 20 150 UMBC 671 Midterm Exam 19 October 2009 Write all of your answers on this exam, which is closed book and consists of six problems, summing to 160 points.

More information

Game Playing State-of-the-Art. CS 188: Artificial Intelligence. Behavior from Computation. Video of Demo Mystery Pacman. Adversarial Search

Game Playing State-of-the-Art. CS 188: Artificial Intelligence. Behavior from Computation. Video of Demo Mystery Pacman. Adversarial Search CS 188: Artificial Intelligence Adversarial Search Instructor: Marco Alvarez University of Rhode Island (These slides were created/modified by Dan Klein, Pieter Abbeel, Anca Dragan for CS188 at UC Berkeley)

More information

Programming an Othello AI Michael An (man4), Evan Liang (liange)

Programming an Othello AI Michael An (man4), Evan Liang (liange) Programming an Othello AI Michael An (man4), Evan Liang (liange) 1 Introduction Othello is a two player board game played on an 8 8 grid. Players take turns placing stones with their assigned color (black

More information

Cassie Huang and Michael D Ambrosio Introductory Digital Systems Laboratory December 14, 2005

Cassie Huang and Michael D Ambrosio Introductory Digital Systems Laboratory December 14, 2005 Cassie Huang and Michael D Ambrosio 6.111 Introductory Digital Systems Laboratory December 14, 2005 For our final project, we developed Pac-Man on the FPGA. The game contains modules to handle both the

More information

Words Mobile Ready Game Documentation

Words Mobile Ready Game Documentation Words Mobile Ready Game Documentation Joongly games 2016 Words Mobile Ready Game Contents Overview... 3 Quick Start... 3 Game rules... 4 Basics... 4 Board... 4 Tiles... 4 Extra Point Values... 4 Game start...

More information

CSC 396 : Introduction to Artificial Intelligence

CSC 396 : Introduction to Artificial Intelligence CSC 396 : Introduction to Artificial Intelligence Exam 1 March 11th - 13th, 2008 Name Signature - Honor Code This is a take-home exam. You may use your book and lecture notes from class. You many not use

More information

5.4 Imperfect, Real-Time Decisions

5.4 Imperfect, Real-Time Decisions 5.4 Imperfect, Real-Time Decisions Searching through the whole (pruned) game tree is too inefficient for any realistic game Moves must be made in a reasonable amount of time One has to cut off the generation

More information

COMP219: COMP219: Artificial Intelligence Artificial Intelligence Dr. Annabel Latham Lecture 12: Game Playing Overview Games and Search

COMP219: COMP219: Artificial Intelligence Artificial Intelligence Dr. Annabel Latham Lecture 12: Game Playing Overview Games and Search COMP19: Artificial Intelligence COMP19: Artificial Intelligence Dr. Annabel Latham Room.05 Ashton Building Department of Computer Science University of Liverpool Lecture 1: Game Playing 1 Overview Last

More information

Playing Othello Using Monte Carlo

Playing Othello Using Monte Carlo June 22, 2007 Abstract This paper deals with the construction of an AI player to play the game Othello. A lot of techniques are already known to let AI players play the game Othello. Some of these techniques

More information

Introduction to Artificial Intelligence CS 151 Programming Assignment 2 Mancala!! Due (in dropbox) Tuesday, September 23, 9:34am

Introduction to Artificial Intelligence CS 151 Programming Assignment 2 Mancala!! Due (in dropbox) Tuesday, September 23, 9:34am Introduction to Artificial Intelligence CS 151 Programming Assignment 2 Mancala!! Due (in dropbox) Tuesday, September 23, 9:34am The purpose of this assignment is to program some of the search algorithms

More information

SCRABBLE ARTIFICIAL INTELLIGENCE GAME. CS 297 Report. Presented to. Dr. Chris Pollett. Department of Computer Science. San Jose State University

SCRABBLE ARTIFICIAL INTELLIGENCE GAME. CS 297 Report. Presented to. Dr. Chris Pollett. Department of Computer Science. San Jose State University SCRABBLE AI GAME 1 SCRABBLE ARTIFICIAL INTELLIGENCE GAME CS 297 Report Presented to Dr. Chris Pollett Department of Computer Science San Jose State University In Partial Fulfillment Of the Requirements

More information

Game Playing State-of-the-Art

Game Playing State-of-the-Art Adversarial Search [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Game Playing State-of-the-Art

More information

CS 188: Artificial Intelligence

CS 188: Artificial Intelligence CS 188: Artificial Intelligence Adversarial Search Prof. Scott Niekum The University of Texas at Austin [These slides are based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley.

More information

Introduction to Spring 2009 Artificial Intelligence Final Exam

Introduction to Spring 2009 Artificial Intelligence Final Exam CS 188 Introduction to Spring 2009 Artificial Intelligence Final Exam INSTRUCTIONS You have 3 hours. The exam is closed book, closed notes except a two-page crib sheet, double-sided. Please use non-programmable

More information

DeepMind Self-Learning Atari Agent

DeepMind Self-Learning Atari Agent DeepMind Self-Learning Atari Agent Human-level control through deep reinforcement learning Nature Vol 518, Feb 26, 2015 The Deep Mind of Demis Hassabis Backchannel / Medium.com interview with David Levy

More information

Computing Science (CMPUT) 496

Computing Science (CMPUT) 496 Computing Science (CMPUT) 496 Search, Knowledge, and Simulations Martin Müller Department of Computing Science University of Alberta mmueller@ualberta.ca Winter 2017 Part IV Knowledge 496 Today - Mar 9

More information

CS 188: Artificial Intelligence

CS 188: Artificial Intelligence CS 188: Artificial Intelligence Adversarial Search Instructor: Stuart Russell University of California, Berkeley Game Playing State-of-the-Art Checkers: 1950: First computer player. 1959: Samuel s self-taught

More information

Analyzing Games.

Analyzing Games. Analyzing Games staffan.bjork@chalmers.se Structure of today s lecture Motives for analyzing games With a structural focus General components of games Example from course book Example from Rules of Play

More information

Enhancements for Monte-Carlo Tree Search in Ms Pac-Man

Enhancements for Monte-Carlo Tree Search in Ms Pac-Man Enhancements for Monte-Carlo Tree Search in Ms Pac-Man Tom Pepels June 19, 2012 Abstract In this paper enhancements for the Monte-Carlo Tree Search (MCTS) framework are investigated to play Ms Pac-Man.

More information

Name: Your EdX Login: SID: Name of person to left: Exam Room: Name of person to right: Primary TA:

Name: Your EdX Login: SID: Name of person to left: Exam Room: Name of person to right: Primary TA: UC Berkeley Computer Science CS188: Introduction to Artificial Intelligence Josh Hug and Adam Janin Midterm I, Fall 2016 This test has 8 questions worth a total of 100 points, to be completed in 110 minutes.

More information

The game of Reversi was invented around 1880 by two. Englishmen, Lewis Waterman and John W. Mollett. It later became

The game of Reversi was invented around 1880 by two. Englishmen, Lewis Waterman and John W. Mollett. It later became Reversi Meng Tran tranm@seas.upenn.edu Faculty Advisor: Dr. Barry Silverman Abstract: The game of Reversi was invented around 1880 by two Englishmen, Lewis Waterman and John W. Mollett. It later became

More information

Inaction breeds doubt and fear. Action breeds confidence and courage. If you want to conquer fear, do not sit home and think about it.

Inaction breeds doubt and fear. Action breeds confidence and courage. If you want to conquer fear, do not sit home and think about it. Inaction breeds doubt and fear. Action breeds confidence and courage. If you want to conquer fear, do not sit home and think about it. Go out and get busy. -- Dale Carnegie Announcements AIIDE 2015 https://youtu.be/ziamorsu3z0?list=plxgbbc3oumgg7ouylfv

More information

Adversarial Search. Hal Daumé III. Computer Science University of Maryland CS 421: Introduction to Artificial Intelligence 9 Feb 2012

Adversarial Search. Hal Daumé III. Computer Science University of Maryland CS 421: Introduction to Artificial Intelligence 9 Feb 2012 1 Hal Daumé III (me@hal3.name) Adversarial Search Hal Daumé III Computer Science University of Maryland me@hal3.name CS 421: Introduction to Artificial Intelligence 9 Feb 2012 Many slides courtesy of Dan

More information

Pacman unblocked games 77

Pacman unblocked games 77 Search Search Pacman unblocked games 77 Hacked Unblocked.com is a store hacked online games. Play hacked unblocked games totaly free include full Arcade Hacked, Rpg Hacked, Armor Hacked and ArcadePrehacked.

More information

Eleventh Annual Ohio Wesleyan University Programming Contest April 1, 2017 Rules: 1. There are six questions to be completed in four hours. 2.

Eleventh Annual Ohio Wesleyan University Programming Contest April 1, 2017 Rules: 1. There are six questions to be completed in four hours. 2. Eleventh Annual Ohio Wesleyan University Programming Contest April 1, 217 Rules: 1. There are six questions to be completed in four hours. 2. All questions require you to read the test data from standard

More information

CS 188: Artificial Intelligence Spring Announcements

CS 188: Artificial Intelligence Spring Announcements CS 188: Artificial Intelligence Spring 2011 Lecture 7: Minimax and Alpha-Beta Search 2/9/2011 Pieter Abbeel UC Berkeley Many slides adapted from Dan Klein 1 Announcements W1 out and due Monday 4:59pm P2

More information

Announcements. CS 188: Artificial Intelligence Spring Game Playing State-of-the-Art. Overview. Game Playing. GamesCrafters

Announcements. CS 188: Artificial Intelligence Spring Game Playing State-of-the-Art. Overview. Game Playing. GamesCrafters CS 188: Artificial Intelligence Spring 2011 Announcements W1 out and due Monday 4:59pm P2 out and due next week Friday 4:59pm Lecture 7: Mini and Alpha-Beta Search 2/9/2011 Pieter Abbeel UC Berkeley Many

More information

AN ABSTRACT OF THE THESIS OF

AN ABSTRACT OF THE THESIS OF AN ABSTRACT OF THE THESIS OF Jason Aaron Greco for the degree of Honors Baccalaureate of Science in Computer Science presented on August 19, 2010. Title: Automatically Generating Solutions for Sokoban

More information

Today. Types of Game. Games and Search 1/18/2010. COMP210: Artificial Intelligence. Lecture 10. Game playing

Today. Types of Game. Games and Search 1/18/2010. COMP210: Artificial Intelligence. Lecture 10. Game playing COMP10: Artificial Intelligence Lecture 10. Game playing Trevor Bench-Capon Room 15, Ashton Building Today We will look at how search can be applied to playing games Types of Games Perfect play minimax

More information

arxiv: v1 [cs.ai] 9 Aug 2012

arxiv: v1 [cs.ai] 9 Aug 2012 Experiments with Game Tree Search in Real-Time Strategy Games Santiago Ontañón Computer Science Department Drexel University Philadelphia, PA, USA 19104 santi@cs.drexel.edu arxiv:1208.1940v1 [cs.ai] 9

More information

game tree complete all possible moves

game tree complete all possible moves Game Trees Game Tree A game tree is a tree the nodes of which are positions in a game and edges are moves. The complete game tree for a game is the game tree starting at the initial position and containing

More information

Artificial Intelligence. Minimax and alpha-beta pruning

Artificial Intelligence. Minimax and alpha-beta pruning Artificial Intelligence Minimax and alpha-beta pruning In which we examine the problems that arise when we try to plan ahead to get the best result in a world that includes a hostile agent (other agent

More information

Tetris: A Heuristic Study

Tetris: A Heuristic Study Tetris: A Heuristic Study Using height-based weighing functions and breadth-first search heuristics for playing Tetris Max Bergmark May 2015 Bachelor s Thesis at CSC, KTH Supervisor: Örjan Ekeberg maxbergm@kth.se

More information

Outline. Game Playing. Game Problems. Game Problems. Types of games Playing a perfect game. Playing an imperfect game

Outline. Game Playing. Game Problems. Game Problems. Types of games Playing a perfect game. Playing an imperfect game Outline Game Playing ECE457 Applied Artificial Intelligence Fall 2007 Lecture #5 Types of games Playing a perfect game Minimax search Alpha-beta pruning Playing an imperfect game Real-time Imperfect information

More information

Adversarial Search. CS 486/686: Introduction to Artificial Intelligence

Adversarial Search. CS 486/686: Introduction to Artificial Intelligence Adversarial Search CS 486/686: Introduction to Artificial Intelligence 1 Introduction So far we have only been concerned with a single agent Today, we introduce an adversary! 2 Outline Games Minimax search

More information

Lab 1. Due: Friday, September 16th at 9:00 AM

Lab 1. Due: Friday, September 16th at 9:00 AM Lab 1 Due: Friday, September 16th at 9:00 AM Consult the Standard Lab Instructions on LEARN for explanations of Lab Days ( D1, D2, D3 ), the Processing Language and IDE, and Saving and Submitting. 1. D1

More information

Announcements. CS 188: Artificial Intelligence Fall Today. Tree-Structured CSPs. Nearly Tree-Structured CSPs. Tree Decompositions*

Announcements. CS 188: Artificial Intelligence Fall Today. Tree-Structured CSPs. Nearly Tree-Structured CSPs. Tree Decompositions* CS 188: Artificial Intelligence Fall 2010 Lecture 6: Adversarial Search 9/1/2010 Announcements Project 1: Due date pushed to 9/15 because of newsgroup / server outages Written 1: up soon, delayed a bit

More information

Solving a Rubik s Cube with IDA* Search and Neural Networks

Solving a Rubik s Cube with IDA* Search and Neural Networks Solving a Rubik s Cube with IDA* Search and Neural Networks Justin Schneider CS 539 Yu Hen Hu Fall 2017 1 Introduction: A Rubik s Cube is a style of tactile puzzle, wherein 26 external cubes referred to

More information

CS 680: GAME AI INTRODUCTION TO GAME AI. 1/9/2012 Santiago Ontañón

CS 680: GAME AI INTRODUCTION TO GAME AI. 1/9/2012 Santiago Ontañón CS 680: GAME AI INTRODUCTION TO GAME AI 1/9/2012 Santiago Ontañón santi@cs.drexel.edu https://www.cs.drexel.edu/~santi/teaching/2012/cs680/intro.html CS 680 Focus: advanced artificial intelligence techniques

More information

Playing CHIP-8 Games with Reinforcement Learning

Playing CHIP-8 Games with Reinforcement Learning Playing CHIP-8 Games with Reinforcement Learning Niven Achenjang, Patrick DeMichele, Sam Rogers Stanford University Abstract We begin with some background in the history of CHIP-8 games and the use of

More information

AI in Computer Games. AI in Computer Games. Goals. Game A(I?) History Game categories

AI in Computer Games. AI in Computer Games. Goals. Game A(I?) History Game categories AI in Computer Games why, where and how AI in Computer Games Goals Game categories History Common issues and methods Issues in various game categories Goals Games are entertainment! Important that things

More information

Tutorial: Creating maze games

Tutorial: Creating maze games Tutorial: Creating maze games Copyright 2003, Mark Overmars Last changed: March 22, 2003 (finished) Uses: version 5.0, advanced mode Level: Beginner Even though Game Maker is really simple to use and creating

More information

UNIT 13A AI: Games & Search Strategies

UNIT 13A AI: Games & Search Strategies UNIT 13A AI: Games & Search Strategies 1 Artificial Intelligence Branch of computer science that studies the use of computers to perform computational processes normally associated with human intellect

More information

In this project you ll learn how to create a game, in which you have to match up coloured dots with the correct part of the controller.

In this project you ll learn how to create a game, in which you have to match up coloured dots with the correct part of the controller. Catch the Dots Introduction In this project you ll learn how to create a game, in which you have to match up coloured dots with the correct part of the controller. Step 1: Creating a controller Let s start

More information

Adversarial Search. Read AIMA Chapter CIS 421/521 - Intro to AI 1

Adversarial Search. Read AIMA Chapter CIS 421/521 - Intro to AI 1 Adversarial Search Read AIMA Chapter 5.2-5.5 CIS 421/521 - Intro to AI 1 Adversarial Search Instructors: Dan Klein and Pieter Abbeel University of California, Berkeley [These slides were created by Dan

More information

Optimal Rhode Island Hold em Poker

Optimal Rhode Island Hold em Poker Optimal Rhode Island Hold em Poker Andrew Gilpin and Tuomas Sandholm Computer Science Department Carnegie Mellon University Pittsburgh, PA 15213 {gilpin,sandholm}@cs.cmu.edu Abstract Rhode Island Hold

More information

MultiPac 24 in 1 Installation Guide and User s Manual

MultiPac 24 in 1 Installation Guide and User s Manual MultiPac 24 in 1 Installation Guide and User s Manual Notice Regarding this Upgrade Warning! Although this upgrade has been tested and the techniques used will not directly cause harm to your video game,

More information

Monte Carlo Tree Search. Simon M. Lucas

Monte Carlo Tree Search. Simon M. Lucas Monte Carlo Tree Search Simon M. Lucas Outline MCTS: The Excitement! A tutorial: how it works Important heuristics: RAVE / AMAF Applications to video games and real-time control The Excitement Game playing

More information

CS 4700: Foundations of Artificial Intelligence

CS 4700: Foundations of Artificial Intelligence CS 4700: Foundations of Artificial Intelligence selman@cs.cornell.edu Module: Adversarial Search R&N: Chapter 5 1 Outline Adversarial Search Optimal decisions Minimax α-β pruning Case study: Deep Blue

More information

TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play

TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play NOTE Communicated by Richard Sutton TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play Gerald Tesauro IBM Thomas 1. Watson Research Center, I? 0. Box 704, Yorktozon Heights, NY 10598

More information

USING A FUZZY LOGIC CONTROL SYSTEM FOR AN XPILOT COMBAT AGENT ANDREW HUBLEY AND GARY PARKER

USING A FUZZY LOGIC CONTROL SYSTEM FOR AN XPILOT COMBAT AGENT ANDREW HUBLEY AND GARY PARKER World Automation Congress 21 TSI Press. USING A FUZZY LOGIC CONTROL SYSTEM FOR AN XPILOT COMBAT AGENT ANDREW HUBLEY AND GARY PARKER Department of Computer Science Connecticut College New London, CT {ahubley,

More information