Board Game AIs. With a Focus on Othello. Julian Panetta March 3, 2010

Transcription

1 Board Game AIs With a Focus on Othello Julian Panetta March 3,

2 Practical Issues Bug fix for TimeoutException at player init Not an issue for everyone Download updated project files from CS2 course page If you ve already started, just copy over new OthelloLib.jar 2

3 Practical Issues How do I run Java? Eclipse IDE Command Line cd src Free from Move your player into the src directory javac -cp../othellolib.jar *.java java -cp :../OthelloLib.jar Main 3

4 Eclipse File ==> New ==> Java Project Create project from existing source 4

5 Python Interaction Java code launches Python interpreter Needs to know where it is! Windows, you might need to: On Mac/Linux, supplied code should work Add directory containing python.exe to your PATH variable or Change line 93 of DobenshaPlayer.java to something like: cmdarray[0] = "C:\\Program Files\\Python\\python"; 5

6 Python not required! We ll see speed matters Python is slow Compile to executable MyPlayer Can use C/C++ or another language if you want! String[] cmdarray = new String[2]; cmdarray[0] = "./MyPlayer"; cmdarray[1] = "1"; Skeleton C player (MyPlayer.c) is now provided. Run make to compile. Please stick with the language with which you re most comfortable. 6

7 Any Questions? 7

8 AIs: The Generic Problem Given: Board state: B Function to find legal moves: moves(b) Game result, result(b) WIN, LOSE, DRAW, NOT_DONE Return: Move that will (eventually) result in our player s result(b) == WIN 8

9 Simple Example Tic-Tac-Toe Board state, B = 3x3 grid of Xs, Os, or Blanks Legal moves, moves(b) = All spaces (i, j) such that B(i, j) == Blank Player X s result(b): WIN if there exists a line of 3 Xs X O X O X O X LOSE if there exists a line of 3 Os DRAW if no moves exist and we haven t won or lost NOT_DONE otherwise 9

10 Toward a Solution At / near the end of a game, this problem is easy: Play the move that makes us win immediately If none exists, make sure our opponent doesn t win Observation: X X X O O If we assume opponent plays perfectly, game becomes solvable ( deterministic ) Predict opponent makes move resulting in your worst future outcome X O X O X 10

11 Toward a Solution This suggests a procedure: For move in moves(b), we: 1. Make move (B = makemove(b, move)) 2. If game ended, consider result(b ) 3. Otherwise, determine future results of perfect opponent playing on B 4. Undo move (B = undomove(b, move)) Return the move that gives us the best outcome (WIN > DRAW > LOSE) 11

12 Toward a Solution Note: Perfect opponent does the exact same thing! Returns move resulting in the best current or future outcome (WIN > DRAW > LOSE) Distinction: Opponent s WIN equivalent to our LOSE and vice versa DRAW means the same for both 12

13 Toward a Solution From our player s perspective We return move resulting in best outcome Opponent returns move resulting in worst outcome Each move brings us one step closer to the game s end Look a bit like recursion? 13

14 Minimax Algorithm Player maximizes its own value: bestmove: For move in moves(b) B = makemove(b, move) if game ended, move.value is -1 for LOSE, 1 for WIN, 0 for DRAW otherwise get move.value from bestopponentmove(b).value B = undomove(b, move) replace maxmove with move if move.value > maxmove.value return maxmove Opponent minimizes player s value: bestopponentmove: For move in opponentmoves(b) B = makemove(b, move) if game ended, move.value is -1 for LOSE, 1 for WIN, 0 for DRAW otherwise get move.value from bestmove(b).value B = undomove(b, move) replace minmove with move if move.value < minmove.value return minmove 14

15 Minimax Algorithm With a little cleverness, you can combine this into a single recursive routine! Termination Assuming no cycles, each move brings us closer to an endgame Every recursive call will eventually reach the endgame and return. 15

16 Tree Representation Minimax algorithm can be thought of as a tree traversal (Depth First Search!) Board State Move1 Move2 Move3... Legend Move1, 1 Move1, bestmove call bestopponentmove call 16

17 Tree Representation Provides a good way to visualize the algorithm s operation Foundation of reasoning that leads to certain optimizations Board State More on this later! Move1 Move2 Move3... Move1, 1 Move1,

18 Example Running Minimax on a partially complete board: X O O X (X s turn) 18

19 Example X 1 O Board State Move1 Move2 Move3 Move4 Move5 2 O X 19

20 Example X X O Board State Move1 Move2 Move3 Move4 Move5 1 O X Move1,1 Move1,2 Move1,3 Move1,4 20

21 Example X X O Board State Move1 Move2 Move3 Move4 Move5 1 O 2 O 4 X Move1,1 Move1,2 Move1,3 Move1,4 LOSE 21

22 Example X X O Board State Move1 Move2 Move3 Move4 Move5 1 O X LOSE Move1,1 Move1,2 Move1,3 Move1,4 LOSE Minimum possible--propagated up. 22

23 Example X 1 O Board State Move1 Move2 Move3 Move4 Move5 2 O 3 LOSE LOSE LOSE X 4 X Move4,1 Move4,2 Move4,3 Move4,4 Skipping ahead a bit... 23

24 Example X 1 O Board State Move1 Move2 Move3 Move4 Move5 2 O 3 LOSE LOSE LOSE X O X Move4,1 Move4,2 Move4,3 Move4,4 WIN WIN WIN WIN Player X can win at next step regardless of O s action. Player X will play either 2 or 4, depending on which is available. 24

25 Example WIN Move4 X 1 O Move1 Move2 Move3 Move4 Move5 2 O 3 LOSE LOSE LOSE WIN Move 4,4 4 5 X Move4,1 Move4,2 Move4,3 Move4,4 WIN WIN WIN WIN Propagate win move up! We know forcing a win is the best we can do, so Move4 from the start is optimal! 25

26 Any Questions? 26

27 How long will this take? For Tic-Tac-Toe, not terribly long! Conservative upper bound of 9! = recursive calls Move checking and evaluation is simple Tic-Tac-Toe is trivially solved Entire game can be planned from start Two perfect players will force a draw 27

28 Other Games? Connect 4 Solved in 1988 by James D Allen First player can force a win! Free perfect players available: Mustrum, Velena, TitOT Source: 28

29 Other Games? Checkers Solved in 2007 Computation spread over 18 years Perfect players force a draw Source: 29

30 Chess No way! Source: 30

31 Othello/Reversi Solved for 4x4 6x6 Second player wins! No solution yet for 8x8 Signs point to a draw... Source: 31

32 What can we do without a solution? Can t play optimally... Introduce a board state valuation: value(b) High value: future win likely Low (negative) value: future loss likely Look as far ahead as possible, and use value in place of certain WIN/LOSE/DRAW when endgame unreached 32

33 Board Valuation: Othello Not all board spaces are equal! Corner squares cannot be captured (good!) Edges can be hard to capture (good!) Spaces adjacent to corners can give away corners (bad!) 33

34 Board Valuation: Othello Capture these effects with board weights High (+) weight ==> good Low (-) weight ==> bad Reward for occupying a good space Penalize for occupying a bad space 34

35 Board Valuation: Othello Weights representing the values we ve discussed: 35

36 Other Considerations Mobility How many (good) moves you have Keep this high! Keep your opponent s mobility low! 36

37 Other Considerations Stability Stable pieces are pieces that can t be captured You want these! Pieces in corners are stable Stable pieces can make other pieces stable 37

38 Termination Condition? How far do we look ahead? Recurse to a constant depth? Must keep track of current depth Recurse until we get short on time? Tricky--don t go over! 38

39 Efficiency Faster search means we can look farther ahead in same amount of time Do we really have to consider every full path? Some can be clearly terrible from the start... Alpha-beta pruning Next lecture! 39