Game playing Chapter 5, Sections 1{5 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 1
} Perfect play } Resource limits } { pruning } Games of chance Outline AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 2
Games vs. search problems \Unpredictable" opponent ) solution is a contingency plan Time limits ) unlikely to nd goal, must approximate Plan of attack: algorithm for perfect play (Von Neumann, 1944) nite horizon, approximate evaluation (Zuse, 1945 Shannon, 1950 1952{57) Samuel, pruning to reduce costs (McCarthy, 1956) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 3
deterministic chance Types of games perfect information imperfect information chess, checkers, go, othello backgammon monopoly bridge, poker, scrabble nuclear war AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 4
choose move to position with highest minimax value Idea: best achievable payo against best play = Minimax Perfect play for deterministic, perfect-information games 2-ply game: E.g., MAX 3 A 1 A 2 A 3 MIN 3 2 2 A 11 A 13 A 21 A 22 A 23 A 32 A 33 A 12 A 31 3 12 8 2 4 6 14 5 2 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 5
Minimax algorithm function Minimax-Decision(game) returns an operator for each op in Operators[game] do end Value[op] Minimax-Value(Apply(op, game), game) return the op with the highest Value[op] function Minimax-Value(state, game) returns a utility value if Terminal-Test[game](state) then return Utility[game](state) else if max is to move instate then else return the highest Minimax-Value of Successors(state) return the lowest Minimax-Value of Successors(state) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 6
Complete?? Optimal?? Time complexity?? Space complexity?? Properties of minimax AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 7
Properties of minimax Complete?? Yes, if tree is nite (chess has specic rules for this) Optimal?? Yes, against an optimal opponent. Otherwise?? Time complexity?? O(b m ) Space complexity?? O(bm) (depth-rst exploration) chess, b 35, m 100 for \reasonable" games For exact solution completely infeasible ) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 8
Resource limits we have 100 seconds, explore 10 4 nodes/second Suppose 10 6 nodes per move ) Standard approach: cuto test depth limit (perhaps add quiescence search) e.g., evaluation function estimated desirability of position = AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 9
Evaluation functions Black to move White slightly better White to move Black winning For chess, typically linear weighted sum of features Eval(s) =w 1 f 1 (s) +w 2 f 2 (s) +:::+ w n f n (s) w 1 =9with e.g., 1 (s) = (number of white queens) { (number of black queens) f etc. AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 10
MAX Digression: Exact values don't matter MIN 1 2 1 20 1 2 2 4 1 20 20 400 Behaviour is preserved under any monotonic transformation of Eval the order matters: Only in deterministic games acts as an ordinal utility function payo AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 11
Cutting o search is identical to MinimaxValue except MinimaxCutoff Terminal? is replaced by Cutoff? 1. 2. Utility is replaced by Eval Does it work in practice? b m =10 6 b =35 ) m =4 4-ply lookahead is a hopeless chess player! human novice 4-ply typical PC, human master 8-ply 12-ply Deep Blue, Kasparov AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 12
{ pruning example MAX 3 MIN 3 3 12 8 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 13
2 2 X X AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 14
14 14 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 15
5 5 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 16
3 2 2 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 17
Pruning does not aect nal result Properties of { Good move ordering improves eectiveness of pruning \perfect ordering," time complexity =O(b m=2 ) With doubles depth of search ) ) can easily reach depth 8 and play good chess simple example of the value of reasoning about which computations A relevant (a form of metareasoning) are AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 18
Why is it called {? MAX MIN...... MAX MIN V is the best value (to max) found so far o the current path If V is worse than, max will avoid it ) prune that branch Dene similarly for min AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 19
The { algorithm Basically Minimax +keep track of, +prune function Max-Value(state, game,, ) returns the minimax value of state inputs: state, current state in game game, game description, the best score for max along the path to state, the best score for min along the path to state if Cutoff-Test(state) then return Eval(state) for each s in Successors(state) do end Max(, Min-Value(s, game,, )) if then return return function Min-Value(state, game,, ) returns the minimax value of state if Cutoff-Test(state) then return Eval(state) for each s in Successors(state) do end return AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 20 Min(, Max-Value(s, game,, )) then return if
Deterministic games in practice Chinook ended 40-year-reign of human world champion Marion Checkers: Tinsley in 1994. Used an endgame database dening perfect play all positions involving 8 or fewer pieces on the board, a total of for positions. 443,748,401,247 Deep Blue defeated human world champion Gary Kasparov in a Chess: match in 1997. Deep Blue searches 200 million positions per six-game uses very sophisticated evaluation, and undisclosed methods for second, some lines of search up to 40 ply. extending human champions refuse to compete against computers, who Othello: too good. are human champions refuse to compete against computers, who are Go: bad. In go, b>300, so most programs use pattern knowledge bases too to suggest plausible moves. AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 21
Nondeterministic games E..g, in backgammon, the dice rolls determine the legal moves Simplied example with coin-ipping instead of dice-rolling: MAX CHANCE 3 1 0.5 0.5 0.5 0.5 MIN 2 4 0 2 2 4 7 4 6 0 5 2 AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 22
Algorithm for nondeterministic games Expectiminimax gives perfect play Just like Minimax, except we must also handle chance nodes: ::: state is a chance node then if ::: return average of ExpectiMinimax-Value of Successors(state) version of { pruning is possible A only if the leaf values are bounded. Why?? but AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 23
Nondeterministic games in practice rolls increase b: 21 possible rolls with 2 dice Dice 20 legal moves (can be 6,000 with 1-1 roll) Backgammon depth 4 = 20 (21 20) 3 1:2 10 9 depth increases, probability of reaching a given node shrinks As value of lookahead is diminished ) { pruning is much less eective uses depth-2 search + very good Eval TDGammon world-champion level AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 24
MAX Digression: Exact values DO matter DICE 2.1 1.3.9.1.9.1 21 40.9.9.1.9.1 MIN 2 3 1 4 20 30 1 400 2 2 3 3 1 1 4 4 20 20 30 30 1 1 400 400 Behaviour is preserved only by positive linear transformation of Eval Hence Eval should be proportional to the expected payo AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 25
Summary Games are fun to work on! (and dangerous) They illustrate several important points about AI } perfection is unattainable ) must approximate } good idea to think about what to think about } uncertainty constrains the assignment of values to states Games are to AI as grand prix racing is to automobile design AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 5, Sections 1{5 26