nnouncements acktracking and Game Trees 15-211: Fundamental Data Structures and lgorithms! HW5 now available!! May do in groups of two.! Review in recitation! No fancy data structures except trie!! Due Monday 11:59 pm Margaret Reid-Miller 5 pril 2005 2 X O X O X O X O O D acktracking acktracking! n algorithm-design technique! Organized brute force "Explore the space of possible answers "Do it in an organized way! Example: maze Try S, E, N, W IN OUT 4 acktracking is useful! when a problem is too hard to be solved directly! 8-queens problem! Knight s tour! when we have limited time and can accept a good but potentially not optimal solution! Game playing (second part of lecture)! Planning (not in this class) acktracking decisions! Develop answers by identifying a set of successive decisions.! Maze! Where do I go now: N, S, E or W?! 8 Queens! Where do I put the next queen?! Knight s tour! Where do I jump next? 5 6
acktracking decisions acktracking decisions! Develop answers by identifying a set of successive decisions.! Develop answers by identifying a set of successive decisions.! Pure acktracking! Decisions can be binary: ok or impossible! Pure acktracking! Decisions can be binary: ok or impossible! annot put an X in two places! an move the rook two spaces forward! Heuristic acktracking! Decisions can have goodness:! Good to sacrifice a pawn to take a bishop! ad to sacrifice the queen to take a pawn 7! ad not to block in tic-tac-toe 8 acktracking technique acktracking technique 9 10 acktracking technique acktracking technique! Undo its consequences! Undo its consequences "Is there a new choice?! If not, you are at another dead-end 11 12
acktracking technique acktracking technique! Undo its consequences 13 14 acktracking technique! Undo its consequences "Is there a new choice?! If so, try that acktracking technique! Undo its consequences "Is there a new choice?! If so, try that! If not, you are at another dead-end E E Search tree 15 16 Search trees No optimality guarantees! The tree "Is rarely explicit as a data structure!! How do we keep track of our search? "Use a stack ( control stack )! Represents a stack of tentative decisions! Tracks decisions to be undone when backtracking.! If there are multiple solutions, simple backtracking will find one of them, but not necessarily the optimal.! No guarantees that we ll reach a solution quickly.! an be implicit in recursive calls 17 18
acktracking Summary! Organized brute force! Formulate problem so that answer amounts to taking a set of successive decisions! When stuck, go back, try the remaining alternatives! Does not give optimal solutions X O X O X O X O O Games 19 Why Make omputers Play Games?! People have been fascinated by games since the dawn of civilization.! Idealization of real-world problems containing adversary situations "Typically rules are very simple "State of the world is fully accessible "Example: auctions 21 What Kind of Games?! No, not Quake (although interesting research there, too)! Simpler Strategy/oard Game! hess! heckers! Othello! Go! Let s look at how we might go about playing Tic-Tac-Toe 22 onsider this position Let s write out all possibilities We are playing X, and it is now our turn. 23 Each number represents a position after each legal move we have. 24
Now let s look at their options Now let s look at their options Here we are looking at all of the opponent responses Opponent options after our second possibility. Not good again to the first possible move we could make. 25 26 Now let s look at their options More interesting case Struggling 27 Now they don t have a way to win on their next move. So now we have to consider our responses to their responses. 28 Our options Finishing it up We have a win for any move they make. So the original position in purple is an X win. 29 They win again if we take our fifth move. 30
!! Summary of the nalysis Tic Tac Toe Game Tree So which move should we make? ;-) 31 32 Tic Tac Toe Game Tree Nodes Denote board configurations Include a score Path Successive by the players Edges Denote legal 33 path in a game tree! KRPOV-KSPROV, 1985 1.e4 c5 2.Nf3 e6 3.d4 cxd4 4.Nxd4 Nc6 5.Nb5 d6 6.c4 Nf6 7.N1c3 a6 8.Na3 d5!? 9.cxd5 exd5 10.exd5 Nb4 11.e2!?N c5! 12.0-0 0-0 13.f3 f5 14.g5 Re8! 15.Qd2 b5 16.Rad1 Nd3! 17.Nab1? h6! 18.h4 b4! 19.Na4 d6 20.g3 Rc8 21.b3 g5!! 22.xd6 Qxd6 23.g3 Nd7! 24.g2 Qf6! 25.a3 a5 26.axb4 axb4 27.Qa2 g6 28.d6 g4! 29.Qd2 Kg7 30.f3 Qxd6 31.fxg4 Qd4+ 32.Kh1 Nf6 33.Rf4 Ne4 34.Qxd3 Nf2+ 35.Rxf2 xd3 36.Rfd2 Qe3! 37.Rxd3 Rc1!! 38.Nb2 Qf2! 39.Nd2 Rxd1+ 40.Nxd1 Re1+ White resigned 34 Properties of Tic Tac Toe Games Not onsidered!!two! players! Deterministic " no randomness, e.g., dice! Perfect information " No hidden cards or hidden hess pieces! Finite " Game must end in finite number of! Zero sum " Total winnings of all players! Poker " Hidden cards! ackgammon " not deterministic (e.g., dice)! Prisoner s dilemma, game of chicken " Not zero-sum Have rich and beautiful theories and algorithms, though. (see S 15-859) 35 36
Two-player games! We can define the value (goodness) of a certain game state (board). 1 0-1?! What about the non-final board? " Look at board, assign value " Look at children in game tree, assign value 37 How do we play?! Traverse the game tree. " Enumerate all possible at each node.! Start by evaluating the terminal positions, where the game is over.! Propogate the values up the tree, assuming we pick the best move for us and the opponent picks the best move for her. 38 How to play? How do we play? 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 39! What is the rule to compute the value of nonterminal node? " If it is our turn (X) then we pick the maximum outcome from the children. " If it is the opponent s turn (O) then we pick the minimum outcome from the children.! This process is known as the imax algorithm.! We play the move that gave us the maximum at the root. 40 More generally! Player (us) maximize goodness.! Player (opponent) minimizes goodness Player maximize Player 9 7 minimize c 8 2 5 a b d Player maximize Games Trees are useful! Provide lookahead to determine what move to make next! uild whole tree = we know how to play perfectly c 8 2 5 a b 9 7 d 41 42
ut there s one problem! Games have large search trees: "Tic-Tac-Toe! There are 9 ways we can make the first move and opponent has 8 possible he can make.! Then when it is our turn again, we can make 7 possible for each of the 8 and so on.! 9! = 362,880 Or chess! Suppose we look at 16 possible at each step! nd suppose we explore to a depth of 50 turns! Then the tree will have 16 50 =10 120 nodes!! DeepThought (1990) searched to a depth of 10 43 44 So Heuristic search! Need techniques to avoid enumerating and evaluating the whole tree! Heuristic search 45 1. Organize search to eliminate large sets of possibilities " hess: onsider major first 2. Explore decisions in order of likely success " hess: Use a library of known strategies 3. Save time by guessing search outcomes " hess: Estimate the quality of a situation! ount pieces! ount pieces, using a weighting scheme! ount pieces, using a weighting scheme, considering threats 46 Let s see it in action (Player ) 10 i- lgorithm (Player ) (Player ) 10-5 10 16 7-5 10 2 12 16 2 7-5 -80 Evaluation function applied to the leaves! 47 48
imax, in reality! Rarely do we reach an actual leaf "Use estimator functions to statically guess goodness of missing subtrees imax, in reality! Rarely do we reach an actual leaf "Use estimator functions to statically guess goodness of missing subtrees 2 7 1 8 2 2 7 1 1 8 49 50 imax, in reality imax! Rarely do we reach an actual leaf "Use estimator functions to statically guess goodness of missing subtrees 2 2 7 2 1 1 8 51! Trade-off "Quality vs. Speed "Quality: deeper search "Speed: use of estimator functions! alancing "Relative costs of move generation and estimator functions "Quality and cost of estimation function 52 imax algorithm - pseudo code Nega lgorithm ( P, depth ) { if( depth == 0 ) return estimator(p) m = -! for each possible move P m = max( m, (P, depth-1) ) return m } ( P, depth ) { if( depth == 0 ) return estimator(p) m =! for each possible move P m = min( m, (P, depth-1) ) return m } 53! Rewrite from the point of view of moving player: nega( P, depth ) { if (depth == 0) return estimator(p) m = -! } for each possible move P v = -nega( P, depth-1 ) if m < v then m = v return m an you prove these two versions are the same? 54
How fast?! imax is pretty slow even for a modest depth.! It is basically a brute force search.! What is the running time? "Each level of the tree has some average b per level. We have d levels. So the running time is O(b d ). an we speed this up?! Let us observe the following game tree. $2 2 %1 2 7 1 What do we know about the root? What do we know about the root s right child? 55 56 Pruning! imax sometimes evaluates nodes that have no impact on the search. lpha eta Pruning! E.g., Evaluating in hess "My 1st move: I m ahead a rook "My 2nd move: Opponent 1st move leaves me down a queen.! No need to look at Opponent s 2nd move to find out it is check mate. lready know my 1st move is better. 57 58 lpha eta Pruning! Idea: Track window of expectations. Two variables: " est score so far at a max node: increases & t a child min node: Parent wants max. To affect the parent s current ", our # cannot drop below ". # est score so far at a min node: decreases & t a child max node. Parent wants min. To affect the parent s current #, our " cannot get above the parent s #. Either case: If " $ #: Stop searching further subtrees of that child. They do not matter!! Start the process with an infinite window (" = -!, # =!). lpha-eta algorithm - pseudo code ( P, depth, ", # ) { if( depth == 0 ) return estimator(p) } for each possible move P && " < # " = max(", ( P, depth-1, ", # )) return " ( P, depth, ", # ) { if( depth == 0 ) return estimator(p) } for each possible move P && " < # # = min(#, ( P, depth-1, ", # )) return # 59 60
lpha-eta algorithm - lternative lpha eta Example ( P, depth, ", # ) { if( depth == 0 ) return estimator(p) for each possible move P && " < # v = -( P, depth-1, -#, -" ) if " < v then " = v 10 # =10 " > #! } return "! In replace " with -# and # with -", and negate the return value. Get. 10 12 10 2 12 " = 12 61 62 lpha eta Example lpha eta Pruning 10 " = 10 " > #!! Does lpha eta ever return a different root value than imax? 10 # =7 7! No! s long as the root value is within the root (lpha, eta) range. 10 12 7 10 2 12 2 7! lpha eta does the same thing imax does, except it is able to detect parts of the tree that make no difference. 63 64 lpha eta Pruning! Does lpha eta ever return a different root value than imax?! No! s long as the root value is within the root (lpha, eta) range.! lpha eta does the same thing imax does, except it is able to detect parts of the tree that make no difference. lpha eta Pruning! Theorem: Let v(p) be the value of position P. Let X be the value returned by (", #).Then one of the following holds: v(p)! " and X! " " < v(p) < # and X = v(p) #! v(p) and X " #! The proof is by induction, and is an easy but tedious case analysis. 65 66
lpha eta speedup Summary! laim: The optimal lpha eta search tree is O(b d/2 ) nodes or the square root of the number of nodes in the regular imax tree. "an enable twice the depth! The speedup is greatly dependent on the order in which you consider at each node. Why? 67! acktracking "Organized brute force "nswer to problem = set of successive decisions "When stuck, go back, try the remaining alternatives "No optimality guarantees 68 Summary! Game playing "Game trees "i-max algorithm "Optimization! lpha-beta pruning! Heuristics search 69