Game Playing. Foundations of Artificial Intelligence. Adversarial Search. Game Playing as Search. Game Playing. Simplified Minimax Algorithm

Transcription

1 Fundatins f Artificial Intelligence Adversarial Search CS47 Fall 007 Thrsten Jachims An AI Favrite structured task Game Playing clear definitin f success and failure des nt require large amunts f knwledge (at first glance) fcus n games f perfect infrmatin Game Playing Game Playing as Search Initial State is the initial bard/psitin Successr Functin defines the set f legal mves frm any psitin Terminal Test determines when the game is ver Utility Functin gives a numeric utcme fr the game MAX(X) Partial Search Tree fr Tic-Tac-Te Simplified Minima Algrithm. Epand the entire tree belw the rt. MIN(O) MAX(X) MIN(O) TERMINAL. Evaluate the terminal ndes as wins fr the minimizer r maimizer (i.e. utility).. Select an unlabeled nde, n, all f whse children have been assigned values. If there is n such nde, we're dne --- return the value assigned t the rt. 4. If n is a minimizer mve, assign it a value that is the minimum f the values f its children. If n is a maimizer mve, assign it a value that is the maimum f the values f its children. Return t Step. UTILITY - 0 +

2 MAX MIN Anther Eample A A A A A A A A A A A A Minima functin MINIMAX-DECISION(game) returns an peratr fr each p in OPERATORS[game]d VALUE[p] MINIMAX-VALUE(APPLY(p,game),game) end return the p with the highest VALUE[p] functin MINIMAX-VALUE(state,game) returns a utility value if TERMINAL-TEST[game](state) then return UTILITY[game](state) else if MAX is t mve in state then return the highest MINIMAX-VALUE f SUCCESSORS(state) else return the lwest MINIMAX-VALUE f SUCCESSORS(state) Imprving Minima: Pruning Features f Evlutin Idea: Avid generating the whle search tree Apprach: Analyze which subtrees have n influence n the slutin Player Oppnent.... Player m Oppnent n If m is better than n fr Player, never get t n in play. α β Search = lwer bund n Ma's utcme; initially set t - = upper bund n Min's utcme ; initially set t + α We'll call prcedure recursively with a narrwing range between and β. Maimizing levels may reset t a higher value; Minimizing levels may reset β t a lwer value. α Search Algrithm. If terminal state, cmpute e(n) and return the result.. Otherwise, if the level is a minimizing level, Until n mre children r β α - υ search n a child i - If υi < β, β υi. Return min( υ i ). Otherwise, the level is a maimizing level: Until n mre children r α β, υ search n a child. i If υi > α, set α υi Return ma( ) υ i

3 Search Space Size Reductins The Need fr Imperfect Decisins Wrst Case: In an rdering where wrst ptins evaluated first, all ndes must be eamined. Best Case: If ndes rdered s that the best ptins are evaluated first, then what? Prblem: Minima assumes the prgram has time t search t the terminal ndes. Slutin: Cut ff search earlier and apply a heuristic evaluatin functin t the leaves. Static Evaluatin Functins Minima depends n the translatin f bard quality int single, summarizing number. Difficult. Epensive. Add up values f pieces each player has (weighted by imprtance f piece). Islated pawns are bad. Hw well prtected is yur king? Hw much maneuverability t yu have? D yu cntrl the center f the bard? Strategies change as the game prceeds. Design Issues fr Heuristic Minima Evaluatin Functin: Need t be carefully crafted and depends n game! What criteria shuld an evaluatin functin fulfill? Linear Evaluatin Functins Design Issues fr Heuristics Minima wf + wf wn fn This is what mst game playing prgrams use Steps in designing an evaluatin functin:. Pick infrmative features. Search: search t a cnstant depth What are prblems with cnstant search depth?. Find the weights that make the prgram play well

4 Backgammn Bard Backgammn - Rules Gal: mve all f yur pieces ff the bard befre yur ppnent des. Black mves cunterclckwise tward 0. White mves clckwise tward 5. A piece can mve t any psitin ecept ne where there are tw r mre f the ppnent's pieces If it mves t a psitin with ne ppnent piece, that piece is captured and has t start it's jurney frm the beginning. Backgammn - Rules If yu rll dubles yu take 4 mves (eample: rll 5,5, make mves 5,5,5,5). Mves can be made by ne r tw pieces (in the case f dubles by,, r 4 pieces) And a few ther rules that cncern bearing ff and frced mves White has rlled 6-5 and has 4 legal mves: (5-0,5-), (5-,9-4), (5-0,0-6) and (5-,-6). Game Tree fr Backgammn Epectiminima MAX DICE MIN DICE MAX /8 /6,, 6,5 6,6 /6, /8, C 6,5 6,6 Epectiminima(n) = Utility(n) fr n, a terminal state ma s Succ(n) epectiminima( s) fr n, a Ma nde min s Succ(n) epectiminima( s) fr n, a Min nde Σ P fr n, a chance nde s Succ( n) ( )*epectiminima( ) TERMINAL 4

5 Evaluatin functin State f the Art in Backgammn. A A A A : BKG using tw-ply (depth ) search and lts f luck defeated the human wrld champin. 99: Tesaur cmbines Samuel's learning methd with neural netwrks t develp a new evaluatin functin (search depth -), resulting in a prgram ranked amng the tp players in the wrld. State f the Art in Checkers 95: Samuel develped a checkers prgram that learned its wn evaluatin functin thrugh self play. 990: Chink (J. Schaeffer) wins the U.S. Open. At the wrld champinship, Marin Tinsley beat Chink. 005: Schaeffer et al. slved checkers fr White Dctr pening (draw) (abut 50 ther penings). State f the Art in G Large branching factr makes regular search methds inapprpriate. Best cmputer G prgrams ranked nly weak amateur. Emply pattern recgnitin techniques and limited search. $,000,000 prize available fr first cmputer prgram t defeat a tp level player. Histry f Chess in AI Legal chess Occasinal player Wrld-ranked Gary Kasparv Early 950's Shannn and Turing bth had prgrams that (barely) played legal chess (500 rank). 950's Ale Bernstein's system, (500 + ε) 957 Herb Simn claims that a cmputer chess prgram wuld be wrld chess champin in 0 years...yeah, right. 966 McCarthy arranges cmputer chess match, Stanfrd vs. Russia. Lng, drawn-ut match. Russia wins. 967 Richard Greenblatt, MIT. First f the mdern chess prgrams, MacHack (00 rating). 968 McCarthy, Michie, Papert bet Levy (rated 5) that a cmputer prgram wuld beat him within 0 years. 970 ACM started running chess turnaments. Chess.0-6 (rated 400). 97 By 97...Slate: It had becme t painful even t lk at Chess.6 any mre, let alne wrk n it. 97 Chess 4.0: smart plausible-mve generatr rather than speeding up the search. Imprved rapidly when put n faster machines. 5

6 976 Chess 4.5: ranking f Chess 4.5 vs.~levy. Levy wins. 980's Prgrams depend n search speed rather than knwledge (00 range). 99 DEEP THOUGHT: Sphisticated special-purpse cmputer; search; searches 0-ply; singular etensins; rated abut DEEP BLUE: searches 4-ply; iterative deepening search; cnsiders billin psitins per mve; regularly reaches depth 4; evaluatin functin has features; singular etensins t 40-ply; pening bk f 4000 psitins; end-game database fr 5-6 pieces. Cncludes Search Uninfrmed search: DFS / BFS / Unifrm cst search time / space cmpleity size search space: up t appr. 0 ndes special case: Cnstraint Satisfactin / CSPs generic framewrk: variables & cnstraints backtrack search (DFS); prpagatin (frward-checking / arc-cnsistency, variable / value rdering 997 DEEP BLUE: first match wn against wrld-champin (Kasparv). 00 IBM declines re-match. FRITZ played wrld champin Vladimir Kramnik. 8 games. Ended in a draw. Infrmed Search: use heuristic functin guide t gal Greedy best-first search A* search / prvably ptimal Search space up t apprimately 0 5 Lcal search Greedy / Hillclimbing Simulated annealing Tabu search Genetic Algrithms / Genetic Prgramming search space 0 00 t Aversarial Search / Game Playing minima Up t ~0 0 ndes, 6 7 ply in chess. alpha-beta pruning Up t ~0 0 ndes, 4 ply in chess. prvably ptimal Why such a central rle? Search and AI Basically, because lts f tasks in AI are intractable. Search is nly way t handle them. Many applicatins f search, in e.g., Learning / Reasning / Planning / NLU / Visin Gd thing: much recent prgress (0 0 quite feasible; smetimes up t ). Qualitative difference frm nly a few years ag! 6