Optimal searching. Best-first search review Advantages

Transcription

1 Optimal searching Best-first search review Advantages Takes advantage f dmain infrmatin t guide search Greedy advance t the gal Disadvantages Cnsiders cst t the gal frm the current state Sme path can cntinue t lk gd accrding t the heuristic functin At this pint the path is mre cstly than the alternate path

2 Branch & Bund Use current cst (past cst) t a nde Pick best (lwest) cst. If f is ur evaluatin functin fr nde n, f(n)= g(n) [g= cst gne s far g 0 B&B: srt queue in rder f lwest f, & make sure nt t pursue identical paths with higher csts than knwn csts The A* Algrithm: cmbining past with future Nw Cnsider the verall cst f the slutin. f(n) = g(n) + h(n) where g(n) is the path cst t nde g= distance gne s far h= future estimated cst think f f(n) as an estimate f the cst f the best slutin ging thrugh the nde n 4 5

3 UCS, BFS, Best-First, and A* f = g + h A* Search h = 0 Unifrm cst search g =, h = 0 Breadth-First search g = 0 Best-First search Admissible Heuristics This is nt quite enugh, we als require h be admissible: a heuristic h is admissible if h(n) < h*(n) fr all ndes n, where h* is the actual cst f the ptimal path frm n t the gal Eamples: travel distance straight line distance must be shrter than actual travel path tiles ut f place each mve can rerder at mst ne tile distance f each ut f place tile frm the crrect place each mve mves a tile at mst ne place tward crrect place

4 Heuristic Functins Tic-tac-te 8 Puzzle Ehaustive Search : 0 = * 0 9 states Remve repeated state : 9! = 6,880 Use f heuristics h : # f tiles that are in the wrng psitin h : sum f Manhattan distance h = h = ++=

5 Heuristics : 8 Puzzle Tic-tac-te Mst-Win Heuristics win 4 win win

6 Rad Map Prblem s g(n ) n h(s) h(n ) n h(n) g Effect f Heuristic Accuracy n Perfrmance Well-designed heuristic have its branch clse t h dminates h iff h (n) h (n), n It is always better t use a heuristic functin with higher values, as lng as it des nt verestimate Inventing heuristic functins Cst f an eact slutin t a relaed prblem is a gd heuristic fr the riginal prblem cllectin f admissible heuristics h*(n) = ma(h (n), h (n),, h k (n))

7 Optimality f A* Let us assume that f is nn-decreasing alng each path if nt, simply use parent s value if that s the case, we can think f A* as epanding f cnturs tward the gal; better heuristics make this cntur mre eccentric Let G be an ptimal gal state with path cst f* Let G be a subptimal gal state with path cst g(g ) > f*. suppse A* picks G befre G (A* is nt ptimal) suppse n is a leaf nde n the path t G when G is chsen if h is admissible, then f* >= f(n) since n was nt chsen, it must be the case that f(n) >= G therefre f* >= f(g ), but since G is a gal, f* >= g(g ) But this is a cntradictin --- G is a better gal nde than G Thus, ur suppsitin is false and A* is ptimal. Cmpleteness f A* Suppse there is a gal state G with path cst f* Intuitively: since A* epands ndes in rder f increasing f, it must eventually epand nde G If A* stps and fails Prve by cntradictin that this is impssible. There eists a path frm the initial state t the nde state Let n be the last nde epanded alng the slutin path n has at least ne child, that child shuld be in the pen ndes A* des nt stp until there are pen list is empty (unless it finds a gal state). Cntradictin. A* is n an infinite path Recall that cst(s,s) > δ Let n be the last nde epanded alng the slutin path After f(n)/δ the cumulative cst f the path becmes large enugh that A* will epand n. Cntradictin.

8 Prperties f A* Suppse C* is the cst f the ptimal slutin path A* epands all ndes with f(n) < C* A* might epand sme f ndes with f(n) = C* n the gal cntur A* will epand n ndes with f(n) > C*, which are pruned! Pruning: eliminating pssibilities frm cnsideratin withut eaminatin A* is ptimally efficient fr any given heuristic functin nther ptimal algrithm is guaranteed t epand fewer ndes than A* an algrithm might miss the ptimal slutin if it des nt epand all ndes with f(n) < C* A* is cmplete

9 A* summary Cmpleteness prvided finite branching factr and finite cst per peratr Optimality prvided we use an admissible heuristic Time cmpleity wrst case is still O(b d ) in sme special cases we can d better fr a given heuristic Space cmpleity wrst case is still O(b d ) Finding heuristics: Rela Optimality (nte: this is nt required material) Gals: Minimizing search cst Satisficing slutin, i.e. bunded errr in the slutin f(s) = (-w) g(s) + w h(s) g can be thught f as the breadth first cmpnent w = => Best-First search w =.5 => A* search w = 0 => Unifrm search

10 Iterative Deepening A* Gals A strage efficient algrithm that we can use in practice Still cmplete and ptimal Mdificatin f A* use f-cst limit as depth bund increase threshld as minimum f f(.) f previus cycle Each iteratin epands all ndes inside the cntur fr current f-cst same rder f nde epansin Games Why games? Games prvide an envirnment f pure cmpetitin with bjective gals between agents. Game playing is cnsidered an intelligent human activity. The envirnment is deterministic and accessible. The set f peratrs is small and defined. Large state space Fun!

11 Games Cnsider Games Tw player games Perfect Infrmatin: nt invlving chance r hidden infrmatin (nt back-gammn, pker) Zer-sum games: games where ur gain is ur ppnents lss Eamples: tic-tac-te, checkers, chess, g Games f perfect infrmatin are really just search prblems initial state peratrs t generate new states gal test utility functin (win/lse/draw) Game Trees Tic-tac-te ply mve

12 Game Trees Eample win lse draw What s a gd mve? win lse draw

13 Game Trees Eample win lse draw Perfect decisins in - persn games Let s name the tw agents (players) MA and MIN MA is searching fr the highest utility state, s when it is MA s mve she will maimize the payff High utility fr MA is lw utility fr MIN, since it s a zer-sum game When it is MIN s mve she will minimize the payff The winning strategy is t maimize ver minimum payff mves.

14 Minima Algrithm Fr the MA player. Generate the game t terminal states. Apply the utility functin t the terminal states. Back-up values At MIN ply assign minimum payff mve At MA ply assign maimum payff mve 4. At rt, MA chses the peratr that led t the highest payff Minima Eample Tw-ply game Ma A A A Min Ma A A A A A A A A A

15 Minima Eample Tw-ply game Ma A A A Min A A A A A A A A A Ma Minima Eample Tw-ply game Ma A A A Min A A A A A A A A A Ma

16 Minima Eample Tw-ply game Ma A A A Min A A A A A A A A A Ma Minima Perfect play fr deterministic, perfectinfrmatin games Ttally impractical since it generates the whle tree Time cmpleity is O(b d )! Space cmpleity is O(b d )

17 The Cmpleity f Minima Fr a given game with branching factr b, searching t depth d require O(b d ) cmputatin and strage chess has a branching factr f arund 5 A -mve search tree fr chess has 5 leaves Say a typical chess game has 00 mves then the number f leaves in the tree is 5 00 = 0 54 Assuming a mdern cmputer can prcess bard psitins a secnd it will take 0 40 years t search the entire tree. g has a branching factr f 60 r mre Partial Search Tree In a real game, we can nly lk ahead a few ply! The depth f search is determined by the time allwed per mve. Suppse we can prcess psitins a secnd and we re allwed ne minute per mve, then we can search 5 ply

18 Minima Cutff Des it wrk in practice? Time cmpleity: O(b m ) Chess: b = 5 Suppse we limit ur search t.5 millin ndes per mve m = 4 4-ply chess player is a lusy player! 4-ply = nvice chess player 8-ply = typical PC, human master -ply = Deep Blue, Kasparv The Evaluatin Functin If we d nt reach the end f the game hw d we evaluate the payff f the leaf states? Use a static evaluatin functin. A heuristic functin that estimates the utility f bard psitins. Desirable prperties Must agree with the utility functin Must nt take t lng t evaluate Must accurately reflect the chance f winning An ideal evaluatin functin can be applied directly t the bard psitin. It is better t apply it as many levels dwn in the game tree as time permits. Eample evaluatin functins: Tic-Tac-Te: # ways t win Chess: value f white pieces/value f black pieces

19 Minima # ways t win heuristic Minima

20 Minima Revised Minima Algrithm Fr the MA player. Generate the game as deep as time permits. Apply the evaluatin functin t the leaf states. Back-up values At MIN ply assign minimum payff mve At MA ply assign maimum payff mve 4. At rt, MA chses the peratr that led t the highest payff

21 Cutting Off Search Because the evaluatin functin is nly an apprimatin it can misguide us. Eample: white appears t have the advantage, but black captures the queen in the net mve. Need t search ne mre ply Often, it makes sense t make depth dynamically decided quiescence search --- g until things seem stable Eample: in chess, dn t stp in psitins where capture mves are imminent Nnquiescent The Hrizn Prblem When a mve by the ppnent causes serius damage, but is ultimately unavidable. Eample: the pawn n the 7 th rw will be queened eventually. The prblem: the player can push this event ff beynd the search hrizn N knwn slutin t the hrizn prblem.

22 Alpha-beta Pruning Efficiency hack n tp f minima: gets same result, but fewer evaluatins Basic idea: keep track f yur best mve's value s far while perfrming minima search Fr Ma, that value is called alpha When Min is eamining its mves, and it gets ne back that is less than alpha (i.e., wrse fr Ma), then its parent, Ma, wuld nt make that mve because the mve that gave alpha is better. S Min can abandn this nde right nw befre eamining any mre mves frm it Ditt fr Min, but the best value s far is called beta (Min wants t make beta as small as pssible) When Ma is epanding its mves, if any are greater than beta (i.e., wrse fr Min) than it can stp early Starts with wrst pssible alpha (negative infinity) and beta (psitive infinity) Alpha-Beta Pruning Tw-ply game Eample Ma - Min Ma 8

23 Alpha-Beta Pruning Tw-ply game Eample Ma Min Ma 8 Alpha-Beta Pruning Tw-ply game Eample Ma Min Ma 8 When Min is eamining its mves, and it gets ne back that is less than alpha (i.e., wrse fr Ma), then its parent, Ma, wuld nt make that mve because the mve that gave alpha is better. S Min can abandn this nde right nw befre eamining any mre mves frm it

24 Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 Ma 8 4 Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 Ma 8 4 5

25 Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 5 Ma Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 5 Ma 8 4 5

26 Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 5 Ma Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 5 Ma 8 4 5

27 Alpha-Beta Pruning Tw-ply game Eample Ma Min 4 5 Ma Alpha-beta pruning Pruning des nt affect final result Alpha-beta pruning Gd mve rdering imprves effectiveness f pruning Asympttic time cmpleity O((b/lg b) d ) With perfect rdering, time cmpleity O(b d/ ) means we g frm an effective branching factr f b t sqrt(b) (e.g. 5 -> 6).

28 Cmpleity f Alpha- Beta Pruning Order the ndes s that best mves fr that player are investigated first tend t get alpha and beta t ptimal values faster s get mre pruning If a decent heuristic fr rdering mves can be fund-- Time cmpleity appraches O(b^(d/)) If mves are randmly rdered, then arund O(b^(d/4)) But these bth assume randmly distributed utilities need empirical wrk with real games Space cmpleity is O(bd) The same as ther depth-first searches Frm here n is ptinal material

29 α β Prcedure pseud-cde minima-α β(bard, depth, type, α, β) If depth = 0 return Eval-Fn(bard) else if type = ma cur-ma = -inf lp fr b in succ(bard) b-val = minima-α β(b,depth-,min, α, β) cur-ma = ma(b-val,cur-ma) α = ma(cur-ma, α) if cur-ma >= β finish lp return cur-ma else (type = min) cur-min = inf lp fr b in succ(bard) b-val = minima-α β(b,depth-,ma, α, β) cur-min = min(b-val,cur-min) β = min(cur-min, β) if cur-min <= α finish lp return cur-min Games That Include an Element f Chance Many games mirrr unpredictability by including a randm element E.g. backgammn

30 Game tree fr a backgammn Decisin Making in Game f Chance Chance ndes Branches leading frm each chance nde dente the pssible dice rlls Labeled with the rll and the chance that it will ccur Replace MA/MIN ndes in minima with epected MA/MIN payff Epectima value f C epectima( C) = P( d ) ma (, i i s S C d )( utility( s)) i Epectimin value epectimin( C) = P( d ) min (, i i s S C d )( utility( s)) i

31 Psitin evaluatin in games with chance ndes Fr minima, any rder-preserving transfrmatin f the leaf values des nt affect the chice f mve With chance nde, sme rder-preserving transfrmatins f the leaf values d affect the chice f mve Psitin evaluatin in games with chance ndes (cnt d) The behavir f the algrithm is sensitive even t a linear transfrmatin f the evaluatin functin.

32 Cmpleity f epectiminima The epectiminima cnsiders all the pssible dicerll sequences It takes O(b m n m ) where n is the number f distinct rlls Whereas, minima takes O(b m ) Prblems The etra cst cmpared t minima is very high Alpha-beta pruning is mre difficult t apply State-f-the-Art fr Chess Prgrams Chess basics 88 bard, 6 pieces per side, average branching factr f abut 5 Rating system based n cmpetitin beginner/legal gd weekend warrir wrld champinship level grand master time limited mves pen and clsing bks available imprtant aspects: psitin, material

33 Chess Ratings Sketch f Chess Histry First discussed by Shannn, Sci. American, 950 Initially, tw appraches human-like brute frce search 966 MacHack average turnament player 970 s discvery that ply = 00 rating pints hash tables quiescence search Chess 4. reaches 000 (epert level), 979 Belle 00, 98 special purpse hardware Cray Blitz and Hitech 00,000 t 0,000 psitin/sec using special purpse hardware

34 IBM checks in Deep thught: 50 chips (M ps/sec /// 6-7M ps/sc) Evaluatin hardware piece placement pawn placement passed pawn eval file cnfiguratins 0 parameters t tune Tuning dne t master s games hill climbing and linear fits rating f 480 === Kasparv beats IBM Ups the Ante Deep Blue is the net generatin parallel versin f deep thught 00 M ps/sec 60B psitins in the minutes alltted fr mve DB = Rs/6000 s with 6 chess prc/nde DB = faster ndes w 8 chess prc/nde (56 prc) message passing architecture search as much as 0-0 levels deep using sing. etensin In 997, Kasparv beaten Kasparv changed strategy in earlier games As much a psychlgical as mental victry

35 Chess Prgrams Tday Deep Blue dismantled --- leaves vid in the wrld f chess prgrams Deep Junir Deep Fritz A cmmercial prduct Pentium III dual prcessing 9 MHz cmputers Analyze 6 millin mves per secnd As strng as Deep Blue Man vs. Machine, Bahrain, Octber 00 Vladimir Kramnik Deep Fritz = = = = = = 8 = = Final 4 4 State-f-the-art fr Checkers Prgrams Checker Arthur Samuel (95) fficial wrld champin Chink Uses etensive mve database

36 State-f-the-art fr Backgammn Prgrams Use a tempral differencing algrithm t train a neural netwrk Strngest Prgrams: TD-GAMMON by Gary Tesaur f IBM, Jellyfish Achieve epert level play State-f-the-art fr Othell Prgrams Prgrams strnger than human players Prgrams use learning techniques t fine-tune the evaluatin functin, the pening bk, and even the search algrithm Strngest prgrams: Lgistell, Hannibal

37 State-f-the-art fr GO Prgrams Branching factr f GO abut 60 Humans lead by a huge margin Many, many prgrams Frm recent G Ladder cmpetitin: G4++, Many Faces f G, Eg, NeurG II, Eplrer, Indig, Glis, Gnu G, Gbble, gttag, Pka, Viking, GLife I, The Turtle, Gg, GL7 State-f-the-art fr Pker Prgrams Pki (University f Alberta) is prbably the strngest pker prgram Nt clse t wrld-class level