CSC242: Intro to AI. Lecture 8. Tuesday, February 26, 13

Transcription

1 CSC242: Intro to AI Lecture 8

2 Quiz 2 Review

3 TA Help Sessions (v2) Monday & Tuesday: 17:00-18:00, Hylan 301 Doodle poll signup before 16:00 Link on BB:

4 Stochastic and Partially Observable Games

5

6 Review

7 Games Toy Problems 9! = =

8 Games Require the ability to make some decision even when calculating the optimal decision is infeasible Penalize inefficiency severely

9 Types of Games Deterministic (no chance) Nondeterministic (dice, cards, etc.) Perfect information (fully observable) Imperfect information (partially observable) Zero-sum (total payoff the same in any game) Arbitrary utility functions

10 Minimax Algorithm Minimax(s) = 8 >< Utility(s) max a2actions(s) Minimax(Result(s, a)) >: min a2actions(s) Minimax(Result(s, a)) if Terminal-Test(s) if Player(s) =max if Player(s) =min

11 Minimax Summary Computes the optimal move assuming opponent also plays optimally (i.e., worstcase outcome) Explores game tree depth-first all the way to terminal states (end of game) Backs up utility values through alternating MIN and MAX (what s best for me is worst for you, and vice-versa)

12 Heuristic Minimax Cutoff search before reaching terminal nodes (time, depth, quiescence ) Use heuristic evaluation function to estimate state utility Backs up utility values through alternating MIN and MAX (what s best for me is worst for you, and vice-versa)

13 Heuristic Minimax H-Minimax(s) = 8 >< h(s) max a2actions(s) Minimax(Result(s, a)) >: min a2actions(s) Minimax(Result(s, a)) if Cutoff-Test(s) if Player(s) =max if Player(s) =min

14 Alpha-Beta Summary Easy bookkeeping modification of basic MINIMAX algorithm Not hard to come up with useful node orderings Even random gets you 33% deeper search Works with other ways of improving game tree search

15 Stochastic and Partially Observable Games

16 Types of Games Deterministic (no chance) Nondeterministic (dice, cards, etc.) Perfect information (fully observable) Imperfect information (partially observable) Zero-sum (total payoff the same in any game) Arbitrary utility functions

17 Non-deterministic (Stochastic) Games A player s possible moves depend on chance (random) elements, e.g., dice

18 Stochastic Games

19 Stochastic Games A player s possible moves depend on chance (random) elements, e.g., dice Can t build game tree since don t know what future legal moves will be

20 MAX CHANCE MIN B 1/36 1,1 1/18 1, /18 1/36 6,5 6,6 CHANCE C... MAX 1/36 1,1 1/18 1,2 1/18 1/36 6,5 6,6... TERMINAL

21 MAX CHANCE MIN B 1/36 1,1 1/18 1, /18 1/36 6,5 6,6 CHANCE C... MAX 1/36 1,1 1/18 1,2 1/18 1/36 6,5 6,6... TERMINAL

22 ? P(H)=0.5 H T P(T)=

23 ? P(H)=0.5 H T P(T)=

24 0 P(H)=0.5 H T P(T)=

25 ? P(H)=0.75 H T P(T)=

26 +0.5 P(H)=0.75 H T P(T)=

27 Expectation Weighted average of possibilities Sum of the possible outcomes weighted by the likelihood of their occurrence What you would expect to win in the long run

28 Expecti-Minimax Same as MINIMAX for MIN and MAX nodes Same backing up utilities from terminal nodes Take expectation over chance nodes Weighted average of possible outcomes

29 MAX CHANCE MIN B 1/36 1,1 1/18 1, /18 1/36 6,5 6,6 CHANCE C... MAX 1/36 1,1 1/18 1,2 1/18 1/36 6,5 6,6... TERMINAL

30 Expecti-Minimax EMinimax(s) = 8 Utility(s) >< max a EMinimax(Result(S, a)) min >: a EMinimax(Result(S, a)) P P (r)eminimax(result(s, r)) r if Terminal-Test(s) if Player(s) =max if Player(s) =min if Player(s) =chance

31 MAX CHANCE MIN B 1/36 1,1 1/18 1, /18 1/36 6,5 6,6 CHANCE C... MAX 1/36 1,1 1/18 1,2 1/18 1/36 6,5 6,6... TERMINAL

32 Expecti-Minimax O(b m n m )

33 Stochastic Games Expectation to handle uncertainty and randomness For example in poker: Pot Odds

34 Partial Observability Some of the state of the world is hidden (unobservable) There is some uncertainty about the state of the world

35 Partially-Observable Games Some of the state of the game is hidden from the player(s) Interesting because: Valuable real-world games (e.g., poker) Partial observability arises all the time in real-world problems

36 Partially-Observable Games Deterministic partial observability Opponent has hidden state Battleship, Stratego, Kriegspiel

37 Partially-Observable Games Deterministic partial observability Opponent has hidden state Battleship, Stratego, Kriegspiel Stochastic partial observability Missing/hidden information is random Card games: bridge, hearts, poker (most)

38 Stochastic Partially Observable Games

39

40 Weighted Minimax For each possible deal s: Assume s is the actual situation Compute Minimax or H-Minimax value of s Weight value by probability of s Take move that yields highest expected value over all the possible deals

41 Weighted Minimax X argmax a s P (s)minimax(result(s, a))

42 Weighted Minimax X argmax a s P (s)minimax(result(s, a)) 26 = 10, 400, =

43 Monte Carlo Methods Use a representative sample to approximate a large, complex distribution

44 Monte Carlo Minimax 1 NX argmax a N i=1 Minimax(Result(s i,a))

45 Summary Non-deterministic games Expecti-MINIMAX: Compute expected MINIMAX value over chance nodes Partially observable games Weighted MINIMAX: Compute expected value over possible hidden states Naive approaches impractical (but stay tuned)

46 For Next Time: AIMA