CS 188: Artificial Intelligence Fall AI Applications

Transcription

1 CS 188: Artificial Intelligence Fall 2009 Lecture 27: Conclusion 12/3/2009 Dan Klein UC Berkeley AI Applications 2 1

2 Pacman Contest Challenges: Long term strategy Multiple agents Adversarial utilities Uncertainty about other agents positions, plans, etc 4 Pacman Contest 60 submissions (over half the class!) Creative names: BalsamicVinegarOfJustice, BigUtilusMaximusAgents, Creative methods: tracking, learning, search 45 qualifiers (a third of the class!) Amazing work by everyone! Final Tournament Double elimination seeded by round-robin Required 15 CPUs for almost a day Final matches: now! 5 2

3 For Third Place Seed 5: NarwhalAgents Kevin Brackbill and Tyler Latzke Probabilistic tracking. Directed expectimax search from imputed game state using a linear feature function. MST for search, dead-end modeling. Really good at beating agents ranked ahead of them. Seed 4: Ducks Nils Reimers Probabilistic tracking. One offense agent that uses minimax in tactical situations. One defense agent that waits in the center of the board and predicts enemy crossing points. 6 For First / Second Place Seed 2: BallerAgents Nick Fraioli and Haotian Bai Blitz! One takes the top, one takes the bottom. Ignores ghosts they can t see, goes after enemies if convenient. Sets food targets with search. Who needs tracking? Seed 4: Ducks Nils Reimers Probabilistic tracking. One offense agent that uses minimax in tactical situations. One defense agent that waits in the center of the board and predicts enemy crossing points. 7 3

4 But Wait! Seed 1: Chris Berner Zelam Ngo Central planner moves agents between offense reflex and defense reflex based on how the game is going Probabilistic tracking with learned transition model for enemies (plan recognition), uses dots in addition to sonar Qualifying Agents Balanced offense / defense agents with basic tracking and avoidance. Seed 3: Andrea Goh (Skynet) Central planner that switches agent roles, assigns target dots, etc. Computes when going into dead ends is guaranteed to be safe Scary AI from the future 8 Results Double Elimination 1 st place: Nils Reimer 2 nd place: Nick / Haotian 3 rd place: Kevin / Tyler Round Robin 1 st Place: Chris Berner / Zelam Ngo 2 nd Place: Nick / Haotian 3 rd Place: Andrea Goh Combined Results 1 st place:, Nils, Chris / Z 2 nd place: Nick / Haotian 3 rd place: Kevin / Tyler, Andrea and congratulations to all! 9 4

5 and Congratulations to All! Amazing work by everyone Record number of entries (60 teams) Record number of qualifications (45!) Lots of mutual support on newsgroup / office hours You should all be proud of what you ve accomplished! 10 Example: Stratagus [DEMO] 5

6 Stratagus Stratagus (similar to Starcraft, etc): example of a large RL task Stratagus is hard for reinforcement learning algorithms > states > actions at each point Time horizon 10 4 steps Stratagus is hard for human programmers Typically takes several person-months for game companies to write computer opponent Still, no match for experienced human players, very fragile Programming involves much trial and error Hierarchical RL Humans break up the task into a multi-level sketch using a partial program Learning algorithm fills in the details [From Bhaskara Marthi s thesis, Berkeley] 12 Solution: Hierarchical Learning (defun top () (loop (choose (gather-wood) (gather-gold)))) (defun gather-wood () (with-choice (dest *forest-list*) (nav dest) (action get-wood) (nav *base-loc*) (action dropoff))) (defun gather-gold () (with-choice (dest *goldmine-list*) (nav dest)) (action get-gold) (nav *base-loc*)) (action dropoff))) (defun nav (dest) (until (= (pos (get-state)) dest) (with-choice (move (N S E W NOOP)) (action move)))) 13 6

7 Hierarchical RL Solution: hierarchical planning and learning Define a hierarchical MDP Each level has Q-functions (one per choice) Learning happens at all levels, all at once State-of-the-art Still not very good at the strategic elements (high level strategy) Very good at balancing resources (mid-level allocation) Excellent at lowest levels of control [DEMO] 14 Pacman: Beyond Simulation? [DEMO] Students at Colorado University: 7

8 AI = Animal Intelligence? Wim van Eck at Leiden University Pacman controlled by a human Ghosts controlled by crickets Vibrations drive crickets toward or away from Pacman s location Bugman? [DEMO] 17 Where to go next? Congratulations, you ve seen the basics of modern AI and done some amazing work putting it to use! How to continue: Robotics / vision / IR / language: cs189 Machine learning: cs281a / cs281b Cognitive modeling: cog sci 131 Vision: cs280 Robotic: cs287 NLP: cs288 Starcraft competition and more; ask if you re interested 18 8

9 That s It! Help us out with some course evaluations Have a good break, and always maximize your expected utilities! 19 9