How Much Memory is Needed to Win in Partial-Observation Games
|
|
|
- Toby McCarthy
- 5 years ago
- Views:
Transcription
1 How Much Memory is Needed to Win in Partial-Observation Games Laurent Doyen LSV, ENS Cachan & CNRS & Krishnendu Chatterjee IST Austria GAMES 11
2 How Much Memory is Needed to Win in Partial-Observation Games Laurent Doyen LSV, ENS Cachan & CNRS stochastic & Krishnendu Chatterjee IST Austria GAMES 11
3 Examples Poker - partial-observation - stochastic
4 Examples Poker - partial-observation - stochastic Bonneteau
5 2 black card, 1 red card Bonneteau Initially, all are face down Goal: find the red card
6 2 black card, 1 red card Bonneteau Initially, all are face down Goal: find the red card Rules: 1. Player 1 points a card 2. Player 2 flips one remaining black card 3. Player 1 may change his mind, wins if pointed card is red
7 2 black card, 1 red card Bonneteau Initially, all are face down Goal: find the red card Rules: 1. Player 1 points a card 2. Player 2 flips one remaining black card 3. Player 1 may change his mind, wins if pointed card is red
8 2 black card, 1 red card Bonneteau Initially, all are face down Goal: find the red card Rules: 1. Player 1 points a card 2. Player 2 flips one remaining black card 3. Player 1 may change his mind, wins if pointed card is red
9 Bonneteau: Game Model
10 Bonneteau: Game Model
11 Game Model
12 Game Model
13 Game Model
14 Game Model
15 Game Model
16 Observations (for player 1)
17 Observations (for player 1)
18 Observations (for player 1)
19 Observations (for player 1)
20 Observations (for player 1)
21 Observation-based strategy This strategy is observation-based, e.g. after it plays
22 Observation-based strategy This strategy is observation-based, e.g. after it plays
23 Optimal observation-based strategy This strategy is winning with probability 2/3
24 Game Model This game is: turn-based (almost) non-stochastic player 2 has perfect observation
25 Interaction General case: concurrent & stochastic Player 1 s move Player 2 s move Players choose their moves simultaneously and independently
26 Interaction General case: concurrent & stochastic Player 1 s move Player 2 s move Probability distribution on successor state -player games
27 Interaction Special cases: Turn-based games player-1 state player-2 state
28 Partial-observation Observations: partitions induced by coloring General case: 2-sided partial observation Two partitions and
29 Partial-observation Observations: partitions induced by coloring General case: 2-sided partial observation Two partitions and Player 1 s view Player 2 s view
30 Partial-observation Observations: partitions induced by coloring Special case: 1-sided partial observation or
31 Strategies & objective A strategy for Player is a function that maps histories (sequences of observations) to probability distribution over actions.
32 Strategies & objective A strategy for Player is a function that maps histories (sequences of observations) to probability distribution over actions. History-depedent randomized
33 Strategies & objective A strategy for Player is a function that maps histories (sequences of observations) to probability distribution over actions. Reachability objective: Winning probability:
34 Qualitative analysis The following problem is undecidable: (already for probabilistic automata [Paz71]) Decide if there exists a strategy for player 1 that is winning with probability at least 1/2
35 Qualitative analysis The following problem is undecidable: (already for probabilistic automata [Paz71]) Decide if there exists a strategy for player 1 that is winning with probability at least 1/2 Qualitative analysis: Almost-sure: winning with probability 1 Positive: winning with probability > 0
36 Example 1 Player 1 partial, player 2 perfect
37 Example 1 Player 1 partial, player 2 perfect No pure strategy of Player 1 is winning with probability 1
38 Example 1 Player 1 partial, player 2 perfect No pure strategy of Player 1 is winning with probability 1
39 Example 1 Player 1 partial, player 2 perfect Player 1 wins with probability 1, and needs randomization Belief-based-only randomized strategies are sufficient
40 Example 2 Player 1 partial, player 2 perfect
41 Example 2 Player 1 partial, player 2 perfect To win with probability 1, player 1 needs to observe his own actions. Randomized action-visible strategies:
42 Classes of strategies rand. action-visible rand. action-invisible Classification according to the power of strategies pure
43 Classes of strategies rand. action-visible rand. action-invisible Classification according to the power of strategies pure Poly-time reduction from decision problem of rand. act.-vis. to rand. act.-inv. The model of rand. act.-inv. is more general
44 Classes of strategies rand. action-visible rand. action-invisible Classification according to the power of strategies pure Computational complexity (algorithms) Strategy complexity (memory)
45 Known results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (belief) [GS 09] exponential (belief) [GS 09] pure???
46 Known results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (belief) [CDHR 06(remark), GS 09] exponential (belief) [GS 09] pure??? [BGG09] Bertrand, Genest, Gimbert. Qualitative Determinacy and Decidability of Stochastic Games with Signals. LICS 09. [CDHR06] Chatterjee, Doyen, Henzinger, Raskin. Algorithms for ω-regular games with Incomplete Information. CSL 06. [GS09] Gripon, Serre. Qualitative Concurrent Stochastic Games with Imperfect Information. ICALP 09.
47 Known results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (belief) [CDHR 06(remark), GS 09] exponential (belief) [GS 09] pure??? Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure???
48 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect
49 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect
50 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect not winning
51 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect not winning
52 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect Neither is winning!
53 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect
54 When belief fails (1/2) Belief-based-only pure strategies are not sufficient, both for positive and for almost-sure winning player 1 partial player 2 perfect This strategy is almost-sure winning!
55 When belief fails (2/2) Using the trick of repeated actions we construct an example where belief-only randomized action-invisible strategies are not sufficient (for almost-sure winning) player 1 partial player 2 perfect
56 When belief fails (2/2) Using the trick of repeated actions we construct an example where belief-only randomized action-invisible strategies are not sufficient (for almost-sure winning) player 1 partial player 2 perfect
57 When belief fails (2/2) Using the trick of repeated actions we construct an example where belief-only randomized action-invisible strategies are not sufficient (for almost-sure winning) player 1 partial player 2 perfect Almost-sure winning requires to play pure strategy, with more-than-belief memory!
58 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (belief) [CDHR 06(remark), GS 09] exponential (belief) [GS 09] pure??? Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure???
59 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (more than belief) exponential (belief) [GS 09] pure exponential (more than belief)?? Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure exponential (more than belief)??
60 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (more than belief) exponential (belief) [GS 09] pure exponential (more than belief)?? Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure exponential (more than belief)??
61 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (more than belief) exponential (belief) [GS 09] pure exponential (more than belief) non-elementary complete? Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure exponential (more than belief) non-elementary complete?
62 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (more than belief) exponential (belief) [GS 09] pure exponential (more than belief) non-elementary complete? Positive rand. act.-vis. player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial Player 1 wins from more states, but needs more memory! rand. act.-inv. pure exponential (more than belief) non-elementary complete?
63 Player 1 perfect, player 2 partial Memory of non-elementary size for pure strategies lower bound: simulation of counter systems with increment and division by 2 upper bound: positive: non-elementary counters simulate randomized strategies almost-sure: reduction to iterated positive Counter systems with {+1, 2} require nonelementary counter value for reachability
64 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (more than belief) exponential (belief) [GS 09] pure exponential (more than belief) non-elementary complete finite (at least nonelementary) Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure exponential (more than belief) non-elementary complete finite (at least nonelementary)
65 Player 1 perfect, player 2 partial Equivalence of the decision problems for almost-sure reach with pure strategies and rand. act.-inv. strategies Reduction of rand. act.-inv. to pure choice of a subset of actions (support of prob. dist.) Reduction of pure to rand. act.-inv. repeated-action trick (holds for almost-sure only) It follows that the memory requirements for pure hold for rand. act.-inv. as well!
66 New results Reachability - Memory requirement (for player 1) Almost-sure player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. exponential (belief) [CDHR 06] [BGG 09] exponential (belief) [BGG 09] rand. act.-inv. exponential (more than belief) finite (at least nonelementary) pure exponential (more than belief) non-elementary complete finite (at least nonelementary) Positive player 1 partial player 2 perfect player 1 perfect player 2 partial 2-sided both partial rand. act.-vis. rand. act.-inv. pure exponential (more than belief) non-elementary complete finite (at least nonelementary)
67 Summary of our results Pure strategies (for almost-sure and positive): player 1 partial: exponential memory, more than belief player 1 perfect: non-elementary memory (complete) 2-sided: finite, at least non-elementary memory Randomized action-invisible strategies (for almost-sure) : player 1 partial: exponential memory, more than belief 2-sided: finite, at least non-elementary memory
68 More results & open questions Computational complexity for 1-sided: Player 1 partial: reduction to Büchi game, EXPTIME-complete Player 2 partial: non-elementary complexity Open questions: Whether non-elementary size memory is sufficient in 2-sided Exact computational complexity
69 Details Details can be found in: [CD11] Chatterjee, Doyen. Partial-Observation Stochastic Games: How to Win when Belief Fails. CoRR abs/ , July 2011.
70
71
72 References Details can be found in: [CD11] Chatterjee, Doyen. Partial-Observation Stochastic Games: How to Win when Belief Fails. CoRR abs/ , July Other references: [BGG09] Bertrand, Genest, Gimbert. Qualitative Determinacy and Decidability of Stochastic Games with Signals. LICS 09. [CDHR06] Chatterjee, Doyen, Henzinger, Raskin. Algorithms for ω-regular games with Incomplete Information. CSL 06. [GS09] Gripon, Serre. Qualitative Concurrent Stochastic Games with Imperfect Information. ICALP 09. [Paz71] Paz. Introduction to Probabilistic Automata. Academic Press 1971.
Qualitative Determinacy and Decidability of Stochastic Games with Signals
Qualitative Determinacy and Decidability of Stochastic Games with Signals INRIA, IRISA Rennes, France nathalie.bertrand@irisa.fr Nathalie Bertrand, Blaise Genest 2, Hugo Gimbert 3 2 CNRS, IRISA Rennes,
Qualitative Determinacy and Decidability of Stochastic Games with Signals
Qualitative Determinacy and Decidability of Stochastic Games with Signals 1 INRIA, IRISA Rennes, France nathalie.bertrand@irisa.fr Nathalie Bertrand 1, Blaise Genest 2, Hugo Gimbert 3 2 CNRS, IRISA Rennes,
Some Complexity Results for Subclasses of Stochastic Games
Some Complexity Results for Subclasses of Stochastic Games Krishnendu Chatterjee Workshop on Stochastic Games, Singapore, Nov 30, 2015 Krishnendu Chatterjee 1 Stochastic Games This talk glimpse of two
Introduction to Game Theory
Introduction to Game Theory (From a CS Point of View) Olivier Serre Serre@irif.fr IRIF (CNRS & Université Paris Diderot Paris 7) 14th of September 2017 Master Parisien de Recherche en Informatique Who
The Complexity of Request-Response Games
The Complexity of Request-Response Games Krishnendu Chatterjee 1, Thomas A. Henzinger 1, and Florian Horn 1,2 1 IST (Institute of Science and Technology), Austria {krish.chat,tah}@ist.ac.at 2 LIAFA, CNRS
Incidents Elie Bursztein and Jean Goubault Larrecq
A Logical Framework for Anticipation of Network Incidents and Jean Goubault Larrecq PhdStudent LSV ENS CACHAN CNRS INRIA DGA Outline Introduction Network Evolution Attack Model Evolution Anticipation game
Some recent results and some open problems concerning solving infinite duration combinatorial games. Peter Bro Miltersen Aarhus University
Some recent results and some open problems concerning solving infinite duration combinatorial games Peter Bro Miltersen Aarhus University Purgatory Mount Purgatory is on an island, the only land in the
Dynamic Programming in Real Life: A Two-Person Dice Game
Mathematical Methods in Operations Research 2005 Special issue in honor of Arie Hordijk Dynamic Programming in Real Life: A Two-Person Dice Game Henk Tijms 1, Jan van der Wal 2 1 Department of Econometrics,
COSE312: Compilers. Lecture 5 Lexical Analysis (4)
COSE312: Compilers Lecture 5 Lexical Analysis (4) Hakjoo Oh 2017 Spring Hakjoo Oh COSE312 2017 Spring, Lecture 5 March 20, 2017 1 / 20 Part 3: Automation Transform the lexical specification into an executable
Practice Session 2. HW 1 Review
Practice Session 2 HW 1 Review Chapter 1 1.4 Suppose we extend Evans s Analogy program so that it can score 200 on a standard IQ test. Would we then have a program more intelligent than a human? Explain.
LECTURE 26: GAME THEORY 1
15-382 COLLECTIVE INTELLIGENCE S18 LECTURE 26: GAME THEORY 1 INSTRUCTOR: GIANNI A. DI CARO ICE-CREAM WARS http://youtu.be/jilgxenbk_8 2 GAME THEORY Game theory is the formal study of conflict and cooperation
Timed Games UPPAAL-TIGA. Alexandre David
Timed Games UPPAAL-TIGA Alexandre David 1.2.05 Overview Timed Games. Algorithm (CONCUR 05). Strategies. Code generation. Architecture of UPPAAL-TIGA. Interactive game. Timed Games with Partial Observability.
CS510 \ Lecture Ariel Stolerman
CS510 \ Lecture04 2012-10-15 1 Ariel Stolerman Administration Assignment 2: just a programming assignment. Midterm: posted by next week (5), will cover: o Lectures o Readings A midterm review sheet will
Advanced Automata Theory 4 Games
Advanced Automata Theory 4 Games Frank Stephan Department of Computer Science Department of Mathematics National University of Singapore fstephan@comp.nus.edu.sg Advanced Automata Theory 4 Games p. 1 Repetition
Multiplayer Pushdown Games. Anil Seth IIT Kanpur
Multiplayer Pushdown Games Anil Seth IIT Kanpur Multiplayer Games we Consider These games are played on graphs (finite or infinite) Generalize two player infinite games. Any number of players are allowed.
Lemmas on Partial Observation, with Application to Phantom Games
Lemmas on Partial Observation, with Application to Phantom Games F Teytaud and O Teytaud Abstract Solving games is usual in the fully observable case The partially observable case is much more difficult;
A game-based model for human-robots interaction
A game-based model for human-robots interaction Aniello Murano and Loredana Sorrentino Dipartimento di Ingegneria Elettrica e Tecnologie dell Informazione Università degli Studi di Napoli Federico II,
CMU-Q Lecture 20:
CMU-Q 15-381 Lecture 20: Game Theory I Teacher: Gianni A. Di Caro ICE-CREAM WARS http://youtu.be/jilgxenbk_8 2 GAME THEORY Game theory is the formal study of conflict and cooperation in (rational) multi-agent
Algorithmique appliquée Projet UNO
Algorithmique appliquée Projet UNO Paul Dorbec, Cyril Gavoille The aim of this project is to encode a program as efficient as possible to find the best sequence of cards that can be played by a single
How to Gamble Against All Odds
How to Gamble Against All Odds Gilad Bavly 1 Ron Peretz 2 1 Bar-Ilan University 2 London School of Economics Heidelberg, June 2015 How to Gamble Against All Odds 1 Preface starting with an algorithmic
Fictitious Play applied on a simplified poker game
Fictitious Play applied on a simplified poker game Ioannis Papadopoulos June 26, 2015 Abstract This paper investigates the application of fictitious play on a simplified 2-player poker game with the goal
Math 152: Applicable Mathematics and Computing
Math 152: Applicable Mathematics and Computing May 8, 2017 May 8, 2017 1 / 15 Extensive Form: Overview We have been studying the strategic form of a game: we considered only a player s overall strategy,
THE GAMES OF COMPUTER SCIENCE. Topics
THE GAMES OF COMPUTER SCIENCE TU DELFT Feb 23 2001 Games Workshop Games Workshop Peter van Emde Boas ILLC-FNWI-Univ. of Amsterdam References and slides available at: http://turing.wins.uva.nl/~peter/teaching/thmod00.html
Hiding Actions in Multi-Player Games
Hiding Actions in Multi-Player Games Vadim Malvone Università degli Studi di Napoli Federico II, Italy vadim.malvone@unina.it Aniello Murano Università degli Studi di Napoli Federico II, Italy murano@na.infn.it
Three Pile Nim with Move Blocking. Arthur Holshouser. Harold Reiter.
Three Pile Nim with Move Blocking Arthur Holshouser 3600 Bullard St Charlotte, NC, USA Harold Reiter Department of Mathematics, University of North Carolina Charlotte, Charlotte, NC 28223, USA hbreiter@emailunccedu
Lecture 19 November 6, 2014
6.890: Algorithmic Lower Bounds: Fun With Hardness Proofs Fall 2014 Prof. Erik Demaine Lecture 19 November 6, 2014 Scribes: Jeffrey Shen, Kevin Wu 1 Overview Today, we ll cover a few more 2 player games
Anavilhanas Natural Reserve (about 4000 Km 2 )
Anavilhanas Natural Reserve (about 4000 Km 2 ) A control room receives this alarm signal: what to do? adversarial patrolling with spatially uncertain alarm signals Nicola Basilico, Giuseppe De Nittis,
Extensive Form Games. Mihai Manea MIT
Extensive Form Games Mihai Manea MIT Extensive-Form Games N: finite set of players; nature is player 0 N tree: order of moves payoffs for every player at the terminal nodes information partition actions
Unit-III Chap-II Adversarial Search. Created by: Ashish Shah 1
Unit-III Chap-II Adversarial Search Created by: Ashish Shah 1 Alpha beta Pruning In case of standard ALPHA BETA PRUNING minimax tree, it returns the same move as minimax would, but prunes away branches
First Cycle Games. Benjamin Aminof (IST Austria) and Sasha Rubin (TU Wien) Strategic Reasoning /20
First Cycle Games Benjamin Aminof (IST Austria) and Sasha Rubin (TU Wien) Strategic Reasoning 2014 1/20 Games in computer science Examples geography, parity games, mean-payoff games, energy games,... Types
2. The Extensive Form of a Game
2. The Extensive Form of a Game In the extensive form, games are sequential, interactive processes which moves from one position to another in response to the wills of the players or the whims of chance.
Domination Rationalizability Correlated Equilibrium Computing CE Computational problems in domination. Game Theory Week 3. Kevin Leyton-Brown
Game Theory Week 3 Kevin Leyton-Brown Game Theory Week 3 Kevin Leyton-Brown, Slide 1 Lecture Overview 1 Domination 2 Rationalizability 3 Correlated Equilibrium 4 Computing CE 5 Computational problems in
Alternation in the repeated Battle of the Sexes
Alternation in the repeated Battle of the Sexes Aaron Andalman & Charles Kemp 9.29, Spring 2004 MIT Abstract Traditional game-theoretic models consider only stage-game strategies. Alternation in the repeated
Artificial Intelligence. Minimax and alpha-beta pruning
Artificial Intelligence Minimax and alpha-beta pruning In which we examine the problems that arise when we try to plan ahead to get the best result in a world that includes a hostile agent (other agent
Lecture 20 November 13, 2014
6.890: Algorithmic Lower Bounds: Fun With Hardness Proofs Fall 2014 Prof. Erik Demaine Lecture 20 November 13, 2014 Scribes: Chennah Heroor 1 Overview This lecture completes our lectures on game characterization.
SUPPOSE that we are planning to send a convoy through
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART B: CYBERNETICS, VOL. 40, NO. 3, JUNE 2010 623 The Environment Value of an Opponent Model Brett J. Borghetti Abstract We develop an upper bound for
Search In Combinatorial Spaces: Fun with puzzles and games
Search In Combinatorial Spaces: Fun with puzzles and games Bart Massey (Physics '87) Asst. Prof. Computer Science Portland State University bart@cs.pdx.edu Why puzzles and games? Vernor Vinge: Usenix 2005
Game theory lecture 5. October 5, 2013
October 5, 2013 In normal form games one can think that the players choose their strategies simultaneously. In extensive form games the sequential structure of the game plays a central role. In this section
CS 4700: Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence Fall 2017 Instructor: Prof. Haym Hirsh Lecture 10 Today Adversarial search (R&N Ch 5) Tuesday, March 7 Knowledge Representation and Reasoning (R&N Ch 7)
Topic 1: defining games and strategies. SF2972: Game theory. Not allowed: Extensive form game: formal definition
SF2972: Game theory Mark Voorneveld, mark.voorneveld@hhs.se Topic 1: defining games and strategies Drawing a game tree is usually the most informative way to represent an extensive form game. Here is one
16.410/413 Principles of Autonomy and Decision Making
16.10/13 Principles of Autonomy and Decision Making Lecture 2: Sequential Games Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology December 6, 2010 E. Frazzoli (MIT) L2:
Solving Coup as an MDP/POMDP
Solving Coup as an MDP/POMDP Semir Shafi Dept. of Computer Science Stanford University Stanford, USA semir@stanford.edu Adrien Truong Dept. of Computer Science Stanford University Stanford, USA aqtruong@stanford.edu
Combinatorics and Intuitive Probability
Chapter Combinatorics and Intuitive Probability The simplest probabilistic scenario is perhaps one where the set of possible outcomes is finite and these outcomes are all equally likely. A subset of the
Game Theory and Randomized Algorithms
Game Theory and Randomized Algorithms Guy Aridor Game theory is a set of tools that allow us to understand how decisionmakers interact with each other. It has practical applications in economics, international
Adversarial Search. CS 486/686: Introduction to Artificial Intelligence
Adversarial Search CS 486/686: Introduction to Artificial Intelligence 1 Introduction So far we have only been concerned with a single agent Today, we introduce an adversary! 2 Outline Games Minimax search
Game Theory Lecturer: Ji Liu Thanks for Jerry Zhu's slides
Game Theory ecturer: Ji iu Thanks for Jerry Zhu's slides [based on slides from Andrew Moore http://www.cs.cmu.edu/~awm/tutorials] slide 1 Overview Matrix normal form Chance games Games with hidden information
Dynamic Games: Backward Induction and Subgame Perfection
Dynamic Games: Backward Induction and Subgame Perfection Carlos Hurtado Department of Economics University of Illinois at Urbana-Champaign hrtdmrt2@illinois.edu Jun 22th, 2017 C. Hurtado (UIUC - Economics)
Reading Robert Gibbons, A Primer in Game Theory, Harvester Wheatsheaf 1992.
Reading Robert Gibbons, A Primer in Game Theory, Harvester Wheatsheaf 1992. Additional readings could be assigned from time to time. They are an integral part of the class and you are expected to read
The tenure game. The tenure game. Winning strategies for the tenure game. Winning condition for the tenure game
The tenure game The tenure game is played by two players Alice and Bob. Initially, finitely many tokens are placed at positions that are nonzero natural numbers. Then Alice and Bob alternate in their moves
Layout design III. Chapter 6. Layout generation MCRAFT BLOCPLAN LOGIC
Layout design III. Chapter 6 Layout generation MCRAFT BLOCPLAN LOGIC Methods for layout design Layout generation Construction algorithms Building a block layout by iteratively adding departments Improvements
Computing Science (CMPUT) 496
Computing Science (CMPUT) 496 Search, Knowledge, and Simulations Martin Müller Department of Computing Science University of Alberta mmueller@ualberta.ca Winter 2017 Part IV Knowledge 496 Today - Mar 9
Multitree Decoding and Multitree-Aided LDPC Decoding
Multitree Decoding and Multitree-Aided LDPC Decoding Maja Ostojic and Hans-Andrea Loeliger Dept. of Information Technology and Electrical Engineering ETH Zurich, Switzerland Email: {ostojic,loeliger}@isi.ee.ethz.ch
10703 Deep Reinforcement Learning and Control
10703 Deep Reinforcement Learning and Control Russ Salakhutdinov Slides borrowed from Katerina Fragkiadaki Solving known MDPs: Dynamic Programming Markov Decision Process (MDP)! A Markov Decision Process
43.1 Introduction. Foundations of Artificial Intelligence Introduction Monte-Carlo Methods Monte-Carlo Tree Search. 43.
May 6, 20 3. : Introduction 3. : Introduction Malte Helmert University of Basel May 6, 20 3. Introduction 3.2 3.3 3. Summary May 6, 20 / 27 May 6, 20 2 / 27 Board Games: Overview 3. : Introduction Introduction
Smyth County Public Schools 2017 Computer Science Competition Coding Problems
Smyth County Public Schools 2017 Computer Science Competition Coding Problems The Rules There are ten problems with point values ranging from 10 to 35 points. There are 200 total points. You can earn partial
Easy to Win, Hard to Master:
Easy to Win, Hard to Master: Optimal Strategies in Parity Games with Costs Joint work with Martin Zimmermann Alexander Weinert Saarland University December 13th, 216 MFV Seminar, ULB, Brussels, Belgium
Math 152: Applicable Mathematics and Computing
Math 152: Applicable Mathematics and Computing April 16, 2017 April 16, 2017 1 / 17 Announcements Please bring a blue book for the midterm on Friday. Some students will be taking the exam in Center 201,
A Fractal which violates the Axiom of Determinacy
BRICS RS-94-4 S. Riis: A Fractal which violates the Axiom of Determinacy BRICS Basic Research in Computer Science A Fractal which violates the Axiom of Determinacy Søren Riis BRICS Report Series RS-94-4
Adversarial Search 1
Adversarial Search 1 Adversarial Search The ghosts trying to make pacman loose Can not come up with a giant program that plans to the end, because of the ghosts and their actions Goal: Eat lots of dots
4. Game Theory: Introduction
4. Game Theory: Introduction Laurent Simula ENS de Lyon L. Simula (ENSL) 4. Game Theory: Introduction 1 / 35 Textbook : Prajit K. Dutta, Strategies and Games, Theory and Practice, MIT Press, 1999 L. Simula
Comparison of Monte Carlo Tree Search Methods in the Imperfect Information Card Game Cribbage
Comparison of Monte Carlo Tree Search Methods in the Imperfect Information Card Game Cribbage Richard Kelly and David Churchill Computer Science Faculty of Science Memorial University {richard.kelly, dchurchill}@mun.ca
UNIVERSITY OF TWENTE. Guard-based Partial-Order Reduction in LTSmin. Formal Methods & Tools.
UNIVERSITY OF TWENTE. Formal Methods & Tools. Guard-based Partial-Order Reduction in LTSmin Alfons Laarman, Elwin Pater, Jaco van de Pol, Michael Weber 8 july 2013 SPIN 13, Stony Brook LTSmin Tool Architecture
Repeated Games. ISCI 330 Lecture 16. March 13, Repeated Games ISCI 330 Lecture 16, Slide 1
Repeated Games ISCI 330 Lecture 16 March 13, 2007 Repeated Games ISCI 330 Lecture 16, Slide 1 Lecture Overview Repeated Games ISCI 330 Lecture 16, Slide 2 Intro Up to this point, in our discussion of extensive-form
SF2972: Game theory. Mark Voorneveld, February 2, 2015
SF2972: Game theory Mark Voorneveld, mark.voorneveld@hhs.se February 2, 2015 Topic: extensive form games. Purpose: explicitly model situations in which players move sequentially; formulate appropriate
Artificial Intelligence
Artificial Intelligence CS482, CS682, MW 1 2:15, SEM 201, MS 227 Prerequisites: 302, 365 Instructor: Sushil Louis, sushil@cse.unr.edu, http://www.cse.unr.edu/~sushil Games and game trees Multi-agent systems
3-2 Lecture 3: January Repeated Games A repeated game is a standard game which isplayed repeatedly. The utility of each player is the sum of
S294-1 Algorithmic Aspects of Game Theory Spring 2001 Lecturer: hristos Papadimitriou Lecture 3: January 30 Scribes: Kris Hildrum, ror Weitz 3.1 Overview This lecture expands the concept of a game by introducing
Adversarial Search. CS 486/686: Introduction to Artificial Intelligence
Adversarial Search CS 486/686: Introduction to Artificial Intelligence 1 AccessAbility Services Volunteer Notetaker Required Interested? Complete an online application using your WATIAM: https://york.accessiblelearning.com/uwaterloo/
Section Summary. Finite Probability Probabilities of Complements and Unions of Events Probabilistic Reasoning
Section 7.1 Section Summary Finite Probability Probabilities of Complements and Unions of Events Probabilistic Reasoning Probability of an Event Pierre-Simon Laplace (1749-1827) We first study Pierre-Simon
Game theory Computational Models of Cognition
Game theory Taxonomy Rational behavior Definitions Common games Nash equilibria Mixed strategies Properties of Nash equilibria What do NE mean? Mutually Assured Destruction 6 rik@cogsci.ucsd.edu Taxonomy
Games, Triangulations, Theory
KTdCW Spieltheorie Games, Triangulations, Theory Oswin Aichholzer, University of Technology, Graz (Austria) KTdCW, Spieltheorie, Aichholzer NIM & Co 0 What is a (mathematical) game? 2 players [ A,B / L(eft),R(ight)
UNIT-III ASYNCHRONOUS SEQUENTIAL CIRCUITS TWO MARKS 1. What are secondary variables? -present state variables in asynchronous sequential circuits 2. What are excitation variables? -next state variables
5.4 Imperfect, Real-Time Decisions
5.4 Imperfect, Real-Time Decisions Searching through the whole (pruned) game tree is too inefficient for any realistic game Moves must be made in a reasonable amount of time One has to cut off the generation
Final Exam, Math 6105
Final Exam, Math 6105 SWIM, June 29, 2006 Your name Throughout this test you must show your work. 1. Base 5 arithmetic (a) Construct the addition and multiplication table for the base five digits. (b)
CS 331: Artificial Intelligence Adversarial Search II. Outline
CS 331: Artificial Intelligence Adversarial Search II 1 Outline 1. Evaluation Functions 2. State-of-the-art game playing programs 3. 2 player zero-sum finite stochastic games of perfect information 2 1
23 Applications of Probability to Combinatorics
November 17, 2017 23 Applications of Probability to Combinatorics William T. Trotter trotter@math.gatech.edu Foreword Disclaimer Many of our examples will deal with games of chance and the notion of gambling.
Reinforcement Learning in Games Autonomous Learning Systems Seminar
Reinforcement Learning in Games Autonomous Learning Systems Seminar Matthias Zöllner Intelligent Autonomous Systems TU-Darmstadt zoellner@rbg.informatik.tu-darmstadt.de Betreuer: Gerhard Neumann Abstract
CS 4700: Foundations of Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence Bart Selman Reinforcement Learning R&N Chapter 21 Note: in the next two parts of RL, some of the figure/section numbers refer to an earlier edition of R&N
Plan. Related courses. A Take-Away Game. Mathematical Games , (21-801) - Mathematical Games Look for it in Spring 11
V. Adamchik D. Sleator Great Theoretical Ideas In Computer Science Mathematical Games CS 5-25 Spring 2 Lecture Feb., 2 Carnegie Mellon University Plan Introduction to Impartial Combinatorial Games Related
CSE548, AMS542: Analysis of Algorithms, Fall 2016 Date: Sep 25. Homework #1. ( Due: Oct 10 ) Figure 1: The laser game.
CSE548, AMS542: Analysis of Algorithms, Fall 2016 Date: Sep 25 Homework #1 ( Due: Oct 10 ) Figure 1: The laser game. Task 1. [ 60 Points ] Laser Game Consider the following game played on an n n board,
Tetsuo JAIST EikD Erik D. Martin L. MIT
Tetsuo Asano @ JAIST EikD Erik D. Demaine @MIT Martin L. Demaine @ MIT Ryuhei Uehara @ JAIST Short History: 2010/1/9: At Boston Museum we met Kaboozle! 2010/2/21 accepted by 5 th International Conference
Computational aspects of two-player zero-sum games Course notes for Computational Game Theory Section 3 Fall 2010
Computational aspects of two-player zero-sum games Course notes for Computational Game Theory Section 3 Fall 21 Peter Bro Miltersen November 1, 21 Version 1.3 3 Extensive form games (Game Trees, Kuhn Trees)
Lecture 14. Questions? Friday, February 10 CS 430 Artificial Intelligence - Lecture 14 1
Lecture 14 Questions? Friday, February 10 CS 430 Artificial Intelligence - Lecture 14 1 Outline Chapter 5 - Adversarial Search Alpha-Beta Pruning Imperfect Real-Time Decisions Stochastic Games Friday,
Today. Nondeterministic games: backgammon. Algorithm for nondeterministic games. Nondeterministic games in general. See Russell and Norvig, chapter 6
Today See Russell and Norvig, chapter Game playing Nondeterministic games Games with imperfect information Nondeterministic games: backgammon 5 8 9 5 9 8 5 Nondeterministic games in general In nondeterministic
ECON 282 Final Practice Problems
ECON 282 Final Practice Problems S. Lu Multiple Choice Questions Note: The presence of these practice questions does not imply that there will be any multiple choice questions on the final exam. 1. How
Ron Breukelaar Hendrik Jan Hoogeboom Walter Kosters. ( LIACS algoritmen )
Ron Breukelaar Hendrik Jan Hoogeboom Walter Kosters ( LIACS algoritmen ) 26-11-2004 23 jun 2006 Tetris? Tetris is NP complete!! what configurations? undecidable Tetris the AI of Tetris www.liacs.nl/home/kosters/tetris/
3 Game Theory II: Sequential-Move and Repeated Games
3 Game Theory II: Sequential-Move and Repeated Games Recognizing that the contributions you make to a shared computer cluster today will be known to other participants tomorrow, you wonder how that affects
Learning to Play like an Othello Master CS 229 Project Report. Shir Aharon, Amanda Chang, Kent Koyanagi
Learning to Play like an Othello Master CS 229 Project Report December 13, 213 1 Abstract This project aims to train a machine to strategically play the game of Othello using machine learning. Prior to
Dice Games and Stochastic Dynamic Programming
Dice Games and Stochastic Dynamic Programming Henk Tijms Dept. of Econometrics and Operations Research Vrije University, Amsterdam, The Netherlands Revised December 5, 2007 (to appear in the jubilee issue
FORMAL MODELING AND VERIFICATION OF MULTI-AGENTS SYSTEM USING WELL- FORMED NETS
FORMAL MODELING AND VERIFICATION OF MULTI-AGENTS SYSTEM USING WELL- FORMED NETS Meriem Taibi 1 and Malika Ioualalen 1 1 LSI - USTHB - BP 32, El-Alia, Bab-Ezzouar, 16111 - Alger, Algerie taibi,ioualalen@lsi-usthb.dz
Imperfect Information. Lecture 10: Imperfect Information. What is the size of a game with ii? Example Tree
Imperfect Information Lecture 0: Imperfect Information AI For Traditional Games Prof. Nathan Sturtevant Winter 20 So far, all games we ve developed solutions for have perfect information No hidden information
The extensive form representation of a game
The extensive form representation of a game Nodes, information sets Perfect and imperfect information Addition of random moves of nature (to model uncertainty not related with decisions of other players).
Artificial Intelligence Adversarial Search
Artificial Intelligence Adversarial Search Adversarial Search Adversarial search problems games They occur in multiagent competitive environments There is an opponent we can t control planning again us!
Extensive Form Games: Backward Induction and Imperfect Information Games
Extensive Form Games: Backward Induction and Imperfect Information Games CPSC 532A Lecture 10 October 12, 2006 Extensive Form Games: Backward Induction and Imperfect Information Games CPSC 532A Lecture
Outline. Game Playing. Game Problems. Game Problems. Types of games Playing a perfect game. Playing an imperfect game
Outline Game Playing ECE457 Applied Artificial Intelligence Fall 2007 Lecture #5 Types of games Playing a perfect game Minimax search Alpha-beta pruning Playing an imperfect game Real-time Imperfect information
Finite games: finite number of players, finite number of possible actions, finite number of moves. Canusegametreetodepicttheextensiveform.
A game is a formal representation of a situation in which individuals interact in a setting of strategic interdependence. Strategic interdependence each individual s utility depends not only on his own
Solution Concepts 4 Nash equilibrium in mixed strategies
Solution Concepts 4 Nash equilibrium in mixed strategies Watson 11, pages 123-128 Bruno Salcedo The Pennsylvania State University Econ 402 Summer 2012 Mixing strategies In a strictly competitive situation
One computer theorist s view of cognitive systems
One computer theorist s view of cognitive systems Jiri Wiedermann Institute of Computer Science, Prague Academy of Sciences of the Czech Republic Partially supported by grant 1ET100300419 Outline 1. The
The next several lectures will be concerned with probability theory. We will aim to make sense of statements such as the following:
CS 70 Discrete Mathematics for CS Fall 2004 Rao Lecture 14 Introduction to Probability The next several lectures will be concerned with probability theory. We will aim to make sense of statements such
Kernels and Support Vector Machines
Kernels and Support Vector Machines Machine Learning CSE446 Sham Kakade University of Washington November 1, 2016 2016 Sham Kakade 1 Announcements: Project Milestones coming up HW2 You ve implemented GD,
Senior Math Circles February 10, 2010 Game Theory II
1 University of Waterloo Faculty of Mathematics Centre for Education in Mathematics and Computing Senior Math Circles February 10, 2010 Game Theory II Take-Away Games Last Wednesday, you looked at take-away