Video Games As Environments For Learning And Planning: What s Next? Julian Togelius

Size: px

Start display at page:

Download "Video Games As Environments For Learning And Planning: What s Next? Julian Togelius"

Kellie Morris
5 years ago
Views:

1 Video Games As Environments For Learning And Planning: What s Next? Julian Togelius

3 A very selective history Othello Backgammon Checkers Chess Go Poker Super/Infinite Mario Bros Ms. Pac-Man Crappy Atari games Montezuma s Revenge Dota 2 StarCraft 2

4 What s next? What s left?

5 How we will think of this

6 Characteristics of games

7 Characteristics of games Number of players 1, 1.5, 2, many; competitive, cooperative, both? Stochasticity Observability Action space and branching factor Time granularity Few turns, many turns, continuous?

8 Branching factor Checkers: 3 Pac-Man: 4 Super Mario Bros: 16 Chess: 35 Go: 350 Civilization:? Fortnite:?

9 Characteristics of the game-ai interface How is the game state represented? Pixels, text, simulated sensors, some internal representation? Is there a forward model? Is it fast? Accurate? Do you have time to train? How much? How many games are you playing?

10 Characteristics of common benchmarks Classic board games: lowish branching factor, few turns, twoplayer adversarial, complete information, no stochasticity, structured game state representation, forward model readily available Poker: lots of hidden information, stochastic, otherwise like above Atari 2600: low branching factor, longish, single-player, mostly complete information, no stochasticity (!), pixels, no/slow forward model StarCraft: insane branching factor, long games, two-player adversarial, hidden information, some stochasticity, pixels (more or less), no forward model

11 Another way of thinking Which cognitive abilities are exercised by a particular game? Which cognitive abilities that humans have are not tested by current game-based AI benchmarks? If we create AI that can play games that require certain abilities of humans, will agents that solve these games need to have the same abilities?

13 Which challenges are next?

14 Wait a minute, why are we doing this? To improve specific AI capabilities? To achieve AGI? To make games better? To keep unemployed AI researchers off the streets?

15 Which challenges are next? Role-playing games Roguelikes Multiplayer games 4X games Open world games Generalization/Transfer Human-like playing Resource-constrained playing Things that are not game-playing Gardening/building games

16 Role-playing games Text as input; very long game length; reasoning about motives, emotions and social practices

17 Text adventure games

18 Roguelikes Handling radical uncertainty, generalization

19 Multiplayer games Multiplayer dynamics (example: Pommerman)

20 4X games Example: Civilization 6. Ridiculous branching factor, n- player, extremely long game time.

22 So many game genres Dating simulators Rhythm games Word games Grand strategy war-games Life simulations Weirdo casual games of all kinds Things that come out of the global game jam

23 Generalization/Transfer Example: Obstacle Tower. Small branching factor, single-player, pixel inputs with very complex visuals, thoroughly randomized puzzles and visuals, potentially endless game time

24 Generalization/Transfer Example: General Video Game AI

25 Human-like playing Playing games to win is one thing - how about playing them like a human? Same strengths, same weaknesses, same preferences Very useful for the game industry

26 Resource-constrained playing How to do it without 5000 TPUs Academics like this Obvious practical applications

27 Things that are not game-playing Procedural content generation Player experience modeling Game modeling Game generation Ultimately, inventing new games for AI to play automatically

Artificial Intelligence and Games Playing Games

Artificial Intelligence and Games Playing Games Georgios N. Yannakakis @yannakakis Julian Togelius @togelius Your readings from gameaibook.org Chapter: 3 Reminder: Artificial Intelligence and Games Making