OLD & NEW APPLICATIONS OF PLANNING IN GAMES

Transcription

1 Stavros Vassos Sapienza University of Rome, DIAG, Italy May 2014 OLD & NEW APPLICATIONS OF PLANNING IN GAMES Character Behavior, Interactive Stories, and more

2 Applications of planning in videogames 2 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

3 Applications of planning in videogames 3 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

4 Characters in videogames 4 Non-player characters (NPCs): Move in game-world space (2D/3D) Act in game-world Pick-up objects, attack, hide, etc. Exhibit behavior Follow the player, get scared and runaway, etc.

5 Characters in videogames 5 Non-player characters (NPCs): Move in game-world space (2D/3D) Act in game-world Pick-up objects, attack, hide, etc. Exhibit behavior Follow the player, get scared and runaway, etc. How do they think?

6 Characters in videogames 6 Non-player characters (NPCs): Move in game-world space (2D/3D) Act in game-world Pick-up objects, attack, hide, etc. Exhibit behavior Follow the player, get scared and runaway, etc. How do they think? Navigation: Pathfinding Action-driven behavior: Finite State Machines / Behavior Trees / Goal Oriented Action Planning / Utility systems

7 Characters in videogames 7 Many more aspects related to the experience of the player Emotions Narrative interactions Interactive storyline Novel interfaces

8 Characters in videogames 8 Game engine, a typical example: C++ Creates game-world objects with (x,y,z) coordinates and calculates what happens to them on every frame E.g., a crate is up in the air on frame1. On frame 2 the game engine will calculate the new position, etc

9 Characters in videogames 9 Game engine, a typical example: C++ Creates game-world objects with (x,y,z) coordinates and calculates what happens to them on every frame E.g., a crate is up in the air on frame1. On frame 2 the game engine will calculate the new position, etc Same for non-player characters (NPCs)!

10 Characters in videogames 10 Game engine, a typical example: Every few frames calls a special thinking method of the NPC game object The thinking method specifies NPC behavior in terms of movement, animation, interaction with the game-world The thinking method can be as simple as a code listing that invokes curated behaviors

11 Characters in videogames 11 int NPC::think(){ if (some condition){ makescarysound(); } else if (some other condition){ chaseplayer(); } else if... }

12 Characters in videogames 12 int NPC::think(){ if (state==onguard && seesmallenemy()){ state=fight; makescarysound(); } else if (state==fight && energy>30){... } else if... }

13 Characters in videogames 13 Commercial game code (you can get it from Steam) HL2-SDK, npc_basezombie.cpp lines switch ( m_npcstate ) { case NPC_STATE_COMBAT: case NPC_STATE_ALERT: }

14 Characters in videogames 14 Finite State Machines as a programming technique for reactive behavior specification Various variants including hierarchical FSMs

15 Characters in videogames 15 (Now) you can experiment yourself! Amazing tools available for (indie) game developers!

16 Characters in videogames 16 Stealth Unity tutorial project unity3d.com/learn/tutorials/projects/stealth

17 Characters in videogames 17 Stealth Unity tutorial project: Enemy AI unity3d.com/learn/tutorials/projects/stealth/enemy-ai void Update () { // If the player is in sight and is alive... if(enemysight.playerinsight && playerhealth.health > 0f) Shooting(); // If the player has been sighted and isn't dead... else if(enemysight.personallastsighting!= lastplayersighting.resetposition && playerhealth.health > 0f) Chasing(); } // Otherwise... else Patrolling();

18 Characters in videogames 18 Reactive behavior Look into the current state of the game-world Use a recipe to decide what the interactive object/ character should do in the next immediate frame Proactive behavior Look into the possible evolutions of the current state under the effect of actions Use evaluation criteria to decide which course of action and corresponding outcome is more desirable

19 Characters in videogames 19 Reactive behavior Look into the current state of the game-world Use a recipe to decide what the interactive object/ character should do in the next immediate frame Proactive behavior Look into the possible evolutions of the current state under the effect of actions Use evaluation criteria to decide which course of action and corresponding outcome is more desirable Thinking (in terms of look-ahead) before acting

20 Characters in videogames 20 So how does planning apply to the videogame setting? Let s see some cases where methods inspired from HTN planning and STRIPS planning have been used in commercial games

21 Characters in videogames 21 So how does planning apply to the videogame setting? Let s see some cases where methods inspired from HTN planning and STRIPS planning have been used in commercial games As you will notice, the requirements and assumptions are different from the ones we typically consider in textbook versions of planning

22 Applications of planning in videogames 22 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

23 Behavior Trees 23 Running example Specify the (part of the) thinking function for a character that decides about how to enter in a room (Very simple.. but it will illustrate the main points) First, we go over an HTN for this simple character behavior

24 HTN-like planning 24 Specify the behavior objective in terms of tasks that can be decomposed to other tasks and atomic actions Get in in room Get in room Free door Free door Move into room Move to door Free door Move into room Unlock door Kick door

25 HTN-like planning 25 Preconditions and effects specify conceptually whether some (task or) atomic action can be executed Get in in room Get in room Free door Free door Move into room Move to door Free door Move into room Unlock door Kick door Pre Eff Move into room Pre Eff Move to door Pre Eff Unlock door Pre Eff Kick door

26 HTN-like planning 26 The tasks and atomic actions form a search tree A solution is a successful (complete) decomposition Get in room Get in room Get in room Pre Eff Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

27 HTN-like planning 27 The tasks and atomic actions form a search tree A solution is a successful (complete) decomposition Get in room Get in room Get in room Pre Eff Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

28 HTN-like planning 28 The tasks and atomic actions form a search tree A solution is a successful (complete) decomposition Pre Eff Move to door Pre Eff Move into room Pre Eff Kick door

29 HTN-like planning 29 The tasks and atomic actions form a search tree A solution is a successful (complete) decomposition Pre Eff Move to door Pre Eff Move into room Actions executed: Move to door Kick door Move into room Pre Eff Kick door

30 Reactive HTN Behavior Trees 30 HTN: Search offline for a solution then execute it Get in room Get in room Get in room Pre Eff Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

31 Reactive HTN Behavior Trees 31 HTN: Search offline for a solution then execute it Behavior Trees: Execute online as you search (!) Get in room Get in room Get in room Pre Eff Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

32 Reactive HTN Behavior Trees 32 HTN: Search offline for a solution then execute it Behavior Trees: Execute online as you search (!) Note that a BT approach actually executes the failing atomic actions Assuming that our world model is accurate, in the running example it would execute the following Actions executed: Move into room Move to door Unlock door Kick door Move into room

33 Reactive HTN Behavior Trees 33 So, BTs are a form of online execution of an HTN, which is typically referred to as reactive planning They introduce a practical way to specify the tree of execution using a common interface among nodes Essentially, the intuition they keep from HTN is the notion of hierarchical decomposition in the sense that parent nodes depend on child nodes

34 Reactive HTN Behavior Trees 34 BT: A tree of tasks Tasks have a common interface: they are given CPU time to execute and they return success/failure/ongoing Leaf tasks: check a condition or execute some code Composite tasks: return value depending on child tasks

35 Reactive HTN Behavior Trees 35 BT: A tree of tasks Tasks have a common interface: they are given CPU time to execute and they return success/failure/ongoing Leaf tasks: check a condition or execute some code Composite tasks: return value depending on child tasks E.g., succeed if the door in front of the NPC is open E.g., kick the door in front of the NPC So, leaf tasks execute online preconditions and effects

36 Reactive HTN Behavior Trees 36 BT: A tree of tasks Tasks have a common interface: they are given CPU time to execute and they return success/failure/ongoing Leaf tasks: check a condition or execute some code Composite tasks: return value depending on child tasks? E.g., succeed if any of the child tasks succeed

37 Reactive HTN Behavior Trees 37 BT: A tree of tasks Tasks have a common interface: they are given CPU time to execute and they return success/failure/ongoing Leaf tasks: check a condition or execute some code Composite tasks: return value depending on child tasks E.g., succeed if all of the child tasks succeed

38 Reactive HTN Behavior Trees 38 BT: A tree of tasks Sequence composite task Selector composite task??

39 Reactive HTN Behavior Trees 39 Preconditions and effects become atomic nodes Decomposition & choice become sequence & selector Get in room Get in room Get in room Pre Eff Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

40 Reactive HTN Behavior Trees 40 Preconditions and effects become atomic nodes Decomposition & choice become sequence & selector Get in room Get in room Door open? Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

41 Reactive HTN Behavior Trees 41 Preconditions and effects become atomic nodes Decomposition & choice become sequence & selector? Door open? Move into room Pre Eff Move to door? Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

42 Reactive HTN Behavior Trees 42 Preconditions and effects become atomic nodes Decomposition & choice become sequence & selector? Door open? Move into room Move to door? Move into room Door locked? Unlock door Kick door Door open?

43 Reactive HTN Behavior Trees 43 NPC Note behavior that no offline based search on more is involved: refined the conditions behavior and strategies tree is executed online as a kind of pre-defined program? Door open? Move into room Move to door? Move into room Door locked? Unlock door Kick door Door open?

44 Reactive HTN Behavior Trees 44 The way the tree is executed depends on the implementation, e.g., always start over, keep track of the current node, etc NPC behavior based on more refined conditions and strategies? Door open? Move into room Move to door? Move into room Door locked? Unlock door Kick door Door open?

45 Reactive HTN Behavior Trees 45 Reactive behavior based on more refined conditions and strategies that are shown visually Pattern: Sequence of actions with preconditions Precond Action 1 Action 2 Action 2

46 Reactive HTN Behavior Trees 46 Reactive behavior based on more refined conditions and strategies that are shown visually Pattern: Complex action with preconditions Precond? Finish Easy way to achieve the complex action Less easy way Last resort before failing

47 Reactive HTN Behavior Trees 47 Reactive behavior based on more refined conditions and strategies that are shown visually Pattern: Complex action with preconditions Precond? Finish Easy way to achieve the Action 1 complex action Less easy way Last resort before failing

48 Reactive HTN Behavior Trees 48 Reactive behavior based on more refined conditions and strategies Pattern: Complex action with preconditions Precond? Finish Easy way to achieve the Action 1 complex action Last resort before failing Less easy way Action 2 Action 3

49 Reactive HTN Behavior Trees 49 Reactive behavior based on more refined conditions and strategies that are shown visually Pattern: Complex action with preconditions Precond? Finish Easy way to achieve the Action 1 complex action Last resort before failing Less easy way Action 2 Action 3

50 Reactive HTN Behavior Trees 50 Reactive behavior based on more refined conditions and strategies that are shown visually Pattern: High-level complex actions under a main selector?

51 Reactive HTN Behavior Trees 51 Reactive behavior based on more refined conditions and strategies that are shown visually Pattern: High-level complex actions under a main selector? Defend Idle Attack Search player Search food

52 Reactive HTN Behavior Trees 52 Reactive behavior based on more refined conditions and strategies that are shown visually Non-deterministic sequence task and selector task ~ ~? ~? ~

53 Reactive HTN Behavior Trees 53 Reactive behavior based on more refined conditions and strategies that are shown visually? Door open? Move into room Move to door ~? Move into room Door locked? Unlock door Kick door Door open?

54 Reactive HTN Behavior Trees 54 Reactive behavior based on more refined conditions and strategies that are shown visually Parallel sequence task (similar to sequence) E.g., perform move actions while also shooting at target Also used to simulate state-like behavior by ensuring that a condition holds

55 Reactive HTN Behavior Trees 55 Reactive behavior based on more refined conditions and strategies that are shown visually Decorator tasks (wrap objects with same interface) Until fail Invert

56 Reactive HTN Behavior Trees 56 Reactive behavior based on more refined conditions and strategies that are shown visually? Until fail Attack enemy Until fail Look around Enemy in sight Invert Enemy in sight

57 Reactive HTN Behavior Trees 57 Resources

58 Reactive HTN Behavior Trees 58 Resources Behavior Trees in Software Engineering (wikipedia.org/behavior_trees)

59 Reactive HTN Behavior Trees 59 Resources Artificial Intelligence for Games, 2nd Ed., Ian Millington and John Funge, Section 5.4 (ai4g.com)? Door open? Move into room Move to door? Move into room Door locked? Unlock door Kick door Door open?

60 Reactive HTN Behavior Trees 60 Resources Behave Unity package by Angry Ant (angryant.com/behave) with integrated editor

61 Reactive HTN Behavior Trees 61 Resources Behavior tree blog post series by Björn Knafla on efficiently implementing and running them in games (altdevblogaday.com:intro-to-bts/)

62 Reactive HTN Behavior Trees 62 Resources Coordinating Agents with Behavior Trees by Crytech (freesdk.crydev.net:behaviortrees) )

63 Reactive HTN Behavior Trees 63 Resources Tank Maneuvers, Damage Models and Motion Planning in RED ORCHESTRA 2 (aigamedev.com:red-orchestra-2)

64 Reactive HTN Behavior Trees 64 Resources Omni-bot in Wolfenstein 3D Enemy Territory (omni-bot.com) Open-source code available for the game (splashdamage.com:wolfenstein3d-et)

65 Reactive HTN Behavior Trees 65 One of the first commercial video games that used BTs is Halo2 (2004) Simple to understand Simple to implement Separation between the work of the programmer and the game designers Offers the visual specification of fine-grained behaviors (e.g., sequencing, parallel tasks, etc)

66 HTNs in videogames 66 BTs and variants of FSMs have dominated NPC behavior specification in character-based games Going back to the HTN formalism, more recently also actual proactive HTN planning has been used in games as well

67 HTNs in videogames 67 Overview on aigamedev.com:planning-in-games

68 HTNs in videogames 68 E.g., Transformers: War for Cybetron (2012) t2thompson.com/2014/03/16/ai-of-transformers/

69 Applications of planning in videogames 69 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Non-linear stories in videogames More!

70 STRIPS planning 70 Let s not write the decompositions ourselves Model the goal condition in terms of state conditions Get in room Get in room Get in room Pre Eff Move into room Pre Eff Move to door Free door Pre Eff Move into room Pre Eff Unlock door Pre Eff Kick door

71 STRIPS planning 71 Let s not write the decompositions ourselves Model the goal condition in terms of state conditions Pre Eff Actions Move into room Get in room Pre Eff Pre Eff Pre Eff Move to door Unlock door Kick door

72 STRIPS planning 72 Let s not write the decompositions ourselves Model the goal condition in terms of state conditions Pre Eff Actions Move into room Get in room State conditions Door open Not Door open Pre Eff Move to door Door locked Not Door locked Pre Eff Unlock door In room Not In room Pre Eff Kick door

73 STRIPS planning 73 Let s not write the decompositions ourselves Model the goal condition in terms of state conditions Pre Eff Actions Move into room Get in room State conditions Door open Not Door open Pre Eff Move to door Door locked Not Door locked Pre Eff Unlock door In room Not In room Pre Eff Kick door Door locked Hold key Unlock door Not Door locked Door open

74 STRIPS planning 74 STRIPS: We essentially only model the requirements for actions and the goal, not how to achieve the goal Initial state Actions Goal Not Door open Pre Move into room Eff In room Not Door locked Pre Unlock door Eff Not In room Pre Eff

75 STRIPS planning 75 STRIPS: We essentially only model the requirements for actions and the goal, not how to achieve the goal I G

76 STRIPS planning 76 Notice that the state space is much bigger compared to the HTN formulation. How can this actually work? I G

77 STRIPS planning 77 STRIPS planning is a graph reachability problem Start from the initial state/node The search graph is generated by the action descriptions The goal condition specifies the node we search for

78 STRIPS planning 78 STRIPS planning is a graph reachability problem Start from the initial state/node The search graph is generated by the action descriptions The goal condition specifies the node we search for STRIPS is a much more informed representation than just the generated search graph because actions are specified using preconditions and add/delete effects Move to door AddEffect: At Door Precond: At Door Kick door

79 STRIPS planning 79 This type of information allows a search algorithm to look into how actions affect each other Move to door Kick door Move into room In fact some exciting work is being done on ways to generate domain-independent heuristics about finding the most promising node to follow in search, exploiting the add/delete list information

80 STRIPS planning 80 E.g., the distinction of add/delete effects allows us to consider a relaxed (i.e., intended to be simpler) problem in which the delete effects are ignored Think of it this way: As actions are applied, only the positive interactions are kept without removing anything from the state, thus making it easier to achieve goals For example, moving does not remove the old position, and using a resource still keeps it available

81 81 STRIPS planning How does this work as a heuristic? Solve this relaxed problem for every next state you consider and the same goal If the goal cannot be found in the relaxed problem then the original problem cannot be solved from this state If the goal is found then the length of the solution on the relaxed problem is a heuristic estimate for the original! It works great! Some of the best heuristics (e.g., FF) are based on this idea (but there are others too now) RP 1 RP 2 RP 3 RP 4

82 STRIPS Goal-Oriented Action Planning 82 OK, so how is this applied in videogames?

83 STRIPS Goal-Oriented Action Planning 83 FEAR (2005), probably the most famous game using GOAP Motivation One main AI programmer responsible for all NPCs behavior Idea: Different behaviors can be achieved among characters by using STRIPS and providing each character with same goals but a different set of available actions

84 84 STRIPS Goal-Oriented Action Planning

85 STRIPS Goal-Oriented Action Planning 85 Simplifying STRIPS planning

86 STRIPS Goal-Oriented Action Planning 86 Simplifying STRIPS planning Literals are stored as variables (essentially having one argument) The state is stored as an array of a fixed size Actions replace the value of some properties Energy: 50 Distance: 76 Holding: Item3 Variable: Value

87 STRIPS Goal-Oriented Action Planning 87 Simplifying STRIPS planning Literals are stored as variables (essentially having one argument) The state is stored as an array of a fixed size Actions replace the value of some properties Energy: 50 Distance: 76 Holding: Item3 Variable: Value The add/delete representation is lost! Regular STRIPS-like heuristics cannot be applied (as-is)

88 STRIPS Goal-Oriented Action Planning 88 Simplifying STRIPS planning Literals are stored as variables (essentially having one argument) The state is stored as an array of a fixed size Actions replace the value of some properties Energy: 50 Distance: 76 Holding: Item3 Variable: Value A* search is used for plans of length at most 3 but the search is more or less similar to uninformed BFS

89 STRIPS Goal-Oriented Action Planning 89 Advantages Easy to manage a large number of generated behaviors Able to achieve different behaviors that satisfy the given requirements under different conditions without explicitly listing the resulting strategies

90 STRIPS Goal-Oriented Action Planning 90 Advantages Easy to manage a large number of generated behaviors Able to achieve different behaviors that satisfy the given requirements under different conditions without explicitly listing the resulting strategies Challenges Need to solve planning problems in real-time! ( = a few frames = a few msec )

91 STRIPS Goal-Oriented Action Planning 91 Advantages Easy to manage a large number of generated behaviors Able to achieve different behaviors that satisfy the given requirements under different conditions without explicitly listing the resulting strategies Challenges Need to solve planning problems in real-time! ( = a few frames = a few msec ) In complex domains with chaining of effects, the goal count heuristic is comparable to blind search

92 STRIPS Goal-Oriented Action Planning 92 One of the first commercial video games that used GOAP is FEAR (2005) Not so simple to understand Not so simple to implement Not so clear separation between the work of the programmer and the game designers The specification of fine-grained behaviors is tricky

93 STRIPS Goal-Oriented Action Planning 93 Overview on aigamedev.com:planning-in-games

94 STRIPS Goal-Oriented Action Planning 94 Overview on web.media.mit.edu/~jorkin/goap.html

95 Characters in videogames 95 General observations about HTN/STRIPS planning States are often represented as normal variables in the target programming language Actions are often modeled as normal functions that operate on the state variables Preconditions as normal if-statements, effects as normal assignments In many cases developers exploit mostly the intuitions about expressing the intended behaviors in terms of actions, and then perform search over the space of action application, rather than exploit offline symbolic tricks like the relaxed heuristic Note1: more difficult to keep track of causal links as in the textbook version of add/delete lists of literals Note2: goal count heuristic can still be applied (check variables if they match with some goal condition)

96 Applications of planning in videogames 96 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

97 Utility-based planning in videogames 97 General observations about utility systems Essentially the state representation is the same as STRIPS-like planning (e.g., with variables and values) but the goal here is to maximize some metric The evaluation function is typically a combination of utility functions, such as energy, hunger, thirst, etc As exhaustive search is typically used, and either the number of available actions is limited or the length of sequences of actions to consider

98 Utility-based planning in videogames 98 Most famous commercial video-game: The Sims series! Basic idea: nearby objects expose available actions to be performed as well as the outcome in utility The Sim evaluates each of the actions and picks the most useful or most urgent 1-step planning of local actions :-)

99 99 Utility-based planning in videogames

100 Utility-based planning in videogames 100 Most famous commercial video-game: The Sims series! Many additional tricks to handle sequences of actions, social interactions based on FSM social states, ageing, Introduced the notion of smart objects Data-driven actions and effects The logic behind the domain is pushed to the objects instead of the NPC Add-on extensions can be done easily

101 Utility-based planning in videogames 101 Nice resources available Original code by Will Wright ( Artificial Intelligence in The Sims series by Yoann Bourse (

102 Applications of planning in videogames 102 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

103 103 Challenge: Hybrid deliberation

104 Challenge: Hybrid deliberation 104 BTs and HTN-like planning share a lot of intuitions Can be used as programs for a STRIPS-like domain Can be mixed to allow for more flexibility Precond? Finish Easy way to achieve the Action 1 complex action Less easy way by HTN-like planning Last resort by STRIPS-like planning

105 Challenge: Hybrid deliberation 105 Quick and dirty Python prototype of HTN/STRIPS hybrid planning available for experimentation

106 Challenge: Hybrid deliberation 106 Pac-man test-bed in Python (used in courses to specify STRIPS planning agents or Golog agents) inst.eecs.berkeley.edu/~cs188/pacman/

107 Challenge: Hybrid deliberation 107 There are other competitions too :-)

108 Applications of planning in videogames 108 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

109 Moving to the story level 109 Each NPC has some action-driven autonomy The overall interaction is emergent based on this Follow player Attack Interact to give item X

110 Moving to the story level 110 Each NPC has some action-driven autonomy The overall interaction is emergent based on this How is the overall story controlled? Follow player Attack Interact to give item X

111 Moving to the story level 111 Linear storytelling: specific events trigger the next chapter/level/area/cut-scene Nonlinear storytelling: the story adapts to the player interaction Follow player Attack Interact to give item X

112 Interactive storytelling in video games 112 Linear storyline similar to the one story of films Strict order of plot points, only one way to move forward Traditional approach in video games 112

113 Interactive storytelling in video games 113 Quest-based approach in an open world Players may visit plot points in any order they choose E.g., World of Warcraft, Grand Theft Auto 113

114 Interactive storytelling in video games 114 Open-world groups that converge to a linear story E.g., in Syberia all chapters start and end in a fixed way but the order of interacting with sub-parts can vary 114

115 Interactive storytelling in video games 115 Branching depending on player action that may lead to different endings E.g., Heavy Rain, Beyond Two Souls 115

116 Interactive storytelling in video games 116 Branching depending on player action that may lead to different endings E.g., Heavy Rain, Beyond Two Souls 116

117 Interactive storytelling in video games 117 Next generation of video games Adapting the story based on player s choices Adapting the story to improve the player s experience, e.g., according to player type: solvers/fighters/wanderers Strong story vs character autonomy AI Drama manager directs the story following a rich collection of parameters

118 Interactive storytelling in video games 118 As the complexity of the increases, hard-coded solutions do not scale well

119 Interactive storytelling in video games 119 Challenge: Generate an interesting adaptive story over a large space of opportunities or story states

120 Interactive storytelling in video games 120 Model story generation as a solution to a planning problem over story states I G

121 Interactive storytelling in video games 121 Model story generation as a solution to a planning problem over story states The domain of stories seems to be better suited for AI work in knowledge representation and automated reasoning than (high-level) NPC behavior High-level symbols are already present Time constraints are less critical A lot of work in major AI conferences (e.g., AAMAS, AAAI, ECAI)

122 Interactive storytelling in video games 122 Some themes

123 Interactive storytelling in video games 123 Some themes Reactive planning for story generation and adaptation Generate stories based on current situation and intended goal condition for the story using PDDL planners Use PDDL features to model time constraints Use more expressive languages to express modalities such as intention of characters and plan also taking these into account Compile modalities into PDDL planning Generate medical soap opera episodes!

124 124 Interactive storytelling in video games

125 Interactive storytelling in video games 125 Stories in games are created by authors/writers and have many more features than events, actors, and time A successful story is one that takes into account relationships between actors, the timing of narrative intensity, etc These open up interesting KR&R problems Let s see one such aspect briefly

126 Interactive storytelling in video games 126 Let s focus on a particular type of stories that are common in video games: war stories Model the story of Iliad as a solution of a planning problem Initial state Available actions Goal condition Motivation: Adaptive Iliad! by means of real-time planners

127 Challenge: First book of Iliad 127 Apollo s priest Chryses comes to the Achaian camp and asks to ransom back his daughter Chryseis, who has been captured by Agamemnon. Chryses, is insulted and sent away. He prays to Apollo to punish the Greeks, which Apollo does by sending a plague upon them. Achilleus calls an assembly to deal with the plague, and the prophet, Kalchas, reveals that Apollo was angered by Agamemnon s refusal to return the daughter of his priest. Agamemnon agrees to give her back, but demands compensation. This provokes Achilleus anger, and, after they exchange threats and angry words, Agamemnon decides to take Achilleus prize, the captive woman, Briseis. The goddess, Athene, prevents Achilleus from killing Agamemnon by promising that he will one day be compensated with three times as many prizes. Agamemnon s men take Briseis from Achilleus, and Achilleus prays to his divine mother, Thetis, for help. He says he will not fight, and he asks her to persuade Zeus to make the battle go badly for the Greeks so they will see that they should not have dishonored him. Odysseus leads a group of Greeks to Chryse to return Chryseis to Chryses. Meanwhile, Achilleus isolates himself from the other Greeks. Thetis, begs Zeus to honor her son, Achilleus, by turning the battle against the Greeks so they will see that they need him.

128 Challenge: First book of Iliad 128 STRIPS is an action-centered representation that models which properties are true at a given state Agamemnon is at the Greek camp Chryseis is kept captive by Agamemnon In Iliad though most crucial to the plot is what the heroes believe, desire, intend Chryses desires that Chryseis is not kept captive by Agamemnon So apart from facts that are true and false we need to reason about modalities over these facts

129 Challenge: First book of Iliad 129 Some facts about Iliad first book in STRIPS At(Agamemnon,GreekCamp) HoldsCaptive(Agamemnon,Chryseis) A desire modality is not over objects of the domain, e.g., people, places, etc, but over facts Desires Chryses ( HoldsCaptive(Agamemnon,Chryseis)) There are logics for representing and reasoning over such modalities, but we want to use existing real-time STRIPS planners, which don t support them

130 Challenge: First book of Iliad 130 There is some recent work particular on intentions IPOCL: Intent-based Partial Order Causal Link planner Riedl and Young, JAIR Narrative Planning: Compilations to Classical Planning Haslum, JAIR

131 Challenge: First book of Iliad 131 A practical solution that would be intuitive also to game developers and story designers Reify modalities into objects of the domain, and use a special mood predicate to represent the disposition of characters toward each other Mood(Agamemnon,Achilleus,Negative) Mood(Chryses,Agamemnon,DesireRelease) Any modalities can be represented as needed, e.g., Angry, Punish, DesireCapture, etc

132 Challenge: First book of Iliad 132 The interpretation of our mood predicate and objects do not follow formal modal semantics, rather than depend on any actions specified by the user We separate actions then in two categories Physical actions modify physical properties E.g., go, capture, release, punish, etc Interaction actions that modify dispositions interactmood(chryses,agamemnon,requestrelease) Also reified over a generic interactmood action that takes arguments of similar form as the Mood predicate

133 Challenge: First book of Iliad 133 Instances of interact-mood actions modify only the disposition between characters The disposition of characters enables different physical actions to be performed This provides a moods layer for representing the gist of the story as a game between the relationships of characters, while physical actions are just ways to materialize the tensions expressed in the moods.

134 Challenge: First book of Iliad 134 The goal of the planning problem is then expressed also solely in terms of the moods of the characters: Mood (Achilleus,Agamemnon,Negative) Mood(Zeus,Agamemnon,Punish) Mood(Chryses,Agamemnon,Good) Hence, at the end of the first book, Achilleus is upset with Agamemnon, Zeus is in the mood of punishing Agamemnon, and Chryses is happy with the former captor of his daughter

135 135 Challenge: First book of Iliad

136 Applications of planning in videogames 136 Characters in videogames: How do they think? Behavior Trees ( Reactive HTN-like planning) Goal-Oriented Action Planning (STRIPS-like planning) Utility-systems (BFS search) Interactive non-linear stories in videogames More!

137 Applications of planning in videogames 137 Conferences Foundations of Digital Games (FDG) Computational Intelligence in Games (IEEE CIG) AI for Interactive Digital Entertainment (AAAI AIIDE)

138 Applications of planning in videogames 138 Two relevant papers from last year NetworkING: using Character Relationships for Interactive Narrative Generation Porteous, Charles, Cavazza, AAMAS, Towards story-based content generation: From plot-points to maps Valls-Vargas, Ontanon, Jichen Zhu, IEEE CIG, = &tag=1

139 Applications of planning in videogames 139 Background Interactive Narrative: An Intelligent Systems Approach Riedl, Bulitko, AI Magazine Applying planning to interactive storytelling: Narrative control using state constraints Porteous, Charles, Cavazza, ACM TIST,