Player Modeling in the Interactive Drama Architecture. Brian S. Magerko

Transcription

1 Player Modeling in the Interactive Drama Architecture by Brian S. Magerko A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science and Engineering) in the University of Michigan 2006 Doctoral Committee: Professor John E. Laird, Chair Professor Edmund H. Durfee Associate Professor Satinder Singh Baveja Assistant Professor Andrew J. Kirschner

2 Brian Sadler Magerko All rights reserved 2006

3 to my loving kitty and our infant daughter ii

4 Acknowledgments The work presented here is the culmination of nearly a seven year journey from start to finish; far longer than I would have guessed when arriving in Michigan in the summer of I would like to thank my advisor, Professor John E. Laird, who has proven to be an excellent mentor and guide during my time as a graduate student. Without him, my writing would be much more obfuscated, my reasoning would be more poorly constructed, and I definitely would not have been as able to pursue such a non-traditional research topic. Prof. Laird gave me the perfect opportunity to apply my academic interests in creativity, improvisation, and artificial intelligence in an exciting new way, which I am wholly grateful for. I would like to thank Professor Edmund H. Durfee, as well as the rest of my committee, for pushing me to improve my work and to think about this research in a more critical light. Thanks to the Institute for Creative Technologies for funding the majority of this work and to the Computer Science and Engineering Department at the University of Michigan for giving me an excellent end to my career as a student. The teachers that I have had throughout my academic career have helped me in innumerable ways to reach this goal. I cannot express the how grateful I am for the aid an inspiration I received from guides like my kindergarten teacher, Charlanne Cress; my favorite librarian, Peggy Clark; my third grade teacher, Tammy Woods; my high school teachers Rick Hess and Marie Cotoya; and my undergraduate advisors Dr. Illah Nourbakhsh, Dr. Brian MacWhinney, and Dr. Herb Simon. I also could not have gotten through this work without the support of my friends and family, especially my roommates at the Country Manor; my faithful friends Jimmy and Richard; and my parents, Rick and Diane. The gifts of music, food, friendship, and love were unparalleled. iii

5 My biggest thanks go to my wife, Kitty, whose partnership through most of this ordeal has been spectacular. She has been an incredible supporter as I pushed through this work in Ann Arbor, Chicago, and finally our first home in East Lansing. Through dating; getting married; and having Lilith, our beautiful baby; I have been surrounded by nothing but love, affection, and unending patience from someone I truly respect and care for. For the short time she has been here, Lilith has made every day better with her infectious smile and joy at being alive. I am excited to put a close to this chapter of our life together and jump head over heels into the many ones left to explore. Now it is time to move on and on. iv

6 Table of Contents Dedication..ii Acknowledgments iii List of Figures...ix List of Tables.x Abstract.xi Chapter 1: Introduction Interactive Drama Contributions of IDA... 7 Chapter 2: Requirements and a Generic Interactive Drama Generic Interactive Drama Requirements for an Interactive Drama Author Requirements Experience Requirements Functional Requirements Chapter 3: Related Work Automated Storytelling Systems TALE-SPIN UNIVERSE MINSTREL BRUTUS Commercial Computer Games First Person Shooters Interactive Fiction and Adventure Games Real-world Storytelling Tabletop Role-Playing v

7 Live-action Role-Play Computational Storytelling DEFACTO MOE Interactive Storytelling Leaders Façade OPIATE MIMESIS Model-Guided Systems User Modeling Intelligent Tutoring Systems Summary Chapter 4: IDA: General Design of an Interactive Drama Architecture Haunt Author Synthetic Characters Chapter 5: Story Representation Requirements Story Representation in IDA Content Structure Implicit Story Elements Chapter 6: Story Director The Director Agent Director Roles Knowledge Maintenance Plot Monitoring Marking Plot Points Variable Instantiation Player Prediction vi

8 Plot Point Selection Executing Direction Director Capabilities Types of Direction Story Direction Reactive Direction Preemptive Direction Director Strategies Example of Director Execution Chapter 7: Player Modeling The Player Model Connecting Modeling to Director Action Example of Using the Predictive Model Chapter 8: Experimental Design and Results Defining Player Archetypes Experimental Design Experimental Biases Director Bias Player Bias Story Bias Results Chapter 9: Discussion Requirements Author Requirements Experience Requirements Functional Requirements Story Representation Future Work Adaptive Models Player Testing Interactive Training vii

9 Authoring Tools Categorization of Director Strategies Contributions Player modeling Representation Evaluation Next Steps Appendix 118 References..120 viii

10 List of Figures Figure 1. Two examples of story graphs...2 Figure 2. A general interactive drama architecture...9 Figure 3. An example of a formal relation for betrayal in BRUTUS. 21 Figure 4. DEFACTO plot manager architecture..28 Figure 5. Screenshot from Façade...33 Figure 6. The Interactive Drama Architecture.41 Figure 7. Synthetic characters in Haunt 2 44 Figure 8. Soar decision cycle...45 Figure 9. Subset of HauntBot goal hierarchy Figure 10. An example of partial-ordering of plot points 52 Figure 11. Entire partially-ordered content for Haunt Figure 12. Director execution cycle.64 Figure 13. Taxonomy of knowledge used in Haunt 2.65 Figure 14. A plot point involving conversation between John and Sally 73 Figure 15. John and Sally are in the lounge while the player explores the house...74 Figure 16. Example of copying information, including hypothesized player knowledge, to the modeling state for prediction..79 Figure 17. Probabilistic model of player behavior in Haunt 2 80 Figure 18. An example of predictive modeling being used.82 Figure 19. Bartle s archetypes.90 Figure 20. A plot point authored in Soar Figure 21. Screenshot of a prototype authoring tool..113 ix

11 List of Tables Table 1. Proposed director actions for attracting player in Figure Table 2. Basic statistics from experimental runs Table 3. ANOVA table comparing means of modeling and no-modeling x

12 Abstract The field of interactive drama attempts to provide a player of a virtual system with a rich dramatic experience that provides him with a large set of consequential actions that directly affect how that experience progresses. As the player makes decisions, the system incorporates those decisions and adapts the environment to continue the ongoing narrative. The amount of content possibly written for such experiences, however, is finite. In a highly interactive world, the player s capabilities may easily exceed the story experiences covered by the authored content (called the story space). A major problem in interactive drama is how to create a large story space and how to believably manage the player s experience so it remains within the boundaries of that story space. I present here the Interactive Drama Architecture (IDA) which addresses this problem. IDA employs a real-time story management agent, called the director, which attempts to subtly guide the player s experience to stay within the boundaries of an authored story space. A key approach to this story management is the use of a model of player behavior and knowledge, which is used to intelligently anticipate problematic situations before they occur. This dissertation describes the functions of the director agent, the story representation, the definition of synthetic characters and how they relate to the director, and an experimental design and results for evaluating the use of prediction for story management in an interactive drama. xi

13 Chapter 1 Introduction Given the inherent interactive nature of computer games, they are an obvious candidate medium for creating interactive stories (Crawford 2004). However, traditional computer games have great difficulty involving the player in a good story with any depth. A computer game that involves the player in a story is likely to be immersive, responsive, participatory, and sometimes even engaging. However, as Chris Crawford notes, the dramatic experience itself within a game may not necessarily be interactive; an engaging, immersive, and / or interactive world can lead to a mostly passive story experience since it offers no guarantee of a structured narrative experience (Crawford 2004). When a game designer attempts to incorporate a story into the experience, the player s dramatic experience is invariably constrained to a fairly linear set of dramatic events; providing interactivity in the story is precisely where story-based computer games falls short. Brenda Laurel s PhD thesis offers the first concrete definition of what it means for a game to be both story-based and interactive: An interactive drama, then, is a first-person experience within a fantasy world, in which the User may create, enact, and observe a character whose choices and actions affect the course of events just as they might in a play. The structure of the system proposed in the study utilizes a playwriting expert system that enables first-person participation of the User in the development of the story or plot, and orchestrates systemcontrolled events and characters so as to move the action forward in a dramatically interesting way (Laurel 1986). The Holodeck as envisioned on the television show Star Trek: The Next Generation is the quintessential example of computational interactive drama (Murray 1997). It is a multi-sensory three-dimensional environment that one or more crew 1

14 members can enter to play out dramatic roles in a story world, such as a 1940 s pulp fiction detective story (Scanlan 1988). One or more of the crew members programs what the environment will look like, what characters are in the world, etc. (which takes astonishing little time on the television show). In the detective story example, the Holodeck is programmed to create an experience for the Enterprise s Captain Picard where he plays the role of Dixon Hill, a 1940 s noir-style detective. He plays through a detective story as he desires, and the Holodeck adapts the world to his interactions with it. The story that is created in the Holodeck is as much influenced by the player as it is by the system. If Picard decides to shoot the femme fatale as she enters his office to give him a case to solve, the system adapts this new event into another storyline (e.g. someone else gives him a case or the story suddenly revolves around her murder by Dixon Hill). This kind of experience is precisely my goal for interactive drama: to create a system that presents story experiences that are directly influenced in consistent, meaningful ways by both the author of the system and the player as he intentionally acts out a role in a dramatic setting. Figure 1. Two examples of story graphs. The first is a very linear graph while the second represents offering the player some choices that branch out before collapsing down again into a small set of states. 2

15 The first concrete approach to representing player involvement in a computational narrative was the use of story graphs (also called branching storylines). This approach has been widely used for storytelling in different media, such as computer games and books, that allow the player to guide how a story progresses (Fahlstein 2005). A typical story graph, as shown in Figure 1, consists of choice points, where the player must make a decision in order for the story to progress forward, and directed edges out of that point that represents some static story event which leads to another choice point or an ending. These story graphs are a straight-forward approach to representing a story in a non-linear fashion where player choice plays a key role in how the story progresses. The use of choice points provide the player with a direct way of affecting plot, but do not offer the interactive experience described by Laurel above. The participation of the User that Laurel describes is minimal at best when using these kinds of structures to represent story because interactivity in a story is directly related to the amount of consequential choices available to the player. The common problem therefore faced in authoring dramatic experiences using story graphs is the combinatorial explosion of content that arises when attempting to cover the space of possible decisions made by the player; more choices equal exponentially more content. A typical approach to dealing with this problem in games is to constrain the possible actions that the player has to choose from so that only actions that are consistent with a linear or near-linear plot are available, such as in the graphs shown in Figure 1 (Fahlstein 2005). The other approach is to offer the player many choices, but to only make a small set of them actually consequential to the story. Neither of these approaches remotely provides the kind of deep and openended narrative experiences that a medium like the Holodeck could provide; the player does not have a large number of choices that can guide the story as he wishes as he could in the Holodeck. The more consequential choices that a player has available to him in a story world (i.e. as Laurel puts it, the choices and actions (that) affect the course of events ), the more interactive his dramatic experience can be; there is a direct connection between dramatic interactivity and agency (Laurel 1986; 3

16 Murray 1997). This assertion illustrates the undesirability of traditional story graphs, which have the aforementioned hindrance of requiring more and more content and edges for each outgoing edge added to a choice point. If content is authored in a less constrained manner (e.g. not having content directly tied to player choices, as in story graphs), then this problem is slightly relaxed. The author for a digital narrative can be viewed as creating a story space with her authored content, which describes the space of possible story experiences for the player that are explicitly detached from player actions / choices (Magerko 2005). For example, representing story content in a partial-order planning language allows the player to fulfill the preconditions of actions in many different ways, as long as the effects of that action logically fulfill conditions in the plan. The use of hierarchical task networks to represent story allows for abstract constructs to be instantiated in different ways (Cavazza et al. 2002). AND / OR trees are another general framework more flexible than traditional story graphs for defining story spaces, which could use a Boolean logic to describe possible transitions through story content. Writing content in terms of story spaces allows the author to rely on Laurel s playwriting expert system to find continuities between player action in the story world and how it relates to the story space (e.g. a planner that sees a precondition has been fulfilled) Interactive Drama In an interactive drama, both the player s decisions and the author s desires should coherently influence the player s individual story experience. Different player interactions with the system should yield different stories, just as different authored content would. By defining a story space, the author of the story content is communicating an artistic vision for the player to take part in. As opposed to having explicit choices for the player to choose from and constraining those choices, interactive drama attempts to offer the player a fluid, continuous dramatic experience, akin to taking part in an improvisational play where the player is the protagonist in the story (Kelso et al. 1993; Laurel 1986). The problem with relying on a playwriting expert system is that it becomes difficult to constrain the interactivity so that the player doesn t perform actions that 4

17 take the experience outside of the boundaries defined by the story space (i.e. the player executes an action that should have a dramatic consequence, such as killing a main character, but no content is written to cover the situation). This is a key problem when attempting to explore more relaxed approaches to interactive drama rather than relying on the use of story graphs. More player choices lead to more opportunities for him to harm the progression of the plot. I term this issue of player actions bringing a dramatic experience outside of the boundaries of authored content in an interactive drama as the boundary problem. Research in the field of interactive drama investigates methods for both providing the player with a greater sense of interactivity and addressing the boundary problem described above. These methods include natural language understanding and generation, which provides the player with a powerful means of interacting with synthetic characters and avoids the boundary problem by using catch-all phrases in response to ambiguous player inputs (Cavazza 2005; Mateas and Stern 2003; Zubek 2005); new story representations, which provide the means for creating larger story spaces (Fairclough 2004; Mateas and Stern 2002; Young et al. 2004); the creation of synthetic characters (Blumberg and Galyean 1997; Gratch et al. 2002; Loyall 1997; Reilly 1996); automated story management (Fairclough 2004; Gordon and Iuppa 2003; Magerko and Laird 2004; Weyhrauch 1997; Young et al. 2004); generating dramatic situations (Bringsjord and Ferrucci 2000; Fairclough 2004; Sgorous 1999); and developing methods for the incorporation or prevention of player actions that cause a boundary problem (Young et al. 2004). These approaches either rely on a) strategies that mainly apply to boundary problems in discourse as opposed to physical boundary problems (discourse), b) assumptions that many different dramatic situations can be reliably generated by rules (generation), or c) strategies that disallow player actions from having logical or expected consequences (prevention). They do not provide a means for subtly addressing the boundary issues that may arise in an interactive drama that arise from physical interactions with the world, such as attempting to kill a character that is key to future plot development. 5

18 My approach to address the boundary problem is to use an omniscient story director agent, implemented in the Interactive Drama Architecture (IDA), which uses a predictive model of player behavior to maintain the plot progression. Much like a human dungeon master does in some table top role-playing games, the director agent works with a pre-written story structure and attempts to guide the player through that story. The director follows along with the plot as it moves along, giving commands (called directions or director actions) to characters when necessary to perform particular plot elements. As opposed to waiting until boundary problems occur, the director agent attempts to predict the player s future behavior so that it can preemptively, though subtly, steer the player away from actions that may endanger the progression of the plot. Haunt 2, which is the game environment used by IDA that has been developed with the Soar Games group at the University of Michigan, consists of a fully structured story, synthetic characters that take part in the story, a 3- D world constructed with the Unreal Tournament engine (Magerko et al. 2004), and the story director agent, which is the focus of this research. The ability to preemptively direct is what distinguishes IDA most from other interactive drama systems, such as the MIMESIS architecture and the Crosstalk framework (Klesen et al. 2003; Young et al. 2004). MIMESIS uses a fully structured plot, represented as a partial-order plan, and either incorporates unplanned player actions into the story or avoids them altogether if incorporating them is infeasible. The CrossTalk framework incorporates plan-based automatic dialogue generation with an author-defined narrative graph. Other approaches to interactive drama have taken a more modular approach to plot construction so that there is no single coherent plot that is explicitly created by the author (Mateas and Stern 2002; Sgorous 1999; Weyhrauch 1997). They rely on heuristically choosing plot elements as the player moves through the space of possible stories. Some systems have also included a player history as a model of player experience to help heuristically choose what plot elements should occur next (Szilas et al. 2003; Weyhrauch 1997). What these systems do not address is how to avoid problematic player actions (i.e. boundary problems) before they occur (Beal et al. 2002). IDA focuses on making use of prediction to subtly guide the player s behavior to stay within the bounds of the authored story 6

19 content, and looks at methods for expanding the number of possible player experiences and thus the size of potential story spaces Contributions of IDA The main contributions of IDA that will be discussed in the subsequent chapters are: 1) the incorporation of semi-autonomous agents into an interactive drama architecture, 2) the use of a complete story director, 3) the development of a story representation to support the director s capabilities, and 4) an evaluation procedure for testing. The creation of semi-autonomous agents (i.e. intelligent agents that can take commands at different levels of abstraction or rely on their own autonomous behaviors) to perform story content is not a novel contribution (Blumberg and Galyean 1997), the incorporation of such agents into an overall architecture for interactive drama is. IDA s story director agent, which coordinates the behavior of these characters and is the focus of this work, has several novel features. The director executes actions in the world to reactively address boundary problems as they occur as well as to preemptively to avoid them. It uses a predictive model of player behavior to inform when to preemptively address boundary problems. The director also maintains a model of player knowledge which is used to inform the predictive model and allows the story representation to express content that is contingent on player knowledge. This story representation is designed to allow for the reactive and director actions as well as to support explicit pacing constraints for use by the author. The final major contribution is the design of a methodology for evaluating the capabilities of the director. Most evaluation in the field of interactive drama is anecdotal in nature. I present a low-cost methodology to show the benefit of using preemptive director actions in an interactive drama architecture. 7

20 Chapter 2 Requirements and a Generic Interactive Drama In order to investigate how useful one approach to interactive drama can be compared to another, it is useful to describe the general approaches and issues addressed in these kinds of systems. A generic approach to interactive drama is used here to discuss multiple frameworks with a common language. A set of requirements is key to examining the contributions and failings of past approaches, to informing the design and creation of a new approach, and to evaluate the success of that new approach. This chapter presents a description of a generic interactive drama, outlining its main features, as well as the requirements laid out for a system that would satisfy my goal of an interactive drama that addresses the boundary problem Generic Interactive Drama As suggested by Laurel s definition in Chapter 1, the main area of research in this field examines how the player can contribute to an interactive drama as the protagonist or main character in a story. Façade, for example, puts the player in the awkward situation of being friends of both sides of a couple as they wrestle through their relationship difficulties in front of him, each trying to involve the player in their side (Mateas and Stern 2003). Other approaches have explored how the player can interactively affect the story as an unseen entity, indirectly affecting the lives of the synthetic characters in the world (Cavazza et al. 2002). The means of realizing these approaches can be described in a general view of an interactive drama architecture, as shown in Figure 2. In this general description, there is a presentation medium, such as a computer, that a human player interacts with. The player can perceive the world presented by the medium and execute actions in that world. There is also an author that writes plot content, which could be character behaviors, story events, or 8

21 even logical rules defining how story should dramatically unfold, which is an input into the presentation medium. The author is viewed both as the creator of story content as well as the designer that creates the experience in the medium, which could mean writing programming code, creating visual art content, etc. These roles could be split across several people or performed by a single individual. The presentation medium takes this input and interacts with the player to create an immersive (i.e. gives the player the experience of actually being in a fictional world) dramatic experience for the player. User percepts actions authored content Author Synthetic Characters Objects Presentation Medium Figure 2. A general interactive drama architecture Requirements for an Interactive Drama The set of requirements for a general interactive drama architecture are organized to describe a) how the architecture supports authoring story, b) the quality of the player experience provided by the architecture, and c) the architecture mechanics that help create an interactive drama. The requirements are for a directorbased interactive drama, similar to Laurel s vision, which attempts how to deal with boundary problems that arise from physical interactions with the world. These 9

22 requirements will be a useful lens for discussing other systems, how the requirements have guided my architectural design, and how they can be used to evaluate for the resulting system. They will be used to point out the strong / weak points of my system during evaluation and direct future research Author Requirements A1) Authoring Story. In order to build an interactive drama, it is obviously necessary for the writer to author the story. The story can be written with more concrete representations, such as with a partial-order planning representation or as conversation graphs (Gordon and Iuppa 2003; Gordon et al. 2004; Mateas and Stern 2002; Young et al. 2004), or by relying on more generative methods (Cavazza et al. 2002; Fairclough 2004; Sgorous 1999). Story can be broken down into two different dimensions: story content and story structure (Rimmon-Kenan 2002). Story content is the who, what, where, why, and how of story. It describes the events that take place in story. Story structure describes how these events relate to each other temporally. It describes when events take place, in terms of fixed or relative time. An author needs to be able to specify story in both of these dimensions. A2) Building the story world. The story needs a virtual environment for the player to experience it. The author must be able to create an environment that is sufficiently complex and flexible for the player to move around and interact with. This includes defining the world s physics, designing and creating the virtual environment, and populating it with characters and objects. A3) Defining player actions in the world. The author needs to be able to define what actions or verbs the player can take in the world. This includes defining how the player can interact across different modalities with the environment, the synthetic characters, the objects in the world, and the player s own virtual entity. The author needs to create game mechanics that answer the question How does the player affect the story? 10

23 A4) Building synthetic characters. Unless the story involves the player as the only speaking character (e.g. a game based on the film Cast Away (Zemeckis 2000)), the author needs to be able to define the characters that will be involved in the story content. This includes a perception and action model, animations, behavior, coordination of behavior across characters with the story content, and the coordination of all of these elements to present a unified dramatic character. How characters coordinate with others to fulfill story-level goals as opposed to their own individual goals is also a design concern (Mateas and Stern 2002). A5) Expressiveness. Just as in any other storytelling medium (e.g. novels, plays, or films), it is important to have a full range of expression to communicate the author s artistic vision. This means being able to specify the canonical who, what, when, where, why and how of authored story content. Any representation that is used to describe a story in an interactive drama should provide for the same degree of expressivity. The author needs to be able to describe what happens in a scene, who is involved and how, where and when the event takes places, and a reason for why the event even occurred (related to Requirements A1 and A4). For example, if a traditional STRIPS planning representation (Weld 1994) is used, it would be difficult to specify the pacing of when events occur; they could only be described temporally and causally in relation to other actions Experience Requirements E1) Interactivity. In an interactive drama, the player should have a wide range of consequential actions available to him during the game (Crawford 2004; Mateas and Stern 2000; Murray 1997). As the player acts in the world, the story should adapt and respond to the player s actions, akin to how a traditional story flows with the choices made by the protagonist. The more interactive a dramatic situation is, the more consequential actions are at the player s disposal and the more possible story futures exist. This means a system should constrain the player's experience only when necessary. The holy grail of interactivity, as illustrated by the capabilities of the Holodeck, would be to offer the player as many choices in different modalities as possible (e.g. natural language interaction, realistic physical interactions with the 11

24 world, affecting social relationship, etc.); any significant player choice would have dramatic consequences on an interesting, personalized storyline (Murray 1997). E2) Believability. The boundary problem occurs when a player executes an action that pushes the story beyond the bounds of what has been written by the author and the progression of the story has halted. If the system makes changes to the world that the player may observe, either directly or indirectly, then it follows that the changes should seem as dramatically realistic and believable as possible within the context of the drama. Any content witnessed by the player should be presented in the context of the dramatic situation if at all possible. For example, having the hand of God suddenly come down from the heavens and point at an important object the player missed would be a poorly transparent and unbelievable system intervention (except perhaps in the case of telling a Greek myth). An interactive drama should provide for the possibility of more subtle direction of the player s experience. While at times very heavy-handed approaches might be necessary, the system should highly prefer more subtle, believable actions whenever possible Functional Requirements F1) Coordinating story and player actions. The system should maintain coherency between what the player is trying to do in the world and the story space created by the author. This means that the system should have some means of monitoring the progression of the story (e.g. observing all of the changes in the story world) and dealing with when the player s actions bring the story experience outside of that covered by plot content (i.e. the boundary problem discussed in Chapter 1). This also includes the performance of plot content by the synthetic characters when necessary (e.g. the characters autonomously go to a room to introduce themselves to the player at the beginning of a story). F2) Preemptively avoiding boundary problems. The system should be able to intelligently anticipate boundary problems before they occur. With this anticipation, the system can selectively take preventive measures to avoid boundary 12

25 problems before they occur. These preventative measures, as discussed in Chapter 1, may be a more subtle approach to boundary problems than waiting until they occur. F3) Standardized, Multi-Level Representation. The representation used by the author to encode story must not be specific to one particular story. In order to build an interactive drama architecture, one must have a story representation that is general enough to allow the description of different interactive stories in many domains. This representation should also not be over-constrained, forcing the author to describe details at a particular level of abstraction, such as writing every single detail. Just as a film director may give highly abstract commands to the background characters (e.g. hang out by the bar ) and very specific commands to the main characters in a scene (e.g. Look tense, say your dialogue, then walk out of the room in a hurry), the author of an interactive drama should be able to do the same. The author should be able to express story events at the desired level of detail as opposed to having to specify every single detail or only have one possible level of abstraction out of many possible ones to use. F4) Semi-Autonomy. The director should avoid relying on high-bandwidth communication with the characters, dictating every possible detail of their performance (Mateas and Stern 2000). It would be difficult, if not impossible, to author every single behavioral response directly in the plot. However, if a leastcommitment representation for story content is to be used, then the story world must be populated with synthetic characters that are able to execute commands at various levels of detail, thus relying on them to interpret abstract commands based on their individual behavioral definitions. Therefore, characters should be both rational and semi-autonomous. This means that the characters actions should be influenced by their internal goals, which in turn are influenced by or represented in story content. Characters could therefore execute behavior along a spectrum of autonomy. They would have individual goals they can try to achieve on their own (Blumberg and Galyean 1995), and could be given plot-specific goals can range from very specific actions (e.g. perform dialogue #111 to Sally, then run out of the room 13

26 screaming ), to very general ones(e.g. explore ), or any level of abstraction in between. 14

27 Chapter 3 Related Work This section examines five different fields that are related to this research: automated storytelling systems, commercial computer games, real world storytelling, computational storytelling, and model-guided systems. Automated storytelling systems are the artificial intelligence approach to generating complete stories from a set of knowledge. Computational storytelling and commercial computer games involve approaches to using the computer as a medium for players to interact with a virtual story world. Real-word storytelling refers to the more interactive, story-based experiences that exist in the entertainment realm. Model-guided systems refer to the systems that employ a model of the user to guide how the system works or presents information. These five fields will be compared to the requirements laid out in Chapter 2 and be used to guide the design of my interactive drama system Automated Storytelling Systems Automated storytelling is the direct digital predecessor of interactive drama. This field has focused on creating programs that use pre-authored knowledge and logical representations of narrative structure to create new stories without any realtime input from a player. The main difference between automated storytelling and interactive drama is the lack of interaction in automated storytelling. The player simply reads or watches what an automated storyteller produces, rather than actively taking part in the experience. However, these systems deal with problems commonly 15

28 found in interactive drama, such as plot representation, character behavior, and story structure. The systems I will discuss, TALE-SPIN, UNIVERSE, MINSTREL, and BRUTUS, are expressive in terms of the kinds of stories that can be told with them. They are based around the creation of a standard representation that can be used to describe characters and their goals, the story s setting, etc. In other words, these systems are provided with the elements of a story so that a complete story can be created. These systems have complete control over their characters, unlike interactive dramas which have a human character typically involved as the protagonist. Approaches used in automated storytelling, aside from the direct control of synthetic characters, do not directly transfer to interactive drama unless the player's actions are constrained down to the specifics in the created story, which then loses any sense of interactivity. However, these systems do supply a standard representation for authoring narrative systems and make suggestions for how generative techniques could be used in interactive drama TALE-SPIN TALE-SPIN was one of the first major contributions to the field of automated storytelling (Meehan 1981). Its goal was to generate complete stories derived from the goals of the simulated story characters. Story content is represented as character goals and operators that can be used to achieve those goals (fulfills Requirement A1 by providing a representation for authoring content). Operators may represent a set of subgoals or an atomic action that changes the world state (partially fulfills the multilevel representation requirement, Requirement F3). The most significant issue with TALE-SPIN is that the storylines are completely driven by the characters internal goals, which are not the same thing as story goals (e.g. plot lines that involve cooperation or a character portraying a behavior for dramatic effect, not necessarily personal gain) (De Beaugrande and Colby 1979; Dehn 1981; Lebowitz 1985; Mateas and Stern 2000). This restricts the kinds of stories that can be told (does not contribute to Requirements A5 and F3). 16

29 A story is based in conflicts between characters goals. The story is comprised of the results of the actions executed by characters as they attempt to achieve their goals. The TALE-SPIN algorithm has several components: an assertion mechanism that records world events, meaning that the event is recorded and all of its consequences are computed and likewise asserted a description of the physical world, the knowledge in that world, and the social relationships between characters an inference mechanism that assesses the effects of the events that it generates The parameters for the story world (e.g. which characters are involved, what props populate the world, what the characters relationship is, etc.) are first determined by a dialogue between the user and the system (explains how the world is built for Requirement A2 and shows limited player involvement in relation to Requirement A3). TALE-SPIN starts a story by using this input to create a problem for the main character (e.g. Sam Bear is hungry), setting the stage, and then reasoning about how the story takes place via an inference mechanism. This mechanism initiates story content by examining initial user input about the initial conditions of the story world along with character goals. It then decides which of a set of planning operators, called planboxes, will achieve the desired character goals. TALE-SPIN s approach, as is typical in story generation algorithms, does not include the player as a character in the story (no player actions or interactivity defined for Requirements A3 and E1). TALE-SPIN s choice of a planning language for a story representation has been used effectively in other story generation and interactive drama systems (Cavazza et al. 2002; Young et al. 2004), and has the flexibility to represent different stories centered around character goals (fulfills Requirement A5 for stories built around character goals only). However, Meehan admits that Good stories are coherent and interesting on many levels of meaning the more, the better. This may be the hardest of the three areas to develop, because it presumes that we know how to represent meaning on many levels. Meehan suggests here that different coherent levels of meaning, such as symbolism, metaphor, or literary themes, are important elements for a good story. This issue is reflected in the 17

30 limitations of TALE-SPIN s choice of only focusing on character goals as the driving force for generation of story content UNIVERSE Lebowitz's UNIVERSE (Lebowitz 1984; Lebowitz 1985) is an automated storyteller that casts story-telling as a planning and learning exercise. UNIVERSE focuses on the story of interpersonal melodrama, namely the kind of plots found in soap operas. It makes use of complex social relationships between characters, having the characters act both consistently and coherently in a drama. To start a new UNIVERSE story, the system first simulates a back story between the set of characters specified (rich character definitions help fulfill Requirement A4). This simulation helps define the individual traits, personal goals, and inter-character relationships for each character. A library of plot fragments, or plans, provide a narrative means for achieving the story goals (story representation to author story fulfills Requirement A1). Plot fragments are broken down into characters, constraints, goals and subgoals. The characters for a plot fragment list the variables that represent the different characters involved in the plot fragment. The constraints are additional restrictions that dictate which characters can be bound to these variables (e.g. the?her variable must be a female). The goals listed are the goals that the plot fragment can be used to achieve. The partially-ordered list of subgoals is the main content of each plot fragment. It is a list of the goals that must be achieved to carry out the plot fragment. This representation allows the author to define heavily character-driven stories that are guided by the set of author-defined goals. Lebowitz stresses that the main difference between UNIVERSE and approaches like TALE-SPIN is that the plot elements should not be viewed as the goals and plans of the characters, but rather those of the author, which may be a human or program (better fulfills Requirement A5 than TALE-SPIN by focusing on story goals as opposed to character goals). This is an important viewpoint of representation that character behavior is directly tied to how the author wishes them to behave as opposed to an intrinsic definition of character (Szilas et al. 2003). This stance is a response to the issues brought up with TALE-SPIN s approach, providing 18

31 author-centric intentionality into the storytelling mechanism rather than relying on character simulations to generate interesting and complexly interleaved plot lines. The story-telling algorithm used in UNIVERSE is similar to that used in TALE-SPIN. The system maintains a precedence graph that records how the various pending author goals and plot fragments relate to each other and previously used events. It then selects an author goal to expand, and recursively continues this process until enough goals reach ground level (i.e. actual events), at which point the events would be told to the reader by using natural language generation methods. UNIVERSE, like TALE-SPIN, does not represent the player as a character in the story (does not fulfill Requirements E1 and A3). However, the plot representation used here, which is a descendant of TALE-SPIN s representation, has had significant effect on representation choices in interactive drama. This author-centric approach significantly contributes directly to my view on plot representation, which is discussed later in Chapter 5, as well as the representations in other systems (Fairclough 2004; Mateas and Stern 2002; Sgorous 1999; Weyhrauch 1997) MINSTREL Turner s MINSTREL system applies a formal theory of the creative process to automatically generate stories about King Arthur and his Knights of the Round Table (Turner 1994). Turner views authoring story as a problem-solving exercise, meaning that he asserts that approaches used for problem solving in non-creative domains can be used in creative ones, such as writing stories. MINSTREL is a type of case-based reasoner, The steps in MINSTREL s problem solving process are: 1) identify a problem to solve, 2) recall a past problem similar to the current problem, 3) adapt the past solution to the recalled problem to the current problem, and finally 4) apply the adapted solution to the current problem. Turner posits that creativity is an integrated process of search and adaptation. This integrated process is realized in MINSTREL in its Transform- Recall-Adapt Methods (TRAMs). Each TRAM is coupled with a corresponding adaptation. For example, TRAM:Cross-Domain-Solution is used to encapsulate the notion that tactics in one domain can be used to solve problems in another (e.g. 19

32 military tactics can be used in the business world). This heuristic, when selected, will guide the search process to find a creative solution to a story problem. The use of TRAMs for adaptation serves Turner s purpose of generating creative stories. MINSTREL also attempts to makes stories that are interesting, understandable, and artistic. MINSTREL has four author-level goals that correspond to these concerns: thematic goals, consistency goals, drama goals, and presentation goals. Thematic goals help drive the selection of story content to center around a chosen theme (e.g. history repeats itself ). Consistency goals focus on assuring that a story is plausible and believable. Drama goals involve employing dramatic writing techniques to make the story more enjoyable. Presentation goals deal with attempting to make the end presentation of the story a pleasurable one for the reader. A major criticism of MINSTREL is that it attempts to logically represent the process of creativity, which Bringsjord and Ferrucci assert is too difficult to represent in a computer program. They posit that creating a program that appears to be creative is a more appropriate and realizable goal (Bringsjord and Ferrucci 2000). Others have speculated that the large amount of code needed (over 27,000 lines of code) to produce only a few stories is indicative that a larger problem with MINSTREL was that it was very difficult to use such a cumbersome application to reliably produce different stories (e.g. Turner reported difficulties in adding a new theme to the story content after completion) (Wardrip-Fruin 2006). Turner s theory of creativity ignores concepts like character development, internal dialogue, rich environment descriptions, etc. (fulfills Requirements A5 and A1). There is also no concept of interactivity in this work, which mirrors the entire field of story generation (does not fulfill Requirement E1) BRUTUS BRUTUS is a story generation approach that relies on a rigorous definition of dramatic principles, such as betrayal or heartbreak, to generate stories that reflect those principles (Bringsjord and Ferrucci 2000). The main goal for building BRUTUS is to generate stories that are sufficiently distant from initial knowledge representations (which they call "creative distance") and vary independently across different dimensions, such as character, literary themes, imagery, etc. The authored 20

33 initial components for the story generation are two large bodies of knowledge, domain knowledge, and literary knowledge (addresses how story is authored for Requirement A1). Domain knowledge is a formal description of the story domain, including objects, attributes, relationships, goals, and events. Literary knowledge is the set of principles that are used to encapsulate telling a good story. This includes using thematic knowledge to engage the reader in classic themes and story grammars to instantiate classic story structures. Thematic knowledge is a formal definition of a thematic concept, such as the one shown in Figure 3. This approach allows the author to guide the generation of stories to any defined literary theme (allows for a broad range of themes for expressiveness related to Requirement A5). A story grammar is a formal grammar that describes how story can be recursively decomposed until atomic statements are reached (e.g. Story Setting + Theme + Plot + Resolution, Setting Characters + Location + Time, etc.). This gives the author the choice of generating story according to a specified structure (flexible enough to address many of the surface elements in Requirement A5, such as who, where, and when). Relation betrayal_p(a_betrayal) If Evil is some goal whose plan is an EvilPlan And whose agent is a Betrayer and Saying is included in the EvilPlan and Saying is some say and Thwarting is included in the EvilPlan and Thwarting is some thwart and Betrayeds_Goal is the prevented_goal of Thwarting and Betrayers_Lie is the theme of the Saying and Betrayers_Lie is some support of the Betrayeds_Goal and Betrayed is some person whose goal is the Betrayeds_Goal and whose beliefs include the Betrayors_Lie. Figure 3. An example of a formal relation for betrayal in BRUTUS. BRUTUS algorithm begins by instantiating a selected thematic concept using a specific domain knowledge base. This identifies the particular objects, events and 21

34 characters involved in the story (called the stage). The system then generates a scenario through planning and simulation, which is comprised of a stage plus a complete set of story events. A story outline, which is a detailed template describing each paragraph and sentence type for the generated text, is produced from a selected story grammar in parallel, independent of generated content. The final process outputs the story text based on the story outline and the generated story events. While BRUTUS does not provide the means for player interaction, the space of possible stories is increased due to BRUTUS focus on creating different stories that may come from the same thematic material (fails to fulfill Requirement E1 and A3). The authors of BRUTUS have also built a system that takes story-related knowledge as input; a human author (or in their case it may be better to say ``knowledge engineer'') is a part of the overall architecture. What is unclear in this approach is how practical it is for authoring many different kinds of stories (failure to meet Requirement F3 for applicability to different story domains). A large investment in knowledge engineering is needed (e.g. domain knowledge-base, character definitions, rigorous definition of thematic concepts, etc.), which the authors admit is incredibly time-intensive. Overall, BRUTUS illustrates the means to center computer-generated drama on core literary themes, and also highlights the difficulty in proceeding with this process from a knowledge-engineering level for a single theme Commercial Computer Games Commercial computer games are a highly interactive form of entertainment that often incorporates storytelling into gameplay. Even in the early history of the industry, interactive fiction games were attempting to involve the player in a story as the main character. Interactive fictions, such as Zork (1980) or the infamous Hitchhiker s Guide to the Galaxy game (1984a), are text-based experiences where the main input to the player is a rich textual description of the environment, items nearby available to him, etc. The player can manipulate objects in the world, interact with scripted characters, and navigate through a very discrete physical space (i.e. maps or dungeons are typically organized as directed graphs, with rooms as nodes and the 22

35 connections allowed between them, such as entryways or trapdoors, as directed edges), but the player s actions ultimately had only limited impact on the evolution of the story. Storytelling in computer games has been involved in each new generation of games. Games that typically include some aspects of storytelling are first person shooters, interactive fiction games, and adventure games. Approaches to interactive fiction heavily influenced the design of graphical-based adventures games, such as the Sierra Games King s Quest series (1984b) or more contemporary adventure games like Grim Fandango (1998), Syberia (2002), or The Indigo Prophecy (2005). The issue with these games and with interactive digital storytelling in general is the continuing tension between interaction and story First Person Shooters This genre of computer game (referred to as FPSs ) involves presenting the player with the first person view of a character in a real-time three dimensional world (virtual environment provided for Requirement A2). The experience is typically centered on combat against hordes of enemies, with the player progressing to the end of each level in the world until he reaches the end of the game. Possible player actions are normally small in number (e.g. walk, shoot, change weapon, jump, get health, run) (player actions defined to fulfill Requirement A3). There is usually a basic plot involved with the experience, but it is completely linear with the player being led through it step-by-step (fails to fulfill Requirement E1 and A1). Though the stories in FPSs are typically quite shallow (along the lines of Aliens are attacking go kill them! ), there have been steps to make FPSs more focused on story. Deus Ex is a prime example (2000). Deus Ex still has combat as a main part of the gameplay, but it also tries to provide a large, richly-defined environment that allows the player to choose one of many different possibilities for accomplishing a given mission (meets Requirement E1 more than most commercial games). By interacting with different characters, or by choosing different objects to use during the game, the player may experience several different plot-lines with variances in how those plot-lines play out. 23

36 Even in the most well designed FPS story experiences, such as Deus Ex, the consequential choices offered to the player are still quite small in number (fails to fulfill Requirement E1). Even though flexibility can be increased, the number of different experiences on a dramatic level is still minimal Interactive Fiction and Adventure Games Interactive fiction games are text-based story games that provide the player with a virtual world that is described by pre-authored text. The player can typically execute commands from a command prompt (e.g. flip switch or north to move north) in a turn-based fashion to move around and affect the world. These types of games offer a more specific dramatic character for the player to assume, which is contrary to the gung-ho, transparent characters that are used in FPSs. The gameplay is more centered on a narrative experience and less around combat. They initially offered rich game text-based experiences at a time when computers had little graphics capabilities; the environment is all described with static text (defines how Requirement A2 is met). Popular titles like The Hitchhiker s Guide to the Galaxy, Planetfall (1983), and Zork offer a first-person dramatic experience that involved the player examining his surroundings, solving puzzles, and interacting with fairly shallow synthetic characters all through simple text commands. Adventure games, like the Myst series (Kelso et al. 1993) or Gabriel Knight 3 (1999), are the graphical descendants of early interactive fiction. These types of games often offer a first-person experience in a three-dimensional world, much like the FPSs discussed in Section (defines how Requirement A2 is met). However, these games focus much more on problem-solving and moving through a narrative structure with the player typically as the protagonist in a mystery story. Interactive fiction and adventure games offer a narrative experience without any large degree of interactivity (fails to meet Requirement E1). For instance, if a player is playing through Gabriel Knight 3 and does not behave exactly how the story describes, then he will effectively be stuck in the plot. The burden is put on the player to figure out what is expected of him in order to advance the story. A more interactive, author-centric system would encourage the player to advance the plot if it noticed that his behavior was lagging with what was desired in the narrative. 24

37 The replay value of these systems has been a major issue in game design. They have yet to reach the point of offering significantly different narratives due to different player behaviors. This has been largely due to the lack of a significant amount of available consequential player actions (fails to meet Requirement E1) and corresponding authored content. Another aspect that these games are lacking in is the story representation used. Story in both genres is represented as a story graph, where the player moves from one state to the next after he has found a new clue, solved a puzzle, or executed some misstep (defines how story is authored for Requirement A1). As discussed in Chapter 1, story graphs are a representational dead-end to authoring interactive drama (they fail to fulfill Requirements E1 and A5). Although modern interactive fiction strives to push on this limitation, the benefits of a good interactive fiction experience are mainly from the superb authorship of different plot lines and the smooth integration of puzzles. This is the largest difference between these genres and interactive drama; the focus on dramatic interactivity suffers in these genres Real-world Storytelling There are several non-digital examples of interactive drama that have existed for some time, such as tabletop role-play (e.g. Dungeons and Dragons (Cook 2003)), live-action role-play (LARP), and improvisational theatre. Each of these entertainment mediums provide an interactive experience where the line between performers, audience and author is blurred Tabletop Role-Playing Tabletop role-play typically involves a set of players who create their own characters within a certain set of game-specified constraints (e.g. the intelligence of the character or race) and a central storyteller called the dungeon master, whose role is to set up the narrative framework for each session of a gaming experience, typically basing the current session on what has occurred previously (Pearce 2001; Toles-Patkin 1986) (the dungeon master mediates between story content and player actions to fulfill Requirement F1). The dungeon master is responsible for molding either a pre-existing narrative structure from a printed game manual or one that she 25

38 has authored on her own to the players characters and actions in the world (which describes how content is authored per Requirement A1). At the same time, the dungeon master is also responsible for ensuring that the players actions don t take the story in an unplanned direction (i.e. railroading the players), such as attempting to kill off an important non-player character (called an NPC). The dungeon master may perform this task by observing the players actions and hypothesizing their intent (which is a method of anticipating boundary problems per Requirement F2). This medium fulfills many of the requirements discussed in Chapter 2 through the use of a dungeon master that coordinates and fills in story content. This provides the players with a flexible, interactive story experience (fulfills Requirements A1 and E1). The story experience itself is limited only by the players imaginations, making it a highly expressive medium (fulfills Requirement A5). The largest missing pieces are the synthetic characters for players to interact with, though the dungeon master can fill that role to an extent, and a virtual world for the story to take place in (fails to fulfill Requirements A2 and A4) Live-action Role-Play Live-action role-playing ( LARPing ) is a similar game genre to table-top role-playing but is more geared around the improvisational performance of the players in a real-world environment. Players act out the part of their character in realtime, following a loosely-authored narrative that the game s storytellers have devised. LARPing depends on a sizeable community to organize an outing, and acquire costuming. The participants imaginations then transform their surroundings (e.g. a state park or a dorm room) into the world their story takes place in (e.g. an elven forest of Tolkein lore (Tolkein 2003) or a sci-fi cantina). While LARPing does make a real-world environment populated with other characters possible, it does come at a large organizational cost (fulfills Requirement A4 with a large cost involved). It also does not have the possibilities of a visual environment that a computer can provide or a finer-grained coordinated story, which affects the kinds of stories that can be experienced (does not fulfill Requirements A2, A5 or F1). Improvisational comedy involves actors on a stage performing for an audience (Johnstone 1987). The audience is usually asked for suggestions to guide particular 26

39 scenes depending on the kind of improvisation being performed (e.g. a theme for the scene, the particular job of one of the characters, or a movie title). In some particular games, one or more audience members are actually brought up on stage to take part in a game. Here the line between audience and performers is blurred; an improvisational theater performance is more about a collaborative dialogue between the audience and the performance. The interaction between audience and story does exist, but in a very restrained manner (slightly fulfills Requirement E1). Each of these mediums gives a hint as to what interactive drama can be. The notion of passive audience and active performance are no longer applicable as in traditional visual storytelling genres, such as theatre or film. The stories experienced are largely guided by the choices of the human players involved. Each medium has an established place in our entertainment culture, but also has a cost associated with it and depends heavily on the internal imagination of the players experiencing versus presenting a tangible story world. They seem to be rich in story, but are sparse in comparison to the kinds of interactive worlds computer games offer Computational Storytelling The computational approaches to interactive drama strive to create a new narrative medium that involves the player as a main character in a play that responds to his choices in the storyworld (see Laurel s definition in Chapter 1 of interactive drama). Computational storytelling offers players the chance to take part in a dramatic situation within a visually-rich computer generated storyworld. I will address the major relevant systems that have been built to date: DEFACTO (Sgorous 1999), MOE (Weyhrauch 1997), the Interactive Storytelling system (Cavazza et al. 2002), Façade (Mateas and Stern 2003), MIMESIS (Young et al. 2004), and OPIATE (Fairclough 2004) DEFACTO DEFACTO approaches interactive storytelling by relying on logical definitions of dramatic concepts to guide the story, much like the story generation system BRUTUS does (Bringsjord and Ferrucci 2000; Sgorous 1999). The purpose of the DEFACTO architecture is to provide a computational framework for 27

40 Figure 4. DEFACTO plot manager architecture (from Sgorous 1999). supporting dynamic generation, management and resolution of interactive plots (Sgorous 1999). The initial inputs to the system describe initial plot conditions, role descriptions for characters, and rules on how the social world in the story acts (defines how Requirement A1 is met). As the player interacts with the world, the Plot Manager continuously executes a decision-making cycle of generation, evaluation, and resolution. This cycle, shown in Figure 4, involves the proposal of possible character interventions in the world, which are different character actions that would 28

41 have an effect on the player's experience. Rules are defined in DEFACTO to evaluate these proposals based on their dramatic merit. An example dramatic situation encoded into DEFACTO is the lifeline. The lifeline rule describes an unfavorable event occurring by the hands of a non-player character followed by a favorable event. A suitable character intervention is chosen and then enacted in the world, leading to the beginning of the cycle again. The DEFACTO approach to interactive drama focuses on the generation of content through dramatically-focused character intervention; there is no directly authored plot constructed beforehand. Instead, the author of the experience defines character roles, social constructs, and dramatic principles in order to indirectly specify the player's experience. While this possibly provides a more interactive experience, it is limited in terms of artistic expression (fails to fulfill Requirement A5). Specifying dramatic rules so they are both general enough to cover many similar situations, but specific enough to have any real meaning, is precisely the difficulty pointed out by Bringsjord and Ferrucci (Bringsjord and Ferrucci 2000) MOE MOE is the part of the Oz architecture that focuses on search-based story direction (Weyhrauch 1997). The goal of MOE s design was to build an automated story director that helped guide the player through a dramatic experience. It relied on searching through possible complete stories to inform its decision about how to guide the player. Story is represented as partially-ordered USER MOVES (defines how Requirement A1 is met). A USER-MOVE is a player action in the storyworld that moves the story along (Weyrauch s example is finding a new clue in solving a mystery). MOE, the name of the system s story director, can affect the player s behavior by executing actions in the world (called MOE-MOVEs) (defines how the system can influence the player to stay within the story space per Requirement F1). The author encodes quantitative and qualitative measures about what a good story progression is in a heuristic function, using such features in the function as intensity and story momentum. MOE employs an adversarial search algorithm to search through possible future states and heuristically evaluates them according to the collection of metrics 29

42 provided by the designer of the drama. Once the best-rated future is chosen, it is up to MOE to choose a MOE-MOVE that will encourage that future. For instance, if the next scene should take place in the study and the player is in the kitchen, a selected MOE-MOVE might be to have a loud crash audible from the study, attracting the player to check out that room. MOE s approach to interaction is to provide a connection between player actions, authorial desires (via the author-defined heuristic for search), and the temporal ordering of content. While MOE was a good step towards exploring interactivity in a dramatic context, it lacks any connection between scene content and player behavior. With MOE, the ordering of the scenes may vary as well as the MOE-MOVES used, but the content of each scene is fully specified beforehand (fails to fulfill Requirement E1). As Nelson and Mateas have discovered, the results of this approach do not appear to generalize across different stories (fails to fulfill Requirement F3) (Nelson and Mateas 2005) Interactive Storytelling Marc Cavazza and Fred Charles' work on Interactive Storytelling has focused on building an interactive storyteller that uses autonomous actors to dynamically interact and generate story in real-time (Cavazza et al. 2002). Their approach to player interaction with the environment is novel; the player does not participate as a character involved in the plot (fulfills Requirement E1, but not with the player as a main character in the story). The character is relegated to the role of a trickster or mischievous entity who is given the power to manipulate objects in the environment. Roles are defined for each synthetic character in the forms of plans, which are broken down as hierarchical task decompositions (fulfills Requirements A4 and F3). For instance, for one character to accomplish his goal of getting a date with a female character, he must fulfill several subgoals, which in turn have subgoals associated with accomplishing them, etc. The player can directly affect the characters by either manipulating physical objects that are on stage or by advising the characters via speech recognition. This system has a heavily emergent plot; authorial control over story is evident only in the starting state of the world. As the characters execute their 30

43 behaviors in the real world, they replan as best they can when obstacles to reaching their goal occur, either because of intervention by the player or as the result of another character's behavior. This approach appears to be similar to that of TALE- SPIN; story is driven by character goals as opposed to authorial ones (limited fulfillment of Requirement A5). Current research work for this system has focused on natural language generation of conversations between synthetic characters (Cavazza 2005). This approach to interactive drama does offer interactivity with a story, but in a very peripheral manner (limited fulfillment of Requirement E1) since the player is not actually in the story Leaders The Institute of Creative Technology s Leaders project is a system that attempts to combine Hollywood-style storytelling techniques with game technology for military training (Gordon et al. 2004). Leaders explores the use of branching dialogue-rich storylines as a representation for story (story representation used to fulfill Requirement A1). It focuses on a particular method of authoring called Outcome-driven Simulations. This approach creates branching storylines where every trainee decision is motivated by a specific training point and every action will lead the trainee to pedagogically important outcomes. Once in a situation, the trainee can interact with synthetic characters in the world by typing English statements. The system is trained with a Naïve Bayes classification algorithm to map player input to system responses. Leaders makes use of the highly constrained vocabulary and behavior typical in a military setting to map out the anticipated questions and decisions that could arise in a situation. The character response to player input is represented either as a unary edge in the graph (e.g. the players asks a question to clarify the situation before making a decision and a character answers) or as a binary edge pointing to a new node (e.g. the player making a firm decision and giving the character an order on how to deal with the problem). Character speech is purely determined by the story graph content (how character behavior is defined for Requirement A4). Leaders relies on a story graph to represent the space of possible conversations between the trainee and the synthetic characters in a highly-constrained 31

44 social situation (i.e. it relies on typical military dialogue instead of having to incorporate possible everyday civilian English conversations). Even then, it is arguable that using a fixed story graph is still inadequate for providing a suitably interactive experience. The author is expected to write all likely paths and conversations in order for the system, which has been pointed out in Chapter 1 as a difficult bottleneck to overcome for authoring interactive drama (fails to fulfill Requirement E1). The system also primarily focuses on conversation as the only means of interaction with the world as opposed to also addressing the boundary problem issues that arise during physical interactions with the world Façade Façade (see Figure 5) is the first fully implemented interactive drama system (Adams 2005; Mateas and Stern 2003). Façade s goal is to offer a complete, realtime dramatic experience with a highly interactive, character-driven story for the player to play a key role in. The player walks into a married couple s apartment at the beginning of the experience. The player, who is introduced as friends of both of the characters, is quickly put into a tense position when they start bickering more and more in front of the player, subtly (and at times not so subtly) vying for the player to be on their side. Façade avoids describing narratives with an explicit branching structure and instead introduces a representation language called ABL to specify both plot and character behavior in an intertwined fashion (Mateas and Stern 2002). The smallest narrative units used are "beats," which are chosen in succession as the story progresses. A beat is defined as any single action/reaction pair that moves plot along. For example, in a scene where the dialog is pushing the action, a beat could simply consist of one character saying a line and the other character reacting. Façade attempts to achieve a coherent experience via the team coordination of the synthetic actors. Within each beat, a joint behavior is specified that tells each character how they may behave, much like the multi-agent coordination work done by Tambe on STEAM (Tambe 1997; Tambe and Zhang 1998) (agent behavior definition that fulfills Requirement A4). A director agent is used to coordinate the 32

45 Figure 5. Screenshot from Façade. behavior of the synthetic characters which, as mentioned earlier, are weakly autonomous characters (fails to fulfill Requirement F4 and fulfills F1). This approach offers one of the more promising frameworks for an interactive drama. It provides a method for coordinating character behavior, animation and dialogue as a dramatically-motivated response to character input. However, the focus on interaction in Façade is primarily in the conversational realm; physical interactions are mainly incidental (fulfills dialogue interactivity for Requirement E1). Consequently, the boundary problem (i.e. the problem that arises when a player executes an action that harms the progression of the authored story content) is not as intrinsic of an issue for presenting an interactive drama. If the player says something non-sequitor or incomprehensible, little work has to be done to have the synthetic characters say Sorry, I didn t understand you, laugh as though the player made a silly joke, or even just ignore it and focus on something else going on. The player may say things that are incredible anti-social or out of content, but the story still can 33

46 go on. If anything, the characters can simply get frustrated with the player and show him the door. This is very different from making a tangible, observable change to the physical environment that logically keeps the plot from progressing. A physical change to the world, such as attempting to shoot an important character, can bring the story to a complete halt (e.g. Imagine Frodo losing the One Ring at the beginning of The Lord of the Rings). If the player says something inappropriate, incomprehensible, or even contrary to the authored content, the situation become awkward, even obviously implausible (e.g. I just told Trip I ve been shot! Why did he act like nothing is wrong? ), but the story can still logically continue. The former breaks the physical conditions for the story to continue, possibly rendering the story s progression unattainable, while the latter may only break social norms or the player s expectation of character behavior (or the expectation of the NLP technology). Façade tackles some difficult problems in interactive drama not addressed in this dissertation, but does not address the main difficulty, the boundary problem, that IDA does OPIATE The OPIATE storytelling system is an amalgamation of Propp s structural analysis of story structure in folktales (Propp 1968), the architectural organization from Oz (Mateas 1997), and the use of case-based reasoning in a story director (Fairclough 2004). Propp s analysis of Russian folktales elicited a library of literary functions that he used to describe the plot structure of a large set of these folktales. These rules were translated into logical rules in OPIATE that were meant to guide story content choice by an omniscient director agent (story representation and management that fulfill Requirements A1 and F1). The director uses case-based reasoning to guide the player s experience through dramatic events defined by Propp s rules and conventions. Case-based reasoning is used to determine the most suitable sub-plot to be enacted given the current state of the characters and the story world, taking into account the characters attitudes towards each other and towards the player. Director moves are defined as a grouping of character functions according to Propp s rules (e.g. villainy or struggle). Moves are selected according to their suitability in a particular situation. 34

47 The director will either choose to use the most suitable case or choose to combine two cases to create a more suitable case. Cases are combined by putting together the chosen cases most useable functions and removing the least suitable ones. The director then instantiates the selected case in the story world, which involves casting the synthetic characters into eight of nine possible roles (the player is always cast in the Hero role) (synthetic character directions that fulfill Requirement A4). Synthetic characters can talk with the player using a very simple discourse grammar. They can refer to other objects and characters in the world, using a small set of predetermined verbs. They can also gossip with each other about events they have witnessed, affecting their attitudes about other characters. The player s actions in the world consist of: moving in the world, picking up objects, and conversing with other characters in the same fashion that they can speak to each other. The main interactions with the world in OPIATE, speaking to others about your knowledge and getting / using / giving items, do not provide a wide expanse of possible dramatic experiences (fails to fulfill Requirement A5) or consequential actions (fails to fulfill Requirement E1). The player s experience is, admittedly by the author, constrained to typical adventure game plots, where the main focus is on the solving of puzzles with no real dramatic involvement on the player s part. While the approach of combining high-level drama management (i.e. the use of case-based reasoning) with the use of logically represented dramatic rules is a promising hybrid of approaches, it still has to tackle the difficulty faced by other systems, such as DEFACTO and BRUTUS, in logically encoding a broad set of dramatic principles at, as Meehan has suggested, different levels of meaning (Meehan 1981) MIMESIS The Liquid Narrative Group s MIMESIS architecture (Young et al. 2004) is an approach that specifically focuses on how to approach the boundary problem using replanning and story mediation techniques. Story content is represented as a plan, authored by a human writer in a STRIPS-style planning language (story representation for Requirement A1). The plan is comprised of actions executed by the player as well as the synthetic characters. 35

48 As the player interacts with the story world, the system records player actions as having a) no effect, b) fulfilling preconditions of operators in the plan, or c) causing threats to preconditions, possibly harming the progression of the story (called an exception). As the player is executing an action, if the effects of that action will cause a threat (i.e. a boundary problem), the system relies on two mediation approaches to keep the story progressing (fulfills Requirement F1). The first, called accommodation, quickly tries to replan, incorporating this new player action into the existing set of operators so the plan s goal may still be achieved. If replanning does not yield a new, usable plan, then the director will execute what Young calls intervention. Intervention involves executing some director action in the world (similar to how MOE affects to the world) to prevent the effects of the player s actions. For example, if the player tries to shoot a main character in the plot, and accommodation is not successful, then the director will make the player s shot miss, or perhaps make the gun jam. The director relies on strategies that deal with problems as they arise, an approach I call reactive direction, to address the boundary problem. MIMESIS is the primary example in these architectures that both provide for a fully-structured authored plot and a mechanism for dealing with the boundary problem. The reliance on replanning is a seamless approach to modify story structure to accommodate player actions. However, when reactive strategies are frequently relied on to deal with more problematic situations, believability suffers (fails to fulfill Requirement E2). The strategies available to the director for reactive direction are limited to methods that can only be used as an immediate response to the boundary problem. Methods that require lead time or anticipation, such as having the player in question yell Don t shoot! I have something to say, because it seems likely to the system that there will be a boundary problem in the near future are not possible under the guise of reactive direction. This approach to story direction has heavily influenced the approach to the boundary problem discussed in this dissertation. 36

49 3.5. Model-Guided Systems There are two fields in computer science and human-computer interaction that are particularly relevant to my exploration of preemptive techniques to address the boundary problem. These two fields, user modeling and intelligent tutoring systems, attempt to affect a system s configuration based on a hypothesis of user intent, knowledge, and / or goals User Modeling User models can be defined as models that systems have of users that reside inside a computational environment (Fischer 2001). They are a branch of humancomputer interaction that focuses on improving a user s experience with a system by adapting the system to the user s individual needs. User models are used to anticipate user actions (Albrecht et al. 1999; Zukerman and Albrecht 2001) (partially fulfills Requirement F2 by anticipating user interactions with the system), adapt software interfaces to meet the needs of the individual user (Horvitz et al. 1998; Langley 1999) ( fulfills Requirement E1 in a non-dramatic environment), and drive intelligent tutoring systems, which are discussed in Section One particular approach of note is Albrecht s work on user modeling for an adventure game (Albrecht et al. 1999). He presents an approach to keyhole plan recognition (i.e. the system gets its information about the user from non-interactive and incomplete observations) for predicting what quest a player is trying to accomplish in an adventure game. They use a Multi-User Dungeon (MUD) as their test domain, gathering observations from the actions that the player executes in the dungeon. These observations are used to train and test a dynamic belief network that predicts which quest the player is attempting to accomplish. This approach, however, has no elements of interaction (fails to fulfill Requirement E1); the system is used only to observe and predict. While it is one of the few examples of employing a user model in a computer game, it does not illustrate how that model could be used to change the player s experience. However, the usefulness of user modeling in this domain and the ones mentioned above show great promise for the use of models in computer game environments, especially interactive drama (Beal et al. 2002). 37

50 Intelligent Tutoring Systems Intelligent tutoring systems (also called cognitive tutors) employ a psychological model of the cognitive processes behind correct and near-correct student behavior in a computer-based learning environment (Anderson et al. 1995). Models are typically written as a collection of declarative and procedural knowledge that attempt to cover the space of possible operations, both correct and incorrect, that a student may try to use for solving a problem. This cognitive model is primarily used for two modeling techniques, model tracing and knowledge tracing (Koedinger et al. 1997). Model tracing is a plan-recognition technique used to hypothesize about how the student is approaching a problem. As a student executes actions in the environment, the tutor matches those actions to productions in the model (describes how anticipation is done to fulfill Requirement F2). When the student asks for help, the tutor can offer advice tailored to the specific strategy or goals it believes the student has. Knowledge tracing is used to keep track of student knowledge across problems, giving the system the information necessary to tailor the teaching content to the individual needs of the student (fulfills Requirement E1). Intelligent tutoring systems point to a specific use and technique for user modeling. Rule-based models are employed to give the system specific cognitive operations to reason about when giving advice or selecting tutoring content. There are definite similarities between these teaching systems and interactive drama. Both attempt to provide an open-ended domain for the user to interactive with. Both have definite boundaries for acceptable and unacceptable behavior. Both have agents that monitor the user s progress and adapt the environment according to its observations (fulfills Requirement A4). What obviously differs is the purpose of the system. For example, intelligent tutoring systems are concerned about students gaining knowledge much more than how interactive or believable the environment is (places Requirements E1 and E2 at lower priorities than pedagogical goals). While these fields have definite differences, there are lessons to be learned from how intelligent tutoring systems model student behavior and make use of that system to adapt teaching content. There have been a few systems that have begun to explore this 38

51 space, though they are much more heavily weighted toward interactive training than storytelling (Johnson et al. 2004; Magerko et al. 2005) Summary The requirements described in Chapter 2 are all addressed by different elements of these related works. The combination of the related components needed to fulfill these requirements with a single system, most notably the use of player prediction in a computational storytelling system, has yet to be explored. Interactivity (Requirement E1) is needed for improvement in storytelling with computer games; non-computational interactive stories, such as table-top role-playing, do not offer the same kind of immersive environments and plethora of tangible characters to interact with as computer games do (fail to fulfill Requirements A2 and A4). Expressiveness and story representation (Requirements A5 and F3) are most formally addressed in story generation and interactive drama systems, with planning and the formalization of dramatic concepts being key approaches. Semi-autonomy (Requirement F4) is an issue not addressed by most systems, since it is a requirement that comes into play after others have been addressed (such as a multi-level representation). Believability (Requirement E2) is an issue at the heart of addressing the boundary problem in a dramatically plausible fashion, as illustrated by the reliance on reactive techniques in the MIMESIS architecture. The explicit authoring of story (Requirement A1) is commonplace in storytelling domains, such as interactive drama and computer games. Little work has been done in user modeling or intelligent tutoring systems in dramatic contexts. The building of story worlds (Requirement A2) is only a concern for digital entertainment. The monitoring of story (Requirement F1) is typical of interactive drama systems that use a director agent for story mediation. This approach is commonplace in systems where the player is a character in the story, as opposed to a behind-the-scenes observer. Player prediction (Requirement F2) is a technique found in commercial and learning domains, but rarely seen in digital entertainment. A system that fulfills all of the structural and behavioral requirements described in Chapter 2 has not yet been designed. While a system has not yet been designed to maximally fulfill these 39

52 requirements (if that is even possible), IDA is presented in Chapter 4 and in more detail in the subsequent chapters as one that does better fulfill these requirements than previous approaches. 40

53 Chapter 4 IDA: General Design of an Interactive Drama Architecture My approach to address the boundary problem discussed in Chapter 1 and to fulfill the requirements outlined in Chapter 2 is the Interactive Drama Architecture (IDA). IDA s structure is similar to the general architecture for an interactive drama described in Section 2.1. As shown in Figure 6, the architecture is comprised of an author, who writes story content for the director, the director agent, the synthetic characters, the environment, and the player. IDA s central mechanism is the director, an agent that is responsible for managing the story in relation to the synthetic characters behaviors and the player s interactions with the world. This chapter focuses on describing the general architecture and its peripheral components, with the integral components, the story representation and story director, being discussed in subsequent chapters Haunt 2 The story world that I have built with the Soar Games Group is called Haunt 2 (Magerko et al. 2004). The story of Haunt 2 puts the player in the role of a ghost. The player wakes up in a bed and breakfast, not knowing how he got there or why he is a ghost. The building is populated by three other characters: the Innkeeper, the man who runs the bed and breakfast; John, a professor staying at the Inn; and Sally, an ex-girlfriend of the Innkeeper s who is visiting the inn. The story, which is a very simple one, unfolds as the player discovers his dead body, leaving it up to him to try 41

54 percepts AI Actor user actions Human player direction story Haunt 2: Built in Unreal Tournament AI Director Author Figure 6. The Interactive Drama Architecture (IDA). and figure out not only who murdered him, but how to encourage the innocent character to come across his body and alert the authorities to the foul deed. We achieved flexibility and low cost in the game environment by following in the footsteps of other projects (MIMESIS (Young et al. 2004), ESC Online (Martin 2001), and Gamebots (Kaminka et al. 2002)) by using the Unreal Tournament (UT) engine. UT provides an affordable, off-the-shelf 3D game engine that can be easily extended. Moreover there are many free level editors available for creating your own virtual environment; furthermore, the game physics and interface are coded in a powerful internal scripting language (Unrealscript) that is completely accessible. Each AI character is implemented as rules in the Soar architecture (Newell 1990). The characters share the same basic knowledge base to support interacting with the world and other characters. Specific characters have different physiologies (see below), goals and background knowledge. This project directly builds on related work for modeling human behavior in military simulation that shares the same core infrastructure, which consists of UT, Soar, and basic navigation knowledge (Wray et al. 2002). Although there are many components in common, the other research emphasized encoding military doctrine and tactics with short but intense interactions among human participants and AI characters. There was no notion of story or director. 42

55 4.2. Author One of IDA s functions is to serve as a communicative tool for the author s artistic vision. The author uses the story representation provided by the architecture to sculpt a story space, which defines the possible stories that the player can experience. This space is defined both by the authored story content (e.g. who, what, how, and where) and by the story structure (i.e. when events happen in relation to other events). A detailed description of the story representation used is given in Chapter 5. The author acts as both artist and programmer, filling in all domain-dependent content for an interactive drama. Aside from the story description given to the director, the author is also responsible for specifying any domain-dependent functions of the director, the environment and art content, as well as the synthetic character behaviors. The author, for my purposes, is viewed as a single person, but can easily be a conglomerate of artists and / or programmers Synthetic Characters The synthetic characters in the world, as illustrated in Figure 7, exhibit rational goal-based behavior. The agents for Haunt 2 (called HauntBots) are authored in Soar and are defined by their long-term knowledge, which is a hierarchical set of goals, short-term knowledge, which are any working memory elements (WMEs) that are constructed from observing and interacting with the world, and preferences, which are relational control knowledge that probabilistically affects how operators are selected. The characters have basic world knowledge, such as how to navigate around the building, how to use objects (e.g. a match can be lit and a thermos can be drunk from), and how to hold a conversation with other characters. As mentioned above, they can connect to the Haunt 2 environment via either sockets or multiple Soar agents can be directly embedded within the UnrealTournament process using a C-based API. 43

56 drink explore greet talk movearea movenode record -path Figure 7. Synthetic characters in Haunt 2. The Soar decision process is a five step cycle of Input, Propose, Elaborate, Select, and Apply (as shown in Figure 8). The Input phase creates new WMEs for any environmental changes observed by the agent. Rules that propose an operator and have a complete match of their conditions to WMEs will fire in parallel during the Propose phase. Elaboration involves the generation of preference knowledge for the proposed operators and any knowledge that can be inferred from existing knowledge. In the Select phase, one of the candidate operators is chosen based on any applicable control knowledge. Elaboration rules can produce operator preferences, such as Operator A is preferred over Operator B, Operator C is most preferred, or Operator A is indifferent and can be randomly selected out of other indifferent items if nothing is preferred more. Once an operator is selected, any application rules for that operator are executed in parallel during the Apply phase. This cycle repeats as the agent continuously gets new inputs from its perception. 44

57 Input propose elaborate select apply Sensory input from the external environment Propose operators that fit current situation Generate preferences and inferred knowledge Select an operator based on preferences or create an impasse Execute selected operator s actions Figure 8. Soar decision cycle. The primary drive for autonomous character behaviors is the decision process described above. Figure 9 shows a subset of the HauntBot goal hierarchy, which was jointly designed with the Soar Games Group. The synthetic characters are capable of exploring the world, using matches to start a fire to warm themselves, and eating or drinking available food or water. Aside from the cognitive drive, the character s physiological drive also plays an important part in affecting the decision-making process (Magerko, et al. 2004). The environment has variable temperature areas (i.e. some areas are colder than others). If a character is too cold, then it will go to some place that is warmer, or try to create a heat source by lighting a fire in the fireplace. Characters also have hunger and thirst measures, causing them to seek out food or drink sources if those levels get too low. These physical drives have not yet played a significant part in IDA s framework. Future work will involve examining how to incorporate physiological and emotional drives into the overall synthetic character approach. The synthetic characters used in IDA are also directable by design. While they may be allowed to act on their own internal (i.e. cognitive and physiological) drives, it is much more common for them to receive commands from the director agent, telling them how they should behave in relation to the plot. A command can be any goal in the agent s hierarchy. They can be given a top-level goal, such as explore, or very specific low-level actions, such as perform dialogue #131 with John in the library and then run away to another room. An agent takes this 45

58 Figure 9. Subset of HauntBot goal hierarchy. input as its new current goal. Once that goal is fulfilled, it moves on to the next commanded goal or goes back to relying on its own goals. This approach directly addresses the requirement for semi-autonomous agents (fulfills Requirement F4). If the author defines plot content that is at a high / low / intermediary level in terms of character behavior, the synthetic characters can take that content as a command and execute it. One issue it does not address is the believability of the characters taking direction (fails to fulfill Requirement E2). The agents may be directable, but they instantaneously take on their new goal as opposed to smoothly transitioning from whatever they were doing to this new possibly completely unrelated behavior (Assanie 2002). For example, if Sally and John are in the middle of a conversation, directing Sally to immediately break off from that conversation and go speak with the Innkeeper may be awkward at best. In terms of believability, it would be much better if Sally received that command and then pardoned herself from her current conversation before moving to another. My current work in IDA does not specifically focus on the design and believability of synthetic character behaviors (as opposed to the believable management and coordination of story events), so this instantaneous goal transition is an acceptable, though less than ideal, approach. The author, who has several artistic and programming duties that may involve more than one actual person, is involved in the agent construction process. Therefore, the author does have knowledge about how an abstract behavior might be elaborated. 46

59 This approach to semi-autonomy in an interactive drama architecture is in direct response to earlier approaches of strongly autonomous and weakly autonomous synthetic characters (Blumberg and Galyean 1995; Cavazza et al. 2002; Douglas and Gratch 2001; Mateas and Stern 2000). The argument for using strongly autonomous characters is that they are more robust in nature and can encapsulate the drives (e.g. cognitive, physiological, emotional, social, etc.) and motivations for their actions. As Mateas and Stern have noted, the use of strongly autonomous characters in a story world is problematic. Synthetic characters are there to execute actions that move the story forward, not to just realistically communicate their internal state and make rational decisions. The current state of the story is a crucial input to any decisions involved in pushing the story forward; therefore, the typical approach of making decisions solely based on a character s state (e.g. goals, emotional state, and local sensing) does not adequately fit into this paradigm. The benefits of semi-autonomy can be best illustrated through an analogy in the directing of a bar scene for a film noir piece. Let us say that there is a scene involving two main characters of this film (the male and female leads), the supporting cast (the male lead s buddies), and the background characters (the bar patrons and employees), all of which are AI agents. If the characters are weakly autonomous, the film s director would have to give the same kind of direction to each character in the room. He would have to tell each character how to act out their role, what dialogue is appropriate at the moment, their staging, etc. The same level of detail in terms of direction would be needed, no matter the character s importance in the scene. If the characters are strongly autonomous, the director would be able to give directions of a granularity specific to the character s involvement in the scene. The background characters would be told to play pool, be social, and drink, letting them rely on their own behaviors to determine how those goals get fulfilled. The supporting cast would be given more specific instructions, such as being told to execute a few lines of dialogue, talking about a party the other night, and their blocking. The main characters would be given very specific instructions, referring to their lines in the screenplay, how those lines are delivered, and exactly how the female lead gets up angrily at the conversation s end and walks out of the bar. In a later scene, those 47

60 same characters may have a different focus and then require different levels of autonomy (e.g. the main characters are background characters in a party while some of the supporting cast are the focus). When using strongly autonomous characters, it is difficult to maintain a coordinated control of group behavior within the context of the story (Mateas and Stern 2000). When using weakly autonomous characters, there is a difficulty when dealing with scale; when there is a sizeable group of characters in a scene, an author should not need to specify each character s behavior at a fine-grain. At the same time, we want the characters to be intelligent, rational, and responsive, regardless of their focus in the scene. These issues all point to the benefit of using semiautonomous agents. This provides the means of specifying scene content at all possible levels of abstraction, while avoiding the use of less-intelligent methods (e.g. finite state machines (Rabin 2005)) for authoring non-main characters. It also easily allows agents to easily respond to scenes where they are more or less-focused on by the story content (as in our film noir example above). A possible downside is that, for each character, complete autonomous behavior has to be authored. Semi-autonomous behavior is either constructed by the author directly or in conjunction between the game designer / programmer and the author. Regardless, the author has to be aware of the agent hierarchies to know what goals can be referred to in the story. One open question on this issue is how much reuse is possible across characters and how scalable this approach is for large story worlds, such as for Massively Multi-player Online Role-Playing Games (MMORPGs). This issue has been investigated slightly with Haunt 2 s characters. These agents use the same code but with different dialogue lines, character skins, and animations used. While this does offer some insight into reuse, how to efficiently build rich synthetic characters is not a focus of this project and is an open research question. 48

61 Chapter 5 Story Representation Per requirements A1, A5, and F3 in Chapter 2, a standardized, expressible story representation is a necessary component of this architecture. An author should be able create many different stories using the representation, not just single stories like Haunt 2. Given the lack of a large set of test domains, a set of requirements to define what the good attributes of a story representation are is crucial in guiding the design for IDA s representation. I present the requirements for the story representation in IDA below Requirements SR1 Expressivity: The author should be able to express a story idea along a series of dimensions, including (but not necessarily restricted to) dialogue, staging, character behavior, pacing, and environmental conditions (e.g. lighting). The author should also be able to describe how the player fits into the story as a character. For example, an author for Haunt 2 should be able to describe a scene introducing the Sally character to the player. He should be able to give her expository dialogue, describe where she should be relative to the player, what other characters could be part of this introductory conversation, and what state the player should be in. Fulfillment of this requirement can be judged by whether or not each dimension is addressed as well as how deeply each dimension is defined. For example, character behavior could be represented at a very shallow level (i.e. position and dialogue), a 49

62 very deep level (i.e. character motivation, emotional state, and social context), or not at all. SR2 Coherency: The author must be able to associate specific plot content with other similar content to assure coherency within the context of the overall narrative. This may be done implicitly or explicitly as long as the space of possible stories that can be experienced by the player only includes coherent, logically unfolding stories. A representation that did not meet this requirement would allow possible orderings of plot content that did not make sense in an overall narrative. For example, the introductory scene with Sally should not be allowed to occur after the concluding scene where the player leads one of the characters to discover the murder. A story representation fulfills this requirement if it provides the author with a means of authoring coherent content (e.g. ordering a scene B that refers to a past conversation in scene A so that B is experienced only after A). SR3 Variability: The story representation should force the author to constrain the player s experience to a single possible narrative. It should support the author to create multiple paths through the space of stories as desired by the author. The more possible orderings of plot content, whether explicitly or implicitly authored by the writer, the better the representation fulfills this requirement. When playing through Haunt 2, for example, different narratives should experienced by the player when different decisions are made. A story representation should also allow for unexpected player actions to contribute positively to the story. In a highly interactive story world, the player may execute a series of unorthodox or unexpected actions that were not planned for by the author, but ultimately keep the story on track. SR4 Explicit Story Space Boundaries: The author should be able to explicitly define a space of stories that carves out her artistic vision. Our goal is to create experiences in which complete dramatic stories can be authored, which is in contrast to systems that depend on a story to emerge (Fairclough 2004; Sgorous 1999). This requirement intends to provide the representation with a means for explicitly specifying when certain plot elements happen, either relative to each other or relative to the overall passing of time in the narrative. 50

63 SR5 Supports Player Prediction: The representation must allow the author to define a space of possible player behavior that a predictive model can search through. As described earlier, player prediction can be a valuable asset to interactive drama. If the drama manager has an accurate hypothesis about what the player is trying to accomplish in both the present and in the future, then that mechanism can make a better-informed decision about how to guide the story s progression. A plot representation that generated future content would not necessarily provide a searchable future from a given point in the plot, thus making it difficult, if not impossible, to reliably compare a predictive model of player behavior with future content Story Representation in IDA A story representation for an interactive drama is comprised of at least two main features, story content and structure (described in story requirements SR1, SR2, and SR4). With story content and structure both defined, the author has encoded the space of story events possible in the system. How that structure is defined can vary from the very rigid, such as the use of dialogue graphs (Gordon et al. 2004) to more flexible approaches, such as constructing dramatic rules that determine plot progression (Bringsjord and Ferrucci 2000; Fairclough 2004; Sgorous 1999; Turner 1994), partially ordering plot content (Jhala 2004; Magerko 2005; Mateas and Stern 2002; Meehan 1981; Young et al. 2004), or defining a heuristic for future plot choice (Mateas and Stern 2002; Weyhrauch 1997) Content IDA s story representation uses a partial ordering of abstract plot points, an example of which is shown in Figure 10. Each story event that takes place in Haunt 2, such as a scene where the player discovers his body, is represented as a node in this graph. This graph structure, G, is represented as G (N, E), where N is the set of nodes (or plot points) in the graph and E is the set of edges connecting them in a partial order. Plot points are defined as N (P, M, A, c), where P is the set of preconditions for a node, which describes a set of world states where every p є P is true, M is the name of the plot point, A is the set of actions for a node, which are the 51

64 At(Sally, Lobby) At(Innkeeper, Lobby) Proximity(Sally, Player, 1 room) Begin: 5 sec. End: 100 sec. Talk(Innkeeper, Sally, Conv. #9) Talk(Sally, Innkeeper, Conv. #9) preconditions actions Figure 10. An example of partial-ordering of plot points. plot events that are performed after all members of P are fulfilled, and c is the timing constraint associated with this plot point, which describes a time span during which every p є P must be true in the story world. A plot point s precondition, p, is a logical statement that describes what should be true in the world in order for the plot point s actions to be executed, similar to preconditions used in STRIPS-style representations. The example plot point shown in Figure 10 illustrates some basic preconditions that are used in the story. The plot point s set of actions represents the performances that are made by the directable characters once the preconditions are fulfilled. This can be viewed as an explicit stimulus / response model of plot events; when the current state of the world is a member of a set of world states that meet a plot point s requirements, the corresponding actions are performed. As shown in Figure 10 where the Innkeeper and Sally are supposed to have a conversation in front of the player, that plot point s preconditions would be fulfilled once everyone is in the same room and then the characters would be instructed to have a specific conversation. Any plot points without parents at the beginning of the experience are labeled as active. A plot point is parent to another if it has a directed edge pointing from itself to the other, its child. The director keeps a list of all active plot points. Once an active 52

65 point has been performed, it is removed from the list and any new points that a) have not been on the list and b) have no parents that have not been performed will be added to the list. The human author can specify the actual content of what happens, where it happens, and with whom through the use of plot points. This capability relates back to SR1, which describes the need for expressivity in a representation along a series of dimensions. The preconditions that define a plot point in Haunt 2 may include the location of particular actors, their physiological and / or mental state (e.g. knowledge that they have), or their inventory. Actions are typically comprised of verbs, character dialogue, staging directions, or some desired change in the character s mental state, such as a new goal to achieve or fact to know (e.g. perform conversation #3 or go to the lounge and fix a snack ). The purpose of providing the author with timing constraints is to give her the means to specify the pacing of the experience she is creating, which is another aspect of our representation that addresses SR1 and is a novel feature of IDA s representation. Just as in other visual media, such as cinema (Field 1994), pacing is an important feature of performance. The timing constraint, c, associated with a plot point can be viewed as a special precondition for that plot point. As shown in Figure 10, a timing constraint has begin and end conditions, denoting the earliest time that the given plot point could be performed, and the latest possible time that the conditions could all be fulfilled. The director will not execute any director actions to fulfill a plot point s preconditions until that point s begin timing constraint has been fulfilled. The begin constraint for a plot point marks the starting point for any directions to be given for that point. If there are preconditions that do not involve the player (e.g. At( Sally, Lobby )), the director will wait until the current time is past the begin constraint before directing the actors involved in that precondition. If all of the preconditions are fulfilled, then the director will wait until the current time is greater than the begin constraint before executing the plot point s action. The end constraint for a plot point represents an upper bound on how much time the player can be left to his own devices before the system intervenes. This defines the deadline by which the 53

66 player and characters must fulfill the plot point s preconditions. If the constraint is violated while there are still unfulfilled preconditions, the flow of the story, as specified by the author, has been violated and direction is needed to reactively reconcile the current world state with the desired state. The author can make the interval relatively short and make the start constraint small if she desires a quicker pacing. This decreases the amount of time between when the plot point becomes first active and when it can actually be performed. The author can make the interval longer and make the start constraint larger for a slower pacing, creating an environment that allows the player more time to observe and interact with the environment and gives the autonomous agents the freedom to pursue their individual goals. For example, the player in Haunt 2 is a ghost because his character was murdered by an unknown assailant at the beginning of the story. The rest of the plot involves the player figuring out who might have murdered him and helping one of the other characters find the hidden body to reveal the murder. As the player gets closer to reaching the main goal of helping another character uncover the body, I as an author would like the pacing of the story to quicken. I can achieve this by defining shorter spans of time with the latter plot points timing constraints. If the player does not act quickly, the system will execute guidance in the world to move the story along (a director capability addressed in detail in Chapter 6) Structure Plot points are connected to each other via directed edges, which creates a partial ordering of these points (see Figure 10). The entire partially-ordered story for Haunt 2 can be seen in Figure 11. These links do not represent paths in a story graph for the player to follow. They describe an explicit partial-ordering of the plot content. Therefore, as opposed to describing a graph for traversal (i.e. the approach uses in branching storylines), this representation describes a space of possible topological orderings (fulfills SR4). In Haunt 2, there are plot points that the author will likely want to have happen before others. The end scene, concerning the player leading another character to the dead body, should happen after the player has learned about the seldom-used library and discovered the body there. The player could learn about this room in a number of different ways (e.g. from overhearing the Innkeeper mention 54

67 it or by stumbling upon the room while exploring the building). However, there are other unrelated events, such as the player being initially introduced to the other characters, which can happen in a flexible ordering and would be structured with fewer ordering constraints than the ending. This structure is similar to the planning language in MIMESIS described earlier. The key difference is that our representation has no explicit concept of causality, which is a novel approach to structuring plot content in an interactive drama. A STRIPS-style language contains both ordering links and causal links between plot operators, whereas our language can be viewed as an incomplete plan; it has no causal links. Our graph contains partially-ordered nodes which have preconditions. However, the nodes do not have explicit postconditions that can be causally linked to subsequent nodes; thus the absence of explicit causality. This inhibits our ability to replan like MIMESIS does as an approach to story management. However, it offers a more streamlined representation that allows us to directly encode actor and director actions in the plot points rather than encoding the effects of those actions. If there is a particular behavior the author wants to see, it is represented in a plot point s actions rather than as a separate operator. This approach is also ignorant of unexpected actions. Since there is no explicit causality, preconditions may be fulfilled by unexpected sequences of actions that may initially seem threatening by a replanner. As long as the preconditions are fulfilled before a timing violation occurs, it does not matter how the player achieved it. This representation forces the writer to consider the temporal flow of events when ordering them. For example, it is possible to author a story that has dialogue in a plot point which refers to an event that hasn t occurred yet. There is no mechanical device, such as a planner, to ensure causality, which is a similar approach to the one used in Façade (Mateas and Stern 2003). This representation therefore partially fulfills SR2 by providing an explicit mechanism to describe temporal coherence, but not necessarily content coherence (partially fulfills Requirement SR2). It is quite possible for the author to write dialogue that refers to a past event (e.g. the Innkeeper warning the guests to leave the ghost they spoke of alone) that has not even occurred yet (e.g. the plot point with this dialogue does not explicitly come after the ghost 55

68 being seen by any of the guests). This does provide the author with the means of creating large, undefined spaces in the player experience. If there is no explicit description of action and effect, there can be any sequence of events that can be combined to fulfill a plot point s preconditions. This helps open up the representation, leading to a larger amount of possible player experiences within the story space (fulfills Requirement SR3). Therefore, the plot does not represent an action-by-action account of what happens in the world; it is a skeletal representation that defines key A (Fireside Chat) At(Match, Fireplace) Hidden(player, True) At(sally, FrontOfFireplace) At(chatter, FrontOfFireplace) B (Fireside Chat Conversation) At(sally, FrontOfFireplace) At(chatter, FrontOfFireplace) Proximity(sally, player, 2) Talk(sally, chatter, FiresideChat) X (Finder Finds Body) Knows(finder, Exists(DeadBody)) C (Small Talk) Y (Finder is Killer) At(sally, FrontOfStair) At(innkeeper, FrontOfStair) Proximity(sally, player, 1) Hidden(player, True) Talk(sally, innkeeper, SmallTalk) Proximity(player, killer, 1) killer == finder Talk(finder, player, KillerFoundBody) End D (Ghost Appears) Z (Finder Is Innocent) Hidden(player, False) Proximity(player, sally, 1) Talk(sally, innkeeper, player, Eeek!) Proximity(sally, sally-ex, 1) Proximity(player, killer, 1) killer!= finder Talk(finder, player, FoundBody) End E (Relationship Talk) Hidden(player, True) Proximity(sally-ex, sally, 1) Proximity(sally-ex, user, 1) Proximity(player, sally, 1) Talk(sally, sally-ex, RelationshipTalk) Figure 11. Entire partially-ordered content for Haunt 2. Y and Z end the story. 56

69 points that the story should gravitate towards. How those plot points are fulfilled depends largely on the player s interactions with the world. An evaluation of this approach is presented in Section 9.2. This explicit use of structure in our representation allows us to employ a predictive player model in IDA (used to fulfill Requirement SR5). We can compare hypothesized player behavior against a well-defined story space. If the player s predicted actions move the experience outside of that story space, then the director can choose to preemptively direct the world, attempting to influence the player s behavior to avoid a boundary problem. It would be much harder to take advantage of this useful story mediation tool without a story space to compare this prediction to. How IDA approaches player prediction is addressed in detail in Chapter Implicit Story Elements As we stated earlier, the story graph represents a space of possible stories. The content defined by the author is obviously the main influence on the narrative experience. However, there are other aspects of story that are less explicitly represented that we have yet to address. One implicit story element is the author s use of uninstantiated plot content. This is the ability to define story content in an abstract manner, allowing the grounded definition of that content to be determined at runtime. The other implicit story element to consider is the contribution of director and character behavior to the narrative. While the story written by the author does define the narrative space for the player s experience, that space is filled in by the actions executed by the director, synthetic characters, and of course the player. A simple, yet effective way for us to expand the possible size of a story space is to allow the authoring of uninstantiated plot content. When the author does not want to have control over a specific detail, she can use a variable in a precondition or action that will be bound at runtime. The author can further constrain what that variable can be bound to. For example, the situation shown in Figure 10 may not need to involve the Innkeeper talking to Sally specifically; it just needs to have the Innkeeper talking to any synthetic character (e.g. John or Sally). Therefore, the author can replace Sally with a story variable (e.g. At(x, Lobby) ) in that plot point, then constrain x!= Player. This variable is globally defined so it can be used in other 57

70 plot points in the story once the first instance of it is bound. All constants in IDA s representation are actually variables that are constrained to be a specific story element. This approach provides the author with a mechanism of least-commitment authorship; she only has to be as specific as she desires in authoring plot content. Important plot content will likely be tightly constrained, but the flexibility of the story representation allows the author to expand the size of the story space (see Requirement A5). This approach has been used to some extent to add variability to synthetic character behaviors, but has not been considered for use by a director in an interactive drama architecture before (Perlin and Goldberg 1996). Which actions are chosen by the director can also implicitly affect the narrative from experience to experience. The director s actions are a set of strategies for directing the characters (including the environment) to perform plot content or to encourage specific player behavior. These actions are categorized according to what kind of situations they are applicable to. Currently-implemented categories in Haunt 2 are: location-actor, location-player, physiology, conversation, drink, proximity, and knowledge. The actions in the location-actor category are strategies for getting an actor to a specific location, such as giving the target actor the goal of moving to that area or giving a second actor the goal of yelling Hey, come over here to the target. Since there is more than one possible way for the director to fulfill a plot point s preconditions or perform its actions, this is another example of variability between experiences (fulfills Requirement E1). The skeletal plot representation can be realized in different ways because the player makes different choices or because the director does. Just as the strategy categories dictate which director actions are appropriate to fulfill a plot point s preconditions or to perform its actions, they also dictate which strategies are appropriate for guiding the player, either as a response to model failure or as a timing constraint violation. The characters that are involved in the story are by design not simple puppets that can only do the specific actions that they are given. As mentioned in Chapter 4, IDA uses semi-autonomous synthetic characters (Blumberg and Galyean 1997) that are developed with the Soar architecture. This distinction is twofold, meaning that not only can they pursue their own goals when not involved in the plot, but they can also 58

71 receive directions with varying degrees of specificity. If a character has no specific directions to execute in between plot points, it may be left to pursue its own goals until directed again. This gives the characters more believable behavior without burdening the director with determining their moment to moment behavior. For example, when an agent is sent a new command from the director, the command can be anything from a very high-level goal (e.g. hang out anywhere ) to a very specific one (e.g. engage in conversation #113 with Sally ). These commands provide a method for the director to control the actors behavior to carry out specific plot points specified by the author. How a character decides to fulfill a high-level goal or spend its time when pursuing its own goals adds to the variability of the player s experiences (fulfills Requirement E1). The use of a story representation that supports the intelligent direction of synthetic characters in a complete interactive drama architecture is an advancement of the use of semi-autonomous characters. 59

72 Chapter 6 Story Director The director s role in IDA is to manage the tension between the forces of interactivity and authorship. The main problem with this task is in dealing with the boundary problem. This refers to the player s ability in a highly interactive world to execute actions that may bring the experience outside of the authored story space, thus halting the progression of the story. The inclusion of this problem in an interactive story world is necessary but problematic. Since authoring content for every possible physical, social, and conversational interaction in a story world is an intractable problem, the system must take one of several approaches to handling this problem. The first is constraining the player s actions so story-relevant choices are allowed (and encouraged), story-irrelevant choices (i.e. actions that don t affect the story s progression positively or negatively, except for perhaps wasting time) are allowed, and story-harming choices (e.g. attempting to leave the city where the entire story takes place) are prohibited. This approach is commonly used in commercial computer games (Fahlstein 2005). The second is generating story content to respond to player decisions. As discussed in Chapter 3, story generation is still an active research area even in the traditional noninteractive storytelling domain. Applying such techniques to interactive drama is a daunting task, though quite possibly part of the future of the field. The third is to recognize and encourage dramatic situations (Fairclough 2004; Sgorous 1999). This approach has some promise: DEFACTO and OPIATE are both systems that have attempted this method, but the main drawback to this approach is the reliance on reliably authoring dramatic situations with a logical representation. This is difficult 60

73 both in terms of the depth of the definitions (e.g. BRUTUS uses different logical rules to represent the single concept of betrayal ) and the sheer number of those definitions needed to cover the space of dramatic situations (see the discussion of BRUTUS approach to codifying dramatic concepts in Section 3.1.4). The fourth is to attempt real-time story mediation. This involves an intelligent agent, typically called the story manager or director, actively observing the player s interactions with the world and dynamically modifying the story and / or environment to either accommodate the player s choices or influence them. This chapter discusses my implementation of a story director that employs this final technique The Director Agent IDA s director agent responds to the boundary problem by attempting to influence the player s behavior (through story direction) to keep the experience within the story world. The director s decision to guide the player s behavior is dependent on both the player s current actions in the world as well as the director s hypothesis of future player actions. Systems like MIMESIS or ALT-SIM have a similar view of this problem: the player can execute actions that are in conflict with story content what can the system do to deal with this? Their approaches focus on responding to player actions as they are being executed. MIMESIS relies on replanning to incorporate an unexpected action into the story plan. If replanning fails, then the effects of that action are disallowed (e.g. forcibly making the player miss when attempting to shoot a critical character with a gun). ALT-SIM attempts to employ guidance strategies that push the player towards a particular event (e.g. drawing the player s attention to an important coffee shop he has walked by and paid little attention to), but doesn t necessarily deal with potentially critical problems, like shooting a main character as MIMESIS does (Gordon and Iuppa 2003). As introduced in Section 3.4.7, IDA employs reactive direction to altering the story or story world to address problematic player actions. The problem with relying on reactive direction is that it puts a premium on the effectiveness of story direction as opposed to preferring more subtle strategies. Once the player attempts to execute a problematic action that creates a boundary problem 61

74 there are only so many strategies that can be used. Moreover, the repetition of these strategies can have an extremely negative effect on believability. For example, you only get one coincidence in a story according to screenwriting theory (Field 1994). One coincidence, such as a character miraculously surviving a fall from a building by landing in a cushy, trash-filled dumpster, is typically acceptable for an audience. Any more than that will likely make the story seem contrived and unbelievable. Therefore, even if a reactive strategy is fairly believable (e.g. missing the targeted character from fifteen yards away), the repeated use of these strategies harms the perceived plausibility of the story experience. IDA addresses this problem by attempting to predict the player s behavior to affect when and how it attempts story guidance, which is a major contribution of this work. If the hypothetical future player behavior indicates that a boundary problem is likely to occur, then the director employs story guidance strategies that are subtler than strategies that apply to when a problem occurs (which I define as preemptive direction). One could construct an agent that always used these subtle strategies whenever possible, but a design that relied on the director to execute actions all the time would harm believability (fails to fulfill Requirement E2). The hypothesis is that the preemptive approach will need to execute fewer directions. Its design is to both use subtle story direction when possible, but only to use it when deemed necessary by the predictive model. This design provides the intelligent use of subtle strategies without having to rely on executing them at every possible moment. Consider an interactive drama that involves the player in a road trip story like in the film Sideways (Payne 2004). The player is to travel with his buddy from San Francisco to Sonoma Valley and craziness ensues once they get there. Suppose that the player has chosen to drive there via Highway 1, a road that has numerous stretches of dangerous sharp curves bordered by the ocean on one side and a mountain wall on the other. The director agent for this system observes them driving much too fast and recklessly down the highway. It is obvious that an accident could easily occur, which would disrupt the flow of the story experience. If the director only uses reactive strategies, then it will passively observe the scene, waiting for an accident to happen. When the player does hit a curve too sharp and goes careening 62

75 off of the cliff, the director observes that the story is definitely disrupted and does something, such as throwing the player and his buddy from the car at the last second to keep the story going. As pointed out earlier, this kind of effective reactive strategy may work once, but the plausibility of the story world is definitely harmed if such a coincidence happens again in the future (the fact that they survived this single accident at all could even put a strain on the plausibility of the story world). A director that uses preemptive direction would observe the scene and predict that there is a high likelihood of an accident. The director would attempt to alter the world to preemptively discourage that problem from occurring in a subtle, believable fashion. The system could spawn a cop behind the player and have the cop pull him over, spawn a truck in front of him around a curve so he has to slow down to the truck s speed, or even have a fuel injector blow in the car s engine, greatly harming the car s capabilities. These strategies, which could not be used for reactive direction, can fit into the game world in a realistic manner (e.g. the player can get pulled over, then later encounter a slow-moving truck that that is blocking both lanes) without straining the dramatic plausibility of the story world. The player s behavior can be subtly guided in this manner, while relying on reactive direction as a last resort (e.g. the player tries to swerve around the truck and goes flying off the road) Director Roles The director has several roles as the story coordination agent in IDA: knowledge maintenance, plot monitoring, executing direction, and scene choice. Figure 12 illustrates how these roles relate to each other. Knowledge maintenance is an ongoing process that involves the director observing changes in the world state. Plot monitoring involves the director checking these changes to determine if there is a change in the world state that is relevant to the story content. The director is also responsible for maintaining the story state (e.g. which plot points are active). The director may choose to execute one of three kinds of direction in the world depending on the results of plot maintenance (e.g. a new active plot point): story direction, reactive direction, or preemptive direction. Preemptive direction is a special case that 63

76 depends on the results of a predictive model of player behavior, which is discussed briefly below and in depth in Chapter 7. Figure 12. Director execution cycle. Plot monitoring actions are selected individually. Control is then passed back on to the knowledge maintenance task or direction is executed Knowledge Maintenance The knowledge maintenance role is the director s task of monitoring the world from an omniscient view, meaning that it can observe the entire state of the world regardless of physical boundaries. It records the observable changes in the state of the world, such as the physical properties of items and objects and the physiological state of the characters. It also hypothesizes character knowledge as the story 64

77 progresses. World knowledge can be categorized as describing physical state, mental state, or physiological state. For example, if the player walks into a room that he has not been in before, the director would hypothesize that the player knows that a) the room exists, b) the items and characters in that room exist and their observable properties, and c) any information that is tagged to the dialogue being performed by the characters. The knowledge used in the plot representation, actor behaviors, and the director s model of player behavior is represented in an overall taxonomy in Figure 13. The purpose of building such a taxonomy is not to construct an exhaustive description of the kinds of information that could be represented in all interactive drama, but rather to organize the knowledge that is used by this particular architecture for reasoning about and describing the world. We can have a better understanding of where IDA s strengths and weaknesses are in terms of expressivity by understanding and organizing the kinds of knowledge that are used in the system. relationships* attraction repulsion long-term world knowledge rules mental knowledge goals short-term story goals actor goals observations dialogue self-awareness emotional* modeled goals inventory physical physiology state w.r.t. environment Figure 13. Taxonomy of knowledge used in Haunt 2. Items marked with an * are currently unimplemented. Knowledge in the director is represented as working memory elements (WMEs) in Soar. These WMEs are represented as attribute value pairs (e.g. ^name Player) that can either store atomic facts or identifiers that point to a more complex data structure (e.g. ^entity <e1> points to all of the information for a specific NPC in 65

78 the world). These WMEs are updated based on both inputs from the environment and internal elaboration of the director s knowledge. They are kept consistent over time by Soar s internal truth maintenance mechanisms. The director s use of a hypothesis of player knowledge is a novel strength of the director that is not used in other interactive drama approaches. This allows the author to specifically write changes in the player s knowledge base (e.g. the player knows where the murder weapon is ). The author can rely on the director s inferences rather than having to encode those inferences into the content itself (e.g. the player is facing the weapon in the same room as the murder weapon and the weapon is visible ). Therefore, she can encode more intentional content, such as the player knows where the murder weapon is, which provides greater ease of use and readability for the story representation Plot Monitoring As the director is recording knowledge about the world, it is constantly comparing the current state of the world to its knowledge about the plot and player behavior to determine what, if anything, it should do next. As shown in Figure 12, this role has several actions associated with it: marking preconditions as true or false, marking plot points as active or to be performed, instantiating variables, recording timing violations, observing when a plot point should be performed, selecting plot points, and predicting player behavior Marking Plot Points A plot point is defined as active if all of its predecessors have been performed. If the state of the world matches with an active plot point s precondition (e.g. At( Player, Lobby ) when the player is in the lobby), then that precondition is marked as true. If it is true and the state of the world disagrees (e.g. At( Player, Lobby ) when the player is in the library), then that precondition is marked as false. Preconditions values can change as many times as needed. 66

79 Variable Instantiation As the director compares its knowledge of the world state to the plot, one of its purposes is to consider when a plot variable should be instantiated. If a plot point contains an uninstantiated variable, and a possible instantiation of that plot point matches with the director s current knowledge of the world, then the director will bind that variable to the appropriate instantiation. For example, if there is a precondition Proximity(Player, x, 2) and the director is right next to both Sally and the Innkeeper, then the director could randomly choose between instantiating x to be Innkeeper or Sally. If the Innkeeper is selected, any plot point that has x in its conditions would then be bound to Innkeeper. For the sake of simplicity in implementation, all objects and entities are represented as variables in the director s knowledge base. Some of these variables are bound at runtime (e.g. the player variable is bound to the player s game representation) and are designated as static (i.e. they cannot be unbound and changed during the course of the story), and others are marked as dynamic (i.e. they can be unbound and changed) and are unbound at startup. Reasoning about unbinding and rebinding variables as a type of director action has been considered in IDA s design, but has yet to be implemented Player Prediction Once a plot point has been performed, the director hypothesizes about future player actions via a look-ahead search of the possible interactions between the player and the environment. Those actions are compared to the story. If a boundary problem is predicted to occur, then the director will execute direction to preemptively avoid the problematic situation. Preemptive direction is discussed in detail in Section while the techniques used in predictive modeling are described in Chapter Plot Point Selection Given the partial-ordering of plot points, it is sometimes unclear which plot point the director should be attending to. There may be more than one plot point that has all of its preconditions filled at the same time. In this instance, it is unclear which plot point should be performed. The director relies on a novel two-tiered method to 67

80 make this decision. At any time during a story experience, there is the set of currently active plot points (i.e. are annotated by the director with ^active true). The director first relies on player actions to dictate where the story will go next. This is the player-choice tier. If the player executes some action(s) in the world that fulfills a precondition of an active plot point, then that plot point is considered to be the one to perform (i.e. is annotated by the director with ^perfoming true). This allows the player s interest to help guide the story, giving a stronger connection between player desires and the story experienced. The second tier in performing plot choice is the contingency strategy for when the player does not fulfill any preconditions before a timing constraint is violated. This is the heuristic-choice tier, which is similar to the heuristic choice of plot content in MOE and Façade (Mateas and Stern 2002; Weyhrauch 1997). Plot points are annotated by the author along a series of author-defined dimensions, such as tension, relevance to the content of its parents, etc. The next plot point to be performed is then selected by an author-defined heuristic. This provides the means for the author to encode their vision of plot development in the story space as a heuristic. This heuristic can be as simple or as complex as desired, though obviously the more complex it is the more computationally expensive the decision may be. For IDA s implementation in Haunt 2, I have chosen a rudimentary ^score attribute to annotate plot points and the heuristic is the Max() function. This simple implementation was done to explore the use of a two-tiered method without delving deeply into the area of heuristic content choice, which has been explored in MOE and Façade. This novel approach puts a premium on player interactions directly guiding the story and allows the author to directly encode her artistic vision for plot selection Executing Direction IDA s director agent is responsible for making changes in the state of the world based on how the story is progressing (called direction or directing). What exact actions are available to the director is domain-dependent; strategies that are used in Haunt 2 are unlikely to be used without alteration in an 1800 s love story. The director, as implemented for Haunt 2, can affect the state of three major 68

81 components of the world: the synthetic characters, the objects in the world, and the environment Director Capabilities The director can give the characters new goals (e.g. get a drink ), information about the state of the world (e.g. the player is in the lobby now ), or specific atomic actions for them to perform (e.g. perform dialogue line #2 now, speaking to Sally). These directions change the working memory of the agents, and therefore alter their behavior. It is also possible for the director to change a character s physiology to indirectly affect behavior (e.g. making a character thirsty or cold). The director has a library of directions to choose from, each labeled to help match it to the appropriate situation. The director can create or remove objects from the world as well as change several physical parameters associated with that object (e.g. location or locked vs. unlocked). This may be especially useful if the player is predicted to alter or has altered an important object in an irreversible manner. For example, the player may have unwittingly destroyed an old book that contained a piece of information key to the story. As opposed to the story coming to a halt, the director can create a new book with the same information in a part of the house the player has not been to yet or place it on the person of one of the characters. There is an important interplay here between the hypothesized knowledge base of the player and what the director can do. Having such a knowledge base as an input into the decision-making process of what can I do to move the story along? adds a check on the believability of the action. Creating a book out of thin air in a room that the player has just left is not as subtle or as believable as creating another copy in the hands of a character that is elsewhere in the building or creating it in a room that the director knows the player has not visited. This is a novel capability of IDA s director agent. In terms of environmental story direction, the director has control over lighting and sound triggers in Haunt 2. If it wishes to attract the player to a particular nearby room, there are sound triggers that are accessible to the director that may be triggered in that room (e.g. the clock chiming in the lounge). Such actions are useful 69

82 as subtle attractors to different areas, objects, or even characters in the world. If the player is in the lobby but is needed in the lounge, the director could chime the clock s bells loudly as a new event in the world that may grab the player s interest. The director can also attempt to attract or repel the player from a particular room by manipulating light levels in the building. If the player is hanging out in the lobby but really should be moving on to the lounge, the director can raise the light level in the lounge, giving some dialogue to the Innkeeper like Ah, that s better! Now I can see who I m talking to. The director could alternatively turn out the lights in the lobby, directing the Innkeeper to say loudly Sounds like we ve blown a fuse downstairs. I ll look into it after we re done with this chat of ours. The actions that the director can execute (e.g. giving the characters new goals, creating objects, etc.) can be used to perform some story mediation strategy, such as the player isn t near an event, so attract him to that event. A full list of the strategies implemented in the director is included in Appendix A1. While IDA makes use of several mediation strategies (typically attraction and repulsion), it makes no claims at covering an even moderately satisfactory amount of the space of possible mediation strategies. Systems like IDA that use story direction, such as MOE and MIMESIS, have not thoroughly examined the mediation strategies used by humans in real-world interactive storytelling domains, such as tabletop role-playing games or D&D-influenced computer games like Neverwinter Nights (2002). Both IDA and the field of interactive drama in general would benefit from ethnographic studies of reallife storytellers to build a taxonomy of usable mediation strategies. I have considered two different approaches for determining which directions are appropriate for any particular descriptor. My first approach to this was similar to that in Weyhrauch s MOE director (Weyhrauch 1997). Descriptors were annotated with specific directions to help fulfill them. When a particular story element needed to be encouraged, it was annotated with the exact direction needed. I have since opted for a more modular approach to representing directions in the agent. Descriptors are annotated with a classification, such as proximity or knowledge. This classification denotes what strategies are most appropriate for a particular descriptor. When a descriptor is marked as needing direction, the director examines the entire set of 70

83 directions, matches on those that are of the same classification, and then chooses between whichever are applicable for this particular situation. For example, there may be two direction rules written in the agent for the proximity class, one that involves only synthetic characters and one that involves a synthetic character and the player. If Proximity(Player, Sally, 1) requires direction, then the director would match that descriptor to the Proximity action that deals with the player and another synthetic character. This approach allows for the reuse of director actions across multiple descriptors, the authoring of actions that can apply to many descriptors or just a single one, and encourages future work in researching the different kinds of strategies humans use in story mediation Types of Direction As shown in Figure 12, the director will execute direction for four different reasons: a precondition of a performing plot point that only refers to non-player characters, all of the preconditions of an active plot point being fulfilled, a timing constraint being violated, and the player modeling indicating a likely future boundary problem. The different kinds of direction that are executed to alter the story world are called story direction, reactive direction, or preemptive direction Story Direction The director will give commands to the characters and / or environment when story content needs to be performed, either because certain preconditions need to be fulfilled by synthetic characters or because a plot point s actions need to be performed. If a plot point is selected to be performed, it may have preconditions that do not involve the player that need to be fulfilled, such as Proximity(Sally, John, 1). Rather than rely on the synthetic characters autonomous behaviors to accidentally fulfill such a precondition, the director issues the necessary commands to the involved characters, such as Goto(Sally, Library) if John is in the library. If there is more than one possible command (e.g. Sally could be told to go to John, or John could be told to go to Sally), the director randomly picks between the candidates and executes only one of them. 71

84 The fulfillment of all preconditions of a plot point that is marked for performance signifies that some actions on the synthetic characters part should be executed. As described in Chapter 5, the actions for a plot point are performed once its preconditions have been met and it is selected for performance. The director then chooses a strategy to fulfill those conditions and issues the appropriate commands to the characters and / or the environment Reactive Direction As described above, the director employs reactive direction in response to immediate problems between the authored plot and the player s interactions with the world. The main signal to the director that a boundary problem has occurred is the violation of a timing constraint. This could come about because the player simply does not know what to do next, has executed some action that has made fulfilling a precondition impossible (e.g. driving off the cliff), or the player is interested in something that is simply not part of the story space (e.g. floats out of the house in Haunt 2 and examines the surrounding area). The director does not differentiate between these different possible causes. An approach that is stronger than using timing constraints as a catch-all would actually notice the problem that is occurring and why it is occurring, thus informing the director on which direction strategy to use (e.g. the replanning / mediation strategy used in MIMESIS (Young et al. 2004)). The director uses timing violations as a means of recognizing when reactive direction is necessary. Once a timing constraint is violated, the director will fire a production that creates a timing violation WME, which records that a timing constraint has been violated and that the director should execute some reactive direction to attempt to rectify the problem. If there is an applicable strategy to apply, the director will execute it. In the example shown in Figure 14, if 100 seconds have passed after this plot point has become active without all of the preconditions being filled, then the director will create a timing violation and direct. If the player is in the nearby piano room, then John and Sally may be directed to walk from the lobby to the piano room and converse there. In other words, if the player is not physically in a suitable place that keeps the experience in the story space, the story can be brought to him. 72

85 At(Sally, x) At(John, x) Proximity(Sally, Player, 1 room) Begin: 5 sec. End: 100 Talk(John, Sally, Conv #9) Figure 14. A plot point involving conversation between John and Sally with the User nearby and invisible Preemptive Direction The director decides to execute preemptive direction when the player is predicted to execute some action(s) that will harm the progression of the plot, which has not been used in any other interactive drama architectures. Specifically, this means that a timing violation is predicted to be violated before the player fulfills any new preconditions of an active plot point. Chapter 7 explains how the predictive model works in detail. In the example given in the previous section, the director waited until a boundary problem occurred before executing direction, but a more preemptive approach to this problem could have been taken. Once this scene became active, Sally and John are located in the lounge (see Figure 15) while the player is off exploring the house. The director might be able to predict that the player will not go to the lounge and meet up with John and Sally. One response to this prediction would be that the director creates a new noise in the lounge, such as one of the characters laughing loudly, to attract the player to the lounge and fulfill the story content. 73

86 The philosophy behind the use of preemptive direction is a direct response to the less subtle reliance on reactive direction (Gordon and Iuppa 2003; Young et al. 2004). As illustrated in our previous examples from Haunt 2 and the highway driving scenario, preemptive direction can have an effective, yet more subtle effect on the Figure 15. John and Sally are in the lounge while the player explores the house. player s experience. How subtlety and effectiveness play into the director s decision making, as well as how the player model is defined, is discussed in the next chapter Director Strategies The strategies that are employed by the director are broken down into several categories: location, proximity, hidden, physiology, conversation, light-fire, and drink. The director maps preconditions and actions to these strategies by the predicates used in the logical statements (e.g. At(x, y) maps to the location strategy). A strategy is a set of possible actions that the director can take. Selection of these actions depends on both the applicability of that action (i.e. is it appropriate now?) and any control knowledge the director may have (e.g. action A is preferred over action B ). 74

87 The use of a pool of strategies allows for reuse of director actions (e.g. a single action can be used for more than one kind of situation) as well as adds to the believability of the player s experience (fulfills Requirement E2). If the director has several different strategies to choose from to approach a problem, then the player is less likely to be exposed to the same one multiple times. If more than one strategy is applicable (e.g. create-sound vs. relocate-player), then one out of a set of candidates could be selected based on some qualitative measures Example of Director Execution This section steps through the director s decision-making process in a specific situation that draws from the examples above. Consider the initial beginning of the story, where the current world state includes: the player beginning as the invisible ghost character in the lobby, Sally in the lobby, the Innkeeper in the lounge, and the corpse in the library. The director creates WMEs that encode these and all other observable world facts (record world state). The director also hypothesizes that the player knows that Sally exists and any of her observable properties (hypothesize entity knowledge). Once knowledge monitoring can no longer execute, the director begins executing plot monitoring functions; several preconditions in plot point C from Figure 11 are marked as true (e.g. Sally and the player are within one room of each other) (mark preconditions as true / false). The director will also bind the variable x to Lobby so that At(Sally, x) is fulfilled as well (instantiate variable). Therefore, the plot point now contains the preconditions At(Sally, x=lobby) and At(John, x=lobby). After 5 seconds have passed from the beginning of the experience, the begin timing constraint for plot points C, D and A is met. Therefore, the director creates an ^active true WME for these plot points (mark plot points as active / inactive). Since the example plot point has a precondition that has been fulfilled by the player (in this case, by virtue of the fact that the constraint was written to match the opening conditions of the story), the director selects it for performance and creates a ^performing true WME for that plot point (select plot point for performing). 75

88 The director now executes story direction to fulfill the sole precondition that is left unfulfilled since it only involves the actions of synthetic characters (i.e. the Innkeeper needs to be in the Lobby) (note precondition with NPCs only). The director creates a goal to execute direction with an augmentation noting that this is specifically story direction. With this goal active, the director proposes to use the only relevant location strategy, actor-to-area. Since it is the only strategy proposed, the director selects and execute actor-to-area. It sends a command to the Innkeeper to create a new goal of going to the Lobby. The Innkeeper immediately switches goals from his current one to this new goal and move to the Lobby. Once the Innkeeper is in the Lobby, the director sets the final precondition, At(John, x=lobby) to true. Now the preconditions for the plot point have been fulfilled within the timing constraints, so the director executes story direction once more (note perfoming plot point fulfilled). The director sends commands to Sally and the Innkeeper to perform the small_talk conversation. The conversation is performed and the plot point is marked as having been performed (mark plot points as performed). According to the story s ordering constraints, only plot points A and D can be active at this point. Since a plot point has just finished, the director runs its predictive model based on the current situation (predict player). For this example, assume that the prediction yields no relevant results (a more detailed example of prediction is given in Chapter 7). The precondition Proximity(player, sally, 1) for plot point D is now fulfilled, leading the director to select this plot point for performance. (select plot point for performing). The remaining unfulfilled condition, Hidden(player, False) cannot be directed via story direction, since it involves player actions. The player executes no actions that shift the director to label a different plot point as performing and time passes. The end timing constraint for plot point D is violated, causing the director to choose to execute reactive direction. The director creates a goal to execute direction with an augmentation for reactive direction. The director proposes all relevant hidden strategies, which are hidden-funny-feeling and hidden-reveal-thyself. Hidden-reveal-thyself is rated highly in effectiveness and low in subtlety while the opposite is true for hidden-funny- 76

89 feeling. Since the director is reactively directing, it puts a higher preference on effective strategies, leading hidden-reveal-thyself to be selected. This strategy involves the director commanding a synthetic character that is within listening distance to eerily chant, Mysterious apparition, reveal yourself! to goad the player into becoming visible. If the strategy works, then all of the preconditions are then fulfilled and the director proceeds to perform story direction. If not, then the system s dealing with the boundary problem has failed and it is on the player to move on to some other active plot point and / or eventually fulfill this currently problematic precondition. 77

90 Chapter 7 Player Modeling 7.1. The Player Model IDA is designed to avoid conflicting player behavior before it occurs. The system models short-term player behavior and treats the results of that model as a hypothesis of player behavior in the near future. If the player is hypothesized to create a boundary problem, then the director will try to influence the player s behavior to avoid that possible future. This hypothesis involves the director a) creating an internal simulation of the game environment, b) running an author-defined player model on that environment, and then c) building a hypothesis of the probability of future boundary problems by observing the model s behavior and comparing it to the plot representation (Laird 2001). This process gives the director a hypothesis about what may happen in the future as well as the probability of that future occurring. Whenever a plot point is finished, the director begins modeling by creating an internal copy of the world state, which includes the director s hypothesis of the player s knowledge base. This internal copy is treated as a virtual representation of the environment that the model can reason about and execute actions in. It is a working memory structure identical to the director s current representation of the world state. Figure 16 illustrates a subset of the information that is copied over during prediction. The identifiers, such as S1 or E8, are unique identifiers in working memory. When a new state is created for modeling (S2), it contains new WMEs that are created for modeling. The structure of working memory and the information stored in those 78

91 Figure 16. Example of copying information, including hypothesized player knowledge, to the modeling state for prediction. WMEs (e.g. L2 represents the list of entities and their attributes that the player has knowledge about) remain the same (e.g. L4 stores the exact same information as L2). The state of the world, NPC s, and the player are all copied over into the modeling state for prediction. The director has an internal probabilistic rule-based model of the player s behavior that is specified by the author (i.e. the model is domain-dependent and need be created by the author as a programmer). This model is intended to represent a general hypothesis of how a player would behave in the story world. Player behavior in the Haunt 2 environment is strictly comprised of physical actions, which makes modeling that behavior a more approachable problem as opposed to, for example, dealing with the player engaging in verbal conversations and attempting to predict speech acts. This approach may be expanded to deal with verbal interactions (e.g. discourse plan recognition), but is not within the current scope of IDA. As shown in 79

92 Figure 17, the model used for Haunt 2 is composed of both an operator hierarchy very similar to the hierarchy used to define the HauntBot agents and the hypothesized knowledge base of the player. This hypothesized knowledge base is the main source of information that the director has about the individual currently playing the game; the player model, as currently designed, is a general model of player behavior. The operators used in the Haunt 2 player are a subset of those used by the HauntBots for autonomous behavior in the game. Operators do not decompose into atomic actions (e.g. move forward one step), but instead only decompose into executable abstract actions, such as move-to-area. The player is modeled to instantaneously move to the target area with some associated cost in time when move-to-area is selected and applied. This is a rudimentary player model used to show the benefits of a predictive director. Future work entails research on defining more rigorous and adaptive models. Figure 17. Probabilistic model of player behavior in Haunt 2. Model operators are run on the simulated environment just as if the director agent were proposing, selecting and applying operators in the real world (as the player). When choosing which model operator should be selected at a given time, the director chooses one of the operators that are applicable in the current situation based on probabilistic control knowledge per the Soar decision cycle described in Chapter 80

93 4. For example, when all top-level operators in the model subset shown in Figure 17 are applicable, explore will be selected 60% of the time. The operators that have conditions matching current WMEs are proposed. This provides a more functional model that represents a probabilistic space of players rather than a more rigid deterministic model. This also allows the director to estimate the probability of the occurrence of model actions. The director runs this player model on the simulated world, executing rules to simulate how the world would respond to the player s actions and observing what plot elements are affected by the model s actions. The model returns a tuple M (R, P). R is the modeling result, which is either a success, meaning that an active plot point s preconditions are fulfilled by player behavior, or a failure, indicating that no active plot points are fulfilled. P is the probability of the sequence of operators selected in that modeling run, P(Run x ) = P(Operator x1, Operator x2,, Operator xn ). For example, after the game has begun and the player has executed a few actions, the director creates a copy of the world, runs the player model on that copy, and returns the result that the player will remain in that room (probably examining objects) until the next plot point s timing constraint is violated in the simulation. Once a modeling run is completed, the director goes through the entire process again, executing a Monte Carlo simulation until some author-determined limit ρ is reached. Once the simulation is completed, the director computes the probability of the player fulfilling plot content, P(F), which is defined as: P(F) = (s/n s ) - (f/n f ) Equation 1. where s is the sum of P values across runs with R = success, n s is the number of successes, f is the sum of P values across runs with R = failure, n f is the number of failures, and -1 <= P(F) <= 1. Therefore, the value P(F) equals the amount of confidence we have in R occurring in the story world, according to the player model. 81

94 Player has been moving to new rooms Predicted goal: Explore Predicted timing constraint violation Direct Sally to cough Figure 18. An example of predictive modeling being used. The result of modeling, P(F), is used to determine if the director should preemptively direct. If P(F) is above some author-defined threshold α, which represents the author s desire for how frequent the director should direct in a particular domain, then the modeling is considered to indicate that the player is likely to contribute to future plot content. If P(F) < α, then the director will decide to preemptively direct the world because the story s progression is likely to be hindered. In Figure 14, we can see that the player, Innkeeper, and Sally should all be near each other. The player knows where the dead body is and should be invisible. Once the preconditions of this plot point are fulfilled, the characters are to reveal more information about their relationship to the player. Suppose that, at this point in the experience, the plot point reflected in Figure 18 has just become active. The player was staying near the other characters for awhile and has now begun searching for clues through the different rooms in the building after the last plot point has ended. The director queries the predictive model, which hypothesizes with a high degree of certainty that the player is exploring and will continue to explore for some 82

95 time, not entering the lounge as required by the plot point content. The director concludes that there is a high probability of a timing violation occurring and should therefore execute some direction in the environment. Since this timing violation has yet to occur, the director selects an action that is high in subtlety and a little low in terms of effectiveness, such as directing one of the characters to cough loudly. The director executes this command and the player, who is exploring the world, may be attracted to this new input and go to the room where the other characters are. Without the use of prediction, the director would have had to either wait until a timing violation had actually occurred or execute a subtle action at every possible opportunity, neither of which fulfills my architectural requirements Connecting Modeling to Director Action The section above shows how predictive modeling is used by the director to decide whether or not it should execute direction. The director s next step after deciding to direct is to decide which direction is most appropriate (i.e. how it directs) based on the modeling results. Director actions are rated by the author in terms of several dimensions, the most important (and currently only ones used by the architecture) being subtlety and effectiveness. The dimensions that the author uses can be specified on a scale of 0 to 1. The selection of dimensions and rating of actions is up to the author (e.g. she may decide to rate actions based on how many characters are involved), which is similar to author-rating in MOE (Weyhrauch 1997). This role for the author allows her to encode her artistic ideal of what is important for selecting director actions. I have decided on subtlety and effectiveness to help illustrate the benefits of using predictive modeling in an interactive drama. In addition to these ratings, different scoring functions are defined to reflect the author s bias in how the director should respond to the modeling result. Scoring functions for IDA are defined as: score = (S 1 * s 1 + S 2 * s S n * s n ) / n Equation 2. 83

96 where n is the number of dimensions used, s 1 to s n are the ratings used by the author, and S 1 to S n are the weights used to reflect the author s desire for the particular scoring function. These weights are assigned at run-time to reflect the appropriate weightings for a given situation. The specific scoring function used for Haunt 2 is: score = (S * sub + E * eff) / 2 Equation 3. where n is the number of dimensions used (n=2 in this case), sub and eff are the subtlety and effectiveness ratings, and S and E are weights. If S > E, then subtlety is more preferred than effectiveness for that function. For example, if α < P(F) < α + pos, where α is an author-defined threshold that defines the line between a positive and negative result and is some small value used to indicate that the result was only a failure or success by a small fraction, then the author might write a scoring function to highly prefer subtlety over effectiveness when the director is heuristically choosing a director action. If α - neg < P(F) < α, then subtlety might be a little less preferred than effectiveness. If P(F) < α - neg, then effectiveness may be preferred heavily over subtlety. These relations are designed to encode the relationship between modeling result and director action selection, but are not based on any preexisting theory. The important factor in this design is that the score for a director action is dependent on the current situation. If a situation is urgent (i.e. requiring reactive direction), then the weights will be assigned to favor effectiveness, thus affecting which direction has the highest score. If the predicted problem occurs off in the future, then the weighting will naturally shift towards more subtle measures. This design allows a connection between the urgency of a boundary problem and the direction that is selected to address it. The cough direction selected in Figure 18 is an example of the selection of director actions based on score. If modeling returns a success (α < P(F)), but the confidence in that result is low (P(F) < α + pos ), then the weights are assigned as S = 0.8 and E = 0.2 to reflect the preference for more subtle measures. If modeling 84

97 returns a failure (P(F) < α) with high confidence (P(F) < α - neg ), then the weights are opposite: S = 0.2 and E = 0.8. If the possible actions that are shown in Table 1 are the ones proposed by the director, then cough would be selected if S = 0.8 and E = 0.2. If reactive direction were being executed, then the more effective strategy, actorto-area, would be chosen out of this set of possibilities. proposed direction sublety Effectiveness cough actor-to-area create-sound Table 1. Proposed director actions for attracting player in Figure 18. The combination of reactive and preemptive direction provides IDA with a unique approach to addressing the boundary problem. While this method does make use of a specific predictive model to show prediction s value in IDA, it uses a simple hand-authored model of general player behavior. More adaptive or empiricallyderived models would most likely better demonstrate IDA s benefits; however, if experiments can show that this basic approach to predictive modeling is useful, then more accurate models must be at least as useful, if not much more so. Chapter 8 evaluates the use of this model and Chapter 9 explores what work may be done in the future to improve on the kinds of models used Example of Using the Predictive Model This section describes an example of the director executing the predictive model in Haunt 2. Consider the active plot points A and D from the story content shown in Figure 11. Once the previous plot point, C, has been performed, the director will create a goal to predict. The director creates a new internal problem space and copies the entire knowledge structure about the world state over to this new space. Once completed, the director initiates running the player model. The model selects the explore goal, since there are many rooms the player does not know about. 85

98 The model proceeds to predict that the player moves from room to room, invisible, as it explores the inn. The simulated clock increases by 10 seconds with each movement. This continues until a timing violation occurs because of the Hidden(player, False) precondition, thus ending the prediction. The director records the results of this run, noting the negative result and the likelihood of it occurring. The director completely erases the problem space and creates a new one, copying over the state information once more. The director then executes the predictive model and records the results. The director repeats this process until it is executed ρ times. Once ρ is reached, the director sees that the result (P(F) = -0.12) indicates that a timing violation is predicted to occur with low confidence (α - neg < P(F) = < α ). Since a timing violation is predicted to occur, the director creates a goal to execute direction augmented to note it is preemptive direction. The director proposes hidden strategies to address the problematic precondition, which are hidden-revealthyself, which is ranked as more effective than subtle, and hidden-funny-feeling, which is ranked as more subtle than effective. Since the prediction was low in confidence, the scoring coefficients are ranked as Eff only being slightly higher than Sub, thus allowing hidden-funny-feeling to be selected. The director then executes this direction in the environment and continues on with the execution cycle (see Figure 11). Two approaches are possible for what the director does after executing preemptive or reactive direction. The director could use a one-time approach, preferring executing fewer director actions to possibly executing several in a short time span. The other approach involves monitoring the success of direction, and executing a more effective strategy until the desired result is reached. In the current implementation of IDA, the first strategy was chosen as a less domineering approach to story direction. It is not clear how players respond to constant intervention versus one-time intervention by the director. However, the second strategy may in retrospect be the more useful design. Since this is an author-centric system, IDA should have the means to monitor director action failure and take the appropriate 86

99 steps to keeping the story within the story space. Future versions of IDA will incorporate this observation. 87

100 Chapter 8 Experimental Design and Results One of the contributions of this work is the quantitative evaluation of the architecture s components usefulness, specifically the added value of using a director that employs preemptive direction to avoid boundary problems. Few examples exist to date of evaluations of the approaches used in interactive dramas (Nelson and Mateas 2005; Weyhrauch 1997). MOE s evaluation consisted of simulating types of players playing in a simulated game world; no actual game was actually created. Nelson and Mateas later implemented MOE s approach in an actual game system and found conflicting results as to the effectiveness of using the adversarial search methods used in MOE. Very little work has been done on evaluating interactive drama systems with actual player inputs. I have created an experimental design that takes one step towards validating the approaches used in IDA with real player testing. A significant constraint to this design, however, is cost. While testing Haunt 2 with a large set of human players would be ideal, the constraints of this project have led us to take a more economical approach. My goal in evaluation is to conduct a low-cost, quantitative experiment that will support my claim of preemptive direction being a useful alternative to reactive direction in an interactive drama by testing the effectiveness of my implementation of IDA. The experimental design used for this evaluation is not the most rigorous that could be done but is a low-cost step in a quantitative evaluation of preemptive direction in an interactive drama. As opposed 88

101 to testing many different types of players with the system, I have investigated the use of player archetypes to define different behaviors for a single playtester. It involves the defining of player archetypes, which are used to define how the experimenter should behave while playing the game. The experimenter then plays through the game, taking on the different game personae defined in the archetypes. The data recorded from playing as these different archetypes, which are defined in Section 8.1, with preemptive direction on and off are then used in the evaluation. The comparison between these two groups is done by comparing the mean number of timing violations that occur, the number of preemptive and reactive directions that are executed by the director, and the average subtlety of those director actions Defining Player Archetypes Player archetypes are canonical definitions of player behavior that are used to represent players across the spectrum of likely player types. Some work has been done in identifying player archetypes for Multi-User Dungeons (MUDs) which identifies four extreme player archetypes: Achiever, Explorer, Socializer, and Killer (Bartle 1996). The use of archetypes in this evaluation is meant to explore a subset of the space of possible player types. Bartle uses archetypes, as shown in Figure 19, to define a player space along the dimensions of action and focus (e.g. whether or not the players focus more on acting on players vs. the world). This space describes how players behave in the world and what their behavior is focused on. Each archetype is intended to represent an extreme in that space, with most player behaviors existing at points somewhere in between. Archetypes are used for evaluating IDA s effectiveness as a means of representing the edges of the space of typical player behavior in Haunt 2. The steps for doing this are first to hypothesize what players may want to do in the world, then attempting to map those to archetypical definitions. 89

102 ACTING Killers Achievers PLAYERS WORLD Socialisers Explorers INTERACTING Figure19. Bartle's archetypes. In order to identify what archetypes could be used to define player behaviors in Haunt 2, a list of possible player goals was hypothesized along with a mapping from each goal to the game actions involved in achieving those goals: Explore: move to new rooms or rooms with new attributes (e.g. player sees an NPC moves to a new room or a hears a new sound from a room) o move-to-area o move-to-area-with-npc o move-to-area-with-sound o hide Chase: follow NPC s around while visible o appear o hide o move-to-area-with-npc Help: help NPC s with needs o get-object o light-fire o hide Learn: listen to NPC s conversations o move-to-area-within-listening-distance o stand-still Coax: coax NPC to room with body o appear o move-to-area-with-npc 90

103 Use item o Get-item o Use-item o Drop-item These goals and actions are used to describe a set of proposed player archetypes. The set of archetypes below represents the plausible behaviors that are defined by the set of goals above: GENERAL (a behavior that reflects that user model's preferences) EXPLORER (focuses on exploring the physical space) PLAYER (focused on figuring out the game) TESTER (pushes the limits of the system, scare the NPC's a lot, etc.) SOCIALIZER (hangs around the characters and tries to listen to them / affect them) SCARER (scares players but doesn t really do much else) These archetypes can further be defined by the goals and actions listed above, providing us with a mapping from archetypes to actual agent behaviors: EXPLORER: High preference to explore High preference to move to new inputs (e.g. a new sound) Low preference to chase Medium preference to learn Low preference to coax PLAYER High preference to explore High preference to move to new inputs Low preference to chase High preference to learn Medium preference to coax TESTER High preference to explore Medium preference to move to new inputs High preference to chase Low preference to learn Medium preference to coax 91

104 SOCIALIZER Medium preference to explore High preference to move to new inputs Low preference to chase High preference to learn Medium preference to coax SCARER Low preference to explore Medium preference to move to new inputs High preference to chase Medium preference to learn Low preference to coax GENERAL Medium preference to explore Medium preference to move to new inputs Medium preference to chase Medium preference to learn Medium preference to coax 8.2. Experimental Design A subset of these archetypes was selected for experimentation: Explorer, Scarer, and General. These archetypes do not represent the complete space of possible player behaviors, but do sufficiently cover wholly different parts of the space defined by the archetypes above based on the different emphases on possible domainspecific player goals. This design defines one experimental group, modeling, and one control group, no-modeling, to test the usefulness of preemptive direction. The modeling group involves the director operating with all of its capabilities intact; it performs story, reactive, and preemptive direction. The no-modeling group has all capabilities intact except for preemptive direction. The director does go through the process of modeling but does nothing with the results of the prediction, so there is no obvious time advantage for the this group by not spending decision cycles on modeling. 92

105 The variables involved in each run are: the type of archtype used (arch), the number of timing violations (num_tv), the number of directions executed by the director (num_dir), and the average & median subtlety of directions (avg_sub & med_sub). Num_tv indicates the number of times the player s behavior has caused a boundary problem. Num_dir gives a rough measure of how often the director has to interfere with the world based on player behavior - the less interference, the better. My hypothesis is that num_tv will be higher on average for the no-modeling group, since preemptive direction should help avoid at least some boundary problems. I also hypothesize that avg_sub should also be higher in no-modeling than for modeling, since modeling attempts to make use of more subtle direction. The director parameters were assigned as follows: α = 0 (on a scale between -1 to 1), neg = -0.12, and pos = These parameter assignments were tweaked during the design process to elicit the desired director behavior. Experimental runs were done 15 times for each archetype in each group. A single run involved me, the experimenter, following the behavioral rules described for the selected archetype. The director logged each measured event (e.g. timing constraints) into a file. Each file was parsed with a simple parser program and the statistics for each experimental run were computed. A more rigorous design would have been to encode these behaviors in an autonomous agent or better yet have another person or people conduct the runs instead of me. The agent approach was ruled out since the same bias would likely exist if I built the agent versus doing the runs myself. The second was ruled out for pragmatic reasons, since the time taken to conduct the runs was over forty hour s time. This design acknowledges the relaxed approach used in this part of the experiment and should be used to indicate where future work can improve on it Experimental Biases Given the low-cost nature of the experimental design, there are several dependencies that may influence the results of this study. These dependencies reflect possible biases in the different components of the IDA architecture: director bias; player bias; and story bias (which includes the story world, content, and synthetic 93

106 character behavior). These components represent the major factors within the closed system of the experiment Director Bias The director s behavior is dictated by the knowledge it gathers about the environment and the player, the parameter settings for modeling and using the modeling result, and the director strategies that are authored for the story domain Knowledge Model The gathering of knowledge is a necessary function of the director that is unlikely to introduce bias into the experimental results, unless the knowledge model is authored incorrectly. The hypothesis of player knowledge could contain error (e.g. just because a player walks into the lounge does not mean that the player sees the match and knows that putting it into the fireplace will start a fire), but this error would be consistent across experimental groups. However, the predictive model does take the knowledge model as an input. Therefore, the accuracy of the knowledge model may affect the accuracy of the predictive model, which in turn may affect the decisions of when to preemptively direct and how Predictive Model Just as the accuracy of the knowledge model may introduce bias, the accuracy of the predictive model also has a similar effect. The less accurate the model, the less reliable the director s decision is to preemptively direct or not, which in turn decreases the accuracy of the comparison of timing violations between groups. IDA s model for player behavior in Haunt 2 is discussed in Section Parameter Values The results of IDA s evaluation depend on the author-assigned parameter values of α, neg and pos, which dictate director behavior. If the modeling result is below the author-defined threshold α+ pos, then the director will perform preemptive direction. This threshold has an effect on the director s decision making process in terms of when to direct. If this threshold is very high (i.e. close to 1) the director will 94

107 almost always decide to direct, since P(F) is very unlikely to be consistently that high, if at all. Conversely, if the threshold is very low (i.e. close to 0), the director will seldom decide that the preemptive direction is needed. In the case of a high threshold, the director behavior for modeling versus no_modeling would certainly differ in the number of directions executed. Unless the predictive model is extremely accurate, the modeling condition would essentially execute preemptive direction every time the model was queried, regardless of player behavior. The director would nearly always come to the conclusion that a boundary problem was going to occur since the modeling result would almost always be below the threshold for assuming that the story will continue. The no_modeling case would be unaffected, since this threshold only affects the decision to preemptively direct. While α+ pos determines the threshold between directing or not, the sizes of α- pos and α+ neg affect the average subtlety of the director actions. As pos increases, the average subtlety and number of directions should increase since the threshold α+ pos is larger and the number of uncertain modeling successes (α < P(F) < α+ pos ) increases. Both num_dir and avg_sub in the results may have been affected by the assignment of these values. As stated earlier, these values were tweaked during the design phase to have the director s behavior match the author s desire (i.e. my desire) of what the director should do and when. This suggests that the parameter values may be dependent on the domain, the specific author, the plot content, and / or the director actions available Director Strategies The director strategies authored for a story domain determine how the director can attempt to affect player behavior. The effectiveness of reactive or preemptive direction could be weakened if the authored strategies do not adequately cover the space of rated dimensions (e.g. subtlety and effectiveness). For example, if the director actions are all very subtle and not effective, then the story may not even continue when reactive direction is executed since the system has no effective strategies to put the story back on track. Conversely, if only effective strategies are 95

108 used, then timing violations would still be addressed, but the average subtlety of direction would increase, therefore reducing the believability of the experience (heavy-handed techniques decrease believability per Requirement E2). As shown in Appendix A1, there are IDA director actions that are spread across the two rating dimensions for Haunt 2, which avoids the extreme conditions described here Player Bias The same experimenter, computer (a Dell Inspiron 8600 laptop), and experiment location was used for all tests as a control. Any possible bias that was introduced by using the experimenter as the subject is the result of the archetype s sensitivity to direction, the experimenter s inherent sensitivity to direction, and the experimenter s awareness of the current experimental group used in a given run Archetype and Individual Sensitivity The weight coefficients S and E represent the author s prediction of the subtleness and effectiveness each direction has on the player. These coefficients directly contribute to which director action is selected in a given situation. The assigned values for these coefficients are unlikely to be accurate across all player archetypes. For instance, creating a new sound in a nearby room might be highly effective for the explorer archetypical player, but not nearly so for the scarer, who is not as focused on exploring new information. As the experiment was executed, this effect was approximated by the experiment, but did not necessarily reflect the true values of the coefficients. Ideally these values would be determined from player testing and adjusted depending on the player s apparent playing style. This discussion is part of a larger issue of players general susceptibility to direction. Different players of the same behavior type (e.g. explorer) may have different responses to specific director actions. The factors that may influence susceptibility include game experience, playing style, and gender (Heeter and Winn 2005). For example, experienced computer gamers may easily notice when the system is trying to avoid or deal with boundary problems, thus decreasing the believability of the experience and perhaps the effectiveness of using direction, while novices of the same archetype are less aware of the system s manipulations. 96

109 These individual differences within archetypes were not modeled in the archetype behaviors definitions for this study. While acting as the experimenter in this study, I attempted to respond to directions as an archetype would. However, my responses to direction remained constant while playing as that archetype; individual differences were not modeled. This lack of variety is most likely different from reality. Therefore, the data collected in this experiment is an idealized data set; there is no variability between runs of how likely a player within an archetype will respond to direction Awareness of Experimental Group If the subject knew whether preemptive direction was on or off for a particular run, the subject s behavior could be biased. Therefore, I was only aware of the desired archetype behavior during any given run. Whether or not preemptive direction was turned on was not directly known by me, nor did I make a conscious effort to surmise this. The only possible bias here was my unconscious determining of the experimental group and letting that affect my behavior. The only means of removing that bias will be to have a more formal experiment with human subjects not involved in the creation of the system Story Bias Authored content (e.g. plot points, predictive model and synthetic character behaviors) can bias the experience to be more amenable to certain player archetypes versus others. If the content matches particularly well with a specific archetype s goals, then that archetype would be expected to have, on average, fewer timing violations than the others. For example, if the Haunt 2 story was focused on a tale of the player chasing occupants out of the building, then the scarer archetype will be less likely to commit timing violations since the archetype s typical behavior would be closer to the story goals. The archetypes were defined to cover the space of possible archetypical behavior for Haunt 2 in an attempt to avoid this problem. 97

110 8.4. Results The key hypothesis to test was that num_tv would be significantly higher for the no-modeling group than the modeling group (H O : μ model = μ no)model, H A : μ model < μ no_model ). The overall statistics of the 90 experimental runs can be seen in Table 2. A univariate analysis of variance was run on the collected data, comparing the means across variables for the modeling and no-modeling groups. The results of the comparison are displayed in Table 3. A significant difference was found between the means of num_tv in the modeling and no-modeling groups (F= 8.444, N = 45, p < 0.01). This finding confirms my hypothesis that the use of predictive modeling does have a significant difference on the number of boundary problems that occur. There is also an insignificant difference for num_dir (F = 0.49) and avg_sub (F =.818). These results suggest that predictive modeling does have an overall positive effect by decreasing the number of timing violations experienced by the player. This is likely due to the use of directions that guide the player away from boundary problems with at least some moderate success. While there is bias in the experimental design due to me playing as the archetypes versus other people playing it, this is definitely a strong result (p < 0.01) that highlights the benefits of the approach. Type # of timing violations # of directions avg subtlety of direction no_modeling Mean N Std. Deviation Median modeling Mean N Std. Deviation Median Table 2. Basic statistics from experimental runs. 98

111 # of timing violations * Type # of directions * Type avg subtlety of direction * Type Between Groups (Combined) Sum of Squares df Mean Square F Sig Within Groups Total Between Groups (Combined) Within Groups 1, Total 1, Between Groups (Combined) Within Groups Total Table 3. ANOVA table comparing means of modeling and no-modeling. ANOVA tests were done within each archetype group to investigate what factors contributed to this positive result. The general and chase groups yielded no significant difference between the means (F= and respectively). The explore group s comparison, however, was overwhelmingly significant (F= 25.27, N = 30, p < 0.01). To confirm the explore group s affect on the results, a final analysis was done on the explore and general groups together. The difference in means for this test was insignificant as well. This result points to the explore group as being the main contributor to the overall ANOVA results. This finding does bring into question the strength of the evaluation s results, but it is more an affect of the predictive model s ability to cover the different archetype behaviors rather than of the effectiveness of the preemptive approach. Ideally, a significant difference in means would be found in all archetypes. The lack of results found in general and chase are likely due to the predictive model s lack of coverage for these behaviors. The explore archetype is arguably the most similar in behavior to the predictive model; therefore, the model was more likely to be accurate for this archetype than the others in retrospect. There were inconclusive results for significant differences in the number of directions (F = 0.049) and average subtlety of the directions (F = 0.818) across groups. My hypothesis is that num_dir in both groups should be roughly equal, since 99

112 the director is preemptively directing for boundary problems that would otherwise occur and thus be directed reactively. The lack of a difference in avg_sub is a little more problematic. One possible reason for this negative result could be the fine grain that was used to rate the directions (i.e. all directions were rated between 0 and 1). Another more likely reason could be the lack of variety in rating the directions; I as an author did not use a wide spectrum of values for assigning subtlety and effectiveness ratings. In retrospect, paying closer attention to the ratings given to directions would have affected this result. However, the most important result by far is the significant difference in num_tv between groups, which was a positive one. This work does operate under the assumption that fewer boundary problems are better for the player; however, future work should empirically verify this assumption. 100

113 Chapter 9 Discussion The results reported in Chapter 8 are encouraging, showing that there was a significant difference between using preemptive direction versus relying on only reactive direction to deal with the boundary problem. This is evidence that Requirement F2 (i.e. the use of player prediction), which is a novel addition to an interactive drama system, is a reasonable structural requirement to have. What is yet to be determined is how much of a better player experience this approach affords. Qualitative studies, while included in the original experimental design, were not carried out during the course of this work. My hypothesis that fewer timing violations yield a more positive player experience is still untested. Further work needs to be done on evaluating whether or not fewer timing violations and fewer director actions are actually correlated with better player experiences. These results, along with an evaluation of how well IDA meets the requirement laid out in Chapter 2, can be used to describe the effectiveness of IDA as a unique approach to interactive drama Requirements The requirements in Chapter 2 were intended to both guide the initial design of IDA as well as be used to evaluate the final design. This section s purpose is to review the requirements initially laid out and compare them to IDA s final design. 101

114 Author Requirements A1 Authoring Story. The author can create story content independently of building the director. An author explicitly defines story content as plot points; the plot points are then given structure by temporal orderings between them. Plot points can be added and ordered without making any changes to director functionality. The author can specify story content through the FOL representation used in preconditions and actions and story structure through temporal orderings and timing constraints. A2 Building the story world. The story world built for Haunt 2 is a complex 3D environment for the player to act in. Synthetic characters controlled by an exterior AI architecture are used to populate the world. Haunt 2 is authored to allow the player to appear or disappear, move, get object, and use object. While Haunt 2 has been used as a test bed for experimentation, IDA has been designed to be domain-independent. However, the IDA approach is intended to explore domains that involve physical interactions with the story space; boundary problems that arise from conversation, social relationships, etc. are not currently addressed. A3 Defining player actions in the world. The actions that the player can execute in the world are defined in the design of the storyworld, separate from the story representation. The player actions are defined by the story world s game mechanics and the user interface to that world. However, the author must be aware of this set of actions in order to author story content. A4 Building synthetic characters. IDA includes the authoring of synthetic characters in the Soar architecture. While the use of Soar is not an explicit requirement, IDA does depend on the characters to be goal-oriented and directable, meaning that they can transition between a goal they are trying to fulfill and one that the director gives to them. HauntBots have a perception and action model provided by the Soar architecture. What IDA has not explored is the integration of story and performance in synthetic characters. Character animations, dialogue, and story are not coordinated in the representation language as it is in ABL (Mateas and Stern 2002). A5 Expressiveness. IDA s story representation provides a range of expressiveness in content (e.g. anything expressible in a first-order logic 102

115 representation), pacing (e.g. timing constraints) and structure (e.g. partial ordering of plot points). Authoring structured content for Haunt 2 was not hindered by the representation s expressiveness. What the representation does not really address is why events occur. There is no explicit model for providing demonstrable causation in the story world, such as asking a character why they just said something. A use of causal constraints in the representation language, akin to those in partial-order planning, would provide an implicit causal model of events, but it would still not address the sort of improvisational behavior to justify past actions or events that is needed from the synthetic characters Experience Requirements E1 Interactivity. As the requirement states, the player should have a wide range of consequential actions available during the game. The largest problem with Haunt 2 as a story world is the lack of a large amount of consequential actions for the player to execute; future environments used with IDA should provide a larger set of actions to both make better use of IDA as well as aid in improving on its design. IDA does, however, provide a design that constrains the player only when necessary. Timing constraints are the only constraint that the story representation places on the player. Otherwise, the player s actions drive the story experience (e.g. the two-tiered selection of plot points relying on player actions first). E2 Believability. The main thrust of IDA s design is to improve the believability of interactive drama experiences that deal with the boundary problem. The use of preemptive direction has been shown in Chapter 8 to reliably avoid some timing violations that would otherwise occur if the system relied on reactive direction alone. Future qualitative studies should show the benefit of these findings. One of the major facets of dramatic believability that IDA does not address is the directability and performance of the synthetic characters. The director assumes that the characters will intelligently switch between goals, as is suggested in Assanie s work (Assanie 2002). However, IDA currently uses synthetic characters that immediately switch between their current goals and any new ones that have been sent by the director. 103

116 Functional Requirements F1 Coordinating story and player actions. The director agent in IDA is used to maintain coherency between the player s actions and the authored story content. The director achieves this by monitoring the world state and comparing it to the plot (plot monitoring) and dynamically altering the environment to avoid boundary problems (executing direction). The director coordinates the story experience by sending new goals to the synthetic characters when dictated by the story content. F2 Preemptively avoiding boundary problems. IDA employs a predictive model of player behavior to preemptively avoid the boundary problem. IDA is not wholly dependant on the type of model that is used, but is designed to use models that generate the likelihood of the occurrence of a boundary problem. F3 Standardized, Multi-Level Representation. IDA s story representation is designed to be a general representation for expressing plot content and structure. It is instantiated in Haunt 2 but is designed to be domain independent. Future work with IDA should show its usability in other story worlds. The representation does support semi-autonomy in the characters by providing a representation that can represent plot events at different levels of abstraction. The author can specify synthetic character performance at any level of abstraction in the character s operator hierarchy. F4 Semi-Autonomy. IDA uses a multi-level representation for story content, which can then in turn be used to give the synthetic characters commands at different level of abstraction. Characters are defined as a goal hierarchy, which allows the director to send abstract commands that correspond with fulfilling high-level goals or low-level commands that match to atomic character actions. The Soar architecture, which is used for defining character behavior, has been used as a suitable representation for semi-autonomous characters in IDA Story Representation The story representation developed for IDA has several key elements that differ from other story representations and directly relate to the general planning 104

117 community: partially-ordered plot points without explicit causation, content variables, and timing constraints. IDA s design was intended to explore the benefits of using non-causally linked plot elements, as opposed to the typical plan representation used in systems like MIMESIS. The hypothesis was that non-causal content would allow for more flexibility in an interactive environment. Using non-causal content was intended to accommodate player innovation (i.e. unforeseen player actions that positively contributed to story content). The intention was to avoid having to explicitly represent all possible player actions that would have to be included in a traditional planning representation. In allowing innovative approaches to fulfilling preconditions, it was deemed necessary to remove causal links from the effects of operators to preconditions of operators. Therefore, IDA s representation checks for a precondition to be fulfilled, ignorant of what action fulfilled it. For example, if the Haunt 2 environment were more richly defined, the player could have possibly fulfilled plot content for starting a fire by using a magnifying lens and newspaper to start a fire in the library. The non-causal design was intended to avoid having to explicitly represent this action and still allow the effects of the action to contribute to the story. The final story for IDA does not adequately illustrate this claim. The action set and possible effects on the world that could be executed by the player and synthetic character were relatively small in number and easily enumerable in a causally-linked representation. If Haunt 2 were more richly-defined, as in the firesetting example above, the main requirement would be that there has to be at least one way of fulfilling every precondition in the story. The author could not include a precondition where a fire is set without representing a way that the event could occur (e.g. the player placing a lit match in the fireplace, spontaneous combustion of a piece of furniture, etc.). The IDA representation places no such explicit requirement, which may be a shortcoming. Without this requirement, the author can include preconditions that are impossible to fulfill. The addition of explicit causality avoids this problem. 105

118 The initial concern of not allowing innovation with a planning representation proved to be unfounded. Techniques used in continuous planning (planning that addresses unforeseen changes in a dynamic environment) provides a model for how IDA can address concerns about player innovation when using causal links (Cox and Veloso 1998; Myers 1999). The typical closed world assumption for planning does not hold in continuous planning; therefore, unexpected changes in the environment can have a positive (or negative) effect on fulfilling goals (Cox and Veloso 1998). If an unexpected event occurs that is not represented in the plan, the effects of that event are monitored and the planner s knowledge is adjusted accordingly (e.g. a threat may now occur or a precondition may be fulfilled). If the player fulfills plot content (represented as a plan) in Haunt 2 in an unexpected manner, then the director would notice that a precondition has been fulfilled by an outside action and mark that precondition as fulfilled, just as it currently does in the non-causal version. The firesetting innovation example for the current non-causal representation would also be possible in a causal representation with a monitor that observed changes in the world and updated the director s knowledge when appropriate. This would require no major functional change in the way IDA monitors the world and updates preconditions. The major change would be in the story representation and the authoring process. Committing to a plan representation does change the authoring process dramatically, however. Systems like IDA and Façade rely on the author to order plot content for coherence, but otherwise let the selection of content to be affected by player decisions and heuristic selection. Ordering constraints are used to capture temporal coherence, but no explicit causal coherence is used. Using a plan representation forces the author to represent plot content with causal effects (i.e. all preconditions for plot events must explicitly be matched by the effects of some other event to create a complete plan). The author constructs story as planning operators and relies on an iterative design process (using a planner) to see if a coherent and consistent story is created. It is unclear whether the non-causal vs. causal approach is better from an author s point of view. No comparison in this field has been done to date to deeply compare story representations from the author s point of view (nor from the player s). 106

119 The overall benefits of using plans, such as the ability to replan and the allowance of player innovation, suggest a possible change in IDA s representation. MIMESIS has shown how replanning can be a subtle, behind-the-scenes reactive approach to dealing with boundary problems (i.e. threats to a story plan) (Young et al. 2004). The benefit of allowing player innovation was the main reason for selecting a non-causal representation during IDA s design. These benefits imply that a planning representation may be more suitable for IDA in the future. However, these benefits will need to be weighed against possible computational factors, such as scalability (would this representation be suitable for large story worlds?) and responsiveness (is a continuous planning approach fast enough to respond in a fast-paced, richly-defined story world?). Variables have been commonly used in planning representations (Russell and Norvig 2002) and for semi-autonomous character behavior (Blumberg and Galyean 1997; Perlin and Goldberg 1996). The initial integration of using variables into IDA s story presented the challenge of having some story elements represented as constants and others as variables, which meant relying on different director functions for processing story content. As opposed to having split functions for working on plot constants and variables, IDA s representation was changed so all parameters in preconditions and actions are variables. Content that was previously represented as a constant was changed to be a static variable (i.e. a variable that is assigned before runtime and cannot be changed). This provides a cleaner director design that can apply the same functions on all plot content. The use of content variables in interactive drama has specific problems when exploring the concept of director variable reassignment (i.e. changing a variable s assignment as a director action), which has been considered but not implemented in IDA s design. If the director decides to reassign a plot variable as a means of dealing with a boundary problem (e.g. the player kills a character that was supposed to be the story s antagonist, but someone else could fill in the antagonist role in the story), the system has to take into account the believability of that director action. In other words, the director cannot change a variable s binding to be bound to another game entity that makes logical sense, but may make no sense to the player (e.g. changing 107

120 the antagonist to be the player s best friend). The issue of believability of variable assignment is not a typical issue in traditional planning domains. The addition of timing constraints is a necessary addition to IDA s story representation provided a means for capturing specific boundary problems that do not appear to be address in MIMESIS. Player inaction, whether due to boredom, indecision, or a lack of important information, can lead to negative player experiences (e.g. playing an adventure game and having the story experience completely halt because the player has missed an important clue). Timing constraints have used for goal deadlines in planning work (André and Rist 1996; Bacchus and Kabanza 1998; Dasarathy 1985), but have not been considered in a narrative medium for dealing with boundary problems due to player inaction. IDA s use of timing constraints is a step forward in addressing this issue Future Work While my work on IDA represents a significant step forward in interactive drama approaches, there are quite a few areas of future research that this work points to: Adaptive Models IDA has shown that using predictive models can be effective as an input to story management. It does not, however, make any claims about how to build good player models. The predictive model used in IDA is a simple baseline approach for experimentation; better models should in turn yield better results. Two approaches could be explored for improving the reliability of IDA s model: 1) having a set of models to match the player s behavior to, and 2) adapting a single model to match the current player s behavior. The first approach could take advantage of models that represent player archetypes (or a set of models in the archetype space) and pigeonhole players as their behavior best matches one of those archetypes. This would allow the director to have specialized strategies for guiding one particular type of player versus another. While an archetype model (or a model that represents a player that is a mix of archetypes) may be more effective than the single general model approach used in IDA thus far, it still has the disadvantage of 108

121 pre-authoring behavioral models that aren t directly representative of the current player. The second approach would be to start with a model or set of archetype models and then adapt those models to the player s behavior. Starting with some base model of behavior, the director could refine and update its hypothesis about what this player is trying to accomplish through observation. This observation could help tune the probabilities in the player model(s) to better encapsulate an individual player s goal preferences (Albrecht et al. 1999). This technique will highly depend on both the number of observations the system can make as well as knowing how to assign credit (i.e. how does the system know what series of actions were executed to reach what goal?) Player Testing The experimental design in this work, as pointed out earlier, is a low-cost approach to evaluation with archetype definitions. An extension of this work would be to do both a quantitative and qualitative study with real human subjects. The quantitative study could use the same measure done in the informal study, such as comparing the number of timing violations in the modeling and no-modeling groups. A qualitative study, such as asking subjects to rate the quality of the experience, could be run to gauge whether or not fewer timing violations actually lead to a more enjoyable player experience Interactive Training While IDA focuses on providing interactive, dramatic experiences, some of the same approaches used in IDA may also be used for experiences that focus on training as well. I have explored a first step for reformulating IDA in the teaching domain, putting pedagogy as the primary goal of the system while having entertainment and engagement as a secondary goal. This approach would take advantage of IDA s approach to drama management while providing more individualized teaching content, as an intelligent tutoring system may do. This concept is currently being explored in the Interactive Storytelling Architecture for Training (ISAT) (Magerko et al. 2005). ISAT builds on IDA s approach to hypothesizing about a player s knowledge by building a model of the 109

122 player s aptitude in the set of skills being trained. The state of the skill model can be used to affect the director s heuristic choice of plot points (e.g. preferring scenes that test skills that the trainee is lacking in), the reasons for executing direction (e.g. giving the player different levels of in-game guidance depending on the player s aptitude in the current skill being tested). The individualization of training content provides a trainee with material that is guided towards addressing his individual needs. For example, if a trainee in a medic training environment has shown a lack of aptitude in applying tourniquets, the director may adapt the environment to provide in-game guidance. Much like intelligent tutoring systems, this provides some of the benefit of having a human tutor teaching each individual student in a realizable, cost-efficient manner. As opposed to intelligent tutoring systems, this approach relies on a rough-grained model of trainee skill in unstructured domains rather than the finer-grained, cognitive model that is appropriate for more mental tasks (e.g. algebra problems) Authoring Tools One of the most difficult aspects of using IDA is the process of authoring, which involves the construction of synthetic characters, the defining of director strategies, and the writing and structuring of story content. Constructing a synthetic character requires the author to define the operator hierarchy, how the operators in that hierarchy are selected, as well as the language for communication between the director and the characters. These tasks are commonplace for creating intelligent agents in the Soar architecture. Authoring tools already exist to aid programmers in creating Soar agents, such as VisualSoar. These tools have served well in the construction of HauntBots for Haunt 2. Creating director strategies, for both plot point selection as well as executing direction, for a story world involves a cyclic process of creating a strategy, debugging it in the story world, and then editing the strategy if necessary and repeating. This process becomes more complicated when several different strategies may be applicable and the author wants to see the proposal of all of them and possibly the effects of all them (i.e. ask what could possibly happen given a particular world state? ). Without tools, this relies on the author to load up the story world, set it to a 110

123 particular to state (e.g. the synthetic characters in one room with the player in another, invisible and holding the thermos object), either see what the director does (or perhaps hardcode it to select a specific strategy), then iterate through this process again to see another strategy. With more than a couple of strategies, this cycle can be very time-consuming for the author. An authoring tool could be constructed to help shorten this process by providing a simulated representation of the environment that the author could directly manipulate. With this tool, the author would be able to specify a desired world state in this simulated world (e.g. moving objects and characters around into a specific configuration in the world, changing their attributes, etc.) then query the director to see how it responds to the situation. The director s proposed or selected actions could be reported to the author in the tool, thus quickly informing her how the director would behave in a given situation. This would provide an immense reduction in the time needed to debug director strategies for a particular domain. The same tool described above could be extended to authoring plot points in IDA. A common process for writing story content currently involves writing a plot point in English on paper, then encoding it as a production in Soar (as shown in Figure 20), running Haunt 2 and seeing how the plot point worked, and iterating as a debugging process between changing Soar code and running Haunt 2. For someone who is very familiar with Soar this process is still a time-consuming one during the debugging phase. As shown in Figures 11 and 14, story content is authored in a very visually disjoint manner, requiring the author to create each logical statement across several lines and maintain consistency with the predicates that are used. An authoring tool that could both speed up the debugging process as well as remove the authoring process from being Soar-dependent would be a significant contribution to the usability of IDA. For example, the authoring tool that is described above could be extended to allow the author to visually create plot points and associate them with particular world states. The author could configure the world, select Create new plot point to insert it into the story, and then add ordering constraints between it and other plot points, much like how Figure 11 shows the visual organization of the plot points. This would avoid having to hand-author story 111

124 content in Soar, which means that an author could encode story content without a working knowledge of Soar. I am currently working on a prototype of this type of authoring tool with a graduate student at Michigan State University. A screenshot of creating plot points in the initial prototype is shown in Figure 21. sp {plot*elaborate*final-story*e (state <s> ^name director ^timing-addition <ta> ^story final-story) --> (<s> ^plot-point <pp>) (<pp> ^name E ^title relationship-talk ^active false ^preconds <pre> ^postconds <post>) (<pre> ^descriptor <d1> ^descriptor <d2> ^descriptor <d5> ^descriptor <d6>) (<d1> ^type hidden ^entity user ^entity scared-remarker ^hidden true ^done false) (<d2> ^type proximity ^entity sally ^entity sally-ex ^distance 1 ^done false) (<d6> ^type proximity ^entity user ^entity sally-ex ^distance 1 ^done false) (<d5> ^type proximity ^entity user ^entity sally ^distance 1 ^done false) (<post> ^descriptor <d3>) (<d3> ^type conversation ^entity sally ^entity sally-ex ^conversation relationshiptalk ^done false) } Figure 20. A plot point authored in Soar. 112

125 Figure 21. Screenshot of a prototype authoring tool for creating interactive drama content Categorization of Director Strategies The director in IDA employs a set of story management strategies that are created by the author. These strategies are simply a set of director actions that the author thinks would be a good idea for this story domain; they have no basis in how an expert human storyteller may approach the problem. Future work could improve the director s strategies in terms of breadth and effectiveness by studying the strategies that real-world storytellers use in interactive domains, such as table-top role-playing. If a set of mediation strategies can be elicited from them, then the authoring of director strategies can be better informed by the experts who actually serve as directors in real life Contributions IDA makes several contributions to the field of interactive drama that set it apart from current approaches in the field: Player modeling IDA s modeling of player knowledge and behavior are both unique steps in the construction of an interactive drama architecture. The director s maintenance of a 113