Behavior Modeling in Commercial Games

Transcription

1 Behavior Modeling in Commercial Games Final Report Produced By: William Ferguson David Diller Alice Leung Brett Benyo Dennis Foley Produced For: Office of Naval Research 800 North Quincy Street Arlington, VA Contract No. N C-0161 April 19, 2006 A BBN Technologies ument

2 Copyright 2006 BBN Technologies Corp. All rights reserved.

3 Contents Contents...i Abstract Introduction Background Behavior Generation Behavior by Genre Action and Shooter Games Strategy Games Adventure Games Role-Playing Games Simulation Games Sports Games God Games Massively Multiplayer On-line Games Components of Behavior Sensation and Perception Motor Control Memory Decision-Making Learning Communication Behavior by Functional Role Teammates Adversaries Basic Units Story Elements or Support Characters Dolls or Subjects Current Behavior Authoring Capabilities and Limitations in Games Parameterization Programming/Scripting Specialized GUI Scripting Trigger and Rule Systems Dialog/Decision Trees Hybrid Approaches Smart Environments Machine Learning Third Implications for the Modeling Community Additional Military Uses For Games Game Engines Content Creation Networking and Multiplayer Capabilities Other and Technologies Matchmaking Community Building Future Directions for Behaviors in Games...18 Acknowledgments...19 Appendix A: Fallacies about the Gaming World...20 Appendix B: Resources...23 Appendix C: References...24 Appendix D: Game Evaluations...26 D1. Game Evaluation Form...26 D1.1. Overview...26 i

4 D1.2. Description...26 D1.3. Buzz...27 D1.4. Team Play...27 D1.5. Game AI...27 D1.6. Content and Scenario Authoring...28 D1.7. Behavior Authoring...29 D1.8. Miscellaneous...29 D1.9. Expansions...30 D1.10. Current Military Applications...30 D1.11. Information Sources...30 D1.12. Extra Comments...30 D2. Game Evaluations...31 D2.1 America s Army...32 D2.2 Age of Empires II...34 D2.3 Black and White...36 D2.4 Black Hawk Down...38 D2.5 Civilization III...40 D2.6 Civilization D2.7 Civilization Call to Power...44 D2.8 CivEvo...46 D2.9 Cloak, Dagger, and DNA...48 D2.10 Counterstrike...50 D2.11 Far Cry...52 D2.12 FreeCiv...54 D2.13 Galactic Civilizations...56 D2.14 HalfLife...58 D2.15 HalfLife D2.16 Hidden and Dangerous...63 D2.17 Mind Rover...65 D2.18 Morrowind...67 D2.19 Neverwinter Nights...69 D2.20Operation Flashpoint...71 D2.21 Quake...73 D2.22 Rise of Nations...75 D2.23 Storytron...77 D2.24 The Sims...79 D2.25 Unreal Tournament D2.26 Vampire: The Masquerade...83 D2.27 Warcraft III...85 ii

5 Abstract This is the final report for the ONR-funded effort to survey commercial game technologies for military relevance. Our study is centered on those technologies related to behavior modeling and the authoring of behavior models. In this paper we attempt to lay a foundation for behavior modelers to understand the gaming industry -- its methods, goals and accomplishments. We extrapolate where the industry is going and what future synergies it is likely to contribute. We focus on behavior generation and other accomplishments that are relevant for training applications and especially for non-programmer authorship of content for training applications. We will argue that the gaming industry s modeling capabilities are currently of little theoretical interest but that, pragmatically, the current accomplishments of the gaming industry can have a substantial positive impact on training and modeling and that this impact will increase over the next few years. 1. Introduction The computer game industry invests a lot of resources into game development, and has achieved impressive advances and a huge player base. The military has long sought to leverage game technology, and both an interest in game-based technologies and actual application of these technologies is on the upswing. Should behavior-modeling researchers jump onto the gaming bandwagon? We will try to answer that question for the benefit of the behaviormodeling community, particularly those with an interest in developing models for use in training. This is the final report of our effort to survey gaming technology for potential military use. In this report we discuss possible benefits to the behavior-modeling community, and draw attention to aspects of the gaming world that are particularly relevant to modeling researchers. Utilization of current game technology already holds promise for improving training efficiency and effectiveness, but future developments in the gaming world are likely to have even more direct application to behavior modeling and benefit to military training. There are many motivations for military interest in incorporating game-based technologies into training applications. Computer games are of interest as low-cost alternatives to other more expensive forms of training. Recent advances in Personal Computers have allowed games to provide increasingly believable, compelling, virtual training environments. Additionally, the pedagogical utility of games has been improved by advances in game modifiability and extensibility through scripting and scenario-editing software. Existing support for multiplayer interactions, such as text messaging, voice chat over the Internet (called Voice Over IP, or VoIP ), or player scheduling and matching capabilities could similarly be utilized in training sessions. Games are also of interest due to their motivating capabilities and the familiarity that today s soldiers have with computers and games. There are also many indirect reasons for behavior modelers to keep an eye on the game industry. First, commercial games provide extensive examples of how simple artificial intelligence (AI) behavior can be used effectively in virtual worlds. Modelers who build training applications may be able to take advantage of the fact that trainees, like game players, will only expect a small slice of behavior from synthetic entities populating the training scenario. By studying human-ai interaction in game contexts, modelers can determine what are the essential requirements for a synthetic entity, and delineate those aspects of behavior that can remain shallow without adversely affecting the trainee experience. Researchers who build models for training applications should be aware of game-driven technologies that are only peripherally related to behavior modeling, but are directly related to training applications. Synthetic entities may be expected to interact with trainees through many of the same channels where multiplayer communications currently occur in games. Second, because games provide a virtual environment for both human users and synthetic entities, modeling researchers can use games as research testbeds. Games can provide a variety of behavior challenges at varying levels of abstraction, ranging from low-level path planning, mid-level tactical behaviors, to high-level strategic choices, while avoiding many of the problems introduced with real hardware, such as noisy or imperfect sensors and effectors. Researchers can use these game testbeds to gather data on human behavior, to experiment with how people and models interact, and to validate model behavior. The simple AI of synthetic characters in games might serve as a base-line case for measuring the utility of behavior models; and utility metrics might be based on how training performance is improved when behavior models are substituted for baseline game AI [Campbell, Bolton, Young & Pew, 2005]. 1

6 Finally, the growing popularity and availability of game authoring tools demonstrates the value that the gaming world places on enabling non-programmers to modify game content. These kinds of authoring tools have made games more useful as training applications, since using them makes it easier to deploy new training scenarios into the games. Behavior modelers are also likely to find that an investment in developing good authoring tools for entity behaviors will result in useful customization of training applications by non-programmers (for example, by subject-matter experts). These may benefit from some authoring technologies used in games. In the following discussion, we begin by comparing the goals of behavior creators in the game world to the goals of military-behavior modelers. We review the range of synthetic-entity behaviors used in commercial games, and the methods and techniques by which these behaviors are developed. We then describe current capabilities and limitations for behavior authoring in games; followed by an inventory of potential uses of games technologies by military-behavior modelers and training systems. We conclude with a discussion of the future directions of behavior creation in games and their potential applicability to the Military. The appendix includes data from our survey of behavior-authoring techniques in games, and an accounting of many of the common fallacies regarding the gaming world. 2. Background Commercial game developers have some of the same goals for the simulation of human behaviors as militarytraining-application developers, such as the creation of an immersive, simulated world. Additionally, behavior creation is attracting increasing attention from game developers. As advanced, high-resolution graphics become commonplace, game developers are increasingly relying on game AI (i.e., behaviors of synthetic entities) to distinguish their game from competitors [Woodcock, 2000]. At the same time, game developers have become increasingly concerned with producing behaviors that are more realistic and robust: The hardest thing in game AI is just making sure that the game never looks dumb. You d be better off having an AI that was just above average all the time rather than one that was brilliant 98 percent of the time and stupid 2 percent of the time. [Steve Rabin quoted by Cass, 2002] Game developers are interested in producing entities that are more adaptive to new situations, more difficult to game 1, less predictable, and more variable. Currently, the gaming community uses synthetic entities to play a range of roles, and makes use of well-crafted scenarios to focus the user experience and highlight appropriate behavioral capabilities while downplaying behavioral imperfections and inadequacies. These needs and intentions for the development of behaviors in synthetic entities are shared with the Military-training community. Primarily, the Military-training community is interested in better ways to build more effective training systems, including 1) methods that can reduce costs by more quickly and easily developing training applications, including building training systems with less reliance on expensive researchers such as computer and cognitive scientists; 2) methods that will produce superior training systems, including better human-performance models for simulated entities, and improved training through immersiveness and realism. Table 1: Notable commercial ( COTS ) games in use by the U.S. Military. COTS Based Game Military User Reference Air Force Delta Storm Air Force [1] Decisive Action Army National defense, 2002 Delta Force 2 (modified) Army, Military Academy Macedonia, 2002a; Warlords, 2002; [1] Falcon 4.0 Air Force [1] Full Spectrum Command Army [1] Full Spectrum Warrior Army [1] Jane s Fleet Command Navy Kauchak, 2003 M1 Tank Platoon Army [1] Medal of Honor Marines [1] Microsoft Flight Simulator Navy, Naval Academy Macedonia, 2002b, Warlords, 2002 Operation Flashpoint Military Academy Warlords, To game the AI, means to take advantage of a synthetic entitiy (or other aspects of the AI), for example, by exploiting its unrealistic, predictable behavior. 2

7 COTS Based Game Military User Reference SOCOM: U.S. Navy Seals Navy [1] Soldier of Fortune Marines [1] Starcraft Air Force Warlords, 2002; [1] Steel Beasts Military Academy Macedonia, 2002a Sub Command Naval Academy Warlords, 2002 TACOPS Army National Defense, 2002 Because of these motivations, there is currently an extensive interest by the U.S. Military (henceforth the Military ) in computer games for education and training. We have found over fifteen COTS ( Commercial Off The Shelf ) games or modifications of COTS games in use by the Military, and again as many games created specifically for the Military. Table 1 lists COTS or COTS-modified games currently in use by the Military. is an active website that is devoted to the use of games in the U.S. Military and provides additional information about many of the games listed here. The military s interest in computer games is not new; but dates back more than twenty years to the late 1970 s with the use of the game Mech War by the Army War College [Macedonia, 2002b]. However, unlike the behavior-modeling community, commercial game developers are interested more in the illusion of intelligence by this we mean that behaviors in games are designed to make synthetic entities appear to be intelligent, rather than actually intelligent [Butcher & Griesemer, 2002; Laird & van Lent, 2000; Scott, 2002]. Paul Tozour, AI Programmer for Deus Ex 2 notes the following regarding games: The whole point is to entertain the audience, so no matter what you do, you need to make sure the AI makes the game more fun. If a game s AI doesn t make the game a better experience, any notions of intelligence are irrelevant. 2 A key similarity between the development of intelligent behavior in games and military training systems is that in both cases there is a need to facilitate cooperation between the content experts who describe and specify the desired behaviors and the programmers who implement them. In the game industry, programmers are responsible for the software and basic functionality (often called the game engine ), while designers are responsible for the content and game-play, the game s goals, and the style of play. In order to enable game designers typically nonprogrammers to control the events and behaviors in a game and work semi-independently of the programmers, tools such as scripting languages are often employed. These tools allow the behaviors of synthetic entities to be defined in a higher-level, English-like language. This shared responsibility is similar to how, in the Military humanbehavior modeling community, a subject-matter expert often works side-by-side with a behavior modeler to develop appropriate behaviors. However, the game community appears to have found methods for enabling game designers to work with less dependence on programmers than is typically seen in military behavior-model development. Another relevant trend in the commercial gaming industry is the increase in content authoring by users themselves. While players have hacked games for years (enabling them to, among other things, cheat in a game by increasing equipment, abilities, or scores), the gaming community is increasingly supporting game modification by providing software tools often the same ones that were used to construct portions of the game itself that facilitate game modification and new-content creation. Games can be modified in a number of ways. Media (e.g., graphic and sound) files can be changed to modify the presentation of objects or to create new objects in the game. Entire levels (scenarios or maps) can be constructed by changing the map files representing the spatial architecture of the game. The rules that govern the execution of the game can be manipulated, and in some instances the behaviors of game characters themselves can be changed. Players dedicated to modifying ( modding ) games often spend hundreds of hours making changes that can range from minor tweaks to virtually new games based on a preexisting game engine. Changes to virtually all aspects of the game are often known as total conversions. Counter-Strike is an example of a conversion; it adds teamoriented, tactical game-play, and is based on the first-person-shooter game Half-Life

8 Web sites such as GameSpy s Fileplanet ( Planet Unreal ( and Neverwinter Vault ( provide a means for millions of gamers to share modifications and news, and to interact in message forums. Game companies encourage these modifications because they increase gameplay and sales. Game companies are now even holding workshops on how to modify their games [Unreal University]. Much has been written about the Military s interest in the gaming world [e.g., Macedonia, 2002b]. This interest has largely stemmed from the academic community s pursuit of behavior-modeling technologies that can play roles in games. The academic community has also become interested in the use of games as testbeds for research into human-level AI [e.g., Laird, & van Lent, 2000; St. Amant & Young, 2001], robotic coordination [Adobbati, et al, 2001], and virtual collaborative interaction [Fröst, Johansson & Warrén, 2001]. It is clear that both the gaming and military-behavior modeling communities share an interest and desire in the development of better behavior-modeling capabilities, and on the surface share some of same needs and intentions. However, it is less clear to what degree a mutually beneficial relationship can arise, given the differences in their goals, current practices, and audience. It is our goal to explore any potential synergies and to elucidate them to the advantage of both communities. 3. Behavior Generation In order to familiarize behavior modelers with game vocabularies we examine the generation and authoring of game-character behaviors from several frames of reference. First, from a gamer s point-of-view, we examine character behaviors in various game genres. We follow with a psychological perspective, looking at how and whether game behaviors are based on the types of processes that underlie human behavior. Finally, we conclude with an evaluation based on the types of roles game characters play in games. 3.1 Behavior by Genre A common method of organizing computer games is through a taxonomy of game genres. Computer games are often classified into the following genres: adventure, strategy, role-playing, shooters (with either a or person player perspective), sports, simulation, and god games. We note that the boundaries between these genres are not hard and fast, and that many games have characteristics from several genres. Additionally, games can be categorized as single player, multi-player tens of simultaneous players or massively multi-player hundreds or thousands of simultaneous players. While massively multiplayer online games (MMOGs) can, in theory, occur as part of any of the aforementioned genres, they are mostly found as role-playing games. Due to a number of unique properties we will address MMOGs as a separate genre. In related work, [Laird & van Lent 2002, online] others have examined the game AI used across game genres Action and Shooter Games Action games are often the most publicized and popular of the game genres. In action games the primary control of a character is through actions taken by the player that translate directly to real-time actions of the character in the game. The majority of early games, such as PacMan and Castle Wolfenstein were of this type. Today, there are a number of sub-genres of action games, including first-person shooters (FPS), where the player s view of the world is through the first-person perspective of his character; fighting games, in which two simultaneous opponents are in hand-to-hand combat against one another and players learn special fighting moves represented by various combinations of keystrokes; arcade games, which are simple, two-dimensional games; and moving-puzzle games such as Tetris. Half-Life and Unreal are current well-known first-person shooter games, and the Mortal Kombat and Tekken series of games are examples of fighting games. Historically, the behaviors of non-player characters (NPCs, also called synthetic characters ) in action games have been based on finite state machines (See section for a brief description), or have been heavily scripted [Cass, 2002]. In this way, game designers created a challenge for the player by manipulating the numbers, strength, and placement of opponents, rather than by making the opponents behave more realistically. However, fewer games are being released with NPCs that make obvious faux pas such as running in place or bumping into things. The 4

9 behaviors of NPCs are becoming increasingly sophisticated often showing coordinated team behaviors, typically as opponents (e.g., Warcraft), but occasionally as player teammates (e.g., Tom Clancy s Ghost Recon). Techniques such as goal-directed reasoning and smart environments (described in section 4) are allowing games such as No One Lives Forever 2 (NOLF 2) to produce more realistic and emergent behaviors [PC Gamer, 2002]. The AI talking to each other about the intruder, flipping over the table for cover, and leaping over the railing are all behaviors that in the past would have been scripted events that were specifically set up and repeated the same way every time you played the game. But we ve designed our systems so that we can essentially wind it up and let it go. The emergent game-play is determined by the circumstances of the setting and the actions of the player. John Mulkey, Lead Level Designer, NOLF 2. The developers at Naughty Dog, creators of Crash Bandicoot, Jak and Daxter, and Jak II have also made extensive use of embedding real-time behaviors into game objects. Interestingly, these were developed using a computer language called GOAL (game oriented assembly lisp) an extension of the computer language LISP that they developed in-house which they suggest allows the behaviors to be developed faster and more compactly than programming in the computer language C. 3 Other AI techniques used in NPC behavior in action games include Fuzzy Finite State Machines (FuSMs), as employed in Unreal; and Fuzzy Logic, as used in S.W.A.T. 2, and Close Combat I & II. 4 Recently, a number of FPS games have begun providing mechanisms by which the behaviors of NPCs can be modified and extended by the player a capability often described as extensible AI. This has proven popular with gamers [Shelley, 2001]. In some cases the game developers have even released the game s source code 5 to the public. Behavior authoring in these types of games (Quake, Half-Life, Unreal) involves directly editing behavior scripts or source code. In some cases, third-party developers have created middleware layers 6 which aspire to be general behavior authoring toolkits (e.g., AI.Implant, SimBionic, Renderware AI). There are some GUI-based tools that allow the parameters of certain types of NPCs to be manipulated. 7 Regardless of the fact that these tools can be easier to use, adding new NPC behaviors with these tools typically requires significant programming experience Strategy Games A strategy game is one in which a player controls multiple characters or units at various locations on a map in order to carry out campaigns or strategies against other opponents. Strategy games exist in various contexts, such as historical reenactments, fantasy worlds, space frontiers, etc. In strategy games the computer often plays the role of an opponent against which the player competes. Additionally, characters or units controllable by the player often have some level of autonomy, which frees the player from mundane decisions. Players can provide varying degrees of control, and the unit executes the low-level details based on the player s directives. Because they are map-based games, strategy games often require computercontrolled entities with route-planning capabilities. To create engaging adversaries, game developers have begun using increasingly complex path-planning algorithms. In this regard, Star Trek: Armada use non-uniform notions of space [Davis, 2000], while Civilization: Call to Power incorporates a history of units destroyed at map locations in order to avoid moving computer-controlled forces into killing fields [Forbus, Mohoney & Dill, 2002]. There are two main sub-genres of strategy games: real-time strategy (RTS) and turn-based strategy. In real-time strategy games, the units typically have rudimentary autonomy they can attack and follow visible enemy units, patrol a series of waypoints, navigate to a far-away point and the player selects the types of actions for the unites to perform. The challenge for the player is that the unit AI is not very competent, and so for better performance the unit needs to be managed directly. For example, unmanaged units will simply attack the nearest enemy unit, whereas a much better strategy might be to have the several units coordinate their fire (possibly moving to get in range) on one target. Players must also manage resource collection, and use resources to build more units. Thus, the 3 LISP is the language in which most AI was programmed prior to about Source: 5 A game s source code is the game s computer program itself, in its original computer language. 6 A middleware layer of code would interact with the source code to affect the game without changing the source code. 7 For example, see the Quake Toolkit ( 5

10 player must divide his or her attention among multiple areas on the map while still reacting quickly to developments. Players formulate strategies before game-play, or while watching game replays. In single-player mode, the AI must also take the place of the opponent, and manage its own units which use weak AI. Current examples are Warcraft and Command and Conquer. In a turn-based strategy game, players take turns issuing orders to units. While issuing orders, the game is paused, so the player has an unlimited amount of time to strategize. After orders have been submitted, the players watch to see the effects, and the next turn commences. Some of these games are played via , where players a turn file back and forth, and the game can take many months to complete. One subgroup of turn-based strategy games are wargames, where historical accuracy is paramount. Examples are the modern-naval simulation Harpoon and the tank game Steel Panthers. Examples of other turn-based strategy games are Heroes of Might and Magic and Civilization. Generally, there are little to no behavior authoring capabilities in real-time strategy games. Unit behaviors are intentionally limited, since part of the challenge of the game is how the player divides his or her attention. Some games, such as Warcraft do allow for a more complex behavior authoring, yet few modders seem to be taking advantage of it. Turn-based strategy games lack much behavior authoring, due to the complexity of the algorithms used to do look-ahead and other deep analysis. Usually, players compete against the AI to learn the game, then compete against another human player for a greater challenge. Some projects have sprung up (FreeCiv and CivEvo) that are aimed at developing better strategy AIs. In addition, some newer turn-based games, like Galactic Civilizations are focusing more on intelligent strategies Adventure Games Adventure games are among the oldest computer games, dating back to 1972 and the game Adventure [Adams, 2002]. 8 Games are often set up as if the player were a character in a story. The emphasis is on character interactions (often scripted as part of the storyline or plot), item accumulation, and puzzle solving. Examples include the Monkey Island series, the Myst family, the King s Quest series, and the Gabriel Knight series. Like in action games, NPCs in adventure games are designed with behaviors to make them more realistic and engaging. However, in adventure games the focus is more on the storyline rather than on active behaviors such as fighting tactics. Typically, players engage and communicate with NPCs either through text based, menu driven, or point-and-click interfaces. In text-based communication, the player types free-form text (e.g., open door ). Early adventure games, before advanced graphics capabilities, often used this format. In menu-driven conversations, the statement to be made by a player is chosen from a list of options. Point-and-click interfaces use the mouse to direct the character to move or to engage objects. Often, games use a combination of these techniques. Most NPC behaviors are highly scripted, and have a linear or limited-branch storyline. There is little to no behavior authoring in adventure games, and the popularity of the genre appears to be waning in favor of the similar genre of roleplaying games. Another cousin to adventure games is interactive fiction, in which the player takes an active role in the computer-generated story. In interactive fiction players interact with NPCs, dynamically changing the storyline Role-Playing Games Role-playing games (RPGs) are often seen as extensions of adventure games, with an additional emphasis on character personality and abilities. Typically, there is a variety of character types to choose from, each with differing skills and attributes. Through actions in the game new skills can be learned and attributes increased. Like adventure games, there is typically a world to explore, quests to fulfill, and enemies to conquer. Examples include Baldur s Gate, Neverwinter Nights, the Ultima series, and Diablo I & II. Like adventure games, the game world in role-playing games often contains a variety of other characters, such as shopkeepers, villagers, and other adventurers with various behaviors and dispositions. Behavior representation techniques are similar to those found in adventure games, and the choices available to the player are often 8 Will Crowther developed Adventure while at BBN when not working on the Internet-precusor ARPAnet. 6

11 influenced by prior actions such as whom you have talked with, what you have said to them, your attributes and statistics, your reputation, who is with you, and where you ve been (e.g., Planscape: Torment 9 ). Until recently, there was little to no behavior or scenario authoring in RPGs. Baldur s Gate was one of the first games released to include the tools used by the developers to create the game s content. Neverwinter Nights (NWN) took this one step farther, and was designed explicitly for players to create their own scenarios. NWN provides a C- like scripting language, powerful scenario authoring and map-drawing tools, along with full access to the scripts used to control AI opponents. Another important feature present in NWN is the ability for a Game Master to dynamically modify the scenario while it is running. This Game Master can take over control of any NPC opponent, add new creatures, and move things around in the game world. The NWN community has grown rapidly, and a large number of user-created scenarios exist, from complex stories providing over a hundred hours of game-play to strategy games where players control AI-unit armies. In addition, there are persistent worlds providing networked play at any time Simulation Games Simulation games provide the player with a simulated environment with which to engage. Among the most popular are flight simulation games and driving games, but they have been associated also with submarines, helicopters, trains, motorcycles, and X-wing star fighters, to name a few. Examples include Microsoft Flight Simulator, Indy Car Racing, and Falcon 4. Sub-genres of simulations include sports games and god games, described below. We are unaware of any simulation games that provide any behavior authoring capabilities accessible to the player Sports Games Most sports games are simulations of actual sports and they exist for virtually every sport imaginable combining aspects of action and simulation game genres. Sports games cover both individual and team play, with many games allowing the player to build their own teams, trade players, etc. In many cases, developers spend significant effort to make synthetic versions of real players as accurate as possible, even going to the effort of viewing hundreds of hours of game tapes in order to accurately represent their signature moves and play characteristics [Ratliff, 2002]. Additionally, many sports games make extensive use of statistics gathered on a variety of the aspects of game-play. Play format is similar to how sports are shown on television complete with replays, a variety of camera angles, half-time reports, and game color commentary. Examples include Madden Football 2003, FIFA Soccer 2004, and PGA Tour Golf. Like strategy games, synthetic entities can include the opposing strategist often thought of as the opposing coach as well as the individual NPC players. Often a player can take the role of an individual on the team, with the remaining teammates controlled by the computer. Sports games are often well suited for behavior-based approaches [Arkin, 1998]. To our knowledge, there are no sports games which provide behavior authoring tools God Games God games have similarities to both real-time strategy games and simulation games. AI units are more autonomous than in real-time strategy games, and the player cannot directly order the AI units about. 10 Instead, the player can make changes to the game world to affect the behaviors of the units. For example, in SimCity, the player plays the role of an urban planner, and must build roads for AI units to travel on, houses for them to live in, and office buildings for them to work in. Examples include Simcity, The Sims, Railroad Tycoon, and Black & White. Black & White was a ground-breaking game for its incorporation of empathic learning where a creature watches the player s actions and attempts to determine their intent. Also, the player can instruct the creature using a punishment and reward system But see The Sims, which allows for more direct control of characters. 7

12 The Sims is the only god game of which we are aware that includes tools for editing game content and behaviors [Forbus & Wright, 2001] Massively Multiplayer On-line Games Emerging from the on-line Multi User Dungeon (MUD) community of the 1980s, massively multiplayer on-line games (MMOGs) burst into the mainstream with the release of Ultima On-line in Most MMOGs are of the role-playing genre., although see There and The Sims Online for alternatives. MMOGs are different from other games due to their social structures and communities. MMOGs and their communities of users can have guilds, entrepreneurs, warring factions, economic classes, crime rings, political groups, even entire cities and nations. 12 While the history of MMOGs includes user-content creation the developers of MUD games were generally glad to accept contributions from well-known players, and many MUDs were the collective work of a large number of people today s MMOGs do not allow player content creation or behavior authoring. Game companies point to technical difficulties and legal issues as roadblocks. However, user-generated content is generally seen as the next big thing in MMOGs. Brad McQuaid, the producer of the MMOG EverQuest and President and CEO of Sigil Games On-line, Inc. predicts the following: User-generated content is going to be huge. It s one of those questions that s more a matter of when than if, because it s coming, and it s coming big time. There are a lot of hurdles and difficulties, but if the players talents and creativity can be harnessed like it has been with FPS mods and the like it will be a huge step forward for MMOGs. Several games such as Second Life and There are now allowing players to define the content in MMOG worlds. While these games are MMOGs, they are not RPGs or FPS, instead they are more like graphically persistent virtual worlds with a large emphasis on social interaction rather than role-playing or adventuring. NPCs are extremely scarce, with no behavior authoring capabilities. 3.2 Components of Behavior The domain in which computer game NPCs must operate although simpler than the real-world often require NPCs to perform a wide range of functions and to integrate capabilities that include sensing the environment, reasoning about its spatial layout, planning and executing actions, as well as communicating and coordinating with other NPCs or players. This has been recognized by John Laird and colleagues [e.g., Laird & van Lent, 2000], who suggests that computer games make an ideal testbed in which to perform research on human-level intelligence. Given the niche oriented and specialized nature of much of the research in cognitive psychology and artificial intelligence, it is unsurprising that the vast majority of human behavior models are specialized for specific behavioral phenomena, and are not integrative across a range of phenomena or behavioral disciplines. Integrative models are rare, with most in existence based on cognitive architectures with long running research programs such as Soar [Rosenbloom, Laird, & Newell, 1993; Newell, 1990] or ACT-R [Anderson, 1993; Anderson & Lebiere, 1998]. In this section, we examine whether, how, and to what extent behavior representations in commercial games include the same types of processes underlying human behavior. We explore game NPCs and their representations of memory, sensation and perception, motor control, decision making, learning, and language processing the basic building blocks of human behavior. 11 The Edith visual editor for The Sims is currently not in public distribution. Other tools can be found at and

13 3.2.1 Sensation and Perception The sensory mechanisms by which game characters see and in some cases hear the world range in complexity from the extremely simple to the surprisingly sophisticated and complex. Compared to the rich and compelling virtual worlds provided to human players, the world representations designed for synthetic characters are quite impoverished. Environments are typically reduced to simplified representations used only for navigation. Object detection occurs by performing collision tests radiating out from the NPCs location. Sensory routines often only perceive entities and events that can influence the NPC s reactions (e.g., Halo) typically these entities and events are human players and the actions they produce. Routines are run periodically in order to gather relevant events. Vision is often represented by a two-dimensional view cone, and a line-of-sight operator is used to detect whether an object is visible to the NPC. Unfortunately, line-of-site detection does not take into account real-world cues such as motion detection, depth perception, and coloration or texture differences. Hearing is often implemented using even simpler models. For example, Halo models the distance that an NPC can hear a sound as a function of the volume of the sound scaled by a hearing sensitivity parameter. Smell is sometimes implemented using the same mechanisms as sound. In contrast to these simple cases, the focus on stealth, evasion, and surprise found in Thief and Thief 2, required a more complex perceptual system. Multiple three-dimensional view cones are used to represent focal and peripheral vision differences each with differing probabilities for the presence, location, and identity of an object. Furthermore, view cones are sensitive to additional factors such as motion or darkness levels. Objects relevant to NPCs have intrinsic visibility values, scaled by lighting levels, degree of motion, and exposure levels. Sound processing in Thief is also fairly sophisticated, complete with directional and attenuation information. In order to develop NPCs that do not rely on preexisting navigational waypoint maps, researchers are creating methods based on techniques developed for robotic exploration (by which an NPC explores a region collecting sensor data), which are used to construct a high-level spatial map of the region [e.g., Best & Lebiere, 2003; Hill, Han, & van Lent, 2002]. Like all other mechanisms, the realism of sensory processes is subservient to game-play. In Halo, an NPCs ability to see a player is constrained so that the NPC can only see the player if the player can see the NPC. This is because players often felt cheated when a player is seen by an opponent they do not see. Additionally, the time required to sense and recognize an object was removed after testing showed people interpreted it as the NPC being broken [Butcher & Griesmer, 2002] Motor Control The kinds of actions that game characters are allowed to perform are often typical of human characters, although in science fiction or fantasy games the characters may have numerous special abilities. While the allowable actions may be human-like, often the constraints on these actions are not. In many instances characters are afforded superhuman capabilities in order to make game-play more interesting. For example, the female assassin character in Halo is able to jump higher than player characters, and she is thereby able to access areas specifically built for her use and not available to the player. Also, game character reaction times have often been manipulated in order to provide challenging opponents Memory Cognitive memory mechanisms in computer game characters are perhaps the least like their human counterparts. Typically, memory mechanisms are simply data structures representing the current state of the game, and data caches designed to reduce processing requirements for activities such as navigation and path finding. Memory limitations are not based on any psychological theory of human memory, but instead on the memory and processing constraints of the computing platform. Differences between short- and long-term memory, memory limitations due to decay and forgetting, and the processes by which memories are rehearsed, encoded, and retrieved, are not found in game characters. Most game characters do not retain memories of encounters with players in the game. Entities outside of sensor range are immediately forgotten. There are, however, some exceptions. For example, characters in Titanic, Adventure out of Time will react differently depending on whether they have previously met the player. Some strategy games such as Civilization: Call to Power, incorporate a memory of the number of units destroyed by a player at any given location and take this information into account when performing path planning [Forbus, 9

14 Mohoney & Dill, 2002]. Note, however, that these capabilities are more along of the lines of clever techniques to enhance game realism, rather than human-like memory mechanisms Decision-Making The most common representations for decision-making processes in game characters are finite-state machines (FSMs). Character behaviors are modeled as a finite set of states with transitions between them in the form of a directed graph. The character resides in only one state at a time, with transitions between states driven by the conditions and actions occurring in the game. FSMs are simple, easily understood, can be graphically represented, are computationally inexpensive, and have compact representations. There are a number of extensions to FSMs which make it possible to develop more complex behaviors, including Stack-based FSMs, Fuzzy State Machines, Hierarchical FSMs, and Probabilistic FSMs. However, current FSM techniques do not scale well 13 and their propositional representation can make behaviors difficult to represent. Rule-based approaches to behavior have also been implemented in games (e.g., Age of Kings). Typically, in these systems a set of if-then expressions is used to encode simple condition-action pairs. In most cases FSMs and rulebased approaches are represented in game-specific script or code, rather than within a general purpose implementation. Another behavior representation technique seeing some use in the game community is that of smart environments (also called smart terrain). In smart environments, the objects in the world contain information on how game characters can interact with them. Section 4.4 describes smart environments in more detail. A few games have developed character behaviors using goal-directed-reasoning techniques. In this technique game characters have a set of goals and actions are chosen in order to best satisfy the most relevant goal or goals. No One Lives Forever 2 implements a simple goal-directed system in which characters have multiple goals and choose which goal to pursue given the current game conditions. However, the type of behavior executed in response to the goal is hard-wired no plan of action is developed to satisfy the goal. Finally, there is a small number of games that have used neural networks or genetic algorithms to specify behaviors. We describe several of these in the following section Learning A number of AI learning techniques have been employed in computer games, including decision trees, neural networks, and genetic algorithms. While many of these techniques are used during the game-development process, few games actually ship with the learning mechanisms turned on for fear that the game will learn poorly and provide a less than desirable game-play experience. There are notable exceptions, however. Black & White has learning as a central component in the game. In Black & White a player controls a Creature that learns from the player s actions and any positive or negative feedback provided by the player. The Creature, based on the Belief- Desire-Intention agent architecture, learns using a variety of learning algorithms, including decision trees based on Quinlan s ID3 [Quinlan, 1986] and neural networks. Relatively few games have used genetic algorithms. A notable exception is Cloak Dagger, and DNA, which used genetic algorithms to evolve opponent strategy. Between battles with the player, the player can allow the opponent s DNA strands to compete and evolve through a series of tournaments Communication In most games, communication between players and NPCs is quite limited, with the dialog consisting of a few scripted phrases. A slightly more complex technique used to enrich game interaction is the use of dialog trees, in which there is a tree of game character statements and possible player responses. The player typically picks a response from a menu interface, which in turn dictates the game character s response, thereby walking through the conversation tree. Some games (e.g., Baldur s Gate, The Elder Scrolls III: Morrowind), make the dialog sensitive to 13 Current FSM techniques do not scale well : they have size limitations; it is difficult to build large systems with them. 10

15 situational context what has been done in the game, with whom the player has talked, current teammates, etc. Games with speech recognition capabilities for giving simple orders to NPCs are just beginning to emerge 14 (e.g., SOCOM: U.S. Navy SEALS, Tom Clancy s Rainbox Six 3) with more rumored to be released shortly. In addition, there are now companies producing speech recognition Software Development Kits (SDKs) for game platforms. 3.3 Behavior by Functional Role As we have described in the previous two sections, the gaming industry often organizes games and associated behaviors by game genre, while cognitive behavior modelers often structure behavior around the underlying psychological principles on which behaviors are based. While each of these categorizations serve their audiences well, they are not natural when applied across the two domains. In order to bridge this gap we suggest a third categorization which can bridge the two domains. We observe that the class of behavior that is interesting to both cognitive behavior modelers and game designers is the behavior that supports interaction between characters, specifically between computer modeled entities and human beings. AI techniques in games which serve purposes besides behavior construction are of less interest to cognitive modelers (e.g., behind the scenes path finding, resource allocation algorithms, algorithms that support active parts of the world which are not cognitive, such as weather and natural processes, or luck and play balance models). Likewise, cognitively motivated models built by behavior modelers are of little direct interest to game world behavior modelers if they do not drive interactions with the human players and otherwise improve game-play. Given this overlap between the two communities, it may be fruitful to characterize behavior models according to the kinds of roles played and the interactions required by them. Cognitive behavior modelers should be interested in the new interaction classes from the gaming world (e.g., dolls, plot elements) as long as they are interested in the behavior itself. Laird and van Lent [Laird, & van Lent, 2000, online] have described the basic roles which game characters can play. We highlight and expand on their classification Teammates It has become common in FPS and RPG genre games to include the ability to play with NPC teammates. These NPCs require some of the most complex behaviors found in computer games today. The NPCs must be capable of autonomous activity, but still responsive to team goals, and they must be able to communicate and coordinate with other teammates (both human and NPC). In many games communication between the player and the NPC can flow in both directions: an NPC can, for example, alert the player to a relevant activity often through audio, and a player can issue orders to the team and individual teammates. These orders are often sensitive to context. For example, in SWAT 3: CQB, you can point your crosshair at objects in the environment, such as a door, and give context-sensitive commands to teammates, such as breach and clear. The NPC must be smart enough to respond with accurate tactics moving and positioning themselves correctly, coordinating target engagement, and covering teammates. To date, there are no games which expose the behavioral mechanisms developed for complex NPC teammates, either by providing behavior authoring tools, or exposing the code which controls these NPCs. Some games do provide in-game mechanisms for instructing teammates about how to behave by applying a small number of policies. For example in Neverwinter Nights, teammates can be instructed to stay close to me which affects following behavior or attack with ranged weapons as opposed to close weapons, like swords. This ability constitutes a step towards a parameterized authoring tool for teammates. It is precisely the complex behaviors required by teammates which are of interest to cognitive behavior modelers. Conversely, it is in the development of these characters that cognitive behavior modelers have the most to offer game developers Adversaries NPCs as adversaries is perhaps the earliest and most common role for NPCs. As adversaries, NPCs compete against, or thwart the player. In early games, opponent NPCs resorted to cheating in order to challenge the player. As such, NPCs were made bigger, faster, or stronger than player characters, or were given additional information, extra units, or played by a different set of rules. Laird and van Lent [Laird, & van Lent, 2000, online] differentiate 14 Seaman by Sega was an early game employing speech recognition technology 11