Basic AI Techniques for o N P N C P C Be B h e a h v a i v ou o r u s: s FS F T S N

Transcription

1 Basic AI Techniques for NPC Behaviours: FSTN

2 Finite-State Transition Networks A 1 a 3 2 B d 3 b D Action State 1 C Percept Transition Team Buddies (SCEE)

3 Introduction Behaviours characterise the possible actions that NPCs or automatic players will perform in reaction to, e.g.: their physical environment (e.g. Lemmings) human player s location or real-time actions (e.g. Doom, Tekken, Air Combat, etc.) current phase of the game (e.g. RISK) Etc.

4 Intelligent Behaviours We here consider behaviours to be restricted to intelligent behaviours (i.e. AI-based) Computation of a trajectory in a racing game would not fall under that concept --- only if based on kinematics. but decisions to undertake/attack other cars/vessels do!

5 Behaviours and Gameplay Complex behaviours implement more intelligent opponents. They can also define a kind of personality for NPCs or opponents. The more you formalise behaviours, the more you make some gameplay concepts explicit.

6 Finite-State Systems They describe devices that can take discrete states. They can represent compiled plans. Two main concepts: States Transitions

7 FSTN 1 B 3 A 3 2 D State 1 C Percept Transition (adapted from A. Whittaker)

8 FSTN (2) A 1 3 a 2 B d 3 b D Action State 1 C Percept Transition (adapted from A. Whittaker)

9 FSTN (3) i ii iii 1 a B 3 b Register Action A 3 2 d D State Percept 1 C Transition (adapted from A. Whittaker)

10 Example: Quake The artificial opponents go through a cycle of: being idle, attacking, reacting to attack, etc. Their behaviour is based on the definition of: the cycle of states and, the transitions between such states.

11 The Original Quake FSA

12 Quake FSA (comments) Compute LOS Compute damage idle sees enemy attack gets killed dead loses interest kills enemy search loses sight of enemy gets hurt hurt Change appearance Compute, e.g. time limit Compute LOS Search procedure retaliate

13 Quake FSA ( (comments) For a given FSA, you can fine-tune behaviour: You can vary the time limit for losing interest to make characters more persistent. You can alter line of sight. You can have them escape if hurt, rather than retaliate. A good formalisation gives you the basis to make consistent changes and/or improvements.

14 Prefer FSA if: Small set of clearly identified states. Meaningful transitions. Clear pre-conditions (action representation). A priori definition, with little need for change or experimentation. Variant: cascaded FSA.

15 FSA: Representing Transitions

16 FSA: from description to interpretation FSA are generally described for parsing sequences of tokens. When they describe (compiled) plans or behaviours, you have to compute the next transition, rather than accepting a token and checking compatible transitions.

17 Developing FSTN Systems The development of FSTN systems comprises several steps: Collecting the data. Formalising FSTN. Choosing an implementation tool/method. Testing. Maintenance.

18 Maintenance?? It is often difficult to manage a large number of FSTN. This is a problem when adding new FSTN: integrating testing FSTN are a procedural formalism (and not declarative) and, as such, rather difficult to maintain

19 FSTN Example

20 Queuing Agents (1) How an agent jumps the queue An available position The violated agent The cheeky agent will need to move to this position before they can advance in the queue The cheeky agent Figure 2.2

21 Queuing Agents (2) Introduce conflict

22

23 Baseline Behaviour

24 Double Skip

25 Successful Skip

26 Cheating Fails

27 Finite-State Transition Networks

28 Finite State Machines (FSMs) Finite State Machines (FSMs) - Implementation Issues -

29 FSMs are a simple and efficient method to implement many game features. may be diagrammed using a standard diagram may be diagrammed using a standard diagram format called a directed graph, which is easy to read and understand, even for non-programmers (this facilitates discussions with the Design team).

30 FSMs (2) describe under which events/conditions a current state is to be replaced by another for example, switching from an attack mode to an escape mode if the NPC is hit. is mostly a design concept i.e. the game has no general FSM interpreter, but the FSMs are usually created using individual entities (scripts, or else).

31 Ghosts in PacMan

32 Ghosts behaviour in PacMan All ghosts have the same Evade state, Each ghost has its own Chase state, the actions of which are implemented differently for each ghost. The input of the player eating one of the power pills is the condition for the transition from Chase to Evade. The input of a timer running down is the condition for the transition from Evade to Chase.

33 Quake-like Bots

34 Bots behaviour Bots include states such as FindArmor, FindHealth, SeekCover, and RunAway. Even the weapons in Quake implement their own mini finite state machines. E.g. a rocket may implement states such as Move, TouchObject, and Die.

35 FIFA 2002

36 Football players in FIFA 2002 Players behaviour can be defined via states such as: Strike, Dribble, ChaseBall, and MarkPlayer. In addition, the teams themselves are often implemented as FSMs and can have states such as: KickOff, Defend, or WalkOutOnField.

37 World of Warcraft

38 RTS NPCs in RTSs (real-time strategy games) such as World of Warcraft also make use of finite state machines. They have states such as: MoveToPosition, Patrol, and FollowPath.

39 Example for RTS genre

40 Example for FPS genre

41 State Transition Tables A better mechanism for organizing states and affecting state transitions. Definition: a table of conditions and the states those conditions lead to.

42 State Transition Tables (2) table can be queried by an agent at regular intervals, enabling it to make any necessary state transitions based on the stimulus it receives from the game environment. Each state can be modeled as a separate object or function existing external to the agent, providing a clean and flexible architecture. One that is much less prone to confusion than embedding all transitions in an if-then/switch

43 Embedded Rules An alternative approach is to embed the rules for the state transitions within the states themselves. States are encapsulated as objects and contain the logic required to facilitate state transitions. Each state is a self-contained unit and not reliant on any external logic to decide whether or not it should allow itself to be swapped for an alternative.

44 Bot class Imagine a Bot class that has member variables for attributes such as health, anger, stamina, etc., and an interface allowing a client to query and adjust those values. A Bot can be given the functionality of a finite state machine by adding a pointer to an instance of a derived object of the State class, and a method permitting a client to change the instance the pointer is pointing to.

45 Process When the Update method of an Agent is called, it in turn calls the Execute method of the current state. The current state may then use the Bot interface to query its owner, to adjust its owner s attributes, or to effect a state transition. In other words, how a Bot behaves when updated can be made completely dependent on the logic in its current state.

46 References Programming Game AI by Example by Mat Buckland ISBN Finite State Machine Tutorial Finite State Machine Tutorial By Nathaniel Meyer

47 Even Better Develop a visual programming interface when FSTN are drawn Generates code corresponding to this FSTN Communication between NPC/Bot s states and those produced by the FSTN code

48 Quake : from FSTN to Rule-based Systems (introducing SOAR-Quakebot by John Laird)

49 Quake III NPC AI Most material from [van Waveren and Rothkrantz, 2001]

50 Quake s FSTN

51 Layered Architecture Team leader AI Misc. AI AI Network Commands Fuzzy Character Goals Navigation Chats Area Awareness System Basic Actions

52 Layered Architecture 1 st layer: I/O layer for the bot (basic actions are the output) 2 nd layer: subconscious intelligence, fuzzy logic for goal selection 3 rd layer: mixture of production rules and AI network (FSTN). High-level reasoning: navigation and fighting 4 th layer: for team leaders (co-ordination knowledge)

53 Game AI Network Seek Long-Term Goal Seek Short-Term Goal Seek Activate Enemy Stand Respawn Battle Fight Battle Chase Battle Retreat Battle Short-Term Goal

54 Integration in the Game Engine Game Bot AI (layers 3-4) Server networking Client Client Game Renderer Bot AI (layers 1-2) Client code provided the IO functionality for Human players 3D image

55 Area Awareness System Provides Information about the current state of the world Includes: navigation, routing, other entities Based on a 3D representation of the world 3D bounded hulls (= areas) Everything the bot senses goes through the AAS

56 Area Awareness System The map is subdivided into convex hulls using Binary Space Partitioning (BSP) Areas specific property: the navigation complexity for travelling between any two reachable points in the area is minimal It is further required to compute reachability

57 Terrain Reasoning Predicting out-of-sight threats Dynamic situations (seeking cover, concealed paths) Waypoint resolution Terrain pre-processing

58 Rule-based decisions IF the bot is fighting AND the bot is low on health AND the bot does not have a powerful weapon THEN retreat from the fight

59 Rule-based decisions IF the bot is fighting AND the bot is not fit enough to fight THEN retreat from the fight (but in Quake III, these are essentially implemented in several procedures, not taking advantage of rule modularity)

60 Fuzzy Coefficients Inspired from Fuzzy Logic Fuzzy relations are used to express the relation between the bot s current state and possible desires/goals Example: based on which weapon the bot is holding, and how much ammo the bot has for the weapon, the bot can attach a fuzzy value to its desire for more ammo Also used in conjunction with rules

61 Bot Characters 25 characteristics: name, gender, aggression, alertness, jumper, walker, attack skills, aim accuracy, weapon weights, item weights, chats, etc.

62 Results Quake III Arena bot Soar-Quakebots Performance vs. human-like ability

63 Soar/Games Project Build an AI Engine around the Soar AI architecture Soar/Quake II Soar/Descent 3 Soar/Half-life Quake II Interface DLL Sensor Data Actions AI Engine (Soar) Knowledge Files Build autonomous, unscripted AIs J. Laird

64 Soar AI engine to support multi-method problem solving Applied to wide variety of tasks and methods Proposed unified theory for modeling human cognition Models a wide variety of human behavior: language, HCI,... Combines reactive and goal-directed symbolic processing Supports very large bodies of knowledge (>100,000 rules) Optimized implementation in ANSI C In the public domain J. Laird

65 Soar Quakebot Goal: Human-like behavior Fun and challenging to play against Not super-human Currently plays as well as good player Completely implemented in Soar > 750 rules Spread across operators for exploring, wandering, chasing, Sensing similar to human Runs on separate PC from game Uses ~5-10% of 400 MHz Win-95/98/NT machine J. Laird

66 Execution Flow of an AI Engine G A M E Sense Think Simulated Perception Self: orientation, position, health Senses: vision, hearing, smell, taste, touch Augmented Senses: radar, night-vision, > 80 sensors in Quakebot Field of Vision Obstacle Act Simulated Actions Movement, weapons use, communication ~20 actions in Quakebot J. Laird

67 Execution Flow of an AI Engine Sense G A M E Think Finite-state machines Subsumption Neural nets? Rule-based systems C code Planning systems Act J. Laird

68 Rule-based Systems Program = set of if-then rules, not sequential code. Use whatever knowledge that is relevant to current situation Rule Memory Tactics, Movement Knowledge,... [ ] [ ][[ ] [ ] [ ][ ] [ ] [ ][[ ] Match [ ] [ ] [ ] Changes Working Memory sensing, elaborations, persistent memories, goals, actions J. Laird

69 Rules Conditions contain variables and other first-order tests If sense a weapon and do not have that weapon, pickup the weapon. (p rule1 (sense weapon?weapon1) (self weapon <>?weapon1) (action pickup?weapon1)) A single rule can have multiple instantiations A single condition can match multiple working memory elements Compilers convert rules into discrimination networks J. Laird

70 Which Rule Should Fire? Matching finds which rules can fire in current situation. Rule Rule Instantiations Memory Match [ ] [ ] [ ] [ ] Conflict resolution? Working Memory [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] Conflict resolution picks which rules actually fire. Based only on syntactic features of rules and data. Must engineer rules so correct rule is selected each time. J. Laird

71 Weaknesses in Rules Conflict resolution based on syntactic features Not knowledge of the task Rules combine three types of knowledge about actions What is possible What should be done (based on conflict resolution) How to do it Leads to duplication if have multiplicity in one of the types No hierarchical organization of knowledge and goals J. Laird

72 Soar s Approach Organize knowledge as operators: Primitive & abstract actions Turn, move, climb, goto-door, get-item, wander, attack, chase Provides trace of behavior comparable to human decisions Use rules to propose, select, and apply operators Rules fire in parallel - act as an associative memory Provides fine-grain, situation-dependent behavior Conditional selection and execution determined dynamically J. Laird

73 Execution Flow of Soar Engine Sense Soar Elaboration G A M E Think Propose Operators Evaluate Proposed Operators Select One Operator Perform Operator Actions Act J. Laird

74 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

75 Soar Quakebot Top Operators Attack Retreat Wander Explore Record Enemy Remove Enemy Chase J. Laird

76 Soar Quakebot Top Operators Attack Retreat Wander Explore Record Enemy Remove Enemy Chase If enemy visible and my health is > very-low-health-value (20%) and his weapon is not much better than mine THEN propose attack If enemy visible and my health is < very-low-health-value (20%) or his weapon is much better than mine THEN propose retreat If no enemy visible or recorded and explored level THEN propose wander If no enemy visible or recorded and not explored level THEN propose explore J. Laird

77 Soar Quakebot Top Operators Attack Retreat Wander Explore Record Enemy Remove Enemy Chase If enemy just became not visible and no enemy visible or recorded THEN propose record-enemy If hear enemy and no enemy visible or recorded THEN propose record-enemy If record-enemy selected THEN record enemy s last position and time + 20 seconds. If time > recorded enemy s saved time THEN propose remove-enemy If remove-enemy is selected THEN remove recorded enemy structure If there is a recorded enemy structure THEN propose chase J. Laird

78 Example Rules Attack Get-item Face-enemy Side-step Approach Shoot IF I see a weapon that is much better than any I have THEN I need that weapon IF my health is less than low-health-value (40%) THEN I need any health object IF attacking an enemy and there is an object I need and that object is closer than attack-get-item-range THEN propose get-item for that object IF two get-item operators are proposed THEN prefer selecting the one for the closer object J. Laird

79 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

80 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

81 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

82 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

83 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

84 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

85 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

86 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

87 Get item Go through door Get item in room Exit room Go to door Face item Stop moving to item Move to item Detect item missing Face door Align with door Record at door Move to door Shoot Slide to door Stop move to door Stop slide to door J. Laird

88 Example Tactics Collect-Powerups Pick up best weapons from spawn locations Remember when missing items will respawn Use shortest paths to get objects Get health/armor if low on health/armor Pickup up other good weapons/ammo if close by to deny enemy Attack Use circle-strafe Move to best distance for current weapon Chase enemy based on sound of running Ambush in corner that can t be seen by enemy Hunt at nearest spawn room after killing enemy

89 Mapping Need information on location of walls, doors, etc. Many tactics require this information Built up through exploration of a level Similar to a robot exploring with range sensors Saved and reused when return to level Represented internally as a graph structure Currently restricted to 2D rectangles J. Laird

90 J. Laird

91