CS 680: GAME AI WEEK 4: DECISION MAKING IN RTS GAMES

Transcription

1 CS 680: GAME AI WEEK 4: DECISION MAKING IN RTS GAMES 2/6/2012 Santiago Ontañón

2 Reminders Projects: Project 1 is simpler than it seems: 1) Implement a basic AI (doesn t have to play well) 2) Pick a path-finding/decision making algorithm and experiment with it Progress self-check indicator: Your progress is good is you already have a basic AI that can play a complete game of S3/Starcraft (whether it wins or not).

3 Outline Student Presentation: Near Optimal Hierarchical Pathfinding Student Presentation: Intelligent Moving of Groups in Real-Time Strategy Games Decision Making Basics: Hardcoded Methods Decision Theory Adversarial Search Project Discussion

4 Outline Student Presentation: Near Optimal Hierarchical Pathfinding Student Presentation: Intelligent Moving of Groups in Real-Time Strategy Games Decision Making Basics: Hardcoded Methods Decision Theory Adversarial Search Project Discussion

5 Decision Making A situation is characterized by: Known information about the state of the world Unknown information about the state of the world Set of possible actions to execute Problem: Given a situation, which of the possible actions is the best?

6 Example: RTS Games Known information: Player data, explored terrain Unknown: Unexplored terrain Enemy strategy Actions: Build barracks Build refinery Build supply depot Wait Explore

7 Example: Final Fantasy VI Known information: Party information Two enemies Unknown: Resistances Attack power Remaining health Actions:

8 Basic RTS AI Diagram Perception Unit Analysis Map Analysis Strategy Strategy Give Orders Economy Logistics Attack Arbiter Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

9 Basic RTS AI Diagram Perception Unit Analysis Decision making is key at these Map Analysis levels Strategy Strategy Give Orders Economy Logistics Attack Arbiter Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

10 Basic RTS AI Diagram And Perception is most important at the Unit Analysis high-level strategy level Map Analysis Strategy Strategy Give Orders Economy Logistics Attack Arbiter Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

11 Example Basic RTS AI: Strategy Finite-State Machine Resource spending: 80% Economy, 20% Military 2 workers wood 1 worker metal Army Composition: 100% footmen After training 4 footmen Resource spending: 20% Economy, 80% Military 2 workers wood 4 workers metal Army Composition: 100% knights If enemy has Flying units Army Composition: 50% knights 50% archers If enemy has no more flying units

12 Outline Student Presentation: Near Optimal Hierarchical Pathfinding Student Presentation: Intelligent Moving of Groups in Real-Time Strategy Games Decision Making Basics: Hardcoded Methods Decision Theory Adversarial Search Project Discussion

13 Finite State Machines Harvest Minerals 4 SCVs harvesting Build Barracks barracks Train marines Less than 4 SCVs 4 SCVs 4 marines & Enemy seen No marines 4 marines & Enemy unseen Train SCVs Attack Enemy Enemy seen Explore

14 Finite State Machines Easy to implement: switch(state) { case START: if (numscvs<4) state = TRAIN_SCVs; if (numharvestingscvs>=4) state = BUILD_BARRACKS; Unit *SCV = findidlescv(); Unit *mineral = findclosestmineral(scv); SCV->harvest(mineral); break; case TRAIN_SCVs: if (numscvs>=4) state = START; Unit *base = findidlebase(); base->train(unittype::scv); break; case BUILD_BARRACKS: }

15 Basic RTS AI Diagram Perception Unit Analysis Map Analysis Harvest Minerals 4 SCVs harvesting Build Barracks barracks Train marines Strategy/ Give Orders Less than 4 SCVs Train SCVs 4 SCVs 4 marines & Enemy seen Attack Enemy No marines Enemy seen 4 marines & Enemy unseen Explore Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

16 Basic RTS AI Diagram For Simple games or simple AIs, Perception we could substitute both Strategy and Give Orders layers by a FSM Unit Analysis Harvest Minerals 4 SCVs harvesting Build Barracks Map Analysis barracks Train marines Strategy/ Give Orders Less than 4 SCVs Train SCVs 4 SCVs 4 marines & Enemy seen Attack Enemy No marines Enemy seen 4 marines & Enemy unseen Explore Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

17 Basic RTS AI Diagram For mode complex AIs, FMSs are Perception too restrictive, and it s Unit Analysis better to use the architecture as we explained it in Week 2 of class Harvest Minerals 4 SCVs harvesting Build Barracks Map Analysis barracks Train marines Strategy/ Give Orders Less than 4 SCVs Train SCVs 4 SCVs 4 marines & Enemy seen Attack Enemy No marines Enemy seen 4 marines & Enemy unseen Explore Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

18 Finite State Machines Good for simple AIs Become unmanageable for complex tasks Hard to maintain, Example: Imagine we want to add the behavior if enemy inside base then attack him with everything we have We will have to add a new state and transitions from every other state!

19 Finite State Machines (Add a new state) Harvest Minerals 4 SCVs harvesting Build Barracks barracks Train 4 marines Less than 4 SCVs 4 SCVs 4 marines & Enemy seen No marines Enemy unseen Train 4 SCVs Enemy Inside Base Attack Enemy Enemy seen Explore Attack Inside Enemy

20 Hierarchical Finite State Machines FSM inside of the state of another FSM As many levels as needed Can alleviate complexity problem to some extent Enemy Inside Base Standard Strategy No Enemy Inside Base Attack Inside Enemy

21 Hierarchical Finite State Machines FSM inside of the state of another FSM As many levels as needed Can alleviate complexity problem to some extent Harvest Minerals 4 SCVs harvesting Build Barracks barracks Train 4 marines Enemy Inside Base Less than 4 SCVs Train 4 SCVs 4 SCVs 4 marines & Enemy seen Attack Enemy No marines Enemy seen Enemy unseen Explore No Enemy Inside Base Attack Inside Enemy

22 Decision Trees In the FSM examples before, decisions were quite simple: If 4 SCVs then build barracks But those conditions can easily become complex Decision trees offer a way to encode complex decisions in a easy and organized way

23 Example of Complex Decision Decide when to attack the enemy in a RTS game, and what kind of units to build We could try to define a set of rules: If we have not seen the enemy then build ground units If we have seen the enemy and he has no air units and we have more units than him, then attack If we have seen the enemy and he has air units and we do not have air units, then build air units etc. Problems: complex to know if we are missing any scenario, The conditions of the rules might grow very complex

24 Example of Complex Decision: Decision Tree The same decision, can be easily captured in a decision tree Enemy Seen yes Does he have Air Units yes Do we have Antiair units yes no Do we have More units Than him Build Antiair units yes no Attack! Build more units no Build more units no Do we have More units Than him yes no Attack! Build more units

25 Decision Trees Intuitive Help us determine whether we are forgetting a case Easy to implement: Decision trees can be used as paper and pencil technique, to think about the problem, and then just use nested if-then-else statements They can also be implemented in a generic way, and give graphical editors to game designers

26 Finite State Machines with Decision Trees In complex FSMs, conditions in arches might get complex Each state could have a decision tree to determine which state to go next S1 C1 C2 S2 S3

27 Example Basic RTS AI: Strategy Finite-State Machine used as an example in Week 2 of class Resource spending: 80% Economy, 20% Military 2 workers wood 1 worker metal Army Composition: 100% footmen After training 4 footmen Resource spending: 20% Economy, 80% Military 2 workers wood 4 workers metal Army Composition: 100% knights If enemy has Flying units Army Composition: 50% knights 50% archers If enemy has no more flying units

28 Other Approaches Rule-based systems: Not extremely common in games, but very well studied in AI (expert systems) Collection of rules plus an inference engine Problems: hard to scale up (rules have complex interactions when there s many of them) Behavior Trees: Combination of Hierarchical FSMs with planning and execution Very popular in modern games (not so popular in RTS games) Covered in Intro to Game AI (offered next quarter)

29 Outline Student Presentation: Near Optimal Hierarchical Pathfinding Student Presentation: Intelligent Moving of Groups in Real-Time Strategy Games Decision Making Basics: Hardcoded Methods Decision Theory Adversarial Search Project Discussion

30 Authoring AI vs Autonomous AI FSMs, Decision Trees, Rule-based Systems, Behavior Trees, etc. are useful to hardcode decisions: Game designer tools to make the AI behave the way they want The AI will never do anything the game designers didn t foresee (except for bugs) We will now change our attention to techniques that can be used to let the AI autonomously take decisions The AI takes decisions on its own, and can generate strategies, not foreseen by game designers

31 Decision Theory Given a situation, decide which action to perform depending on the desirability of its immediate outcome Desirability of a situation: utility function U(s) Decision theory is based upon the idea that there is such utility function Example utility function, Chess: Score of a player: 10 points for the queen + 5 points per rook, + 3 points per knight or bishop + 1 point per pawn. Utility for white pieces: U w (s) = Score(white) Score(black) Utility for black pieces: U b (s) = Score(black) Score(white)

32 Example Utility Function for RTS games: Similarly to chess, we can do: U(s) = X t w t (c friendly t c enemy t ) Enemy units must be estimated This is oversimplified, but it is good as a first approach. It can be improved by adding: Resources: (minerals, gas, gold, wood, etc.) Research done Territory under control

33 Is it Realistic to Have a Utility Function? Yes: In hardcoded approaches (FSM, Decision trees, etc.): The AI designer has to tell the AI HOW to play Utility function: Only captures the goal of the game In the simplest form, utility is simply 1 for win, -1 for loss, and 0 otherwise But the more information conveyed in the utility function, the better the AI can decide what to do The AI designer has to tell the AI WHAT is the goal It is easier to define a utility function, than to hardcode a strategy, since utility function has less information (only WHAT, not HOW)

34 Decision Theory Effect of an action a on the state s: Result(s,a) Since we might not know the exact state in which we are, we can only estimate the result of an action: P(Result(s,a) = s e) (e is the information we know about s) Example: a = attack enemy supply depot with 4 marines e = we haven t observed any enemy unit around the supply depot Result(s,a) = supply depot destroyed in 250 cycles, 4 marines intact But we don t know if there were cloaked units, so we can only guess the result of a

35 Maximum Expected Utility Principle (MEU) Select the action with the expected maximum utility: EU(a e) = X s 0 P (Result(a, s) =s 0 e)u(s 0 ) Requires: Utility function (hardcoded or machine learned) Estimation of action effects (hardcoded or machine learned) The AI has to know what the actions do!

36 Example: Target Selection Utility function: 60 points per footman 400 points per barracks 200 points per lumber mill Which action? Attack enemy footman Attack enemy barracks Attack enemy lumber mill Player = blue Enemy = red

37 Example: Target Selection Utility function: 60 points per footman 400 points per barracks 200 points per lumber mill Which action? Attack enemy footman: 2 footmen can kill 1 footman U(s ) = 2* = -480 Player = blue Enemy = red

38 Example: Target Selection Utility function: 60 points per footman 400 points per barracks 200 points per lumber mill Which action? Attack enemy barracks: During the time it takes to destroy the barracks, the enemy footman can kill our 2 footmen U(s ) = = -660 Player = blue Enemy = red

39 Example: Target Selection Utility function: 60 points per footman 400 points per barracks 200 points per lumber mill Which action? Attack enemy lumber mill: During the time it takes to destroy the lumber mill, the enemy footman can kill our 2 footmen U(s ) = = -660 Player = blue Enemy = red

40 Example: Target Selection Utility function: 60 points per footman 400 points per barracks 200 points per lumber mill Which action? Attack enemy footman: -480 Attack enemy barracks: -660 Attack enemy lumber mill: -660 Player = blue Enemy = red

41 Basic RTS AI Diagram Decision Theory can be useful Perception at these three levels. Unit Analysis Map Analysis Notice that, as presented here, it only considers one Strategy action at a time. Strategy Give Orders Economy Logistics Attack Arbiter Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder If used in the Attack module, it is more natural to have one decision theoretic module per squad, so each can take individual decisions.

42 Value of Information How do we know when is it worth spending resources in exploring? Value of perfect information: VPI e (E) = X k P (E = e k )EU (a k e, E = e k )! EU (a e) i.e.: How much utility can we expect to gain if we knew the value of an unknown variable E (that can take k different values e 1,, e k )

43 Example Starcraft: Player force: 4 marines (60 points each) 1 Enemy Command Center spotted (500 points) Enemy defenses: unknown command center? Which action to perform? Attack Train more marines 4 marines

44 Example Starcraft: Player force: 4 marines (60 points each) 1 Enemy Command Center spotted (500 points) Enemy defenses: unknown command center? Which action to perform? Attack: U(s ) = 0.5 U(winning) U(losing) = U(s ) = 0.5 * (4 marines) (-1 command center) = -230 Train more marines: U(s ) = 6 marines (1 command center + 2 SCVs) = marines

45 Example Starcraft: Player force: 4 marines (60 points each) 1 Enemy Command Center spotted (500 points) Enemy defenses: unknown Which action to perform? Attack: U(s ) = 0.5 * (4 marines) (-1 command center) = -130 Train more marines: U(s ) = 6 marines (1 command center + 2 SCVs) = -380 command center 4 marines? If we know no defenses, then for Attack: U(s ) = 4 marines = 240 If we know there are defenses, then for Attack: U(s ) = -1 command center = -500 EU(Attack Defenses) = -500 EU(Attack no Defenses) = 240

46 Example Starcraft: Player force: 4 marines (60 points each) 1 Enemy Command Center spotted (500 points) Enemy defenses: unknown command center? Which action to perform? Attack: U(s ) = 0.5 * (4 marines) (-1 command center) = -230 Train more marines: U(s ) = 6 marines (1 command center + 2 SCVs) = marines VPI(defenses) = (0.5 * EU(More Marines Defenses) * EU(Attack No Defenses) ) (EU(Attack))

47 Example Starcraft: Player force: 4 marines (60 points each) 1 Enemy Command Center spotted (500 points) Enemy defenses: unknown command center? Which action to perform? Attack: U(s ) = 0.5 * (4 marines) (-1 command center) = -230 Train more marines: U(s ) = 6 marines (1 command center + 2 SCVs) = marines VPI(defenses) = (0.5 * * 240 ) (-130) = = 60

48 Decision Theory Basic principles for taking rational decisions Goal of game AI is to be fun: Utility function doesn t have to be tuned to optimal play, but to fun play For example, utility function may penalize too many attacks to the player in a given period of time, in order to let the player breathe. Deals only with immediate utility. No look ahead, or adversarial planning: To deal with that: adversarial search (next section)

49 Outline Student Presentation: Near Optimal Hierarchical Pathfinding Student Presentation: Intelligent Moving of Groups in Real-Time Strategy Games Decision Making Basics: Hardcoded Methods Decision Theory Adversarial Search Project Discussion

50 Adversarial Search Decision theory is good for lower reactive decisions (e.g. Attack module) At a higher level (strategy), better decisions could be made if the AI could plan for future actions: E.g.: If I attack with 4 marines, I ll be left with 2, then the enemy will overpower me with his 4 Dragoons. In that case I could lure them into the upper defenses, and annihilate them with my 8 tanks. Solution: game tree search

51 Game Tree Decision theory deals with immediate decisions: Current Situation Player 1 action U(s) U(s) U(s)

52 Game Tree Decision theory deals with immediate decisions: Current Situation Player 1 action Pick the action that leads to the state with maximum expected utility U(s) U(s) U(s)

53 Game Tree Game trees capture the effects of successive action executions: Current Situation Player 1 action Player 2 action U(s) U(s) U(s) U(s) U(s) U(s)

54 Game Tree Game trees capture the effects of successive action executions: Current Situation Player 1 action Pick the action that leads to the state with maximum expected utility after taking into account what the other players might do Player 2 action U(s) U(s) U(s) U(s) U(s) U(s)

55 Game Tree Game trees capture the effects of successive action executions: In this example, we look ahead only one Current Situation player 1 action and one player 2 action. But we could grow the tree arbitrarily deep Player 1 action Player 2 action U(s) U(s) U(s) U(s) U(s) U(s)

56 Minimax Principle Positive utility is good for player 1, and negative for player 2 Player 1 chooses actions that maximize U, player 2 chooses actions that minimize U Current Situation Player 1 action Player 2 action U(s) = -1 U(s) = 0 U(s) = -1 U(s) = 0 U(s) = 0 U(s) = 0

57 Minimax Principle Positive utility is good for player 1, and negative for player 2 Player 1 chooses actions that maximize U, player 2 chooses actions that minimize U Current Situation Player 1 action Player 2 action (min) U(s) = -1 U(s) = 0 U(s) = -1 U(s) = 0 U(s) = 0 U(s) = 0

58 Minimax Principle Positive utility is good for player 1, and negative for player 2 Player 1 chooses actions that maximize U, player 2 chooses actions that minimize U Current Situation Player 1 action Player 2 action (min) U(s) = -1 U(s) = -1 U(s) = 0 U(s) = -1 U(s) = 0 U(s) = -1 U(s) = 0 U(s) = 0 U(s) = 0

59 Minimax Principle Positive utility is good for player 1, and negative for player 2 Player 1 chooses actions that maximize U, player 2 chooses actions that minimize U Current Situation Player 1 action (max) Player 2 action (min) U(s) = -1 U(s) = -1 U(s) = 0 U(s) = -1 U(s) = 0 U(s) = -1 U(s) = 0 U(s) = 0 U(s) = 0

60 Minimax Algorithm Minimax(state, player, MAX_DEPTH) IF MAX_DEPTH == 0 RETURN U(state) BestAction = null BestScore = null FOR Action in actions(player, state) (Score,Action2) = Minimax(result(action, state), nextplayer(player), MAX_DEPTH-1) IF BestScore == null (player == 1 && Score>BestScore) (player == 2 && Score<BestScore) BestScore = Score BestAction = Action ENDFOR RETURN (BestScore, BestAction)

61 Minimax Algorithm Needs: Utility function U Way to determine which actions can a player execute in a given state MAX_DEPTH controls how deep is the search tree going to be: Size of the tree is exponential in MAX_DEPTH Branching factor is the number of moves that can be executed per state The higher MAX_DEPTH, the better the AI will play There are ways to increase speed: alpha-beta pruning

62 Successes of Minimax Deep Blue defeated Kasparov in Chess (1997) Checkers was completely solved by Jonathan Shaeffert (2007): If no players make mistakes, the game is a draw (like tick-tack-toe) Go: Using a variant of minimax, based on Monte Carlo search (UCT), In 2011 The program Zen19S reached 4 dan (professional humans are rated between 1 to 9 dan)

63 Game Tree Search in RTS Games Classic minimax assumes (Chess, Checkers, Go ): 2 players Perfect information Turn-taking game Given a state and an action, we can predict the next state It is easily generalizable to a multiplayer turn-taking game (max^n algorithm) RTS games: Real-time, not turn-taking, simultaneous actions Lots of possible actions: branching factor too large! We cannot exactly predict the next state Imperfect information

64 Game Tree Search in RTS Games Problem: Lots of possible actions, branching factor too large! Solution:??? Problems: real-time, no turn taking, simultaneous actions Solution:???

65 Game Tree Search in RTS Games Problem: Lots of possible actions, branching factor too large! Solution: Sampling (Monte-Carlo Search) Problems: real-time, no turn taking, simultaneous actions Solution:???

66 Monte-Carlo Tree Search: UCT Monte-Carlo Search: For each possible action: play N games at random until the end starting with each action If N is large, the average win ratio converges to the expected utility of the action Upper Confidence Tree (UCT) is a state of the art, simple variant of Monte-Carlo Search, responsible for the recent success of Computer Go programs Idea: Instead of opening the whole Minimax tree or play N random games: Open only the upper part of the tree, and play random games from there

67 Minimax vs Monte-Carlo Minimax: Monte-Carlo: U U U U U U U U U U U U U U U U

68 Minimax vs Monte-Carlo Minimax: Monte-Carlo: U U U U U U U U U U U U U U U U Minimax opens the complete tree (all possible moves) up to a fixed depth. Then, the Utility function is applied to the leaves.

69 Minimax vs Monte-Carlo Minimax: Monte-Carlo: U U U U U U U U U U U U U U U U Monte-Carlo search runs for each possible move at the root node a fixed number K of random complete games. No need for a Utility function (but it can be used), Complete Game

70 UCT Tree Search 0/0 Current State Monte-Carlo Search Current state w/t is the account of how many games starting from this state have be found to be won out of the total games explored in the current search

71 UCT Tree Search 1/1 Monte-Carlo Search win

72 UCT Tree Search 1/2 0/1 Monte-Carlo Search loss At each iteration, one node o the tree (upper part) is selecte and expanded (one node adde to the tree). From this new nod a complete game is played ou at random (Monte-Carlo)

73 UCT Tree Search 2/3 1/1 0/1 Monte-Carlo Search At each iteration, one node o the tree (upper part) is selecte and expanded (one node adde to the tree). From this new nod a complete game is played ou at random (Monte-Carlo) win

74 UCT Tree Search 3/4 2/2 0/1 1/1 Monte-Carlo Search The counts w/t are used to determine which nodes to explore next. Exploration/Exploitation: 50% expand the best node in the tree 50% expand a node at random win

75 UCT Tree Search 3/5 2/3 0/1 1/1 0/1 Monte-Carlo Search The tree ensures all relevant actions are explored (greatly alleviates the randomness that affects Monte-Carlo methods) loss

76 UCT Tree Search 3/5 2/3 0/1 1/1 0/1 Monte-Carlo Search loss The random games played from each node of the tree serve to estimate the Utility function. They can be random, or use an opponent model (if available)

77 UCT After a fixed number of iterations K (or after the assigned time is over), UCT analyzes the resulting trees, and the selected action is that with the highest win ratio. UCT can search in games with much larger state spaces than minimax. It is the standard algorithms for modern (from 2008 to present) Go playing programs

78 Game Tree Search in RTS Games Problem: Lots of possible actions, branching factor too large! Solution: Sampling (Monte-Carlo Search) Problems: real-time, no turn taking, simultaneous actions Solution: Strategy simulation, rather than turn-based action taking

79 Strategy Simulation: Example Assume we want to use UCT for the Strategy module of an RTS AI game Perception Unit Analysis Map Analysis Strategy Strategy Give Orders Economy Logistics Attack Arbiter Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

80 Strategy Simulation: Example Assume we want to use UCT for the Strategy module of an RTS AI game Define a collection of high level actions (or strategies) that make sense for the game. For example, in S3: S1: Attack with the units we have S2: Train 4 footmen S3: Train 4 archers S4: Train 4 catapults S5: Train 4 knights S6: Build 2 defense Towers S7: Build 2 defense Towers around a Gold Mine S8: Build 2 defense Towers around a group of Trees S9: Bring units back to the base S10: Train 2 more peasants to gather resources

81 Strategy Simulation: Example Instead of taking turns in executing actions, we assign a strategy to each player, and simulate it until completion: Player 1, Action 1 Player 2, Action 2 Player 1, Action 3 Standard Minimax Strategy Simulation Player 1: S2 (ETA 240) Player 2: S3 (ETA 400) Player 1: S1 Player 1: S1 (ETA 400) Player 2: S3 (ETA 160) Player 2: S1 Player 1: S1 (ETA 240) Player 2: S1 (ETA 400)

82 Strategy Simulation Requires: A way to simulate strategies: typically a very simplified model E.g. battles just decided by who has more units, or added damage of units (taking into account air/ground units) No pathfinding, etc. Abstracted version of the game, e.g.: divide map in regions, and just count the number of unit types in each region Utility function (optional): If available, there is no need to simulate games till the end when using Monte-Carlo If not available, simply simulate games to the end

83 UCT for RTS Games Applicable to: Strategy (previous example) Attack: where the high-level actions are things like attack enemy X, retreat, etc. Economy In Turn-based games, minimax is executed each turn For RTS games: execute each K cycles (e.g. once per second), or once the current action has finished, or an important event happened (e.g. new enemy sighted) State of the art: No current commercial games use it Research in experimental games shows its potential

84 Overview of Decision Making Hardcoded: FSM/Decision Trees: good for simple AIs and for game designers to author the exact behavior they want the AI to have AI author needs to decide HOW they AI plays Autonomous: Decision Theory: good for reactive behavior (e.g. unit/squad control, Attack module) Adversarial Search: good for high-level strategy AI author only needs to decide WHAT the AI wants to accomplish, the AI will figure out HOW automatically

85 RTS Game AI Overview Week 2: How to create a basic RTS AI Hierarchical system: Strategy: FSM Giving Orders: Different modules (Economy, Logistics, Attack) Pathfinding Building Placer Week 3: Pathfinding A* or TBA* / LRTA* for small games D* Lite for larger games with very dynamic environments Week 4: Decision Making FSMs / Decision Trees: for hardcoding AI behavior (useful for simple games where the game designers can control what they AI does) Decision Theory / Minimax: to let the AI decide what to do in order to maximize the utility function

86 RTS Game AI Overview Perception Unit Analysis Map Analysis Strategy Strategy Give Orders Economy Logistics Attack Arbiter Execute Orders Building Placer Unit Unit AI Unit AI AI Pathfinder

87 Outline Student Presentation: Near Optimal Hierarchical Pathfinding Student Presentation: Intelligent Moving of Groups in Real-Time Strategy Games Decision Making Basics: Hardcoded Methods Decision Theory Adversarial Search Project Discussion

88 Project 1: RTS Games Issues with Starcraft / BWAPI? Issues with S3? Questions about Pathfinding? Options for Project 1 (pick one): Pathfinding: TBA*, LRTA*, D* Lite Decision Making: Utility Function for High-level strategy or attack. Game Tree Search (minimax, Monte Carlo, or UCT) for strategy or attack Other (with permission from instructor)

89 Project 2: Drama Management IFGameEngine demo