AI GDC Game AI Brett Laming

1. From the Ground Up: A.I.
Architecture and Design
Patterns
Dr Brett Laming
Rockstar Leeds

2. Contents
• Basic Principles
• Think/Act - The I/O Divide
• An Algorithmic Approach
• A Functional Approach
• Movement
• Animation
• Conclusion

3. Basic Principles

4. Why Do We Care?
• It’s a Competitive
Market…
– Higher Expectations
– Tighter Schedules
– Multiple Platforms
– Simultaneous Titles
• …But Patterns Are
Everywhere
– Identify, generalise,
reuse, evolve
> Reliability
> Production Speed
⇒ More Fun Stuff

5. Inspirations
Marvin Minsky
Different representations
for different views
Noam Chomsky
Hierarchical
decomposition
Structure vs Meaning
“Colorless green ideas
sleep furiously”
Daniel Dennet
Behaviour can be
viewed at the physical,
design and intentional
levels
David Marr
Computational,
algorithmic and
implementational
Chris Hecker
Style vs Structure [2008]
What is the textured
triangle of A.I.
Craig Reynolds
Simple rules
Complex behaviour
Damian Isla
Cognitive Maps
Spatial Relations
Semantics

6. What Am I Looking For?
Best
Practice
Hierarchy
Reusability
Commutability
Concepts
Components

7. How Do I Find Them?
• Observation
– How might it work?
• Introspection
– What would I do?
• Generalisation
– I’ve done this before
– They do the same
• Bad Experience
– Lets not do that
again
• Background
– I studied this once?
– Could I apply this?

8. Best Practice

9. Best Practice
• Prototype new ideas where possible
– Get visual and design direction
• Mock-ups
– Prove (or disprove) the concept
• Quick and dirty programming
• Play to peoples strengths
– Maths ⇒ Physics Guys
– Navigation Mesh ⇒ Collision Guys

10. Best Practice
• Program Defensively
– Assert and Unit Test
– Automated Scripts as soak tests
– One co-ordinate system and S.I. units
• Maximise Workflow
– Cater tools to their needs
– Put new functionality on bypass
• Think of the man-hour cost!
– Minimise potential for human error

11. Best Practice
• Build A Debugging Suite
– Instant Pause
– Flyable Camera
– Layered Information
– Action Histories
• Maintain Player Immersion
– A.I. should not be too bad or too good
– Limit ourselves to what the player would know
– Constrain them to the same actions

12. The Algorithmic Approach

13. • Key tenants
– Use only what
we might know
– Mimic the player
The Think-Act Loop
Sensory Receptors
Brain
Controller
Game
Player
Sensory Data
Think
Controller
Act
A.I.

14. Sensory Data
Think
Controller
Act
Sensory Input

15. Sensory Data
• Detail level
– Depends on genre and
perceived communication
• Thief vs Medieval Total War II
• General Model
– Visual component an arc
– Auditory component a radius
• Auditory targets less official
– Occlusion too expensive?
• Shoot a weapon to get info
– Theorise using ghost images

16. Blackboards
• Used To Share Information
– Static blackboard stores defined types of info
– Dynamic blackboard stores arbitrary data
• Agents write to the board
– Generally read it as well
Agent
A
Agen
t B
{10,20,15}
{-30,20,13}
{-10,15,12}
{17,11,5}
Scout {3, 17, 10}
Cover {17, 11,5}

17. cSensorCone
Think
Controller
Act
Control Output

18. Virtual Controller (yoke)
• Purpose
– Carries control instructions
– Provides a strict I/O Divide
• Notice the const correctness
– Unifies player and A.I.
• Control mapping for the player
• A.I. fills it in from Think()
• Key Properties
– Never stored in it’s entirety
– Created on the stack
– Lifetime of a single Process()
Process()
Think(yoke&) const
Controller
Act(const yoke&)
cVirtualYoke yoke

19. … So Actually
Process()
Think(yoke&) const
Act(const yoke&)
cVirtualYoke yoke
cSensorCone
cBlackboard

20. Virtual Yoke
• Composite
– Collection of smaller yokes
– Allows selective storage
cVirtualYoke
cLocomotionYoke mLocomotion;
cWeaponYoke mWeapons;
cAnimYoke mAnimation;
cLocomotiveYoke
fp32 mGas;
fp32 mSteering;
fp32 mPitch;
cEntity *mpDontAvoid;
• Context Based
Controller Input
• Logic Flow Control
cWeaponYoke
enum eFire
{
F_PRESSED,
F_WHEN_IN_CONE,
F_WHEN_LOCKED
};
eFire mFireButton;
• Conditional
signals

21. Taking Action
Process()
Think(yoke&) const
Act(const yoke&)
cVirtualYoke yoke
cSensorCone
cBlackboard

22. Act
• Applies Yoke Commands
– Composite yokes →
Subsystems
• Object Models
– Supply common interfaces
• Could be a turret mounted
weapon or my pistol.
• Could be driving a car, a plane,
a boat or myself!
Act()
yoke
iWeaponMgr*
iLocomotive*

23. Object Model
• Self Contained
– Instructions for Think
– Actions for Act
– Commutable
• Plug and Play
• Downloadable
Content
– Maybe broadcast use
• Think Sims 2!
iLocomotive
// Fills in yoke
ComputeMotion(const cTarget &,
cLocomotiveYoke&)
// Computes forces
ApplyMotion(const cTarget &,
const cLocomotiveYoke&,
v3 &force,
v3 &torque)

24. More Coolness
ApplyMotion(const cTarget& targ,
cLocomotiveYoke& yoke)
{
v3 force(Zero);
v3 torque(Zero);
if (Drivable())
{
Drivable()->ApplyMotion(
targ, yoke, force, torque);
}
if (Floatable())
{
Floatable()->ApplyMotion(
targ, yoke, force, torque);
}
ApplyForceTorque(force, torque);
}
Compute
Apply
Compute
Apply
Compute
Apply
Compute
Apply

25. Frame Time
Process()
Think(yoke&, dt) const
Act(const yoke&, dt)
cVirtualYoke yoke
cSensorCone
Update(dt)
cBlackboard
dt()

26. dt()
• Work into everything
– Including fixed time steps
– No more s+=v, s+=v*dt()
• Benefits
– Integration
• Implicit forward Euler
– Rough Smoothing
– Closed Feedback
– Pause dt=0
– Level Of Detail dt=2dt
– Special Effects
dt()
smoothing
feedback control

27. The Functional Approach

28. Problem Domain
• Examine the Terminology
– Feeling, Knowledge, Goals, Beliefs, Needs
• Examine the Concepts
– Decisions, Facts, Uncertainty, Exploration, Verb-Noun
Actions, Repetition, Sequencing
Needs a problem 

29. What Are We Doing?
• Goal based reasoning
– Working to solve a goal
– Thinking about and realising smaller tasks
– Taking a hierarchical approach
– Using a limited number of short verb-noun pairings
to form a plan
• We’ve seen this before
– An old pattern
• Colossal Cave Adventure and MUDs
• Verb-noun actions like “get axe, wield axe” separated by
movement “n, e, e, s, e”
– We use a container object model

30. Applying The Pattern
Know of “Mine, Smithy”
1. Goto “Mine”
Know of “Wall”
2. Get Ore From “Wall”
3. Goto “Smithy”
Know of “Door”
4. Use “Door”
5. Play “Open Door”
6. Warp Inside
Know of “Owner”, “Forge”
7. Play “Close Door”
8. Goto “Owner”
9. Put 10 gp In “Owner”
10. Get Time From “Owner”
11. Goto “Forge”
12. Put Time In “Forge”
13. Put Iron In “Forge”
14. Use “Forge”
15. Use Enchant
16. Use “Forge”
17. Get Sword From “Anvil”
Taking the computational to the algorithmic

31. Easy Questions
• Why did I choose to do this again?
– Because we were driven to it by personality
and need
• What happens when I get the treasure?
– I’ll probably choose to do something else
depending on my mood
• Why stop at get iron?
– Because its reached the atomic level -
there are no more questions, just results

32. Putting It Together
Ambient
Controller
“Get treasure”
Planning
“Get magic sword”
Explicit
Order
“Script here,
Please greet
the player”

33. Explicit Orders
• Script commands
– Script on A.I.
• Autopilot
– Player on Player
• Player instructions
– Player on A.I.
• Squad Commander
– A.I. on A.I.
Ambient
Controller
“Get treasure”
Planning
“Get magic sword”
Explicit
Order
“Script here,
Please greet
the player”

34. Scripting Notes
• Don’t mix styles
– Script has immediate control
– Script waits for an opportunity
• Keep common properties separate
– No sharing memory locations
• A script global population density
• A code global population density
– Maintain a set order of calculation
• Generally consistent with style

35. Ambient Controller
• Generates sensible
actions autonomously
– Maybe Idle
– Maybe Full Daily Routine
• Daily Routines
– Character properties
– Needs/Drives
– Scheduling
– Time of day
⇒ nice emergent behaviour
Ambient
Controller
“Get treasure”
Planning
“Get magic sword”
Explicit
Order
“Script here,
Please greet
the player”

36. Daily Routine
Sleep
Goto Work
Work
Leave Work
Relax
Go Home
Drive Model
Time Of Day
Schedule
Housework 5%
Tavern 50%
Brothel 30%
Shopping 10%
Study 5%
Character
Hunger Libido
dt()

37. Planning
Ambient
Controller
“Get treasure”
Planning
“Get magic sword”
Explicit
Order
“Script here,
Please greet
the player”
Goto Position
“Goto Forge”
Perform Action
“Use Forge”
Planning
Continuous bar dt()
⇒Defines Think()

38. Plan Components
• Play “Animation”
– Waits on dt()
• Use “Object”
– Object model again
– Broadcast actions.
– Change world based on
state
– May wait on dt()
• Get/Take “Object” From
“Container”
– Primary world manipulation
– Contents determine state
• Goto “Location”
– Waits on dt()
– Key A.I. output
– Complex
– Warrants special attention
later

39. Search Based Planning
• Traditional academic approach
– See STRIPS, Hierarchical Task Networks, Bratko
• The Good
– Mimics our introspective reasoning
– Seeks to fully realise a plan to the goal
• Directed search for optimal solutions
• Post processing even more so
• The Bad
– Knowledge representation
• Scalability - difficult for video games
• Lots of storage

40. Procedural Planning
• Industry preferred approach
• Hierarchical
• Easy to comprehend
• Limited Language
– Goals
– Sub-goals
– Conditions
– Actions
• Transitions
– Sequential
– Decision Based
• Powerful Results
Get Object
Short of
money?
Make
Thief
skills?
Steal
Buy
Get OreGoto Mine

41. Procedural Planning - Issues
• Competing children
– We have to make a best guess from the
options
– A* might help
• But we could still end up with case of a basic
sword being bought but not affording the forge.
• Incomplete plan means no post process
– Not good for player supporting A.I.
– More action for generic A.I.

42. The Curve Ball
• Task Interruption
– I’m returning to my gang hideout
– I see an enemy
• I engage the enemy
–I roll out of the way of a car
–I recover to my feet
• I re-engage the enemy
– I continue to return to my gang hideout
• … Is A Key Requirement of our A.I.

43. Finite State Machines
• No plan history
⇒No idea of context
⇒No generalised exit.
⇒Hideous state history
workarounds
• Don’t scale well
– Many transitions
Death
Look At
Gain Range
Attack
Kill Examine
Death
Look At
Gain Range
Attack
Kill Examine
• HFSM came along
– Eased transitions
– But history still an issue

44. Behaviour Trees
Goto Source
Get Material
Take Material
Create
Buy
Steal
Get Object
Crime
Allowed?
Low
money?
Ok
money?
Thief
Class?
Sequencing
Selection
Preconditions
Actions
Decorators

45. Modelling Interruption?
Goto
Location
Kill
Enemy
Return
Home
Locate
Hideout
Gain
Range
Fire
Weapon
Ambient
Behaviour
Parallel?

46. Behaviour Trees
• Simple and powerful
– Limited vocabulary
– Most situations handled
• Highly flexible
– Plug and play
– Customisable
– Nice design tools
– Handy child evaluation
• Lends itself to directed
decision making
• Issues
– Interruption handling
• Where to return
– Amount of flexibility
• Trees get complicated

47. MARPO
Goto Source
Get Material
Take Material
Create Buy Steal
Get Object
Not enough money? Money?
Thief class?
Thieving allowed?

48. ReactiveLong term
MARPO
Get Object
Create
Get Material
Take Material
Immediate

49. ReactiveLong term
MARPO
Get Object
Take Material
Kill Enemy
Gain Distance
Immediate
Get Material
Create

50. ReactiveLong term
MARPO
Get Object Kill Enemy
Immediate
Roll
Gain Distance

51. • Behavioural Bamboo Forest
– Stack based tasks
• Suppression model
– Multiple threads of execution
• Keeps only one stack in memory down to current task
– Decision logic lies with the parent
• Higher level parameterised building blocks
– Authoring is by script not tool.
• Winding Ability
– Allows auto-recovery from any state
– Respects script orders on an immediate basis
MARPO

52. Movement

53. Goto Position
• What Is Position?
– A world co-ordinate
– An entity
– A navigation point
– An offset off an entity
– A radius off an offset,
off an entity
• Still World Positions!
– So generalise
cTarget
void Set(…);
void SetArrivalConditions(…);
v3 WorldPos() const;
v3 InterceptPos(…) const;
bool HasArrived(…) const;
• Lots for free
– Entity intercept
– Completion checks

54. The Problem
• Dynamic obstacles
• … on the move
• Terrain type
• Usable surfaces
• Static geometry

55. Stage 1 - Navigation
• Navigation Mesh
– Industry standard [Tozour, 2008]
– Handles static geometry
• Searching with A*
– Optimise search space, not A*
• Improve Data Format
– Make it Hierarchical
– Allow for reference spaces [Isla, 2005]

56. Navigation
• Quick To Search
⇒ Live Updates
• Zones
⇒ Spatial Reasoning [Isla, 2005]

57. cNavPos
Components
cNavId
mPolyId;
mPolySource;
cNavPoly
… indices
… edge info
cNavId mParent
cNavId mFirstChild
mChildCount
cPolyCoords
… local co-ords
cNavigator
Pos()
FinalTarget()
CurrentTarget()
HasArrived()
Update( cTarget )
cNavPos mFinal
cNavPos mCurrent
cNavPos mPos

58. Stage 2 - Optimisation
• String Pulling
– Bevel uncrossable
edges to a radius
– Tighten points

59. Stage 3 - Dynamic Avoidance
• At rest
– Mesh binding
• On move
– Intercept calculation

60. Stage 4 - Locomotion
• Differing approach
– Vehicle
– Humanoid
• Can we generalise?

61. Locomotion
iLocomotion
ComputeMotion(…)
ApplyMotion(…)
• Generalised by iLocomotion
• Same interface
• Different yoke instructions
• Vehicles
– Vehicles supply gas, steering
– Difficulty is in mapping target to gas and steering
• Actors
– Actors supply ideal position, velocity, direction
– Difficulty is in animation to hit position.

62. Locomotion - Vehicles
Target
Position
Top SpeedSpeed For
Deceleration
Speed For
Steering
Min
Speed to Gas
Maths
PID Controller
Yoke
Yoke
Angle To Steering
Maths
PID Controller

63. Some Hints 
a
d
vmax
u
v
s1 s2
Speed For
Deceleration
• Equations of Motion
Speed to Gas
Angle To Steering
• PID • Response Curves
Manual Shift

64. Animation

65. Locomotion - Character
• Animation ⇒ Position
• Challenges
– Sensible manoeuvre
choice
• Animation Graphs
• Navigation Mark Up
– Natural fluidity
• Foot positioning
• Hand positioning
Idle
Walk
Run
Crouch
Crouch Walk
Crouch Run

66. Goal Based
• Directed Search of Animation Graph
– Incorporates interesting A.I. along the way
– Think Gears of War, A*
• Interested ⇒ Champandard
• Personal Issues
– Need to find a full animation plan to goal
• This would need to sit with path finding
• Makes it more expensive
– Painful Heuristic Balancing
• Switching to the time domain probably partially solves it
• But how do we prevent repeated actions

67. “Carrot On A Stick” Method
• Keep iLocomotion interface
• ComputeMotion() computes
velocity
• ApplyMotion() animates for
velocity
• dt() helps minimise error
• Want to use a cover point?
– Explicit decision
⇒a new target
iLocomotion
ComputeMotion(…)
ApplyMotion(…)
• Given the animation
properties
• We can solve by
mathematics

68. Cover
Under fire?
Find cover?
Route to
cover
Kill
Nav.mesh
Goto
Set piece
“Carrot On A Stick” Method
Anchor Point

69. “Carrot On A Stick” Method
s
θ
Direction for
Animation
off=ƒ(θ,s)
• Offset based on θ, s
• Changes our target
– Hierarchical changes
• Changing over dt
• Smoothes our arrival
– Think high jump!
– Never perfect
– But mistakes made
are human
– Constrained by
navigation mesh

70. “Carrot On A Stick” Method
s
u
V = 0?
• Equations of Motion
– They’re back!
• Give us our aim
velocity vaim
• What about a and d?
vmax
a
d
u
v
s
vaim
dt

71. • Animations
– distance (s) in time (t)
⇒ an average velocity
– Assume velocity bands
– Look up vfor animation
– Dead zones indirectly
determine blending
• a and d now character
properties
• Actively correct for v
– Use tricks like light I.K.
• Implicitly correct by dt()
v
t
Crouch run
Crouch walk
v
Crouch walk
Crouch run
“Carrot On A Stick” Method

72. “Carrot On A Stick” Method
• Animation graph
– Used for information query only
• Locomotive cycle driven by vaim
• Free things!
– Foot positioning
• Obtained through cycle alteration and dt()
– Blending
– Character movement properties
• Obtained through a and d
– Swim, crawl, crouch, walk
• Same procedure, different velocity bands
– Human like arrival mistakes

73. Conclusion

74. Conclusion
– Best Practices
– Strict interfaces
– Multiple takes on
problems
– Concepts
– Commutability
– Components
– Algorithms
– Hierarchy
– Pattern Recognition
– Human Sciences
Good reusable AI is about

75. Conclusion
• Style vs Structure - Hecker 2008
– “Texture Mapped A.I. Triangle”
– Style
• No idea where it is – annoyingly
– Structure
• Not one single triangle
• But many triangle like pieces
• Represented differently
• But all essentially the same piece

76. References
• Key References
– Laming, B. [04] “The Art of Surviving A Simulation Title”, A.I. Wisdom 2, Charles River Media
– Laming, B. [08] “The MARPO Methodology: Planning And Orders”, A.I. Wisdom 4, Charles River Media
– Isla, D. [05], “Dude, where’s my Warthog?”, www.aiide.org/aiide2005/talks/isla.ppt, 13.3.08
– Tozour, P. [08] “Fixing Pathfinding Once and For All”, http://www.ai-blog.net/archives/000152.html, 13.3.08
– Champandard, A., http://aigamedev.com, 13.3.08
– A.I. Wisdom Books in general
• Introduction
– Reynolds, C. [87] “Boids”, http://www.red3d.com/cwr/boids/, 13.3.08
• Intercept Calculation and Dynamic Avoidance
– Stein, N. [02] “Intercepting a Ball”, A.I. Wisdom 1, Charles River Media. [HINT ]
– Tozour, P. [02] “Building a Near-Optimal Navigation Mesh”, A.I. Wisdom 1, Charles River Media
• Best Practice
– Tozour, P. [02] “Building an AI Diagnostic Toolkit”, A.I. Wisdom 1, Charles River Media
• Sensory and Blackboards
– Isla, D. and Blumberg, B. [02] “Blackboard Architectures”, A.I. Wisdom 1, Charles River Media
• Planning
– Champandard, A [08], “Getting Started With Decision Making and Control Systems”, A.I. Wisdom 4, Charles River Media
– Champandard, A., “Understanding Behaviour Trees”, http://aigamedev.com/hierarchical-logic/bt-overview, 13.3.08
– Yiskis, E. [04] “A Subsumption Architecture for Character-Based Games”, A.I.Wisdom 2, Charles River Media
• Navigation
– Numerous references, almost all A.I. Wisdom books.
• Locomotion – Vehicle
– Alexander, B. [02] “The Beauty of Response Curves”, A.I. Wisdom 1, Charles River Media
– Forrester, E. [04] “Intelligent Steering Using PID Controllers”, A.I. Wisdom 2, Charles River Media
• Conclusion
– Hecker, C. [09] “Structure vs Style”, http://chrishecker.com/Structure_vs_Style, 13.3.08