This is a short course that aims to provide an introduction to the techniques currently used for the decision making of non-player characters (NPCs) in commercial video games, and show how a simple deliberation technique from academic artificial intelligence research can be employed to advance the state-of-the art. For more information and downloading the supplementary material please use the following links: http://stavros.lostre.org/2012/05/19/video-games-sapienza-roma-2012/ http://tinyurl.com/AI-NPC-LaSapienza

1. Stavros Vassos, University of Athens, Greece stavrosv@di.uoa.gr May 2012
INTRODUCTION TO AI
STRIPS PLANNING
.. and Applications to Video-games!

2. Course overview
2
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

3. STRIPS planning
3
 What we have seen so far
 The STRIPS formalism for specifying planning problems
 Solving planning problems using state-based search
 Progression planning
 Simple heuristics for progression planning
 Can we take advantage of the information that
action schemas hold to do better?

4. Planning graphs
4
 Action schemas provide useful information about the
interaction between actions
 E.g., action A cannot take place right after B
because A cancels a precondition of B
 There are many more (and more complex)
conditions that would be valuable to identify!

5. Course overview
5
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

6. Planning graphs
6
 Planning graph
 Special data structure
 Consists of a sequence of levels
 Stores the effects of all applicable actions at every
level as if they were all happening concurrently
 Stores some basic mutual exclusion constraints
between actions and literals

7. Planning graphs
7
Α
Β Β
Α Β C Α C C
On(Α,Table) On(Α,Table) On(Α,Β)
On(Β,Table) On(Β,C) On(Β,C)
On(C,Table) On(C,Table) On(C,Table)
Clear(Α) Clear(Α) Clear(Α)
Clear(Β) Clear(Β) Clear(Table)
Clear(C) Clear(Table)
Move(Β,Table,C) Move(Α,Table,Β)
s0 s1 s2

8. Planning graphs
8
??? ???
Α Β C
On(Α,Table)
On(Β,Table)
On(C,Table)
Clear(Α) ??? ???
Clear(Β)
Clear(C)
Α0 Α1
s0 s1 s2

9. Planning graphs
9
Β
???
Α Β C Α C
On(Α,Table)
On(Α,Table) On(Β,Table)
Move(Β,Table,C)
On(Β,Table) On(C,Table)
On(C,Table) Clear(Α)
Clear(Α) Clear(Β) ???
Clear(Β) Clear(C)
Clear(C) On(Β,C)
On(Β,Table)
Clear(C)
Α0 Α1
s0 s1 s2

10. Planning graphs
10
Α
???
Α Β C Β C
On(Α,Table)
On(Α,Table) On(Β,Table)
On(Β,Table) On(C,Table)
On(C,Table) Clear(Α)
Clear(Α) Move(Α,Table,C) Clear(Β) ???
Clear(Β) Clear(C)
Clear(C) On(Α,C)
On(Α,Table)
Clear(C)
Α0 Α1
s0 s1 s2

11. Planning graphs
11
ΑΒ
???
Α Β C Β ΑC C
On(Α,Table)
On(Α,Table) On(Β,Table)
Move(Β,Table,C)
On(Β,Table) On(C,Table)
On(C,Table) Clear(Α)
Clear(Α) Move(Α,Table,C) Clear(Β) ???
Clear(Β) Clear(C)
Clear(C) On(Α,C)
On(Β,C)
On(Α,Table)
On(Β,Table)
Clear(C)
Clear(C)
Α0 Α1
s0 s1 s2

12. Planning graphs
12
On(Α,Table)
On(Β,Table)
On(C,Table) ???
Α Β C Clear(Α)
Clear(Β)
Clear(C)
On(Α,Table) On(Β,C)
Move(Β,Table,C)
On(Β,Table) On(Β,Table)
On(C,Table) On(Α,C)
Clear(Α) Move(Α,Table,C) On(Α,Table) ???
Clear(Β) Clear(C)
Clear(C)
Α0 Α1
s0 s1 s2

13. Planning graphs
13
ΑΒ
ΒΒ ???
Α Β C Move(Α,Table,Β) Β Α C CΒΒ
Α C
Α C
Α C
Α C
Move(C,Table,Β)
On(Α,Table)
On(Α,Table) On(Β,Table)
Move(Β,Table,C)
On(Β,Table) On(C,Table)
On(C,Table) Clear(Α)
Clear(Α) Move(Α,Table,C) Clear(Β) ???
Clear(Β) Clear(C)
Clear(C) Move(Β,Table,Α) On(Α,C)
On(Β,C)
On(Β,C)
On(Β,C)
On(Α,Table)
On(Β,C)
On(Β,Table)
On(Β,C)
Move(C,Table,Α) On(Β,Table)
On(Β,Table)
Clear(C)
On(Β,Table)
On(Β,Table)Α
Clear(C)
Α0 Clear(C)
Clear(C)
Clear(C)
Clear(C)
1
s0 s1 s2

14. Planning graphs
14
 Planning graph
 Special data structure
 Consists of a sequence of levels
 Stores the effects of all applicable actions at every
level as if they were all happening concurrently
 Stores some basic mutual exclusion constraints
between actions and literals
 .. Let’s see an (even) simpler example!

15. Planning graphs
15
 Init( Have(Cake) )
 Goal( Have(Cake)  Eaten(Cake) )
 Action( Eat(Cake)
PRECONDITIONS: Have(Cake)
EFFECTS: Have(Cake)  Eaten(Cake) )
 Action( Bake(Cake),
PRECONDITIONS: Have(Cake)
EFFECTS: Have(Cake) )

16. Planning graphs
16
 Planning graph
 Consists of a sequence of levels that specify how the initial
state is transformed under the effects of actions
 At each level i we specify
A list of literals Si
 A list of actions Ai
 4 kinds of constraints or mutual exclusion links between literals
in Si and actions in Ai

17. Planning graphs
17
 Level 0
 S0: the positive literals of the initial state as well as the
negative literals implied by the closed world assumption
Have(C)
 Eaten(C)
Α0 Α1
s0 s1 s2

18. Planning graphs
18
 Level 0
 We will see now how to specify Α0 and S1
Have(C)
 Eaten(C)
Α0 Α1
s0 s1 s2

19. Planning graphs
19
 Level 0
 Α0: the applicable actions in the initial state
Have(C)
E(C)
 Eaten(C)
Α0 Α1
s0 s1 s2

20. Planning graphs
20
 Level 0
 S1: the effects of actions that appear in Α0
Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)
Α0 Α1
s0 s1 s2

21. Planning graphs
21
 Level 0
 We’re not done yet!
Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)
Α0 Α1
s0 s1 s2

22. Planning graphs
22
 Level 0
 We’re not done yet!
 Also add persistence actions that denote “inaction”
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

23. Planning graphs
23
 Level 0
A persistent action specifies that a literal does not
change truth value between levels, e.g., here  Eaten(C)
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

24. Planning graphs
24
 Level 0
 We’renot done yet!
 Mutual exclusion links
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

25. Planning graphs
25
 Mutual exclusion links (mutex)
 Inconsistent effects
 Interference
 Inconsistent support
 Competing needs

26. Planning graphs
26
 Two actions have inconsistent effects when:
 One action cancels the effect of the other action
 E.g.,action E(C) and the persistent action for Have(Cake) have
inconsistent effects
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

27. Planning graphs
27
 Two actions have inconsistent effects when:
 One action cancels the effect of the other action
 Same for action E(C) and the persistent action for Eaten(C)
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

28. Planning graphs
28
 Two actions have an interference when:
 One effect of one action is the negation of a precondition
for the other action
 E.g., action E(C) and the persistent action for Have(C)
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

29. Planning graphs
29
 Two literals have inconsistent support when:
 One literal is the negation of the other literal
 E.g., Have(C) and Have(C)
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

30. Planning graphs
30
 Two literals have inconsistent support when:
 Every possible pair of action that have these literals as
effects are marked as mutually exclusive
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

31. Planning graphs
31
 Two actions have competing needs when:
A precondition of one action is mutually exclusive with a
precondition of the other action
 Does not arise in this domain
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

32. Planning graphs
32
 Level 0
 We are (finally) done!
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

33. Planning graphs
33
 What kind of information does the graph provide so
far?
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

34. Planning graphs
34
 A pair of mutually exclusive literals cannot be realized
from the actions of Level 0!
 E.g., the goal cannot be achieved with these actions
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

35. Planning graphs
35
 Level 1
 We will specify A1 and S2
Have(C) Have(C)
 Have(C)
E(C)
Eaten(C)
 Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

36. Planning graphs
36
 Level 1
 Α1: the applicable actions in S1 (at least those in Α0)
 S2: the effects of actions in Α1 (at least those in S1)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

37. Planning graphs
37
 Level 1
 Α1: the applicable actions in S1 (and more!)
 S2: the effects of actions in Α1
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

38. Planning graphs
38
 Level 1
 Α1: the applicable actions in S1 (and more!)
 S2: the effects of actions in Α1
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

39. Planning graphs
39
 Level 1
 Mutual exclusive links
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

40. Planning graphs
40
 Level 1
 Mutual exclusive links
 Inconsistent effects between persistence actions
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

41. Planning graphs
41
 Level 1
 Mutual exclusive links
 Inconsistent effects between B(C), E(C) and persistence actions
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

42. Planning graphs
42
 Level 1
 Mutual exclusive links
 Inconsistent effects between B(C), E(C) and persistence actions
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

43. Planning graphs
43
 Level 1
 Mutual exclusive links
 Competing needs between persistence actions!
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

44. Planning graphs
44
 Level 1
 Mutual exclusive links
 No more mutexes between actions
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

45. Planning graphs
45
 Level 1
 Mutual exclusive links
 There are mutexes between literals in S2 though
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

46. Planning graphs
46
 Level 1
 Mutual exclusive links
 Between literals Have(C) and Eaten(C) in S2
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

47. Planning graphs
47
 Level 1
 Mutual exclusive links
 Between literals Have(C) and Eaten(C) in S2
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

48. Planning graphs
48
 Level 1
 Mutual exclusive links
 Between literals Have(C) and Eaten(C) in S2
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

49. Planning graphs
49
 Level 1
 We are (finally) done!
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

50. Planning graphs
50
 What information can we get from the graph now?
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

51. Planning graphs
51
 What information can we get from the graph now?
 Note that literals Have(C) and Eaten(C) are not mutually
exclusive in S2 !!!
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

52. Planning graphs
52
 What information can we get from the graph now?
 Note that literals Have(C) and Eaten(C) can be realized in A1
by the actions {B(C), persistence of Eaten(C)}
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

53. Planning graphs
53
 What information can we get from the graph now?
 In turn actions {B(C), persistence of Eaten(C)} require that
 Have(C) and Eaten(C) hold in S1
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

54. Planning graphs
54
 What information can we get from the graph now?
 Note that literals Have(C) and Eaten(C) can be realized in A0
by the action E(C)
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

55. Planning graphs
55
 What information can we get from the graph now?
 In turn E(C) requires that Have(C) holds in S0 which is true!
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

56. Planning graphs
56
 What information can we get from the graph now?
 So, actions {E(C)} and {B(C), persistence of Eaten(C)}
can actually achieve the goal!
Β(C)
Have(C) Have(C) Have(C)
 Have(C)  Have(C)
E(C)
E(C)
Eaten(C) Eaten(C)
 Eaten(C)  Eaten(C)  Eaten(C)
Α0 Α1
s0 s1 s2

57. Planning graphs
57
 Planning graph

58. Planning graphs
58
 Planning graph
 When do we stop calculating levels?
 When two consecutive levels are identical *
 How do we know this will happen at some point?
 Literals and actions increase monotonically, while
mutexes decrease monotonically (why is this so?)

59. Planning graphs
59
 Planning graph
 Special data structure
 Easy to compute: polynomial complexity!
 Can be used by the GRAPHPLAN algorithm to search
for a solution (following similar reasoning as in the
example)
 Can be used as a guideline for heuristic functions for
progressive planning that are more accurate than the
ones we sketched in Lecture 2

60. Bibliography
60
 Material
 Artificial
Intelligence: A Modern Approach 2nd Ed. Stuart Russell,
Peter Norvig. Prentice Hall, 2003 Section 11.4