This document outlines a 6-lecture course on AI planning techniques and their applications to video games. It introduces the STRIPS planning framework and describes how planning domains can be specified using PDDL. It then presents the SimpleFPS planning benchmark, a PDDL domain for modeling NPC behavior in first-person shooter games. This domain represents an FPS game world using predicates and defines basic actions an NPC can perform. The document discusses how problem instances are generated for evaluating planners on SimpleFPS and presents preliminary results using two award-winning planners.

1. Stavros Vassos, University of Athens, Greece stavrosv@di.uoa.gr May 2012
INTRODUCTION TO AI
STRIPS PLANNING
.. and Applications to Video-games!

2. Course overview
2
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

3. Course overview
3
 Lecture 1: Game-inspired competitions for AI research,
AI decision making for non-player characters in games
 Lecture 2: STRIPS planning, state-space search
 Lecture 3: Planning Domain Definition Language (PDDL),
using an award winning planner to solve Sokoban
 Lecture 4: Planning graphs, domain independent
heuristics for STRIPS planning
 Lecture 5: Employing STRIPS planning in games:
SimpleFPS, iThinkUnity3D, SmartWorkersRTS
 Lecture 6: Planning beyond STRIPS

4. SimpleFPS planning benchmark
4
 Focus on first-person shooter (FPS) games and the
non-player characters (NPCs) that act against the
human player.
 Focus on goal-oriented action planning (GOAP) for
NPC behavior.

5. SimpleFPS planning benchmark
5
 Focus on first-person shooter (FPS) games and the
non-player characters (NPCs) that act against the
human player.
 Focus on goal-oriented action planning (GOAP) for
NPC behavior.
 SimpleFPS is a first step towards evaluating how the
existing academic approaches for planning would
perform if directly applied in an FPS setting.

6. SimpleFPS planning benchmark
6
 Focus on first-person shooter (FPS) games and the
non-player characters (NPCs) that act against the
human player.
 Focus on goal-oriented action planning (GOAP) for
NPC behavior.
 SimpleFPS is a first step towards evaluating how the
existing academic approaches for planning would
perform if directly applied in an FPS setting.

7. STRIPS Planning
7
 Given:
 Initial State
 Goal
 Available actions
 Find:
A sequence of actions that satisfy the goal
 E.g.: [Left, Down, Left, Up, …]

8. Planning Domain Description Language
8
 Language for specifying STRIPS planning problems
 Formal syntax like a programming language
 Initial State (:init …)
 Goal (:goal …)
 Actions (:action name
:parameters (?from ?to ?dir)
:preconditions (…)
:effects (…)
)

9. Planning Domain Description Language
9
 Language for specifying STRIPS planning problems
 Formal syntax like a programming language
 Initial State (:init …)
 Goal (:goal …)
 Actions (:action name
:parameters (?from ?to ?dir)
:preconditions (…)
:effects (…)
)
 Literals like Lisp, e.g., bot-at(area1) (bot-at area1)

10. Planning Domain Description Language
10
 Language for specifying STRIPS planning problems
 Formal syntax like a programming language
 Predicates (:predicates …)
 Actions (:action name
:parameters (?from ?to ?dir)
:preconditions (…) Planning
:effects (…) Domain
)
 Objects (:objects …) Planning
 Initial State (:init …) Problem
 Goal (:goal …)

11. Planning Domain Description Language
11
 Planning Domain Description Language
 SAT Plan
Planning Domains in PDDL:
 TL Plan Blocks world,
Storage, Trucks, …
 FF
 BlackBox Planning Problems in PDDL
for these domains
 SHOP2
 TALPlanner
… Comparisons
Evaluation
Conclusions

12. Planning Domain Description Language
12
 Planning Domain Description Language
 SAT Plan
Planning Domains in PDDL:
 TL Plan SimpleFPS
Domain
 FF
 BlackBox Planning Problems in PDDL
for SimpleFPS domain
 SHOP2
 TALPlanner
… Comparisons
Evaluation
Conclusions

13. Motivation for SimpleFPS
13
 Planning in academia: extensively tested
 Planning in FPS video games: not extensively tested
 SimpleFPS: A PDDL domain for evaluating academic
planning techniques for NPCs in First-Person Shooters

14. Motivation for SimpleFPS
14
 Planning in academia: extensively tested
 Many PDDL planning domains and problems available
 Many off-the-shelf PDDL planners available
 Planning in FPS video games: not extensively tested
A few success stories but not clear if the same works
under different assumptions or what is the best approach
 SimpleFPS: A PDDL domain for evaluating academic
planning techniques for NPCs in First-Person Shooters

15. SimpleFPS planning benchmark
15
 SimpleFPS_PDDL_Domain.txt
 SimpleFPS_PDDL_ProblemGenerator.c

16. SimpleFPS planning benchmark
16
 SimpleFPS_PDDL_Domain.txt:
 Specifies the predicates that can be used to describe
the initial state of the game-world and the goal
condition for the NPC.
 Specifies a list of available actions that the NPC can
perform, along with their preconditions and effects in
terms of the predicates of the domain.
 SimpleFPS_PDDL_ProblemGenerator.c:
A tool to generate problem instances, i.e., an initial
state of the game-world and a goal for the NPC.

17. SimpleFPS planning benchmark
17
 Summer student project at the University of Athens
 Michail Papakonstantinou
 AIIDE-2011 workshop paper
 TheSimpleFPS Planning Domain:
A PDDL Benchmark for Proactive NPCs
 Code/datasets available online
 http://code.google.com/p/simple-fps-pddl/

18. SimpleFPS domain
18

19. SimpleFPS domain
19
 Represent only very simple high-level features:
 Game-world consists of interconnected areas, each of
which has a number of points of interest (POIs).
A few types of items located at POIs (weapons, ammo,
med-kits, keycards).
 NPC can perform basic actions (move between areas
or POIs, pick-up/use items, attack, take cover).

20. A SimpleFPS problem instance
20
 (:init …)

21. A SimpleFPS problem instance
21
 (:init …)
6 areas, some of them
connected through doors
a1 a3 and corridors:
a2
 (area a1)
 (area a2)
a4  (area a3)
 (area a4)
a6 a5
 (area a5)
 (area a6)

22. A SimpleFPS problem instance
22
 (:init …)
6 areas, some of them
connected through doors
a1 and corridors:
a2
 (poi door1 a1)
 (waypoint door1)
 (connected a1 a2 door1)
 (closed door1)
 (opens door1 keycard1)

23. A SimpleFPS problem instance
23
 (:init …)
 For each area a number
of POIs are listed along
with their properties:
a2
 (poi door1 a2)
 (poi c1 a2)
 (poi c2 a2)
 (connected a2 a1 door1)
 (cover-point c1)
 (cover-point c2)

24. A SimpleFPS problem instance
24
 (:init …)
 For each area a number
of POIs are listed along
with their properties:
 (poi g1 a5)
 (poi amm1 a5)
 (gun g1)
 (unloaded g1)
a5
 (ammo amm1 g1)

25. A SimpleFPS problem instance
25
 (:init …)
 For each area a number
of POIs are listed along
a1 a3 with their properties:
a2
 knife
 med-kit
a4  control-box
 night-vision-gun
a6 a5
…

26. SimpleFPS domain: predicates
26
 NPC-related  Area-related  Item-related:
 (npc-at ?a)  (area ?a)  (med-kit ?m)
 (npc-close-to ?p)  (conn ?a1 ?a2 ?w)  (knife ?k)
 (npc-covered)  (waypoint ?w)  (gun ?g)
 (npc-uncovered)  (lighted ?area)  (loaded ?g)
 (npc-holding ?o)  (dark ?area)  (unloaded ?g)
 (npc-injured)  (poi ?p ?a)  (ammo ?i ?g)
 (npc-full-health)  (control-box ?p)  (night-vision ?g)
 (npc-aware)  (cover-point ?p)
 (npc-unaware)  (item ?p)

27. SimpleFPS domain: actions
27
 (: action …)
 Available NPC actions:
a1 a3  move-to-area
a2
 move-to-poi
 pick-up-item
 use-item
a4  take-cover
a6 a5  un-cover

28. SimpleFPS domain: move-to-point
28
(:action move-to-point
:parameters (?area ?point)
:precondition (and
(npc-at ?area)
(point-of-interest ?point ?area)
)
:effect (and
(npc-close-to ?point)
)
)

29. SimpleFPS domain: reload
29
(:action reload
:parameters (?gun ?item)
:precondition (and
(npc-holding ?gun) (gun ?gun) (unloaded ?gun)
(npc-holding ?item) (ammo ?item ?gun)
)
:effect (and
(not (unloaded ?gun))
(loaded ?gun)
(not (npc-holding ?item))
)
)

30. The SimpleFPS domain: actions
30
 Location-related:  Attack-related:
 moving-to-patrol  make-contact
 moving-to-take-position  take-cover
 move-away-from-point  uncover
 move-to-point  use-med-kit
 move-to-point-from-point  reload
 make-accessible  attack-melee
 place-in-inventory  attack-ranged
 turn-on-lights  sneak-kill
 turn-off-lights

31. A SimpleFPS problem instance
31
 (:goal …)
 NPC goals:
a1 a3  g1: (player-wounded)
a2
 g2: (npc-covered)
 g3: (npc-full-health)
a4  g4: (and g1 g2 g3)
a6 a5

32. A SimpleFPS problem instance
32
 blackbox -o problemssfps-domain.txt -f
problemssfps-problem1.txt
 1. (move-to-point area3 door3-2)
 2. (move-to-point area2 control-box2)
 3. (turn-on-lights area2 control-box2)
 4. (make-contact area2 p)
 5. (move-to-point-from-point area2 knife2 control-box2)
 6. (place-in-inventory area2 knife2)
 7. (move-to-point area2 p)
 8. (attack-melee area2 knife2 p)
 9. (move-to-point-from-point area2 door2-0 p)
 10. (moving-to-take-position area2 area0 door2-0)
 11. (move-to-point area0 coverpoint1)
 12. (take-cover area0 coverpoint1)

33. SimpleFPS problem generator
33

34. SimpleFPS problem generator
34
 Takes as input:
 -a number of areas,
 -c the probability that two areas are connected
 -n total number of points of interest
 -g the goal condition as one of g1, g2, g3, g4
 -l the number of instances to be generated
 Generates problem instances also using some rules:
 Card-keysare added for locked doors
 Ammo is added for guns that are unloaded
…

35. SimpleFPS datasets
35
 Used the tool to generate 3 datasets:
5 areas
 7 areas
 10 areas
 For each dataset we generated 10 instances with:
 10 items
…
 100 items
 For each of the 4 goals:
 g1, g2, g3, g4

36. Preliminary results with SimpleFPS
36
 Used two award-winning planners in these datasets:
 BlackBox [Kauts, Selman 1999]
 FastForward [Hoffman 2001]
 Run the planners on an average laptop
 1.4GHz
 2 GB RAM

37. Preliminary results with SimpleFPS
37
 BlackBox, FastForward
 Problems planners always return an answer within 1.5sec
BB: up to 5 areas/50 POIs FF: up to 10 areas/70 POIs

38. Preliminary results with SimpleFPS
38
 BlackBox,Different planning techniques paper
FastForward: Two graphs in the
 Problems planners a lot of difference within 1.5sec
make always return an answer
BB: up to 5 areas/50 POIs FF: up to 10 areas/70 POIs

39. Preliminary results with SimpleFPS
39
 Investigate further the available heuristics for
progression planning using FastDownward as a
framework

40. Thorough results with SimpleFPS
40
 Left: best-first search with FF heuristic
 Right: weighted A* search with a combination of FF
and landmark cut heuristic
 Average (total) run-time of planner

41. Thorough results with SimpleFPS
41
 Left: best-first search with FF heuristic
 Right: weighted A* search with a combination of FF
and landmark cut heuristic
 Average (total) run-time of planner

42. Thorough results with SimpleFPS
42
 Left: best-first search with FF heuristic
 Right: weighted A* search with a combination of FF
and landmark cut heuristic
 Average run-time of planner doing search

43. Thorough results with SimpleFPS
43
 Left: best-first search with FF heuristic
 Right: weighted A* search with a combination of FF
and landmark cut heuristic
 Average run-time of planner doing search

44. Thorough results with SimpleFPS
44
 Left: best-first search with FF heuristic
 Right: weighted A* search with a combination of FF
and landmark cut heuristic
 Average plan length

45. Thorough results with SimpleFPS
45
 Left: best-first search with FF heuristic
 Right: weighted A* search with a combination of FF
and landmark cut heuristic
 Average plan length

46. Some conclusions
46

47. Some conclusions
47
 Relatively small-sized problems wrt FPS games:
 10 areas/100 POIs
 The planner takes a lot of resources:
A lot of time: up to 1.4 sec to respond using 100%
of the CPU resources of a laptop
 A lot of memory: up to 6.5MB for each problem

48. Some conclusions
48
 Relatively small-sized problems wrt FPS games:
 10 areas/100 POIs
 The planner takes a lot of resources:
A lot of time: up to 1.4 sec to respond using 100%
of the CPU resources of a laptop
 A lot of memory: up to 6.5MB for each problem
 Different assumptions in academia and FPS games

49. Some conclusions
49
 There is a lot of room for improvement
 Take advantage of pre-processing
 6.5 MBs is too much for 1 character but how about 100?
 Maintain the pre-processed state and update instead of
re-computing it each time
 Guide the search method
 The Golog family of languages

50. Bibliography
50
 References
 The SimpleFPS Planning Domain: A PDDL Benchmark for Proactive NPCs.
Stavros Vassos, Michail Papakonstantinou. In Proceedings of the Non-
Player Character AI workshop (NPCAI-2011), 2011
 PDDL - The Planning Domain Definition Language. Drew McDermott,
Malik Ghallab, Adele Howe, Craig Knoblock, Ashwin Ram, Manuela
Veloso, Daniel Weld, David Wilkins. Technical report, Yale Center for
Computational Vision and Control, TR-98-003, 1998.
 Unifying SAT-Based and Graph-Based Planning. Henry Kautz, Bart
Selman. In Proceedings of the International Joint Conference on
Artificial Intelligence (IJCAI), 1999
 The FF planning system: Fast plan generation through heuristic search.
Jörg Hoffmann, Bernhard Nebel. Artificial Intelligence Research, Vol.
14, 2001
 The Fast Downward Planning System. Malte Helmert. Artificial
Intelligence Research (JAIR), Vol. 26, 2006