1. 1
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
This
work
is
licensed
under
a
CreaBve
Commons
AEribuBon-­‐NonCommercial-­‐ShareAlike
4.0
InternaBonal
License.
Chapter
1
Introduc0on
Dana S. Nau and Vikas Shivashankar
University of Maryland

2. 2
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
3. A systematic arrangement of
elements or important parts; a
configuration or outline: a seating
plan; the plan of a story.
4. A drawing or diagram made to
scale showing the structure or
arrangement of something.
5. A program or policy stipulating a
service or benefit: a pension plan.
plan n.
1. A scheme, program, or method
worked out beforehand for the
accomplishment of an objective:
a plan of attack.
2. A proposed or tentative project or
course of action: had no plans for
the evening.
Some
Dic0onary
Defini0ons
of
“Plan”
[a representation] of future behavior …
usually a set of actions, with temporal
and other constraints on them, for
execution by some agent or agents.
– Austin Tate, MIT Encyclopedia of
the Cognitive Sciences, 1999

3. 3
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Introduc0on
Ø AI planning has made huge advances
over the past several decades
● Languages (PDDL, RDDL,
ANML) for specifying problems
● Mature technology for finding
plans in large state spaces
● Several well-known applications
Ø But less practical impact than one
might hope
Ø Less than several other areas of AI
● Machine learning
● Data mining
● Natural language processing
Ø Why?

4. 4
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
s s’Predict
Search
What
Does
Planning
Involve?
Ø Planning a course of action = Prediction + Search
● Predict what effects various
actions may have
● Search through alternative
sets of actions
● Look for plan that’s best
for the intended purpose

5. 5
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
“Conven0onal”
AI
Planning
Ø Focus: domain-independent techniques for
generating plans of action
Ø Given:
● Domain model (descriptions of
the states and actions)
● Planning problem
▸ Initial state s0
▸ Goal g
Ø Find a plan (sequence of actions)
▸ e.g., ⟨go-­‐leM,
load,
go-­‐right⟩
or a policy (set of state-action pairs)
▸ e.g., {(s0, go-­‐leM), (s1, load), (s2, go-­‐right)}
● Should be executable starting in s0
● Should produce a state that satisfies g
load
go-­‐leM
go-­‐right
loc1
loc2
loc1
loc2
loc1
loc2
loc1
loc2
loc2
s2
s1
s0
s3

6. 6
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Key
Assump0ons
Ø Assumptions used in conventional AI planning
● Domain is static except for the planned
actions
▸ No other source of change in the world
● Planning is done offline
▸ Enough time to generate a complete plan
● Actions are directly executable primitives
▸ Planner doesn’t need to worry about how
they’ll be executed
● Actions can be represented by simple models
telling what they will do
▸ Required to be trivial to compute
• e.g., add/delete elements from sets
● Action models assumed correct and
complete
• e.g., include all possible outcomes
load
go-­‐leM
go-­‐right
loc1
loc2
loc1
loc2
loc1
loc2
loc1
loc2
s2
s1
s0
s3

7. 7
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Interleaved
Planning
and
Execu0on
Ø Some good work on this topic
● But it still assumes actions are directly executable
Tracker
Target
Planner Executor

8. 8
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Delibera0ve
Ac0ng
Requires
More
Ø Planner is part of a larger system: the actor
Challenges in P&A
‣ Open problems in addressed issues
‣ Several unaddressed issues
Monitoring
Goal reasoning
Acting Observing
Learning
Models, data &
knowledge bases
Environment
Mission
Criteria
Objectives
Monitoring
actions
Control
variables
Signals
Sensing
actionsFeedback
Feedback
Users
Planning
Robot’s platformExecution platform

9. 9
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Example:
the
Key
to
Room
215
Dana Nau and Malik
Ghallab having breakfast
at a hotel near Paolo
Traverso’s home
Dana left before Malik was
finished, went to hallway,
started toward stairwell
Dana came back, told
Malik he had forgotten his
room key, picked it up
Went to hallway, saw
elevator was there, took it
to his floor

10. 10
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Example:
the
Key
to
Room
215
Must
finish
tasks
in
my
room,
then
catch
a
bus
▸ Is
there
enough
Bme?
▸ Only
if
I
go
now
What
do
I
need
to
do?
▸ I’ll
need
my
room
key
▸ I
leM
it
on
the
table
▸ Go
back
and
get
it
Elevator
is
here
now
▸ My
room
is
2
floors
up
▸ If
I
use
elevator
instead
of
stairs,
I’ll
recover
some
lost
Bme
Dana Nau and Malik
Ghallab having breakfast
at a hotel near Paolo
Traverso’s home
Dana left before Malik was
finished, went to hallway,
started toward stairwell
Dana came back, told
Malik he had forgotten his
room key, picked it up
Went to hallway, saw
elevator was there, took it
to his floor

11. 11
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Example:
the
Key
to
Room
215
Must
finish
tasks
in
my
room,
then
catch
a
bus
▸ Is
there
enough
Bme?
▸ Only
if
I
go
now
What
do
I
need
to
do?
▸ I’ll
need
my
room
key
▸ I
leM
it
on
the
table
▸ Go
back
and
get
it
Elevator
is
here
now
▸ My
room
is
2
floors
up
▸ If
I
use
elevator
instead
of
stairs,
I’ll
recover
some
lost
Bme
Dana Nau and Malik
Ghallab having breakfast
at a hotel near Paolo
Traverso’s home
Dana left before Malik was
finished, went to hallway,
started toward stairwell
Dana came back, told
Malik he had forgotten his
room key, picked it up
Went to hallway, saw
elevator was there, took it
to his floor
● Retrieve an abstract
temporal plan
Ø Do rough evaluation
from past experience
● Refine concurrently
with execution
Ø Predictive simula-
tion and monitoring
Ø Online replanning
and execution
● Event-driven comparison
of abstract plans
Ø Choose new plan
Ø Refine concurrently
with execution

12. 12
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
General
Characteris0cs
Ø Continual online planning
● Plans remained partial and abstract
until the details were needed
Ø Multiple levels of abstraction
● Go upstairs, go into room
▸ What route to take?
• How to open the door?
Ø Heterogeneous reasoning, heterogeneous models
● How much time will my tasks take?
● How to get to the stairwell?
● What to tell Malik when I go back to get my key?
Ø Is this just human reasoning, or is any of it needed in AI systems?

13. 13
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Service
Robot
ungrasp
grasp
knob
turn
knob
maintain
move
back
pull
monitor
identify
type
of
door
pull
monitor
move
close
to
knob
open door
……
get out close door
respond to user requests
……
bring o7 to room2
go to
hallway
deliver
o7
…… … …
…
move to door
fetch
o7
navigate
to room2
navigate
to room1
Activities for a service robot to perform
Ø Multiple levels of abstraction
● Different state spaces and action spaces
● How to translate among them?
Ø Conventional AI planning can be
used for some of the tasks
● e.g., bring o7
to room2
Ø For others, need other
deliberation techniques
● e.g., how to open a door
Ø How to verify & maintain
consistency among different
action models?

14. 14
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Harbor
Management
……
manage incoming shipment
unload unpack store
… ……
…… …
registration
manager
storage
assignment
manager
release
manager
booking
manager navigation
…
…
await order prepare deliver
storage area
C manager
storage area B
manager
storage area A manager
Ø Harbor management system based on
one for Bremen Harbor
● Imports/exports cars
Ø World changes dynamically
● Need to plan reactions
Ø Multiple levels of abstraction
● Top level can be planned offline
● The rest is online, based
on current conditions
Ø Operations are carried out by
different components that must
interact with each other
● Need operational models of the
components
● On-the-fly synthesis of automata
to control the interactions

15. 15
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
General
Characteris0cs
Ø Same characteristics as in the summary of the “Room 215” example
Ø Continual online planning
● Plans remain partial and abstract
until the details were needed
Ø Multiple levels of abstraction
● Manage shipment
▸ Storage assignment
• Booking manager
Ø Heterogeneous reasoning, heterogeneous models
● Different kinds of action models
● Interactions among processes

16. 16
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Focus
of
the
Book
(and
the
Course)
Ø Planning
Ø Acting
Ø How to combine them
Deliberation
components
Execution platform
Commands Percepts
Other
actors
Objectives
Messages
External World
SignalsActuations
Actor
Deliberation components
Execution platform
Planning
Acting
Queries
Plans

17. 17
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Outline
of
the
Book
Ø Chapter 1: Introduction
● That’s what we’re covering right now J
Ø Chapter 2: Deliberation with deterministic models
● The “conventional” AI planning techniques mentioned earlier
▸ Sometimes called “classical” AI planning
● Action models, planning algorithms
● Integrating them with acting
Ø Chapter 3: Deliberation with refinement methods
● Hierarchical decomposition of larger problems in to smaller ones
● Reactive execution algorithm
● Planning algorithm
● Combining them
Ø Chapter 4: Deliberation with temporal domain models
● Modeling the time required to perform an action
● Algorithms for planning and acting

18. 18
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Outline
of
the
Book
Ø Chapter 5: Deliberation with nondeterministic domain models
● Action models in which the actions have multiple possible outcomes
● Different kinds of solutions (safe/unsafe, cyclic/acyclic)
● Planning techniques (determinization, model checking)
● Acting techniques (I/O automata, synthesis of controllers)
Ø Chapter 6: Deliberating with probabilistic domain models
● Actions with multiple possible outcomes, and probabilities of those outcomes
● Stochastic Shortest Path problems (generalization of MDP problems)
● Planning algorithms (policy and value iteration, other more advanced
approaches)
● Acting with probabilistic models
Ø Chapter 7: Other deliberation functions
● Perceiving, monitoring, goal reasoning, interaction, learning, hybrid models,
ontologies
Ø Chapter 8: Conclusion

19. 19
Dana
Nau
and
Vikas
Shivashankar:
Lecture
slides
for
Automated
Planning
and
Ac0ng
Updated
1/23/15
Any
Ques0ons?