4. PROBLEM STATEMENT
• Proactive approach:
• Take uncertainty into account during planning. Computationally
expensive
• Reactive approach:
• Plan optimistically and replan if necessary. Computationally cheaper
6. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
7. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
8. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
9. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path If path is blocked, update the map and
go to step 3
10. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path If path is blocked, update the map and
go to step 3
11. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path If path is blocked, update the map and
go to step 3
12. HIGH LEVEL OPERATIONS
1. Input: Map of the environment
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path If path is blocked, update the map and
go to step 3
14. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
15. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
16. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
17. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
18. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
19. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
20. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
21. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
22. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
23. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
24. HIGH LEVEL OPERATIONS
1. Input: Optimistic map (assume there are no obstacles)
2. Discretize the search space
3. Find a shortest path to the goal
4. Move along path. If path is blocked, update the map and
go to step 3
25. Instead of searching for a shortest path from
scratch, reuse information from the previous
search to speed up the current search
INCREMENTAL SEARCH
First search Second search
reusing information
from the first search
...
28. BACKGROUND: A*
• Maintains three values for each cell s
• g(s): cost from start to s
• h(s): estimated cost from s to goal
• f(s) = g(s) + h(s): estimated cost from start to goal
• Maintains two lists
• CLOSED: Contains expanded cells (i.e., cells whose shortest path from
start is known)
• OPEN: Priority queue that contains generated cells
29. BACKGROUND: A*
1 5
2
1 6
2
0 7
6
1 8
2
1 4
2
1 5
2
1 6
2
1 7
2
1 4
2
1 5
2
1 6
2
1 2
2
1 3
2
1 4
2
1 5
2
1 1
2
1 2
2
1 3
2
1 4
2
1 4
2
1 3
2
0 2
2 S
1 0
2 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
30. BACKGROUND: A*
1 5
2
1 6
2
0 7
6
1 8
2
1 4
2
1 5
2
1 6
2
1 7
2
1 4
2
1 5
2
1 6
2
1 2
2
1 3
2
1 4
2
1 5
2
1 1
2
1 2
2
1 3
2
1 4
2
1 4
2
1 3
4
0 2
2 S
1 0
2 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
31. BACKGROUND: A*
1 5
2
1 6
2
0 7
6
1 8
2
2 4
6
1 5
2
1 6
2
1 7
2
1 4
2
1 5
2
1 6
2
1 2
2
1 3
2
1 4
2
1 5
2
1 1
2
1 2
2
1 3
2
1 4
2
2 4
6
1 3
4
0 2
2 S
1 0
2 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
32. BACKGROUND: A*
3 5
8
1 6
2
0 7
6
1 8
2
2 4
6
3 5
8
1 6
2
1 7
2
1 4
2
1 5
2
1 6
2
1 2
2
1 3
2
1 4
2
1 5
2
1 1
2
1 2
2
1 3
2
1 4
2
2 4
6
1 3
4
0 2
2 S
1 0
2 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
33. BACKGROUND: A*
3 5
8
4 6
10
0 7
6
1 8
2
2 4
6
3 5
8
4 6
10
1 7
2
4 4
8
5 5
10
1 6
2
6 2
8
5 3
8
6 4
10
1 5
2
7 1
8
6 2
8
7 3
10
1 4
2
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
34. BACKGROUND: A*
3 5
8
4 6
10
0 7
6
1 8
2
2 4
6
3 5
8
4 6
10
1 7
2
4 4
8
5 5
10
1 6
2
6 2
8
5 3
8
6 4
10
1 5
2
7 1
8
6 2
8
7 3
10
1 4
2
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
36. ALGORITHM 1: AA*
• Adaptive A* (AA*) [Koenig & Likhachev, AAMAS ’05]
• Uses standard A* searches
• Updates h-values to make them more informed, i.e. better estimates
• Speeds up the A* searches
38. ALGORITHM 1: AA*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
39. ALGORITHM 1: AA*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
40. ALGORITHM 1: AA*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
41. ALGORITHM 1: AA*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
43. ALGORITHM 1: AA*
• Results:
• Up to 20% faster than A* on grids with Manhattan distance as
h-values
• Typically, the worse the h-values, the larger the speedup
• Well-suited for high dimensional search spaces, where h-values
are often bad
44. ALGORITHM 1: AA*
Other algorithms in this class:
• MT-AA* [Koenig, Likhachev & Sun, AAMAS ’07]
• Extension to search paths when the target moves
• GAA* [Sun, Koenig &Yeoh, AAMAS ’08]
• Extension to handle decreasing costs
• Applied to control the robotic arm at the International Space Station by
the Canadian Space Agency [Belghith, Kabanza & Hartman, AIS ’10]
• Trea-AA* [Hernandez, Sun, Koenig & Meseguer, AAMAS ’11]
• Extension to speed up the search by reusing search trees
46. ALGORITHM 2: G-FRA*
• Generalized Fringe-Retrieving A* (G-FRA*)
[Sun,Yeoh & Koenig, AAMAS ’08]
• Uses standard A* searches
• Identifies portions of the search tree of the previous search that
can be reused
• Starts the search with these portions
47. BACKGROUND: A*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
48. BACKGROUND: A*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
49. BACKGROUND: A*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
8 G
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
50. BACKGROUND: A*
3 5
8
4 6
10
2 4
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
2 4
6
1 3
4
0 2
2 S
8 0
0 G
C3 C4
D2 D3 D4
E3
F2 F3 F4
C6
D6
E6
F6
C5
D5
E5E2
while(goal ∉ CLOSED)
{
s = argmins∈OPEN f(s)
move s to CLOSED
for all neighboring cells c of s
{
if(c ∉ OPEN ∪ CLOSED)
set g(c) and f(c)
add c to OPEN
else if(c ∈ OPEN)
update g(c) and f(c)
}
}
A
E
D
C
B
G
F
1 2 3 4 5 6 7
g h
f f = g + h CLOSEDOPEN parent pointers
55. ALGORITHM 2: G-FRA*
4 6
10
0 7
6
1 8
2
3 5
8
4 6
10
1 7
2
4 4
8
5 5
10
1 6
2
6 2
8
5 3
8
6 4
10
1 5
2
7 1
8
6 2
8
7 3
10
1 4
2
8 0
0 G
A
E
D
C
B
G
F
1 2 3 4 5 6 7
C3 C4
D2 D3 D4
E2 E3
F2 F3 F4
C6
D6
E6
F6
C5
D5
E5
High-level pseudocode:
1. Identify reusable portion of the
search tree
2. Remove the unreusable portion
3. Add the cells in the fringe to the
OPEN list
4. Restart the search
56. ALGORITHM 2: G-FRA*
0 7
6
4 6
10
0 7
6
0 7
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
0 7
6
0 7
6
0 7
6
8 0
0 G
C3 C4
D2 D3 D4
C6
D6
E6
F6
C5
D5
E5
A
E
D
C
B
G
F
1 2 3 4 5 6 7
High-level pseudocode:
1. Identify reusable portion of the
search tree
2. Remove the unreusable portion
57. ALGORITHM 2: G-FRA*
0 7
6
4 6
10
0 7
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
0 7
6
0 7
6
0 7
6
8 0
0 G
C3 C4
D2 D3 D4
C6
D6
E6
F6
C5
D5
E5E3
A
E
D
C
B
G
F
1 2 3 4 5 6 7
High-level pseudocode:
1. Identify reusable portion of the
search tree
2. Remove the unreusable portion
3. Add the cells in the fringe to the
OPEN list
58. ALGORITHM 2: G-FRA*
0 7
6
4 6
10
0 7
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
0 7
6
0 7
6
0 7
6
8 0
0 G
C3 C4
D2 D3 D4
C6
D6
E6
F6
C5
D5
E5
High-level pseudocode:
1. Identify reusable portion of the
search tree
2. Remove the unreusable portion
3. Add the cells in the fringe to the
OPEN list
4. Restart the search
E3
A
E
D
C
B
G
F
1 2 3 4 5 6 7
59. ALGORITHM 2: G-FRA*
0 7
6
4 6
10
0 7
6
3 5
8
4 6
10
4 4
8
5 5
10
6 2
8
5 3
8
6 4
10
7 1
8
6 2
8
7 3
10
0 7
6
0 7
6
0 7
6
8 0
0 G
C3 C4
D2 D3 D4
C6
D6
E6
F6
C5
D5
E5E3
A
E
D
C
B
G
F
1 2 3 4 5 6 7
High-level pseudocode:
1. Identify reusable portion of the
search tree
2. Remove the unreusable portion
3. Add the cells in the fringe to the
OPEN list
4. Restart the search
60. ALGORITHM 2: G-FRA*
• Results:
• Up to one order of magnitude faster than GAA*
• Typically, the larger the reusable portion of the search tree, the
faster the algorithm
• Well suited for problems that change only slightly between
search iterations
61. ALGORITHM 2: G-FRA*
Other algorithms in this class:
• FRA* [Sun,Yeoh & Koenig, IJCAI ’07]
• Optimized version of G-FRA* for navigation in 2D grids
• D* [Stentz, ICRA ’94], D* Lite [Koenig & Likhachev, AAAI ’02]
• Applied to the navigation of the Mars rovers
• Applied to the navigation of UGVs in the DARPA Urban Challenge
• MT-D* Lite [Sun,Yeoh & Koenig, AAMAS ’10]
• Combination of D* Lite and G-FRA* to speed up the search
62. SUMMARY
• Incremental search algorithms
• Reuses information to speed up the search
• Useful when one needs to search repeatedly for shortest paths
in a graph
• Often applied to robotics applications for autonomous
navigation