Mission Planning and Execution for the Unmanned Rotorcraft ARTIS

Mission Planning and Execution for the
Unmanned Rotorcraft ARTIS
Florian-Michael Adolf
German Aerospace Center (DLR)
Dept. Unmanned Aircraft
Braunschweig, Germany

Background
Support Acquisition of Situational Awareness in Hazardous Environments
Tepco Fukushima Daiichi Reactor, Japan 2011
[Air Photo Service + Rotomotion/Hélipse]
Earthquake, Chile 2010
Texas City disaster April 16, 1947:
Complex docks building.
[Special Collections, University of Houston Libraries]
www.DLR.de • Chart 2

Mobile Ground Control Station
Unmanned Vehicle
Unmanned Aircraft System
Remote Operator and Unmanned Vehicle

Autonomous Rotorcraft Testbed for Intelligent Systems (ARTIS)
Unmanned rotorcraft midiARTIS (MTOW 14 kg)
shown with stereo-based obstacle detection.

Planning Problems
Given:
Autonomous vehicle that can
perform waypoint navigation
UAV operator specifies single
waypoints or “tasks”
Desired:
Planning of collision free paths
in 3D
Optimization of waypoint
ordering
In case of more UAVs,
automatically assign tasks
MAYBE given:
3D environment with
known obstacles
3D obstacle sensor
Travelling Salesman problem,
known to be NP-hard too
Proven to be NP-hard
(1987, Canny)
Local planners needed, in order to
define actual waypoint set

Mission Planning Problem
3D Path Planner Task Planner
Generate waypoints for tasks
(e.g. search areas)
Optimize the order in which
waypoints are visited
Assign paths to each vehicle
Find collision free connections
between each pair of waypoints
Smooth effective path if possible
Highly coupled problem domains:
Task Planning Path Planning
“…and me,
the task
ordering”
“I need the
costs…”

Strongly Decoupled Approach
Trajectory
Following
Control
Trajectory
Generation
Behaviour
Generation
Graph
Search
Task
Planning
MRAC
Control
World
Model
Predefined knowledge
(e.g. Obstacles and Flying
areas)
Operator-driven:
goal(s) of a specific task
(e.g. “low level flight from A to B”
or “search object C”)
Automated onboard the
UAS
Transition region:
Automated or manual
(path-)commands

Specification of missions goals
(Waypopints, dedicated Subtasks)
obstacles
search area
Mission Planning
3D Motion Planning
3D Path Planning
Trajectory Optimization
Task Scheduling
Optimization towards missions goals
A B
C
Star
t
A B
C
Star
t
Roadmap-based 3D path planning
Fast 3D offline path smoothing
Mission Planning and Execution (MiPlEx)

e.g. Terra-SAR-X DOM w/ 1m x 1m x 0.1m Res.
Triangulated Height Map -> Closed Neighborhood Mesh
Terrain Acquisition

Terrain Acquisition
Sources in 3D
(OSG,WaveFront…)
Memory efficiency and
approximation by polygonal,
irregular mesh
Independent polygonal objects
Sources in 2.5D

Roadmap-based Global Path Planner
B
GoalA
Start

„Classical“ Pseudo random sample distribution (PRM) Lattice grid sample distribution (LRM):
non-orthogona + non-uniform
Quasi-random sample distribution (QRM):
steered randomness using Halton sequences
World Sampling
Roadmap-based Path Planner

- QRM sampling is relatively slow
- PRM may lead to disconnected graphs
+ LRM presents regular neighborhood
+ QRM has optimal discrepancy
+ LRM has near-optimal discrepancy
World Sampling

Graph-based Path Search, Spline-based Smoothing
• maximum height 25m
• 780m x 400m area
• 278 QRM samples

World Model loaded in 15.015 sec.
Quasi random Halton sampling for [x=1748,y=1396,z=125.2]
Roadmap build time 57.468 sec
Roadmap path planning in 74.608 sec
Spline-based smoothing in 0.61 sec
Mission planned in 75.249 sec

3D Path Planner
Graph-based Path Search
Eval. of different graph searchers:
A* - considerably slow for path replanning
D* Lite – uses previous path result
⇨ faster than A* + woth same output A*
ARA* - replans from scratch
⇨ mostly suboptimal paths
AD* - uses previous path result
⇨ improves path towards optimum over time

Conventional Grid
Impact of Sampling on Path

Quasi-Random Halton Sampling
Impact of Sampling on Path

-P1
-P2
Flight below obstacles (bridge, roof etc.)
-Enhanced low level sampling
- Detection of free space „between
- Example: Parking deck
-Sampling:
- Low resolution global
- High resolution local

Task Scheduling and Planning
Scheduling using 2-Optmod that finds a suitable task ordering
c(v1,v2) + c(w1,w2) > c(v1,w2)+c(v2,w1)

Simulated Annealing as Meta Heuristic combined with 2-Optmod

Scheduling using 2-Optmod that finds a suitable task ordering

Complexity Problem: Task to Waypoints
Path Planning for Complex Tasks
obstacle
search area
sym. Matrix n x n
In this example:
ninit=30 → nopt=72
⇨ 72²=5184 combinations!
waypoint
sequence,
variant 0
0
n-1
..

Mission Execution: Behavior-based
Automatic payload directed flight for constrained search area with onboard recovery functions.

Sequenceofbehaviorcommands
ID 20070510
TO -5
HV 0 0 -3 180
avoidance on
WT 10
HV 33.3 3.51 -7 28.9
HV 37.1415 1.63484 -10 90
avoidance off
WT 5
object tracker on
HV 37.1415 48.688 -10 90
HV 37.1415 48.688 -10 180
HV 31.81 48.688 -10 180
HV 26.4785 1.63484 -10 180
HV 26.4785 1.63484 -10 90
HV 26.4785 48.688 -10 90
HV 26.4785 48.688 -10 180
HV 21.147 48.688 -10 180
object tracker off
LD
WO
MiPlEx: A Priori Planning

ID 20070510
TO -5
HV 0 0 -3 180
WT 10
HV 33.3 3.51 -7 28.9
HV 37.1415 1.63484 -10 90
WT 5
object tracker on
HV 37.1415 48.688 -10 90
HV 37.1415 48.688 -10 180
HV 31.81 48.688 -10 180
HV 26.4785 1.63484 -10 180
HV 26.4785 1.63484 -10 90
HV 26.4785 48.688 -10 90
HV 26.4785 48.688 -10 180
HV 21.147 48.688 -10 180
object tracker off
LD
WO
Basis
capability
Complex
behaviour
MiPlEx: A Priori Planning

Sequencing Layer: Compile behaviors into plans Mission Time
t0
Take Off Fly To
t1 t2a
Hover
t3
Hover Fly To
t5 t6
Hover
t7
Land
Track
t2b
Track
t4bt4a
Hover TurnTake off
Reactive Layer: Movement capabilities (skills)
Hover To Fly To Pirouette
Search and Track
Deliberate Layer: High-level Behaviors using Task-specific Planners
Fly Home
Land
ID 20070510
TO -5
HV 0 0 -3 180
avoidance on
WT 10
HV 33.3 3.51 -7 28.9
HV 37.1415 1.63484 -10 90
avoidance off
WT 5
object tracker on
HV 37.1415 48.688 -10 90
HV 37.1415 48.688 -10 180
HV 31.81 48.688 -10 180
HV 26.4785 1.63484 -10 180
HV 26.4785 1.63484 -10 90
HV 26.4785 48.688 -10 90
HV 26.4785 48.688 -10 180
HV 21.147 48.688 -10 180
object tracker off
LD
WO
MiPlEx: Plan Execution

Plan Execution: Behavior-based
Automatic payload directed flight for constrained search area with onboard recovery functions.

view with updated
camera alignment
flight
trajectory
> Florian Adolf • > 26.04.2012www.DLR.de • Folie 31
Mapping of gate in world reference system, path replanning, precise gate passing.
Mission Execution: Behavior-based + Sensor-guided

> Florian Adolf • > 26.04.2012www.DLR.de • Folie 32
Mapping of gate in world reference system, path replanning, precise gate passing.
a-priori gate knowledge
estimated true gate position
vehicle
position
waypoints to
fly through gate
initial
waypoints
p3
p4
p1 p2
Mission Execution: Behavior-based + Sensor-guided

Mission Manager for IMAV
- Missionsmanager for AscTec Pelikan UAV
- Behavior-based Approach to accomblish mission task elements
Pelikan
Maestro2
Mission
Manager
ARTIS
Mission
Manager

Problem: Online Execution
State of terrain
a priori unknown
Remote control link
may be disturbed when
flying out-of-sight
Intermediate paths
depend on acquired
terrain data
Repetitive path changes
UAV with terrain
mapping sensor
3-D structures with
overhangs might exist!
Task-based Mission Execution

Closed Loop Perception > Navigation > Control
Online Mapping and Multi-Query Path Planning
UAV with terrain
mapping sensor
“Raw” obstacle data
(e.g. point cloud, depth image)
Online Mapping
[Andert et al., 2009 / Krause 2010]
Geo-referenced
polygon obstacles
Online Path Replanning
[F.Adolf et al., 2010]
Path Following + Flight Control
[S.Lorenz et al., 2010]
Path updates
+ Replans efficiently on the way from „A to B“
+ Efficient multiple path queries
1. Roadmap expansion into unknown terrain
2. Decision making: „Best next“ waypoint „B“
Sensor
FOV

Stereo-Based In-flight Map Building
- Goal: Automatic environment perception,
sensor fusion and mapping resulting in a 3D
model of the immediate helicopter
surroundings
- Total weight of stereo camera and image
processing computer: 1,5 kg
- Lightest passive system known for
environment perception of UAVs

Obstacle Detection and Mapping
[Andert et al., 2009 / Krause 2010]

y
x
z
3
3
3
3
3
3
1 1
1 1
1
2 2
22
4
2
1
3
1
3
y
x
zy
x
z
y
x
z
y
x
z
1
2
3
y
x
z
3
3
1
1 2
Roadmap Updates for Online Planning

DLR’s test site “Rosenkrug”: Flight test obstacle data fed into the roadmap
-Test site “Rosenkrug”
Roadmap Updates for Online Planning

Roadmap Connection Strategy
UAV

Initial flight tests showed problems at the event of
re-planning

Quasi-random
Roadmap with
30 m sample
distance

Roadmap Expansion Required
Initial roadmap
and its perimeter
1 B
Goal vertex
Acquired
during flight
B unreachable!
Non-traversable
roadmap edges
B
UAV
New obstacles
Issues Exploration from “A to B”
Resampling time hard to predict!
2b
Obstacle-based
resampling
B
UAV

Roadmap Expansion Strategy
Initial roadmap
and its perimeter
B
Goal vertex
A
UAV
New
obstacles
3
B
Increase chance
to find path:
Connection
strategy as for
initial roadmap
Exploration from “A to B”
1
B
Resampled
‚unknown‘
partial volumes

Roadmap-based Path (Re-)Planning
B
A Initial path
Online
Polygon
Updates
Non-traversable
roadmap edges
B
A
Replanned path
*) Results presented at AHS-Forum 68, 2012
Problem:
Linear free-space representation
is not an ideal path geometry for
fast(er) navigation*

Finite Horizon Cubic Spline (FHCS)
1) Revise connection
strategy:
Case dependent steering
of vertex in front of the
rotorcraft
2) Generate collision free
and smooth geometry
within field of view
3) Consider sensor FOV and
hover capability: Special
cases for multiple goal
waypoints
UAV
dstop
Linear extrapolated q‘
UAV
dstop
Heuristic
extrapolation(s)
B
A
B
A
B
A

Simulation Setup
3-D LIDAR Model
50 m detection range
180 degree
scan plane 360 degree rotation @1Hz
of 2-D scan plane
Vehicle state update
ARTIS Closed Loop Simulation
Laser beam
collision detection
A Priori ‘Unknown’ Polygons
Extracted Terrain Polygons
Velocity Command
Roadmap-Based Planner
Closed Loop Flights in “Unknown” Terrain

OFN Benchmark: Simple and Urban Scenarios
OFN Benchmark Files from [Mettler et.al., AHS 2010], s.a. http://aem.umn.edu/people/mettler/projects/AFDD/AFFDwebpage.htm
San Diego, CA

Parameters:
vmax = 3 m/s
vvert = 1.5 m/s
amax = 0.5 m/s/s
rmax = 90 deg/s
dclear = 8 m
dsample = 20 m
dsense = 50 m
freplan = 2 Hz
fsense = 5 Hz
Urban Scenario A1 to A: Runtime Example
Sensor FOV
Model
Linear Roadmap Path
FHCS Path
UAV

Trajectories for Simple Cases
Baseline
FHCS
Linear

Timeline for “Wall Baffle”
Baseline FHCS Linear
SpeedSmoothnessSafetyDifficulty

Trajectories for Urban Cases
A
Baseline
FHCS
Linear

Urban Scenario A
Accumulated terrain over all six “A” test cases.

Timeline Urban Scenario A1 to A
Baseline
FHCS Linear
SpeedSmoothnessSafetyDifficulty

Relative Performance
Scenarios FHCS [s] Linear [s] Relative
Difference
[%]
Out and back 80.1 83.8 -4.4%
Point 40.3 50.4 -20%
Wall 46.8 57.8 -19%
Cube 51.5 53.9 -4.5%
Wall Baffle 51.4 59.5 -13.6%
Cube Baffle 65.8 79.7 -17.4%
Sum 335.9 385.1 -12.8%
Scenarios FHCS [s] Linear [s] Relative
Difference
[%]
A1 93.6 100.2 -7.4%
A2 105.1 110.5 -4.7%
A3 98.6 105.8 -6.8%
A4 74.7 78.9 -5.3%
A5 117.2 140.4 -16.5%
A6 100.9 102.1 -1.2%
Sum 590.1 637.5 -7.4%
FHCS vs. Linear Path Following
Results presented
at AHS-Forum 68, 2012

libOFN2010 344.4 +22.6% libOFN2010 521.5 +8.6%
Performance Comparison
FHCS vs. libOFN2010 vs. Baseline
Scenarios FHCS [s] Baseline [s] Relative
Difference
[%]
Out and back 80.1 78.8 +1.6%
Point 40.3 39.3 +2.5%
Wall 46.8 39.3 +19%
Cube 51.5 42.1 +22.3%
Wall Baffle 51.4 41.7 +23.3%
Cube Baffle 65.8 39.8 +65%
Sum 335.9 281.2 +19.4%
Scenarios FHCS [s] Baseline
[s]
Relative
Difference
[%]
A1 93.6 92.7 +1%
A2 105.1 76.2 +37.9%
A3 98.6 74.6 +32.2%
A4 74.7 70.9 +5.4%
A5 117.2 87.4 +34.1%
A6 100.9 81.7 +23.5%
Sum 590.1 483.6 +22,8%
43.8 s
Online performance depends on fsense, dsense, changes in flight
direction etc
=> If FHCS planner is fully informed a priori, it is close to baseline

CPU Time Overhead
Urban
Scenarios
Relative Difference
of FHCS to Linear
Mode [%]
A1 5.2%
A2 5.8%
A3 6.7%
A4 3.3%
A5 4.1%
A6 3%
Mean +4.7%
FHCS vs. Linear Path Following
1) Smoothing and velocity
profiling uses 4.7% of CPU
usage time during mission
2) CPU time for collision
detection less than 0.1%
Note:
Smoothing over a longer distance that dstop would
increase the percentage and is not required

0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
8 m 10 m 12.5 m 15 m 17.5 m 20 m
total (avg)
replan (avg)
max
min
CPU time over Planning Resolution
CPU time for different sample distances.

Online & Multiple Goals: Terrain Mapping
Geo-referenced
point cloud
Structure of interest
Area of interest
[Stefan Krause, 2010]
Remotely Piloted Aircraft System (RPAS)

Roadmap-Based Decision Making
Roadmap perimeter defines
volume to be mapped
A
Greedy Mapping: Select „Best Next“ Waypoint „B“
Uniform
edge costs
A
Bmap
Mapping
vertex
1 2
A2
Bmap
„Mapped“
vertices
A1
A0
Current „A to B“ path,
no path segment to Bmap

Roadmap-Based Decision Making
Strategies to mark vertices as mapped:
1. Visited:
Physically passed or reached by the
vehicle.
2. Scanned:
All edges to and from a vertex have
been inside sensor FOV.
3. Uninformative:
If vertex is detected by mapping sensor,
it is not considered to provide useful
information anymore.
Greedy Mapping: Select „Best Next“ Waypoint „B“
proximity
radius
threshold
1) Visited
3) Uninformative
Mapping
sensor
FOV
All
edges
2) Scanned
Edge at least
once completely
within FOV

A
Exploration Scenario
Urban Terrain “Berlin Potsdamer Platz”

Simulation Results
Rotating
LIDAR
sensor
UAV
Initial roadmap
perimeter
Exploration of Urban Terrain

Simulation Result
Remaining narrow corridor
(width < 20 m)
UAV
Rotating
LIDAR
Flown path
Current mapping path

Simulation Results
Influence of terrain detail level (1m)

Simulation Result
Efficient
replanning
Terrain almost
fully mapped
Total mission time within max. flight time of ARTIS
Trajectories
always well clear
of obstacles

Exploration of Urban Terrain – Experiment
Influence of sample distance
duration: 441 seconds, search volume mapped: 86% duration: 443 seconds, search volume mapped: 90%
20 m distance 30 m distance

Summary
Prev. static roadmap-based motion planner with online sensor-based
extensions can unify (online) task planning and path planning
1. Mission Planning => Combinatorial Motion Planning
2. Spline-based trajectory time reduction approaches
3. Extensions for planning under uncertainty required
a. Online Roadmap Topology
b. Online Spatial Indexing
c. Local Connection Strategy with Dynamic Constraints
Papers: http://elib.dlr.de/view/authors/Adolf,_Florian-Michael.html

Questions?
Feedback?
Comments?
…
Thank you for your attention!

Extension for Fixed Wing

Dubins Roadmap for Fixed Wing
Draufsicht einer Roadmap

Local Connection Model
Horizontal: DUBINS Paths
- 2D
- Shortest path
- Constant ground track velocity
- Minimal curven radius
Vertical: Double Integrator
- Time optimal approach
- Two point controller
- Consivers: and

Local Connection Model
From ARTIS‘ 3D Roadmap, 4D Roadmap:
Additional sampling of possible vehicle heading angles
0
4
8
12
16
0 60 120 180 240 300
Anzahl[n]
Heading [°]

Dubins Roadmap
Parameters:
Sample Distance
Neighborhood Radius
Example:
Hannover Airport

Roadmap for Fixed Wing
-? AIAA Infotech (Juni
2013)

Task Order Optimization
With this roadmap > task scheduling with
n > 4: k-opt-Verfahren with k = 2

Missions Manager for Fixed Wing Aircraft
-AIAA Infotech (Juni 2012)

Shooting Method in Control Space
a. „Shoot“ trials in control space (velocity commands) using closed
loop simulations of ARTIS
b. Reject resulting position samples (world space) that are too close
to each other
c. Allow for motion primitive-based local planning while minimizing
number of samples

Motion Primitive-based Single Shot Planning
Urban benchmark scenario San Diego A1
Iterations: 450 Zeit: 37 s
Pathlength: 1099 m
Empty World (no obstacles > no collision checks)
Iterations: 167 Time: 5,05 s
Path length: 2178 m

(Linear) Reactive Obstacle Avoidance

CPU time over Planning Resolution
FHCS2012 Rel. Perf.:
Baseline libOFN2010 Linear2012
1,65% -5,21% -4,42%
2,54% -18,09% -20,04%
19,08% -13,49% -19,03%
22,33% -1,34% -4,45%
23,26% -2,10% -13,61%
65,33% 26,78% -17,44%
19,54% -2,47% -12,78%
0,97% -4,39% -6,59%
38,11% 31,87% -4,89%
37,71% 20,69% -6,81%
5,36% -4,60% -5,32%
34,10% 18,38% -16,52%
23,50% 18,85% -1,18%
22,84% 13,15% -7,49%
- MiPlEx with FHCS outperforms
Linear version
- In simple cases, FHCS performs
similar to libOFN
- In difficult urban scenarios, spline
smoothing degrades

Scenarios Spline FHCS (s) libOFNminirisk(s) Relative
Difference (%)
Out and back 80.1 84.5 -5.2%
Point 40.3 49.2 -18.1%
Wall 46.8 54.1 -13.5%
Cube 51.5 52.2 -1.9%
Wall Baffle 51.4 52.5 -2.1%
Cube Baffle 65.8 51.9 +26.8%
UAV
Collision
Spline path
Local Planner Improvements

Mission Planning and Execution for the Unmanned Rotorcraft ARTIS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Mission Planning and Execution for the Unmanned Rotorcraft ARTIS

Similar to Mission Planning and Execution for the Unmanned Rotorcraft ARTIS (20)

More from Florian-Michael Adolf

More from Florian-Michael Adolf (8)

Recently uploaded

Recently uploaded (20)

Mission Planning and Execution for the Unmanned Rotorcraft ARTIS