Bosch Expert Days

Machine Vision and Perception Group @ TUM

Perception in Robotic Applications

Darius Burschka
Machine Vision and Perception Group
Department of Informatics
Technische Universität München
MVP

Copyright © 2011 Darius Burschka

Research of the MVP Group http://mvp.visual-navigation.com
Visual navigation
Perception for manipulation

Biologically motivated
perception

Rigid and Deformable The Machine
Registration Vision and Perception Group
@TUM works on the aspects of
visual perception and control in
medical, mobile, and HCI
applications

Photogrammetric Visual Action Analysis
Exploration of physical monocular reconstruction
object properties
MVP


Brook's Layered Control Architecture

awareness complexity/
speed Brooks, 1986 introduced the
paradigm of layered control to
the robotics community,
implementing many of the
features of Minsky's agents.
Rather than decomposing the
autonomous control problem
into separate layers in a
sense/model/plan/act scheme,
Brooks advocates an
architecture where each layer
tightly couples sensing to
actuators.
MVP


“Instincts” Modeling as low-level Action Planner

Sensors
Camera,IMU,VGPS,Laser map-based action planning Actuators

M
M
MVP

Reactive behavior
e.g., Obstacle avoidance,…


Navigation Alternatives

Map-based Navigation Vision-Based Control
data is stored and fused in 3D direct coupling of perception to
representations which are useful action
MVP

for mission planning
Οi = ℑ−1 ⋅ Ιi


Visual Localization for hybrid Environmental Modeling

Real-Time Localization

The high accuracy allows direct
stiching of images along the
trajectory without bundle
adjustment

Resulting hybrid (appearance
and geometry) model allows
path planning and prediction of
sensor views for e.g. attention
MVP

research


Example of a hybrid model reconstruction
MVP


Result of a visual localization

6DoF pose of the robot is calculated fom single
camera view
MVP


Reconstruction of a hybrid (appearance/geometry) model
MVP


Visual Localization for
Visual Environment Modeling(collaboration with Meier/Steinbach)]

localization result

surprise
trigger
MVP

w

10

Per-pixel Depth Maps

Pixel get an additional attribute -> the depth
MVP


Visual Homing

The system is capable of
registration to previously
observed trajectories
MVP


Motivation

global registration without
external devices
MVP

13

Video: Motion Model + IMU Support

Video Link
MVP

14

Stereo Initialization

KLT based stereo initialization
MVP

15

Feature Set Management

Video Link
MVP

16

Embedded Vision at an Example of a Flying Application
MVP


Work on Optimal Sensor Models

Video Link
MVP


New Vision System– with Canberra National University,
Australia and University Bielefeld
MVP


Biologically Motivated Navigation

Obstacle avoidance

Decoupling of Rotation
and Translation
MVP

Perception-based loop closure


Direct Localization

Direct calculation of rotation
MVP


Machine Vision and Perception Group @ TUM MVP

„Simple“ Image Acquisition

60 images taken with a standard low cost digital camera
MVP


Estimation of the 6 Degrees of
Freedom

Estimation of 3 rotational angles Estimation of a translation vector
MVP


Action parsing
MVP


Perception for manipulation
Example of the Manipulation System in Action

Video Link
MVP


Knowledge Representation

Human demonstration

• Atlas: Atlas
parametric grasp actions and
– Long-term memory shape description points handling

– Experience of the system
• Working memory:
– Short-term memory
Mapping of
– Experience grounded in a Knowledge

given environment Foreground Experience
segmentation abstraction
• Temporal handling information background foreground Episodic buffer
model model of actions
MVP

Working Memory


Knowledge Representation

Mapping of
Knowledge
MVP


Grounding of Actions: Ontology→Working Memory
MVP


Basic Experiments:

Functionality Maps (Tracking Data)

Milk carton
Cup - Handle
MVP

Red = goal, green = push, magenta = arbitrary


Estimation of Physical Properties
in Predict-Act-Perceive loop

Estimation of the Center of mass
Estimation of Mass and Friction Force
Estimation of Mass Distribution

predict act perceive

F

r

τ
MVP

simulator


How to reconstruct 3D under poor texture
condition?

Problem: texture information is more sparse
MVP

39

What can we do if the texture information is
almost non-existent?

→ photogrammetric approach
MVP

40

Reconstruction Example

Works well under static lighting conditions and
roughly Lambertian surfaces
MVP

Ruepp and Burschka. Fast recovery of weakly textured surfaces from monocular image
sequences. ACCV 2010


Conclusions

Perception needs to be coupled to technical
systems at the lowest possible level to avoid
unnecessary error accumulation in the
abstraction process

Perception is not only based on texture and
stereo

Sensor-fusion is important for robust perception
MVP


Bosch Expert Days

Recommended

Recommended

More Related Content

Similar to Bosch Expert Days

Similar to Bosch Expert Days (16)

Bosch Expert Days