Thesis Defence

1. Monocular Simultaneous Localization
and Generalized Object Mapping with
Undelayed Initialization
資訊工程所 蕭辰翰
1 Robot Perception and Learning Lab 2010/7/23

2. Outline
 Introduction
 State Vector Definition
 Proposed Classification Algorithm
 Simulations and Real experiment
 Conclusion
2 Robot Perception and Learning Lab 2010/7/23

3. Outline
 Introduction
 State Vector Definition
 Proposed Classification Algorithm
 Simulations and Real experiment
 Conclusion
3 Robot Perception and Learning Lab 2010/7/23

4. EKF-based SLAM
4 Robot Perception and Learning Lab 2010/7/23

5. Monocular SLAM
 Camera as the only sensor
 Andrew J. Davison et al. proposed a EKF-based
SLAM approach
• Andrew J. Davison, Ian Reid, Nicholas Molton and Olivier Stasse:
MonoSLAM: Real-Time Single Camera SLAM, IEEE TRANSACTIONS
ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.
29, NO. 6, JUNE 2007
 Feature states are the 3D position vectors of the locations
of point features
Multiple images acquired must be
combined to achieve accurate depth
estimates
5 Robot Perception and Learning Lab 2010/7/23

6. Inverse Depth Parametrization
6 Robot Perception and Learning Lab 2010/7/23

7. Inverse Depth Parametrization
 High degree of
linearity
 Ability to cope with
features far from the
camera
 Undelayed
initialization
J.M.M. Montiel, Javier Civera and Andrew J. Davison:
“Unified Inverse Depth Parametrization for Monocular SLAM”.
Robotics: Science and Systems Conference 2006.
7 Robot Perception and Learning Lab 2010/7/23

8. Dynamic Environments
 Inclusion of moving features
=> Degrade performance
 Prior knowledge
 Avoid moving objects
• SomkiatWangsiripitak, David W. Murray: Avoiding moving outliers in
visual SLAM by tracking moving objects, ICRA2009
8 Robot Perception and Learning Lab 2010/7/23

9. Performance Degrade
SLAM with static features SLAM with static features
and a new moving feature
Chieh-ChihWang, Ko-Chih Wang, Chen-Han Hsiao, Kuen-Han Lin and Yi-Liu Chao.:
Monocular Vision-based Simultaneous Localization, Mapping and Moving Object Tracking,
submit to journal
9 Robot Perception and Learning Lab 2010/7/23

10. Classification Stage
 Accumulate temporal
information
 Multiple images
required
 Abnormal negative
inverse depth
Chieh-ChihWang, Ko-Chih Wang, Chen-Han Hsiao, Kuen-Han Lin and Yi-Liu Chao.:
Monocular Vision-based Simultaneous Localization, Mapping and Moving Object Tracking,
submit to journal
10 Robot Perception and Learning Lab 2010/7/23

11. Contributions in this thesis
 SLAM with generalized objects
 Proposed parametrization
 Undelayed Initialization
 Classification algorithm
 based on velocity
11 Robot Perception and Learning Lab 2010/7/23

12. Outline
 Introduction
 State Vector Definition
 Proposed Classification Algorithm
 Simulations and Real experiment
 Conclusion
12 Robot Perception and Learning Lab 2010/7/23

13. State vector definition
 EKF-based SLAM with generalized objects
 State vector:
 Camera:
 Generalized object:
 The inverse depth parametrization
 Proposed parametrization
 With motion model
13 Robot Perception and Learning Lab 2010/7/23

14. Dynamic Inverse Depth
Parametrization
 9-dimension state value
 Position
 Velocity
 3D Location w.r.t. XYZ coordinate system:
14 Robot Perception and Learning Lab 2010/7/23

15. Feature parametrization and
motion prediction
15 Robot Perception and Learning Lab 2010/7/23

16. Dynamic Inverse Depth
Parametrization
 Motion Predict
 (constant velocity assumption)
16 Robot Perception and Learning Lab 2010/7/23

17. Measurement Model
 The observation of a point feature
17 Robot Perception and Learning Lab 2010/7/23

18. Undelayed Feature Initialization
 Initialized using only one image
 First observed frame
18 Robot Perception and Learning Lab 2010/7/23

19. Initial value of inverse depth and
velocity
 Initial value of inverse depth
 Range of depth: [ d min , ]
1
 Range of inverse depth: [ 0 , ]
d min
 To cover its 95% acceptance region: ˆ 0
1 1
,
2 d min 2 d min
 Initial value of velocity
 Range of velocity: [ | v | max , | v | max ]
 To cover its 95% acceptance region: v 0
| v | max
ˆ 0, v
2
19 Robot Perception and Learning Lab 2010/7/23

20. Outline
 Introduction
 State Vector Definition
 Proposed Classification Algorithm
 Simulations and Real experiment
 Conclusion
20 Robot Perception and Learning Lab 2010/7/23

21. Classification Cue
 Conduct simulation to show the convergency of
velocity
 3 targets in the simulation
 Coded in dynamic inverse depth parametrization
21 Robot Perception and Learning Lab 2010/7/23

22. Classification Cue:
Velocity Convergency
22 Robot Perception and Learning Lab 2010/7/23

23. Classification Cue:
Velocity Convergency
23 Robot Perception and Learning Lab 2010/7/23

24. Classification Cue:
Velocity Convergency
24 Robot Perception and Learning Lab 2010/7/23

25. Score function
for classifying static objects
 Given the velocity distribution:
 Probability density function value of the velocity
distribution at
 The relative likelihood at
 Classification by thresholding
 Cs(X ) ts => classify as static object
25 Robot Perception and Learning Lab 2010/7/23

26. Threshold selection on ts
 PDF value of static
object at
is expected higher
 Threshold selection t s
26 Robot Perception and Learning Lab 2010/7/23

27. Score function
for classifying moving objects
 Given the velocity distribution:
 Mahalanobis distance function
 The velocity of moving objects is expected to converge
away from
 Classification by thresholding
 Cm (X ) tm => classify as moving object
27 Robot Perception and Learning Lab 2010/7/23

28. Threshold selection on tm
 M-dist of a moving
object at
is expected to larger
 Threshold selection t m
28 Robot Perception and Learning Lab 2010/7/23

29. Classification State Initialized feature
 Generalized objects in
state vector
 Unknown state Unknown
state
 Static state
 Moving state Cs(X ) ts Cm (X ) tm
 Low computational
classification Static Moving
algorithm state state
29 Robot Perception and Learning Lab 2010/7/23

30. State transition
Unknown state to Unknown state to
Static state Moving state
 Change the label  Change the label
 Adjust values to  Keep the same values
satisfied the property
v 0, v
0
SLAM with generalized object is achieved.
30 Robot Perception and Learning Lab 2010/7/23

31. Issue on unobservable situations
disability of monocular system to find an unique trajectory
of an object under the constant-velocity assumption
31 Robot Perception and Learning Lab 2010/7/23

32. Issue on unobservable situations
 Conduct simulation to show the convergency of
velocity
 3 targets in the simulation
 Coded in dynamic inverse depth parametrization
32 Robot Perception and Learning Lab 2010/7/23

33. Ambiguation under unobservable
situations
33 Robot Perception and Learning Lab 2010/7/23

34. Ambiguation under unobservable
situations
 Cannot distinguish the
state according to the
velocity distribution
 Ambiguation of
 Static object
 Constant speed
parallel-moving object
34 Robot Perception and Learning Lab 2010/7/23

35. Non-parallel moving object under
unobservable situations
35 Robot Perception and Learning Lab 2010/7/23

36. Non-parallel moving object under
unobservable situations
 95% confidence
region do not cover
(0,0,0)
 No ambiguation with
static object
36 Robot Perception and Learning Lab 2010/7/23

37. Classification under unobservable
situations
 Ambiguation
 Static object
 Constant speed
parallel-moving object
 Non ambiguation
 Constant speed
non parallel-moving
object
37 Robot Perception and Learning Lab 2010/7/23

38. Outline
 Introduction
 State Vector Definition
 Proposed Classification Algorithm
 Simulations and Real experiment
 Conclusion
38 Robot Perception and Learning Lab 2010/7/23

39. Simulation
Observable situation Unobservable situation
39 Robot Perception and Learning Lab 2010/7/23

40. Simulation setting
 50 Monte Carlo simulations
 300 static landmarks and 288 moving landmarks
 In each simulation
40 Robot Perception and Learning Lab 2010/7/23

41. Classification error ratio
Observable situation Unobservable situation
41 Robot Perception and Learning Lab 2010/7/23

42. Classification Result
 Classification Result of 50 Monte Carlo
simulations under threshold
 Observable situation
42 Robot Perception and Learning Lab 2010/7/23

43. Convergency of
SLAM with generalized objects
 Convergency of camera
43 Robot Perception and Learning Lab 2010/7/23

44. Convergency of
SLAM with generalized objects
 Convergency of static objects
44 Robot Perception and Learning Lab 2010/7/23

45. Convergency of
SLAM with generalized objects
 Convergency of moving objects
45 Robot Perception and Learning Lab 2010/7/23

46. Real Experiment
 NTU PAL7 robot
 Wide-angle camera
 79.48 degree view
angle
 640 × 480
 Laser scanner
 At the basement of
CSIE
 1793
images,13.65fps, 131
seconds
46 Robot Perception and Learning Lab 2010/7/23

47. Real Experiment at basement of
CSIE
47 Robot Perception and Learning Lab 2010/7/23

48. Real Experiment
 Classification result
48 Robot Perception and Learning Lab 2010/7/23

49. Video of Real Experiment
49 Robot Perception and Learning Lab 2010/7/23

50. Comparison of estimation and
ground-truth (Topview)
50 Robot Perception and Learning Lab 2010/7/23

51. Comparison of estimation and
ground-truth (Sideview)
51 Robot Perception and Learning Lab 2010/7/23

52. Outline
 Introduction
 State Vector Definition
 Proposed Classification Algorithm
 Simulations and Real experiment
 Conclusion
52 Robot Perception and Learning Lab 2010/7/23

53. Conclusions
 Achieve SLAM with generalized objects
 Simulations
 Real experiments
 Adopt un-delayed initialization
 Provide a low computational classification
algorithm
 Competitive performance to laser-based
approach
53 Robot Perception and Learning Lab 2010/7/23