IEEE/RSJ IROS 2008 Real-time Tracker

1,555 views

Published on

This slides were for presentation in IROS 2008 conference.

Published in: Technology, Art & Photos
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,555
On SlideShare
0
From Embeds
0
Number of Embeds
46
Actions
Shares
0
Downloads
35
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

IEEE/RSJ IROS 2008 Real-time Tracker

  1. 1. Real-time 3D Object Pose Estimation and Tracking for Natural Landmark Based Visual Servo Seung-Min Baek and Sukhan Lee Sungkyunkwan University Intelligent System Research Center Changhyun Choi Georgia Tech College of Computing
  2. 2. Contents <ul><li>Introduction </li></ul><ul><ul><li>Motivation </li></ul></ul><ul><ul><li>Related Works </li></ul></ul><ul><li>Proposed Approach </li></ul><ul><ul><li>System Overview </li></ul></ul><ul><ul><li>Problem Definition </li></ul></ul><ul><ul><li>Initial Pose Estimation </li></ul></ul><ul><ul><li>Local Pose Estimation </li></ul></ul><ul><li>Experimental Results </li></ul><ul><li>Summary & Conclusion </li></ul><ul><li>Future Work </li></ul>IEEE/RSJ IROS 2008, Sept 25
  3. 3. Introduction <ul><li>In Visual Servo Control, </li></ul><ul><li>Object Recognition </li></ul><ul><li>Pose Estimation </li></ul><ul><li>are key tasks. </li></ul>IEEE/RSJ IROS 2008, Sept 25
  4. 4. Introduction Many systems still use Artificial Landmark . Unnatural in human environment IEEE/RSJ IROS 2008, Sept 25
  5. 5. Introduction <ul><li>We need Natural Landmarks . </li></ul><ul><li>Natural Landmarks are visual features objects inherently have. </li></ul>IEEE/RSJ IROS 2008, Sept 25
  6. 6. Introduction <ul><li>Modern recognition methods </li></ul>SIFT about 200~300 ms on a modern PC Structured light several seconds IEEE/RSJ IROS 2008, Sept 25
  7. 7. Motivation <ul><li>How to apply these state-of-the-art recognition methods to visual servo control? </li></ul><ul><li>How to overcome the time lag? </li></ul><ul><li>How to solve the real-time issue? </li></ul>IEEE/RSJ IROS 2008, Sept 25
  8. 8. Related Works <ul><li>Monocular </li></ul><ul><li>Model-based </li></ul><ul><li>Use keyframe information as prior knowledge </li></ul><ul><li>Use sparse bundle adjustment technique </li></ul>[ L. Vacchetti et al., PAMI 04 ] Input image should be close enough to the prior knowledge! IEEE/RSJ IROS 2008, Sept 25
  9. 9. Related Works <ul><li>Active Contour </li></ul><ul><li>Local curve fitting algorithm </li></ul><ul><li>Initialize by SIFT keypoint matching </li></ul>[G. Panin and A. Knoll, JMM 04 ] Potential danger in background having same color with tracking object! IEEE/RSJ IROS 2008, Sept 25
  10. 10. Our Idea <ul><li>Use prior knowledge (object models) </li></ul><ul><ul><li>2D images </li></ul></ul><ul><ul><li>3D points obtained from structured light system </li></ul></ul><ul><li>Use scale invariant feature matching for accurate initialization </li></ul><ul><li>Use KLT (Kanade-Lucas-Tomasi) tracker for fast local tracking </li></ul>IEEE/RSJ IROS 2008, Sept 25
  11. 11. System Overview <ul><li>Add text </li></ul>IEEE/RSJ IROS 2008, Sept 25
  12. 12. Two Modes <ul><li>Mono Mode </li></ul><ul><ul><li>Using mono camera </li></ul></ul><ul><ul><li>Better computational performance </li></ul></ul><ul><li>Stereo Mode </li></ul><ul><ul><li>Using stereo camera </li></ul></ul><ul><ul><li>More accurate pose result </li></ul></ul>IEEE/RSJ IROS 2008, Sept 25
  13. 13. Problem Definition – Mono Mode <ul><li>Given 2D-3D correspondences and a calibrated mono camera, find the pose of the object with respect to the camera. </li></ul>IEEE/RSJ IROS 2008, Sept 25
  14. 14. Problem Definition – Stereo Mode <ul><li>Given 3D-3D correspondences and a calibrated stereo camera, find the pose of the object with respect to the camera. </li></ul>IEEE/RSJ IROS 2008, Sept 25
  15. 15. Initial Pose Estimation <ul><li>Add text </li></ul>IEEE/RSJ IROS 2008, Sept 25
  16. 16. Initial Pose Estimation <ul><li>Extract SIFT keypoints </li></ul><ul><li>Matching with model knowledge </li></ul><ul><li>Estimate initial pose </li></ul><ul><li>Get a convex hull of a set of matched SIFT keypoints </li></ul><ul><li>Generate KLT tracking points within the convexhull </li></ul><ul><li>Calculate 3D coordinates of KLT points </li></ul>IEEE/RSJ IROS 2008, Sept 25
  17. 17. Initial Pose Estimation <ul><li>Mono Mode </li></ul><ul><ul><li>Use the POSIT algorithm ( 2D-3D ) </li></ul></ul><ul><li>Stereo Mode </li></ul><ul><ul><li>Use the closed-form solution using unit quaternions ( 3D-3D ) </li></ul></ul>R,t R,t IEEE/RSJ IROS 2008, Sept 25
  18. 18. Initial Pose Estimation <ul><li>Extract SIFT keypoints </li></ul><ul><li>Matching with model knowledge </li></ul><ul><li>Estimate initial pose </li></ul><ul><li>Get a convex hull of a set of matched SIFT keypoints </li></ul><ul><li>Generate KLT tracking points within the convexhull </li></ul><ul><li>Calculate 3D coordinates of KLT points </li></ul>IEEE/RSJ IROS 2008, Sept 25
  19. 19. Initial Pose Estimation <ul><li>3D coordinates of each KLT points are required for subsequent local pose estimation </li></ul><ul><li>Stereo Mode </li></ul><ul><ul><li>Straightforward in a calibrated stereo rig </li></ul></ul><ul><ul><li>Triangulate 3D points </li></ul></ul><ul><li>Mono Mode </li></ul><ul><ul><li>Use approximation with the knowledge of model </li></ul></ul><ul><ul><li>Get 3D coordinates by using three nearest neighboring SIFT points </li></ul></ul>IEEE/RSJ IROS 2008, Sept 25
  20. 20. Initial Pose Estimation + : SIFT points • : KLT points IEEE/RSJ IROS 2008, Sept 25
  21. 21. Initial Pose Estimation Treat the surface as locally flat IEEE/RSJ IROS 2008, Sept 25
  22. 22. Local Pose Estimation <ul><li>Add text </li></ul>IEEE/RSJ IROS 2008, Sept 25
  23. 23. Local Pose Estimation <ul><li>Estimate pose with KLT tracking points and their 3D points </li></ul><ul><li>Pose estimation algorithms are same </li></ul><ul><ul><li>Mono Mode </li></ul></ul><ul><ul><ul><li>Use the POSIT algorithm ( 2D-3D ) </li></ul></ul></ul><ul><ul><li>Stereo Mode </li></ul></ul><ul><ul><ul><li>Use the closed-form solution using unit quaternions ( 3D-3D ) </li></ul></ul></ul>R,t R,t IEEE/RSJ IROS 2008, Sept 25
  24. 24. Removing Outliers IEEE/RSJ IROS 2008, Sept 25
  25. 25. Outlier Handling <ul><li>KLT tracking points are easy to drift </li></ul><ul><li>Drifting points result in inaccurate pose </li></ul><ul><li>Use RANSAC to remove outlier </li></ul><ul><li>Re-initialize when there are no sufficient # of inliers </li></ul>IEEE/RSJ IROS 2008, Sept 25
  26. 26. Tracking Results IEEE/RSJ IROS 2008, Sept 25
  27. 27. Experiment Mono Mode Stereo Mode IEEE/RSJ IROS 2008, Sept 25
  28. 28. Tracking Results - translation IEEE/RSJ IROS 2008, Sept 25
  29. 29. Tracking Results - rotation IEEE/RSJ IROS 2008, Sept 25
  30. 30. RMS Error RMS errors over the whole sequence of image Z IEEE/RSJ IROS 2008, Sept 25
  31. 31. Computational Time Computational times of pose estimation IEEE/RSJ IROS 2008, Sept 25
  32. 32. Computational Time Computational times of each module IEEE/RSJ IROS 2008, Sept 25
  33. 33. Summary & Conclusion <ul><li>A method for tracking 3D roto-translation of rigid objects </li></ul><ul><ul><li>using scale invariant feature based matching </li></ul></ul><ul><ul><li>KLT (Kanade-Lucas-Tomasi) tracker </li></ul></ul><ul><li>Mono mode </li></ul><ul><ul><li>guarantees higher frame rate performance </li></ul></ul><ul><li>stereo mode </li></ul><ul><ul><li>shows better pose results </li></ul></ul>IEEE/RSJ IROS 2008, Sept 25
  34. 34. Future Work <ul><li>To decrease the computational burden </li></ul><ul><ul><li>Use GPU-based implementation of KLT tracker and SIFT </li></ul></ul><ul><ul><ul><li>GPU KLT </li></ul></ul></ul><ul><ul><ul><li>SiftGPU </li></ul></ul></ul><ul><ul><li>Unifying the contour based tracking </li></ul></ul>IEEE/RSJ IROS 2008, Sept 25
  35. 35. Thank you <ul><li>Any Questions? </li></ul><ul><li>Any Suggestions? </li></ul><ul><li>Any Comments? </li></ul>IEEE/RSJ IROS 2008, Sept 25

×