Seminar報告_20150520

701 views

Published on

2015.05.20的seminar報告,就當作提早把碩論要講的內容大致整理一次。

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
701
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Seminar報告_20150520

  1. 1. 3D Pose Estimation for Transparent Objects Presenter: 賴柏任 Advisor:羅仁權教授 05.20.2015
  2. 2. Motivation • Transparent objects are everywhere • If we know he pose, we can grasp it! 2
  3. 3. Problems 3 Color of transparent object changes Hard to locate transparent objects Edge of transparent objects are blur Hard to estimate pose of transparent objects
  4. 4. Effective cure 4 Color of transparent object changes Edge of transparent objects are blur
  5. 5. Kinect v.s. Color changes • Transparent objects produce NaN in depth map 5 Ref: I. Lysenkov and V. Rabaud, "Pose estimation of rigid transparent objects in transparent clutter," in Robotics and Automation (ICRA), 2013 IEEE International Conference on, 2013, pp. 162-169.
  6. 6. Graphcut v.s. Blur edge • Given foreground & background clue 6 Ref: C. Rother, V. Kolmogorov, and A. Blake, "Grabcut: Interactive foreground extraction using iterated graph cuts," ACM Transactions on Graphics (TOG), vol. 23, pp. 309-314, 2004.
  7. 7. Graphcut v.s. Blur edge • Generate the prob. distribution 7 Ref: C. Rother, V. Kolmogorov, and A. Blake, "Grabcut: Interactive foreground extraction using iterated graph cuts," ACM Transactions on Graphics (TOG), vol. 23, pp. 309-314, 2004.
  8. 8. Graphcut v.s. Blur edge • Use distance to compensate 8 Ref: C. Rother, V. Kolmogorov, and A. Blake, "Grabcut: Interactive foreground extraction using iterated graph cuts," ACM Transactions on Graphics (TOG), vol. 23, pp. 309-314, 2004.
  9. 9. Graphcut v.s. Blur edge • OpenCV implementation 9
  10. 10. A coarse pipeline 10 Detect NaN area in depth map Feed the area to Graphcut Segment the edge
  11. 11. How to determine pose? • Model-based matching • Rotate in x & y axis and store the edge 11 Z-axis Y-axis The problem becomes a 2D- 2D matching problem
  12. 12. Where is the model? • Kinect Fusion 12
  13. 13. Where is the model? Wrap your object with paper Use Kinect Fusion to construct the model Store the model 13
  14. 14. What if there are some other NaN objects? • Some non-transparent objects also produce NaN in depth map 14
  15. 15. What if there are some other NaN objects? • Use characteristics of transparent object to rule out non-transparent objects 15 Transparent objects produce highlights Color of transparent object is similar to peripheral area
  16. 16. What if there are some other NaN objects? • Transparent objects produce highlights 16 Ref: K. McHenry, J. Ponce, and D. Forsyth, "Finding glass," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 2005, pp. 973-979.
  17. 17. What if there are some other NaN objects? • Transparent objects produce highlights 17 Ref: K. McHenry, J. Ponce, and D. Forsyth, "Finding glass," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 2005, pp. 973-979. Threshold the image from 0-255 Compute the perimeter in each image Compute the threshold by line fitting (from 255 to 0)
  18. 18. What if there are some other NaN objects? • Color of transparent object is similar to peripheral area 18
  19. 19. What if there are some other NaN objects? • Color of transparent object is similar to peripheral area 19 Hue histogram
  20. 20. A fine pipeline 20
  21. 21. Some results • Pose Matching 21
  22. 22. Some results • Total retrieved candidates are over 200 22 Method Recall Precision Only NaN 86.11% 38.24% Characteristics 86.11% 93.93% Recall = (2/2)*100% =100% Precision=(2/5)*100% =40%
  23. 23. Some other problems • How to let robot grasp? • Is there any choice other from Kinect? 23
  24. 24. How to let robot grasp? • Teach and Play 24 Grasp points
  25. 25. Is there any choice other from Kinect? • Extract the visual word of transparent objects 25
  26. 26. Is there any choice other from Kinect? 26 Ref: M. Fritz, G. Bradski, S. Karayev, T. Darrell, and M. J. Black, "An additive latent feature model for transparent object recognition," in Advances in Neural Information Processing Systems, 2009, pp. 558-566.
  27. 27. Is there any choice other from Kinect? • The result can be the input of Graphcut 27 Ref: M. Fritz, G. Bradski, S. Karayev, T. Darrell, and M. J. Black, "An additive latent feature model for transparent object recognition," in Advances in Neural Information Processing Systems, 2009, pp. 558-566.
  28. 28. 28 Thank you!

×