Introduction to RoboticsPerception II<br />CSCI 4830/7000<br />February 15, 2010<br />NikolausCorrell<br />
Review: Sensing<br />Important: sensors report data in their own coordinate frame<br />Examples from the exercise<br />Acc...
Today<br />Perception using vision<br />Practical angle:<br />Why is vision hard<br />Basic image processing<br />How to c...
Why is Vision Hard?The difference between seeing and perception.<br />Gary Bradski, 2009<br />4<br />What to do?  <br />Ma...
<ul><li>Depth discontinuity
Surface orientation discontinuity
Reflectance discontinuity (i.e., change in surface material properties)
Illumination discontinuity (e.g., shadow)</li></ul>Slide credit: Christopher Rasmussen<br />5<br />But, What’s an Edge?<br />
To Deal With the Confusion, Your Brain has Rules...That can be wrong<br />
We even see invisible edges…<br />
And surfaces …<br />
We need to deal with 3D Geometry<br />9<br />Perception is ambiguous … depending on your point of view!<br />Graphic by Ga...
And Lighting in 3D<br />Which square is darker?<br />
Lighting is Ill-posed …<br />Perception of surfaces depends on lighting assumptions<br />11<br />Gary Bradski (c) 2008<br ...
Contrast<br />12<br />Which one is male and which one is female?<br />Illusion by: Richard Russell,Harvard University<br /...
Frequency<br />
Color<br />http://briantobin.info/2009/06/lost-and-found-visual-illusion.html<br />
Pin-hole Model<br />
Pin-Hole Camera<br />A. Efros<br />
Aperture<br />
Increasing Aperture: Lens<br />
Thin Lens<br />Objects need to have the right distance to be in focus -&gt; Depth-from-Focus method<br />
Thresholds<br />20<br />20<br />http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm<br />Screen shots by Gary Bradski, 20...
Canny Edge Detector<br />21<br />Gary Bradski (c) 2008<br />21<br />
Morphological Operations Examples<br />Morphology - applying Min-Max. Filters and its combinations<br />Dilatation IB<br ...
Stereo Calibration<br />Screen shots  and charts by Gary Bradski, 2005<br />Gary Bradski (c) 2008<br />23<br />23<br />
3D Stereo Vision<br />Find Epipolar lines:<br />Align images:<br />Triangulate points:<br />Depth:<br />
Example: Tomato-Picking Robot<br />Challenges<br />Foliage<br />Reflections<br />Varying size and shape<br />Varying color...
Upcoming SlideShare
Loading in …5
×

Lecture 05: Vision

584 views
483 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
584
On SlideShare
0
From Embeds
0
Number of Embeds
52
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture 05: Vision

  1. 1. Introduction to RoboticsPerception II<br />CSCI 4830/7000<br />February 15, 2010<br />NikolausCorrell<br />
  2. 2. Review: Sensing<br />Important: sensors report data in their own coordinate frame<br />Examples from the exercise<br />Accelerometer of Nao<br />Laser scanner<br />Treat like forward kinematics<br />
  3. 3. Today<br />Perception using vision<br />Practical angle:<br />Why is vision hard<br />Basic image processing<br />How to combine image processing primitives into object recognition<br />OpenCV / SwisTrack<br />
  4. 4. Why is Vision Hard?The difference between seeing and perception.<br />Gary Bradski, 2009<br />4<br />What to do? <br />Maybe we should try to find edges ….<br />Gary Bradski, 2005<br />
  5. 5. <ul><li>Depth discontinuity
  6. 6. Surface orientation discontinuity
  7. 7. Reflectance discontinuity (i.e., change in surface material properties)
  8. 8. Illumination discontinuity (e.g., shadow)</li></ul>Slide credit: Christopher Rasmussen<br />5<br />But, What’s an Edge?<br />
  9. 9. To Deal With the Confusion, Your Brain has Rules...That can be wrong<br />
  10. 10. We even see invisible edges…<br />
  11. 11. And surfaces …<br />
  12. 12. We need to deal with 3D Geometry<br />9<br />Perception is ambiguous … depending on your point of view!<br />Graphic by Gary Bradski<br />
  13. 13. And Lighting in 3D<br />Which square is darker?<br />
  14. 14. Lighting is Ill-posed …<br />Perception of surfaces depends on lighting assumptions<br />11<br />Gary Bradski (c) 2008<br />11<br />
  15. 15. Contrast<br />12<br />Which one is male and which one is female?<br />Illusion by: Richard Russell,Harvard University<br />Russell, R. (2009) A sex difference in facial pigmentation and its exaggeration by cosmetics. Perception, (38)1211-1219<br />
  16. 16. Frequency<br />
  17. 17. Color<br />http://briantobin.info/2009/06/lost-and-found-visual-illusion.html<br />
  18. 18. Pin-hole Model<br />
  19. 19. Pin-Hole Camera<br />A. Efros<br />
  20. 20. Aperture<br />
  21. 21. Increasing Aperture: Lens<br />
  22. 22. Thin Lens<br />Objects need to have the right distance to be in focus -&gt; Depth-from-Focus method<br />
  23. 23. Thresholds<br />20<br />20<br />http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm<br />Screen shots by Gary Bradski, 2005<br />
  24. 24. Canny Edge Detector<br />21<br />Gary Bradski (c) 2008<br />21<br />
  25. 25. Morphological Operations Examples<br />Morphology - applying Min-Max. Filters and its combinations<br />Dilatation IB<br />Opening IoB= (IB)B<br />Erosion IB<br />Image I<br />Closing I•B= (IB)B<br />TopHat(I)= I - (IB)<br />BlackHat(I)= (IB) - I<br />Grad(I)= (IB)-(IB)<br />
  26. 26. Stereo Calibration<br />Screen shots and charts by Gary Bradski, 2005<br />Gary Bradski (c) 2008<br />23<br />23<br />
  27. 27. 3D Stereo Vision<br />Find Epipolar lines:<br />Align images:<br />Triangulate points:<br />Depth:<br />
  28. 28. Example: Tomato-Picking Robot<br />Challenges<br />Foliage<br />Reflections<br />Varying size and shape<br />Varying color<br />Partly covered fruits<br />http://swistrack.sourceforge.net<br />N. Correll, N. Arechiga, A. Bolger, M. Bollini, B. Charrow, A. Clayton, F. Dominguez, K. Donahue, S. Dyar, L. Johnson, H. Liu, A. Patrikalakis, T. Robertson, J. Smith, D. Soltero, M. Tanner, L. White, D. Rus. Building a Distributed Robot Garden. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1509-1516, St. Louis, MO.<br />
  29. 29. Filter-based object recognition<br />Filter image<br />Sobel<br />Hough transform<br />Color<br />Spectral highlights<br />Size and shape<br />Weighted sum of filters highlights object location<br />Color<br />Sobel<br />Hough<br />Spectral<br />Highlights<br />
  30. 30. Group exercise<br />Object recognition<br />Goal<br />Players<br />Ball<br />Field<br />
  31. 31. Homework<br />Read sections 4.2-5 (pages 145-180)<br />Questionnaire on CU Learn<br />

×