URAI 2012, Session TD4: Robot & Future Info Device   Calibration Issues in FRC:Camera, Projector, Kinematics based        ...
Agenda• Introduction: FRC• Example Application: Robotic Spatial AR (RSAR)• Calibration Issues: Camera, Projector, Kinemati...
ETRI FRC 2010      3    Joo-Haeng Lee (joohaeng at etri.re.kr)
ETRI FRC 2011      4    Joo-Haeng Lee (joohaeng at etri.re.kr)
ETRI FRC 2012      5    Joo-Haeng Lee (joohaeng at etri.re.kr)
ETRI FRC 2012• Major components for RSAR  • RSAR = Robotic Spatial Augmented Reality
ETRI FRC 2012• Major components for RSAR  • RSAR = Robotic Spatial Augmented Reality   Robotis                            ...
Examples of RSAR PixelFlex        Ubiquitous Display(MIT, 2001)   (Ritsumeikan Univ. , 2006) LuminAR             Beamatron...
Examples of RSAR PixelFlex        Ubiquitous Display(MIT, 2001)   (Ritsumeikan Univ. , 2006) LuminAR             Beamatron...
FRC RSARImage in the world:   Body outline                         8       Joo-Haeng Lee (joohaeng at etri.re.kr)
FRC RSARImage in the world:    Image from R Prj:   Body outline            Skeleton                               8       ...
FRC RSARImage in the world:    Image from R Prj:       Image from L Prj:   Body outline            Skeleton               ...
FRC RSAR
FRC RSAR
Motivation• Calibration really matters in RSAR!  • camera to capture the geometry of the world  • projector to display on ...
Calibration: Camera
Calibration: Camera• Camera Model  • Qc = Mc Xwc G    • Qc: image in the camera    • Mc: camera internal    • Xwc: camera ...
Calibration: Camera• Chang’s method in OpenCV  • internal and external parameters + lens distortion
Calibration: Camera• Chang’s method in OpenCV  • internal and external parameters + lens distortion• Issues  • geometric c...
Calibration: Projector
Calibration: Projector• Projection Model  • Qp = Mp Xwp Gp  • Gp = Xwp-1 Mp-1 Qp    • Qp: image to be projected    • Mp: p...
Calibration: Projector• Chang’s method in OpenCV  • If a projector is not moving or well aligned, we can    apply Chang’s ...
Calibration: Projector• Chang’s method in OpenCV  • If a projector is not moving or well aligned, we can    apply Chang’s ...
Calibration: Projector• Tsai’s method with custom implementation  • Can handle lens shift: no need to specify the image   ...
Calibration: Projector• Tsai’s method with custom implementation  • Can handle lens shift: no need to specify the image   ...
Calibration: Projector• Tsai’s method with custom implementation  • Can handle lens shift: no need to specify the image   ...
Calibration: Projector• Tsai’s method with custom implementation          Fig. An optical center of a projector (in green)...
Calibration: Projector-Camera• Projector-camera system  • Calibrated camera: Mc and Xwc  • Calibrated projector: Mp and Xwp
Calibration: Projector-Camera• Projector-camera system  • Calibrated camera: Mc and Xwc  • Calibrated projector: Mp and Xw...
Calibration: Projector-Camera• Projector-camera system  • Calibrated camera: Mc and Xwc  • Calibrated projector: Mp and Xw...
Calibration: Projector-Camera• Projector-camera system  • Calibrated camera: Mc and Xwc  • Calibrated projector: Mp and Xw...
Calibration: Projector-Camera• Projector-camera system  • Calibrated camera: Mc and Xwc  • Calibrated projector: Mp and Xw...
Calibration: Kinematics• Precise calibration of kinematics is required for  the quality of RSAR application in FRC  • (ex)...
Calibration: Kinematics• Need to consider the difference between the  CAD model and the actual assembly
Calibration: Kinematics• Vision-based kinematics calibration (in progress)
Calibration: Kinematics• Vision-based kinematics calibration (in progress)
Calibration: Kinematics• Vision-based kinematics calibration (in progress)
Calibration: Kinematics• Vision-based kinematics calibration (in progress)
Calibration: Kinematics• Vision-based kinematics calibration (in progress)
Summary
Summary• Calibration Process  • Camera: Chang’s method  • Projector: Tsai’s method  • Projector-Camera: Xcp  • Kinematics:...
Summary• Calibration Process  • Camera: Chang’s method  • Projector: Tsai’s method  • Projector-Camera: Xcp  • Kinematics:...
Q &Ajoohaeng at etri dot re dot kr
Memo
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approach (URAI 2012)
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approach (URAI 2012)
Upcoming SlideShare
Loading in …5
×

Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approach (URAI 2012)

980 views

Published on

Presentation slides for URAI 2012.
http://www.kros.org/urai2012/
http://www.kros.org/urai2012/04program07.php

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
980
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approach (URAI 2012)

  1. 1. URAI 2012, Session TD4: Robot & Future Info Device Calibration Issues in FRC:Camera, Projector, Kinematics based Hybrid Approach Joo-Haeng Lee ETRI Kosuke Maegawa Ritsumeikan University Jong-Seung Park Ritsumeikan University Joo-Ho Lee Ritsumeikan University
  2. 2. Agenda• Introduction: FRC• Example Application: Robotic Spatial AR (RSAR)• Calibration Issues: Camera, Projector, Kinematics• Summary• Q&A 2 Joo-Haeng Lee (joohaeng at etri.re.kr)
  3. 3. ETRI FRC 2010 3 Joo-Haeng Lee (joohaeng at etri.re.kr)
  4. 4. ETRI FRC 2011 4 Joo-Haeng Lee (joohaeng at etri.re.kr)
  5. 5. ETRI FRC 2012 5 Joo-Haeng Lee (joohaeng at etri.re.kr)
  6. 6. ETRI FRC 2012• Major components for RSAR • RSAR = Robotic Spatial Augmented Reality
  7. 7. ETRI FRC 2012• Major components for RSAR • RSAR = Robotic Spatial Augmented Reality Robotis Logitech Optoma Dynamixel HD Pro Webcam PK-320 MX-28 C920
  8. 8. Examples of RSAR PixelFlex Ubiquitous Display(MIT, 2001) (Ritsumeikan Univ. , 2006) LuminAR Beamatron(MIT; 2010) (Microsoft Research, 2012)
  9. 9. Examples of RSAR PixelFlex Ubiquitous Display(MIT, 2001) (Ritsumeikan Univ. , 2006) LuminAR Beamatron(MIT; 2010) (Microsoft Research, 2012)
  10. 10. FRC RSARImage in the world: Body outline 8 Joo-Haeng Lee (joohaeng at etri.re.kr)
  11. 11. FRC RSARImage in the world: Image from R Prj: Body outline Skeleton 8 Joo-Haeng Lee (joohaeng at etri.re.kr)
  12. 12. FRC RSARImage in the world: Image from R Prj: Image from L Prj: Body outline Skeleton Vessels 8 Joo-Haeng Lee (joohaeng at etri.re.kr)
  13. 13. FRC RSAR
  14. 14. FRC RSAR
  15. 15. Motivation• Calibration really matters in RSAR! • camera to capture the geometry of the world • projector to display on the real-world surface • kinematics to control and sense the motion
  16. 16. Calibration: Camera
  17. 17. Calibration: Camera• Camera Model • Qc = Mc Xwc G • Qc: image in the camera • Mc: camera internal • Xwc: camera external • G: geometry in the world
  18. 18. Calibration: Camera• Chang’s method in OpenCV • internal and external parameters + lens distortion
  19. 19. Calibration: Camera• Chang’s method in OpenCV • internal and external parameters + lens distortion• Issues • geometric constraints should be considered for precise calibration of other components such as kinematics
  20. 20. Calibration: Projector
  21. 21. Calibration: Projector• Projection Model • Qp = Mp Xwp Gp • Gp = Xwp-1 Mp-1 Qp • Qp: image to be projected • Mp: projector internal • Xwp: projector external • Gp: projected area in the world
  22. 22. Calibration: Projector• Chang’s method in OpenCV • If a projector is not moving or well aligned, we can apply Chang’s method as in the camera case
  23. 23. Calibration: Projector• Chang’s method in OpenCV • If a projector is not moving or well aligned, we can apply Chang’s method as in the camera case• Issues • However, for a moving projector, we need to handle lens shift, which cannot be solved using Chang’s.
  24. 24. Calibration: Projector• Tsai’s method with custom implementation • Can handle lens shift: no need to specify the image size
  25. 25. Calibration: Projector• Tsai’s method with custom implementation • Can handle lens shift: no need to specify the image size • Constrained concave programming based on Lagrangian multiplier method: Qp = P G
  26. 26. Calibration: Projector• Tsai’s method with custom implementation • Can handle lens shift: no need to specify the image size • Constrained concave programming based on Lagrangian multiplier method: Qp = P G • RQ decomposition: P = Mp Xwp
  27. 27. Calibration: Projector• Tsai’s method with custom implementation Fig. An optical center of a projector (in green) that is approximated from camera frustums from data set #2. Each frustum is aligned in the common coordinate frame of a camera. The optical center of a camera (in white) is the origin of the frame. Optical centers of a projector computed using the previous method (assuming no-lens shift) are in gray. A pair of red and blue points is the closest points between two frustum Extended rectangles (in orange) from the partial rectangles (in blue).(in gray) of a Fig. edges. The average is the Six optical centers approximate optical center of a projector assuming the no lens-shift. projector (in green). Computed using the fixed internal
  28. 28. Calibration: Projector-Camera• Projector-camera system • Calibrated camera: Mc and Xwc • Calibrated projector: Mp and Xwp
  29. 29. Calibration: Projector-Camera• Projector-camera system • Calibrated camera: Mc and Xwc • Calibrated projector: Mp and Xwp • Transformation between projector and camera
  30. 30. Calibration: Projector-Camera• Projector-camera system • Calibrated camera: Mc and Xwc • Calibrated projector: Mp and Xwp • Transformation between projector and camera • Xcp = Xwp Xcw = Xwp Xwc -1
  31. 31. Calibration: Projector-Camera• Projector-camera system • Calibrated camera: Mc and Xwc • Calibrated projector: Mp and Xwp • Transformation between projector and camera • Xcp = Xwp Xcw = Xwp Xwc -1 • Transformation from the world to the projector
  32. 32. Calibration: Projector-Camera• Projector-camera system • Calibrated camera: Mc and Xwc • Calibrated projector: Mp and Xwp • Transformation between projector and camera • Xcp = Xwp Xcw = Xwp Xwc -1 • Transformation from the world to the projector • Xwp(t) = Xcp Xwc(t)
  33. 33. Calibration: Kinematics• Precise calibration of kinematics is required for the quality of RSAR application in FRC • (ex) inverse kinematics
  34. 34. Calibration: Kinematics• Need to consider the difference between the CAD model and the actual assembly
  35. 35. Calibration: Kinematics• Vision-based kinematics calibration (in progress)
  36. 36. Calibration: Kinematics• Vision-based kinematics calibration (in progress)
  37. 37. Calibration: Kinematics• Vision-based kinematics calibration (in progress)
  38. 38. Calibration: Kinematics• Vision-based kinematics calibration (in progress)
  39. 39. Calibration: Kinematics• Vision-based kinematics calibration (in progress)
  40. 40. Summary
  41. 41. Summary• Calibration Process • Camera: Chang’s method • Projector: Tsai’s method • Projector-Camera: Xcp • Kinematics: vision-based approach
  42. 42. Summary• Calibration Process • Camera: Chang’s method • Projector: Tsai’s method • Projector-Camera: Xcp • Kinematics: vision-based approach• Future Works • Better precision from hybrid calibration approach
  43. 43. Q &Ajoohaeng at etri dot re dot kr
  44. 44. Memo

×