Asian Conference on Design and Digital Engineering 2012 (ACDDE 2012)                Geometric Computing and CAD Workshop  ...
Outline• Problem definition• Outline of proposed solution• Illustrative example• Theory: coupled line cameras• Experimental...
QUIZ 1. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?
QUIZ 1. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a)    ...
QUIZ 1. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?       ...
QUIZ 1. You have some image quadrilaterals     taken from a camera. Which of the following     is the image of any rectang...
QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?
QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a)    ...
Problem Definition• Given: (1) a single image of a scene  rectangle of an unknown aspect ratio; (2)  a simple camera model ...
Problem Definition• Given: (1) a single image of a scene  rectangle of an unknown aspect ratio; (2)  a simple camera model ...
Proposed Solution                    Joo-Haeng Lee (joohaeng@etri.re.kr)
Proposed Solution1. An analytic solution based on coupled   line cameras is provided for the   constrained case where the ...
Proposed Solution1. An analytic solution based on coupled   line cameras is provided for the   constrained case where the ...
Proposed Solution3. We also provide a determinant to tell if an   image quadrilateral is a projection of any   scene recta...
Proposed Solution3. We also provide a determinant to tell if an   image quadrilateral is a projection of any   scene recta...
Illustrative Example                   Joo-Haeng Lee (joohaeng@etri.re.kr)
l is a projection of a scene rectangle.                                                                         G         ...
diagonmple what we can do!                                                       Constracamera      Illustrative= Example ...
•  Image center on the                    two lin                 principal axis                         share t          ...
Illustrative Example3. Find a centered quad Q using the   vanishing points of Qg   w1                                     ...
is given,                           Qg                                     Soluti                                         ...
Q                           Expere if the   Illustrative Example           •  Determinant: D                              ...
vertic                                                               relaticonstruct Illustrative Example                 ...
Line Camera              Joo-Haeng Lee (joohaeng@etri.re.kr)
aint!@etri.re.kr          !       Line Camera I, KOREA !         • Given: (1) 1D image of a scene line            denoted ...
pc (1) 1D image of a                  Line Cameraine denoted by l0 and                                           l2       ...
pc (1) 1D image of aine denoted by l0 and  Line Camera              l2                                                   y...
Coupled Line Cameras                  Joo-Haeng Lee (joohaeng@etri.re.kr)
se                                       l0 + l 2e camera       Coupled Line Cameras       • Given:v(1) a centered quad Q;...
Cameras a special pin-hole camera model!       Coupled Line Cameras                                          pc           ...
Cameras adspecial = G; (3) a each                     cos θpin-hole θ1 G mmodel!                           Q 30           ...
Cameras a special  G; (3) a                     3                        Q pin-hole camera model!  G                      ...
Cameras a special pin-holeu camera model!should                           0                                u             1...
Coupled Line Cameras    • Solution: an analytic solution to the pose       estimation of coupled line cameras             ...
pc scene, which will be projected as a line u0 u2 in the line camera C0 . Especially, we are interested in the posi-      ...
QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a)    ...
Qg is given,                         Qg            Solu                                                   solu       QUIZ ...
QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a)    ...
of G: φ (1) for each raint:                              cos θ 0 cos θ1 v                            u3 d = 2             ...
QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?       ...
Experimental Results• Synthetic Data• Real Data                    Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data•                Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data•                Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
Real Data            Joo-Haeng Lee (joohaeng@etri.re.kr)
d; (2) these               u0                                               u1      u3ras shouldcipal axis.               ...
0                        1   i         i                                    → G → pc               Synthetic Dataperforman...
Gerr; (3)         5                  4vi-vm|, φ ,       3                  2                  1                           ...
Gerr; (3)         5                  4vi-vm|, φ ,       3                  2                  1                           ...
Summary• We proposed an analytic solution to  reconstruct a scene rectangle of an  unknown aspect ratio from a single  ima...
AcknowledgementThis research has been partially supported byKMKE & KRC 2010-ZC1140 and KMKE ISTDPNo.10041743              ...
AcknowledgementThis research has been first presented at ICPR2012 (Int. Conf. Pattern Recognition), Tsukuba,Japan. Nov., 20...
Q &Ajoohaeng at etri dot re dot kr                                 Joo-Haeng Lee (joohaeng@etri.re.kr)
Memo
Note on Coupled Line Cameras for Rectangle Reconstruction (ACDDE 2012)
Note on Coupled Line Cameras for Rectangle Reconstruction (ACDDE 2012)
Note on Coupled Line Cameras for Rectangle Reconstruction (ACDDE 2012)
Upcoming SlideShare
Loading in …5
×

Note on Coupled Line Cameras for Rectangle Reconstruction (ACDDE 2012)

620 views
532 views

Published on

The presentation file for the talk in ACDDE 2012.
http://www.acdde2012.org/
It deals with the research result published in ICPR 2012 with the title as "Camera Calibration from a Single Image based on Coupled Line Cameras and Rectangle Constraint"
https://iapr.papercept.net/conferences/scripts/abstract.pl?ConfID=7&Number=70

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
620
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Note on Coupled Line Cameras for Rectangle Reconstruction (ACDDE 2012)

  1. 1. Asian Conference on Design and Digital Engineering 2012 (ACDDE 2012) Geometric Computing and CAD Workshop Dec.6-8, 2012, Niseko, Hokkaido, JapanNote on Coupled Line Cameras for Rectangle Reconstruction Joo-Haeng Lee Robot & Cognitive Systems Dept. ETRI
  2. 2. Outline• Problem definition• Outline of proposed solution• Illustrative example• Theory: coupled line cameras• Experimental results• Q&A Joo-Haeng Lee (joohaeng@etri.re.kr)
  3. 3. QUIZ 1. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?
  4. 4. QUIZ 1. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a) Rhombus (b) Parallelogram Isosceles(c) Trapezoid___ (d) Trapezoid
  5. 5. QUIZ 1. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle? Parallelogram Isosceles (d) Trapezoid
  6. 6. QUIZ 1. You have some image quadrilaterals taken from a camera. Which of the following is the image of any rectangle? Reconstructed Rectangle Given Image Quadrilateral v0 v3 u0 u3 l0 l3 r f l1 l2 Parallelogramu1 u2 v1 v2 Isosceles (d) Trapezoid Reconstructed Projective Structure
  7. 7. QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?
  8. 8. QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a) Rhombus (b) Parallelogram Isosceles(c) (d) Trapezoid
  9. 9. Problem Definition• Given: (1) a single image of a scene rectangle of an unknown aspect ratio; (2) a simple camera model with unknown parameter values Joo-Haeng Lee (joohaeng@etri.re.kr)
  10. 10. Problem Definition• Given: (1) a single image of a scene rectangle of an unknown aspect ratio; (2) a simple camera model with unknown parameter values• Problem: (1) to reconstruct the projective structure including the scene rectangle; (2) to calibrate unknown camera parameters Joo-Haeng Lee (joohaeng@etri.re.kr)
  11. 11. Proposed Solution Joo-Haeng Lee (joohaeng@etri.re.kr)
  12. 12. Proposed Solution1. An analytic solution based on coupled line cameras is provided for the constrained case where the center of a scene rectangle is projected to the image center. Joo-Haeng Lee (joohaeng@etri.re.kr)
  13. 13. Proposed Solution1. An analytic solution based on coupled line cameras is provided for the constrained case where the center of a scene rectangle is projected to the image center.2. By prefixing a simple pre-processing step, we can solve the general cases without the centering constraint above. Joo-Haeng Lee (joohaeng@etri.re.kr)
  14. 14. Proposed Solution3. We also provide a determinant to tell if an image quadrilateral is a projection of any scene rectangle. Joo-Haeng Lee (joohaeng@etri.re.kr)
  15. 15. Proposed Solution3. We also provide a determinant to tell if an image quadrilateral is a projection of any scene rectangle.4. We present the experimental results of the proposed method with synthetic and real data. Joo-Haeng Lee (joohaeng@etri.re.kr)
  16. 16. Illustrative Example Joo-Haeng Lee (joohaeng@etri.re.kr)
  17. 17. l is a projection of a scene rectangle. G Illustrative Exampletrate the performance of the proposedh synthetic and real data. Q pa a 1. Assume a simple camera model with diExample what we can do! (ex) pinhole camera unknown parameters: Cmple camera •  Square pixel: fx= fy diunknown •  No skew: s = 0 ca •  Image center on the tw principal axis shage So Joo-Haeng Lee (joohaeng@etri.re.kr)
  18. 18. diagonmple what we can do! Constracamera Illustrative= Example •  Square pixel: f f x y diagonown •  No skew: s = 0 can be 2. Given an image quadrilateral Qg, •  Image center on the two lin principal axis share th s given, Qg Solutio solution estimat camera uad Qng points Joo-Haeng Lee (joohaeng@etri.re.kr)
  19. 19. •  Image center on the two lin principal axis share t Illustrative Example s given, Find a centered quad Q using the 3. Qg Solutio vanishing points of Qg solutio estimat camera uad Qng points Q Exper Synthe e if the •  Determinant: D ! 100 rand Q is A +A ±2 A A Joo-Haeng Lee (joohaeng@etri.re.kr)
  20. 20. Illustrative Example3. Find a centered quad Q using the vanishing points of Qg w1 w0 u3 u3 g Qg u2 g um Q u0 u0 u1 Joo-Haeng Lee (joohaeng@etri.re.kr)
  21. 21. is given, Qg Soluti solutio Illustrative Example estima camerquad Qng points can determine if the the centered 4. We quad Q is the image of a scene rectangle.Expe Q Synthne if the •  Determinant: D ! 100 ra d Q is A0 + A1 ± 2 A0 A1 Gref; (cene D± = F1(li ) = >0 A1 − A0 within vertice relativ p , an Joo-Haeng Lee (joohaeng@etri.re.kr)
  22. 22. Q Expere if the Illustrative Example •  Determinant: D Synthet 100 ran Q is ! A0 + A1 ± 2 A0 A1 Gref; (2) ne 5. If so, ±weF1(li ) = D = can reconstruct the scene >0 A1 inA0 metric sense within rectangles, G and Gg, − a before camera calibration. vertices relativenstruct pc, andemetric Real: (1era Gg G with a k ratio is desk: A Joo-Haeng Lee (joohaeng@etri.re.kr)
  23. 23. vertic relaticonstruct Illustrative Example pc, anene a metricFinally, we can calibrate camera 6. Real: Gg Gmera parameters: (1) focal length f, (2) external with params: [R|T] ratio desk: calibrate (2) in ters: recon calibs: [R|T] Joo-Haeng Lee (joohaeng@etri.re.kr)
  24. 24. Line Camera Joo-Haeng Lee (joohaeng@etri.re.kr)
  25. 25. aint!@etri.re.kr ! Line Camera I, KOREA ! • Given: (1) 1D image of a scene line denoted by l0 and l2; (2) the principalpecial linear camera model! axis passes through the center m of a scene line. pce of aby l0 and y2 y 0 l2axis l0 enter of s2 d s0 q0 v2 m v0 Joo-Haeng Lee (joohaeng@etri.re.kr)
  26. 26. pc (1) 1D image of a Line Cameraine denoted by l0 and l2 y2 y 0he principal axis l0 s2 dthrough the center an analytic solution to the pose0 of s • Solution: q0 line. estimation of a line camera v2 v0 mn: an analytic l0 − l 2 cos θ 0 = d = dα 0n to the pose l0 + l 2ion of a line camera v2 m v0 c Joo-Haeng Lee (joohaeng@etri.re.kr)
  27. 27. pc (1) 1D image of aine denoted by l0 and Line Camera l2 y2 y 0he principal axis l0 s2 dthrough the center an analytic solution to the pose0 of s • Solution: q0 line. estimation of a line camera v2 v0 mmera model!n: an analytic l0 − l 2 pc cos θ 0 = d = dα 0n to the pose l0 + l 2ion of a line camera y 2 y0 l2 l0 s2 d s0 q0 v2 m v0 v2 m v0 c Joo-Haeng Lee (joohaeng@etri.re.kr)
  28. 28. Coupled Line Cameras Joo-Haeng Lee (joohaeng@etri.re.kr)
  29. 29. se l0 + l 2e camera Coupled Line Cameras • Given:v(1) a centered quad Q; (2) the 2 m v0 c principal axis passes through the center Cameras a φ"of G pin-hole camera model! of a scene rectangle G; (3) a diagonal angle special pc u1 red quad Qal axis l1 re center of u2 l2 l0 u0 v1 v0 G; (3) a Q l3 G m G: φ u3 v2 v3 Joo-Haeng Lee (joohaeng@etri.re.kr)
  30. 30. Cameras a special pin-hole camera model! Coupled Line Cameras pc u1red quad Qal axis l1 r center• Constraint: (1)0 for each diagonal of Q, a of u2 l2 l u0 v1 v0G; (3) a line camera lcan be defined; (2) these two Q 3 G mG: φ line cameras share a2 principal axis.v3 u3 v each cos θ 0 cos θ1 d= = = F2 (θ 0 ,θ1 ,li )ne camera α0 α1) these u0 u1 u3hould θ0 axis. u2 v0 v1 θ1 v2 v3 Joo-Haeng Lee (joohaeng@etri.re.kr)
  31. 31. Cameras adspecial = G; (3) a each cos θpin-hole θ1 G mmodel! Q 30 cos camera φ G: camera = = F2 (θ 0 ,θ1 ,li )ne α 0 u3 αv2 v3 pc 1 these red quad each hould Coupled Line Cameras d= ucos θ u0 1 0 cos θ1 = u1 Q=u3F2 (θ 0 ,θ1 ,li )al axisine camera u l1 αr α1 θ0 0 axis. of u2 l v0 v1 θ1 center• Solution:2an analytic 1solution to the pose 2 u02) these u0 l0 v v0G; (3) a estimation of3 coupled line 3camerasv should v2 Q l u1 G u mG: φ 3 θ0 u v0 l axis. u2 3 v2 v1 θ1 v3 ic θ0 tan θ = cos θ) = D± cos 0 F1(li eeach vd = 2 2 = 1 = F2 (θ 0 ,θ1 ,li ) v3 ne camera led line α0 α1ytic θ 0 → d θ 0 θ 1 → ψ i → si → φ → ) these u0 tan = F1(li ) = D± se hould →G→ c 2 p u1 u3pled line axis. u2 θ → θ0 d →vθ → ψ θ1 s → φ 0 → 0 1 v1 i ierformance of the proposed method! →G→ p v2 c v3 Joo-Haeng Lee (joohaeng@etri.re.kr)
  32. 32. Cameras a special G; (3) a 3 Q pin-hole camera model! G m came G: φ u3 v2 v3d quad Q Coupled Line Cameras pc ucos θ cos θ1hingquad red points each 1 Q= F (θ ,θ ,l ) d= 0 =al axis ine camera l1 αr 0 Qα1 2 0 1 i Expe center• Solution:l2an analytic 1solution to the pose2) these of u2 u l0 u0 v v0 0G; (3) a estimation of3 coupled line 3cameras should Q l u1 G u Synt ine if the u •  Determinant: D mG: φ l axis. !2 θ0 u v0 3 v2 v1 θ1 v3 100uad Q is A0 + A1 ± 2 A0 A1 Gref;scene each D± = cosiθ 0= cos θ1 vd = F (l ) = > 03 2 1 = A (θ 0 ,θ1 ,li ) v A1 − F20 with ne camera α0 α1 vertiytic θ0 ) these u0 tan = F1(li ) = D± relat se hould 2 u1 u3 construct u2pled line pc, a axis. θ0 → θ0 d →vθ → ψ θ1 s → φ 0 v1 i → i ene 1 a metric v2 → G → pc v3 Joo-Haeng Lee (joohaeng@etri.re.kr)
  33. 33. Cameras a special pin-holeu camera model!should 0 u 1 3 θ0l axis. u2 v0 θ1 Coupled Line Cameras pc v1 u1red quad Qal axis v2 l1 r v3 center• Solution:l2an θl0 u0 v1solution to the pose tic of u2 analytic v0G; (3) a estimation of3 2 = F1(li line cameras tanl = ) G D± 0 e Q coupled mG: φ linepled u3 v2 v3 θ 0 → d → θ 1 → ψ i → si → φ each cos θ 0 cos θ1 d = → G → pc = = F2 (θ 0 ,θ1 ,li ) ne camera α0 α1 ) these erformance of the proposed method! u3 u0 u1 hould θ0 axis.erated u2 v0 v1 θ1 Qg Ggngles: v2 Q v3 Joo-Haeng Lee (joohaeng@etri.re.kr)
  34. 34. Coupled Line Cameras • Solution: an analytic solution to the pose estimation of coupled line cameras s0 Ψ0 x k Β Θ0 pc t0 Α0 d y zQ Φ G t1 Α1 Ψ1 Θ1 s1 Ρ Joo-Haeng Lee (joohaeng@etri.re.kr)
  35. 35. pc scene, which will be projected as a line u0 u2 in the line camera C0 . Especially, we are interested in the posi- Ψ2 Ψ 0 rectangle G in a pin-hole camera with the center of pro- l2 tion pc and the orientation θ0 of C0 when the principal l0 jection at pc . Note that the principal axis passes through s2 d s0 axis passes through the center vm of v0 v2 and the center vm , um and pc . Θ um = (0, 0, 1)0of image line. v2 v0 v Using this configuration of coupled line cameras, we m 2 m v0 c To simplify the formulation, we assume a canon- find the orientation θi of each line camera Ci and the ical configuration where Trajectory of the centerand vm is (a) Line camera (b) vm vi = 1, of projection length d of the common principal axis Line Camera C1 (a) Pin-hole Camera (b) Line Camera C0 (c) from a given placed at the1:origin of the worldacoordinate system: Figure A configuration of line camera quadrilateral H. Using the lengths of partial diagonals, vm = (0, 0, 0). For derivation, we define followings: Figure 2: A pin-hole camera and its decomposition into li = um ui , we can find the relation between the cou- coupled line cameras. d = pc vm , li = um ui , ψi = ∠vm pc vi , and pled cameras Ci from (1): s i = pc v i . tan ψ1 l1 sin θ1 (d − cos θ0 )scene,this configuration, we can derive theu2 in the line In which will be projected as a line u0 following re- = = (3)camera C0 . Especially, we are interested in the posi- lation: tan ψpin-hole camera (d − the center of pro- rectangle G in a 0 l0 sin θ0 with cos θ1 ) l2tion pc and the orientation cosof0C0 when the principal = d−θ θ 0 = d0 (1) jection at pc . Note that the principal axis passes through Manipulation of (2) and (3) leads to the system of non-axis passes through l0 thed + cosvθ0 of vd1 2 and the center center m 0v vm , um and pc . linear equations:um = (0, 0, 1)d − cos θ0 = s0 cos ψ0 and d1 = d + where d0 = of image line. Using this configuration of coupled line cameras, we β sin θ0 cos θ1 − cos θ0 sin θ1 cos θ cos θ1 cos θ0 simplify the . formulation, we assume a between To = s2 cos ψ2 We can derive the relation canon- find the orientation θi of each line = d= camera 0 i and the C=icaland d from (1): where vm vi = 1, and vm is θ0 configuration length d of β sincommon θ1 the θ0 − sin principal axisα0 α1 from a given (4)placed at the origin of the world coordinate system: quadrilateral H. Using the lengths of partial diagonals, where β = l1 /l0 . Using (4), the orientation θ0 can bevm = (0, 0, θ0 =For(l0 − l2 )/(l0we ldefine followings: cos 0). d derivation, + 2 ) = d α0 (2) li = um ui , we can find the relation between the cou- represented with coefficients, α0 , α1 , and β, that ared = pc vm , li = um ui , ψi = ∠vm pc vi , and pled cameras Ci from (1): solely derived from a quadrilateral H:si = pαvi = (li − li+2 )/(li + li+2 ), which is solely where c i . tan ψ1 l1 sin θ1 (d − cos θ0 ) derived from a image line ui ui+2 . Note following re- In this configuration, we can derive the that θ0 and d = + A ± 2√ A A = θ0 ψ0 A0 l0 1 sin θ0 (d0− 1 θ1 ) (3) are sufficient parameters to determine the exact positionlation: tan tan = cos = D± (5) l2 of pc in 2D. When α0 d − cos θpc is defined on a certain is fixed, 0 d0 2 A 1 − A0 = = (1) Manipulation of (2) and (3) leads to the system of non- sphere as in Fig l0 Once θi and d are 1 1b. d + cos θ0 d known, additional where linear equations:where d0 =can − cos θ0determined:ψtan ψi d1 sin θi /d parameters d be also = s cos 0 0 and = = d+ A0 sin B0cos2B1 , cos θ0 1 =θB0= cos 1 0 = cos θ1 β = θ0 + θ1 − A sin 1 − 2Bθcos θsi = s cos/ sinWe. can derive the relation between and = sin θi ψ . ψi 0 2 2 d=θ0 and d from (1): B0 = 2(α0 − 0 −(α0 θ1 α1 ) − 4α0 (α1 − 1)2 βα1 β sin θ 1)2 sin + 2 2 2 α0 22.2. Coupled Line Cameras (4) (α0 − . 2 (α0 (4), the orientation θ0 can be where1β== l1 /l0 1)Using− α1 )(α0 + α1 ) B cos θ = d (l − l )/(l + l ) = d α (2)
  36. 36. QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a) Rhombus (b) Parallelogram Isosceles(c) (d) Trapezoid
  37. 37. Qg is given, Qg Solu solu QUIZ 2. You have some image quadrilaterals estim taken from a camera. Which of the following camd quad Q is the image of any rectangle?hing points Q Exp (a) Rhombus (b) Parallelogram Syntmine if the •  Determinant: D ! 100uad Q is A0 + A1 ± 2 A0 A1 Gref; scene D± = F1(li ) = >0 A1 − A0 Isosceles with (c) (d) verti Trapezoid relat p,a
  38. 38. QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle?(a) Rhombus (b) Parallelogram D>0 D<0 Isosceles(c) (d) Trapezoid D<0 D<0
  39. 39. of G: φ (1) for each raint: cos θ 0 cos θ1 v u3 d = 2 v = = F2 (θ 0 ,θ1 ,l 3 nal of Q, a line cameraθ cos 0 cos θ1 α0 α1ordefined; (2) 2. You have some imageFquadrilaterals e each QUIZ these d= = u0 = 2 (θ 0 ,θ1 ,li ) α0 α1 line camerashould camera. Which of the following u1 u3 ne cameras from a taken (2) these the imageu0 anyurectangle? v0 is the principal axis. of 2 θ0 v1 θ1 u1 u3 s should θ0pal axis. u2 v v20 v1 θ1 on: an analytic Rhombus v2 θ0 (a) tan Parallelogram = D± = F1(li ) v3on to the pose D>0 θ0 2lytic of coupled linetan = F (l ) = Dationose 2 θ 01 → d → θ±1 → ψ i → si → φ irasupled line → G → pc φ θ → d →θ →ψ → s → Isosceles 0 1 i Trapezoid i → G → pc riments performance of the proposed method!
  40. 40. QUIZ 2. You have some image quadrilateralstaken from a camera. Which of the followingis the image of any rectangle? v0 v3 f(a) Rhombus Parallelogram D>0 v1 v2 Isosceles Trapezoid
  41. 41. Experimental Results• Synthetic Data• Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  42. 42. Real Data• Joo-Haeng Lee (joohaeng@etri.re.kr)
  43. 43. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  44. 44. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  45. 45. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  46. 46. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  47. 47. Real Data• Joo-Haeng Lee (joohaeng@etri.re.kr)
  48. 48. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  49. 49. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  50. 50. Real Data Joo-Haeng Lee (joohaeng@etri.re.kr)
  51. 51. d; (2) these u0 u1 u3ras shouldcipal axis. Synthetic Data u2 θ0 v0 v1 θ1 v2 v3nalytic θ0 1. Generated 100 random rectangles Gref and corresponding image quads = refD± tan = F1(li ) Q ; pose 2coupled 2. Get image θ → d →by adding noisesφto Qref line quad Qg θ → ψ → s → within dmax 0 pixels; 1 i i → G → pc 3. Reconstruct Gg from Qg;ts performance of the errors between Gref and G 4. Measured proposed method! g. generated Gg Qgectangles: Qd noises Gref G Joo-Haeng Lee (joohaeng@etri.re.kr)
  52. 52. 0 1 i i → G → pc Synthetic Dataperformance of the proposed method!nerated Gg Qgangles: Qoises Gref Gs to the Error H%L 6 err; (3) 5 4 -vm|, φ , 3 2 1 dmax 1 2 3 |vi-vm| φ pc f Aspect Ratiogle 1.46 Ê Ê ‡ Raw Compensated 1.45 Joo-Haeng Lee (joohaeng@etri.re.kr)
  53. 53. Gerr; (3) 5 4vi-vm|, φ , 3 2 1 1 Real Data |vi-vm| 2 φ pc 3 f dmax Aspect Rationgle 1. A rectangle with a known aspect ratio is 1.46 Ê Ê ‡ Raw Compensated pect moving on a desk: (ex) A4-sized paper; 1.45 1.44 Ê Ê 1.43on the 2. Take pictures to get 9 image quads; Ê ‡ ‡ 1.42 Ê ‡ ‡ Ê ‡φ = 1.414 1.41 ‡ Ê ‡ ‡ Ê 1.40 Ê ‡ y 3. Reconstructed and calibrated for each 1 2 3 4 5 6 7 8 Rect 9 ID case. Reconstructed aspect ratio: φ Merged frustums dcases. A moving A4 paper 1 2 3 4 5 6 7 8 9 Joo-Haeng Lee (joohaeng@etri.re.kr)
  54. 54. Gerr; (3) 5 4vi-vm|, φ , 3 2 1 1 Real Data |vi-vm| 2 φ pc 3 f dmax Aspect Rationgle 1.46 Ê Ê ‡ Raw Compensated 1.45 pect 1.44 Ê Ê 1.43on the Ê ‡ ‡ 1.42 Ê ‡ ‡ Ê ‡φ = 1.414 1.41 ‡ Ê ‡ ‡ Ê 1.40 Ê ‡ Rect y 1 2 3 4 5 6 7 8 9 ID Reconstructed aspect ratio: φ Merged frustums dcases. A moving A4 paper 1 2 3 4 5 6 7 8 9 Joo-Haeng Lee (joohaeng@etri.re.kr)
  55. 55. Summary• We proposed an analytic solution to reconstruct a scene rectangle of an unknown aspect ratio from a single image quadrilateral.• Our method is based on novel formulation of coupled line cameras and rectangle constraint. Joo-Haeng Lee (joohaeng@etri.re.kr)
  56. 56. AcknowledgementThis research has been partially supported byKMKE & KRC 2010-ZC1140 and KMKE ISTDPNo.10041743 Joo-Haeng Lee (joohaeng@etri.re.kr)
  57. 57. AcknowledgementThis research has been first presented at ICPR2012 (Int. Conf. Pattern Recognition), Tsukuba,Japan. Nov., 2012. Poster #5, Session TuPSAT2, ICPR 2012! 21st International Conference on Pattern Recognition (ICPR 2012) November 11-15, 2012. Tsukuba, Japan Camera Calibration from a Single Image based on ! Coupled Line Cameras and Rectangle Constraint! Joo-Haeng Lee joohaeng@etri.re.kr ! Robot & Cognitive Systems Dept., ETRI, KOREA! Camera Calibration from a Single Image based on Coupled Line Cameras and Rectangle Constraint Summary! Line Camera a special linear camera model! pc Given: (1) 1D image of a Given: (1) an image of a scene rectangle of an unknown scene line denoted by l0 and y2 y Joo-Haeng Lee aspect ratio; (2) a simple camera model with unknown l2 0 l2; (2) the principal axis l0 Robot and Cognitive Systems Research Dept., ETRI parameter values: focal length, position, and orientation s2 d s0 passes through the center m Problem: (1) to reconstruct the projective structure q0 joohaeng@etri.re.kr of a scene line v0v2. v2 v0 m including the scene rectangle; (2) to calibrate unknown camera parameters Solution: an analytic l0 − l 2 cos θ 0 = d = dα 0 solution to the pose l0 + l 2 Proposed Solution: Abstract Several approaches are based on geometric prop- estimation of a line camera erties of a rectangle or a parallelogram. Wu et al. 1.  Analytic solution based on coupled line cameras is Given a single image of a scene rectangle of an un- proposed a calibration method based on rectangles of provided when the center of a scene rectangle is known aspect ratio and size, we present a method to known aspect ratio [4]. Li et al. designed a rectangle projected to the image center. v2 m v0 c reconstruct the projective structure and to find camera landmark to localize a mobile robot with an approxi- 2.  By prefixing a simple pre-processing step, we can solve parameters including focal length, position, and orien- mate rectangle constraint, which does not give an ana- the general cases without the centering constraint. tation. First, we solve the special case when the center lytic solution [5]. Kim and Kweon propose a method to Coupled Line Cameras a special pin-hole camera model! 3.  We also provide a determinant to tell if an image of a scene rectangle is projected to the image center. We estimate intrinsic camera parameters from two or more quadrilateral is a projection of a scene rectangle. pc u1 Given: (1) a centered quad Q formulate this problem with coupled line cameras and rectangle of unknown aspect ratio [6]. A parallelogram 4.  We demonstrate the performance of the proposed Q; (2) the principal axis l1 r present the analytic solution for it. Then, by prefixing constraint can be used for calibration, which generally method with synthetic and real data. passes through the center m u2 l2 l0 u0 v1 v0 a simple preprocessing step, we solve the general case requires more than two scene parallelograms projected of an unknown scene l3 G Q m without the centering constraint. We also provides a in multiple-view images as in [7, 8, 9]. rectangle G. (Diag. angle=φ ) determinant to tell if an image quadrilateral is a pro- Our contribution can be summarized as follows. Illustrative Example what we can do! u3 v2 v3 Constraint: (1) for each cos θ 0 cos θ1 jection of a rectangle. We demonstrate the performance Based on a geometric configuration of coupled line d= = = F2 (θ 0 ,θ1 ,li ) of the proposed method with synthetic and real data. cameras, which models a simple camera of an unknown (1) Assume a simple camera •  Square pixel: fx= fy diagonal of Q, a line camera α0 α1 model with unknown •  No skew: s = 0 can be defined; (2) these u0 focal length, we give an analytic solution to reconstruct u1 u3 parameters. •  Image center on the two line cameras should a complete projective structure from a single image of θ0 principal axis share the principal axis. u2 v0 v1 θ1 1. Introduction an unknown rectangle in the scene: no prior knowledge is required on the aspect ratio and correspondences. v2 v3 (2) When an image Camera calibration is one of the most classical topics Then, the reconstruction result can be utilized in finding quadrilateral Qg is given, Solution: an analytic θ0 Qg tan = F1(li ) = D± in computer vision research. We have an extensive list the internal and external parameters of a camera: focal solution to the pose 2 of related works providing mature solutions. In this pa- length, rotation and translation. The proposed solution estimation of coupled line θ 0 → d → θ 1 → ψ i → si → φ per, we are interested in a special problem to calibrate a also provides a simple determinant to tell if an image cameras quadrilateral is a projection of a scene rectangle. (3) Find a centered quad Q → G → pc camera from a single image of an unknown scene rect- In a general framework for plane-based camera cal- using the vanishing points angle. We do not assume any prior knowledge on cor- ibration, camera parameters can be found first using of Qg. Q Experiments performance of the proposed method! respondences between scene and image points. Due to the limited information, a simple camera model is used: the image of the absolute conic (IAC) and its relation Synthetic: (1) generated Gg the focal length is the only unknown internal parameter. with projective features such as vanishing points [2, 10]. (4) We can determine if the •  Determinant: D Qg ! 100 random rectangles: Q Then, a scene geometry can be reconstructed using a the centered quad Q is A0 + A1 ± 2 A0 A1 G The problem in this paper has a different nature with Gref; (2) added noises Gref the image of a scene D± = F1(li ) = >0 Error H%L classical computer vision problems. In the PnP prob- non-linear optimization on geometric constrains such A1 − A0 within dmax pixels to the orthogonality, which cannot be formulated as a closed- rectangle.! vertices of Gref; (3) 6 lem, we find transformation matrix between the scene 5 4 3 and the camera frames with prior knowledge on the form in general. relative errors between 2 1 correspondences between the scene and image points (5) If so, we can reconstruct Gref and reconstructed 1 2 3 dmax the centered scene Gg: |vi-m|, φ , pc, and f. |vi-m| φ pc f as well as the internal camera parameters [1]. In cam- 2. Problem Formulation era resection, we find the projection matrix from known rectangle Gg in a metric Real: (1) a rectangle Aspect Ratio 1.46 Ê Raw Gg G Ê sense before camera ‡ Compensated 1.45 correspondences between the scene and image points 2.1. Line Camera with a known aspect 1.44 calibration. Ê Ê 1.43 ratio is moving on the Ê without prior knowledge of camera parameters [2]. 1.42 ‡ ‡ Ê ‡ ‡ Ê ‡ desk: A4 paper φ = 1.414; 1.41 ‡ Ê ‡ ‡ Ê Camera self-calibration does not rely on a known Eu- A line camera is a conceptual camera, which follows 1.40 Ê ‡ (6) Finally, we can calibrate (2) independently 1 2 3 4 5 6 7 8 Rect 9 ID clidean structure, but requires multiple images from the same projection model of a standard pin-hole cam- camera parameters: Reconstructed aspect ratio: φ Merged frustums camera motion [3]. era. (See Fig 1a.) Let v0 v2 be a line segment in the reconstructed and •  focal length: f calibrated for 9 cases. A moving 1 2 3 4 A4 paper •  external params: [R|T] 5 6 7 8 9 978-4-9906441-1-6 ©2012 IAPR 758 Joo-Haeng Lee (joohaeng@etri.re.kr)
  58. 58. Q &Ajoohaeng at etri dot re dot kr Joo-Haeng Lee (joohaeng@etri.re.kr)
  59. 59. Memo

×