Ch14.ppt

Digital Camera and Computer Vision Laboratory
Department of Computer Science and Information Engineering
National Taiwan University, Taipei, Taiwan, R.O.C.
Chapter 14
Analytic Photogrammetry
Presented by 王夏果 and Dr. Fuh
R94922103@ntu.edu.tw
0937384214

DC & CV Lab.
NTU CSIE
Analytic Photogrammetry
 Make inferences about :
 3D position
 Orientation
 Length of the observed 3D object parts
in a world reference frame from
measurements of one or more 2D-
perspective projections of a 3D object

DC & CV Lab.
NTU CSIE
Analytic Photogrammetry (cont.)
 These inference problems can be construed
as nonlinear least-square problems
 Iteratively linearize the nonlinear functions
from an initially given approximate solution

DC & CV Lab.
NTU CSIE
Photogrammetry
 Provide a collection of methods for
determining the position and orientation of
cameras and range sensors in the scene and
relating camera positions and range
measurements to scene coordinates
 GIS: Geographic Information System
 GPS: Global Positioning System

DC & CV Lab.
NTU CSIE
Exterior Orientation
 Determine position and orientation of camera
in absolute coordinate system from
projections of calibration points in scene
 The exterior orientation of the camera is
specified by all parameters of camera pose,
such as perspectivity center position, optical
axis direction.

DC & CV Lab.
NTU CSIE
Exterior Orientation (cont.)
 Exterior orientation specification: requires 3
rotation angles, 3 translations

DC & CV Lab.
NTU CSIE
Interior Orientation
 Determine internal geometry of camera
 The interior orientation of camera is specified
by all the parameters that determines the
geometry of 3D rays from measured image
coordinates

DC & CV Lab.
NTU CSIE
Interior Orientation (cont.)
 The parameters of interior orientation relate
the geometry of ideal perspective projection
to the physics of a camera.
 Parameters: camera constant, principal point,
lens distortion, …

DC & CV Lab.
NTU CSIE
Interior Orientation (cont.)
 With interior and external orientation, we can
complete specify the camera orientation.

DC & CV Lab.
NTU CSIE
Relative Orientation
 Determine relative position and orientation
between 2 cameras from projections of
calibration points in scene
 Calibrate relation between two cameras for
stereo
 Relates coordinate systems of two cameras
to each other, not knowing 3D points
themselves, only their projections in image

DC & CV Lab.
NTU CSIE
Relative Orientation (cont.)
 Assume interior orientation of each camera
known
 Specified by 5 parameters: 3 rotation angles,
2 translations

DC & CV Lab.
NTU CSIE
Absolute Orientation
 Determine transformation between 2
coordinate systems or position and
orientation of range sensor in absolute
coordinate system from coordinates of
calibration points
 Convert depth measurements in viewer-
centered coordinates to absolute coordinate
system for the scene

DC & CV Lab.
NTU CSIE
Absolute Orientation (cont.)
 Orientation of stereo model in world
reference frame
 Determine scale, 3 translations, 3 rotations
 Recovery of relation between two coordinate
system

DC & CV Lab.
NTU CSIE
Symbol Definition

DC & CV Lab.
NTU CSIE
Rotation Matrix

DC & CV Lab.
NTU CSIE
Rotation Matrix (cont.)

DC & CV Lab.
NTU CSIE
World Frame to Camera Frame
 (x, y, z)’ in world frame represented by
(p, q, s)’ in camera frame:

DC & CV Lab.
NTU CSIE
Pinhole Camera Projection
 Pinhole camera with image at distance f
from camera lens, projection:
where f is a camera constant, related to
focal length of lens

DC & CV Lab.
NTU CSIE
Principal Point
 Origin of measurement image plane
coordinate
 Represented by (u0, v0)

DC & CV Lab.
NTU CSIE
Perspective Projection Equations
 Collinearity equation:

DC & CV Lab.
NTU CSIE
Perspective Projection Equations
(cont.)
 Show that the relationship between the
measured 2D-perspective projection
coordinates and the 3D coordinates is a
nonlinear function of u0, v0, x0, y0, z0, ω, ψ,
and κ

DC & CV Lab.
NTU CSIE
Take a Break

DC & CV Lab.
NTU CSIE
Nonlinear Least-Square
Solutions
 Noise model:

DC & CV Lab.
NTU CSIE
Nonlinear Least-Square
Solutions (cont.)
 Maximum likelihood solution: β1, …, βM
maximize Prob(α1, …, αk | β1, …, βM )
 In other words, this solution minimizes
least-squares criterion:
where

DC & CV Lab.
NTU CSIE
First-Order Taylor Series
Expansion
 First-order Taylor series expansion of gk
taken around βt:

DC & CV Lab.
NTU CSIE
First-Order Taylor Series
Expansion (cont.)

DC & CV Lab.
NTU CSIE
Exterior Orientation Problem
 Determine the unknown rotation and
translation that put the camera reference
frame in the world reference frame.

DC & CV Lab.
NTU CSIE
Exterior Orientation Problem
(cont.)

DC & CV Lab.
NTU CSIE
One Camera Exterior Orientation
Problem
 Known: (xn, yn, zn)’ and (un, vn)’
(un, vn)’ is the corresponding set of 2D-
perspective projections, n = 1, …, N
 Unknown: (ω,ψ,κ) and (x0, y0, z0)’

DC & CV Lab.
NTU CSIE
Other Exterior Orientation
Problem
 Camera calibration problem: unknown
position of camera in object frame
 Object pose estimation problem: unknown
object position in camera frame
 Spatial resection problem in
photogrammetries: 3D positions from 2D
orientation

DC & CV Lab.
NTU CSIE
Nonlinear Transformation For
Exterior Orientation

DC & CV Lab.
NTU CSIE
Standard Solution
 By chain rule,

DC & CV Lab.
NTU CSIE
 In matrix form,

DC & CV Lab.
NTU CSIE
Standard Solution (cont.)

DC & CV Lab.
NTU CSIE
Auxiliary Solution
 Not iteratively adjust the angles directly
 Reorganize the calculation such that we
iteratively adjust the three auxiliary
parameters of a skew symmetric matrix
associated with the rotation matrix
 Then, we determine the adjustment of the
angles

DC & CV Lab.
NTU CSIE
Quaternion Representation
 From any skew symmetric matrix,
we can construct a rotation matrix R by
choosing scalar d: R = (dI + S)(dI - S)-1
which guarantees that R’R = I

DC & CV Lab.
NTU CSIE
Quaternion Representation (cont.)
 Expanding the equation for R:
parameters a, b, c, d can be constrained to
satisfy a2 + b2 + c2 + d2 = 1

DC & CV Lab.
NTU CSIE
Quaternion Representation (cont.)

DC & CV Lab.
NTU CSIE
Relative Orientation
 The transformation from one camera station
to another can be represented by a rotation
and a translation
 The relation between the coordinates, rl and
rr of a point P can be given by means of a
rotation matrix and an offset vector

DC & CV Lab.
NTU CSIE
 Relative orientation is typically with the
determination of the position and
orientation of one photograph with respect
to another, given a set of corresponding
image points

DC & CV Lab.
NTU CSIE
 Relative orientation specified by five
parameters: (yR - yL), (zR - zL), (ωR - ωL),
(ψR - ψL), (κR - κL)
 Assumption:
 Camera interior orientation known
 Image positions expressed to identical scale and
with respect to principal point

DC & CV Lab.
NTU CSIE
Standard Solution
 Let Q’L and Q’R be the rotation matrices with
the exterior orientation of the left and the
right image:

DC & CV Lab.
NTU CSIE
 fR: distance between right image plane and
right lens
fL: distance between left image plane and
left lens
 From perspective collinearity equation

DC & CV Lab.
NTU CSIE
 Hence,
where

DC & CV Lab.
NTU CSIE
Quaternion Solution
 Instead of determining the relative orientation
of the right image with respect to the left
image, we aligns a reference frame having its
x-axis along the line from the left image lens
to the right image lens

DC & CV Lab.
NTU CSIE
Quaternion Solution (cont.)
 The relative orientation is then determined by
the angles (ωR, ψR, κR), which rotate the right
image into this reference frame, and the
angles (ωL, ψL, κL), which rotate the left
image into this reference frame

DC & CV Lab.
NTU CSIE
Interior Orientation
 A camera is specified by:
 Camera constant f: distance between image
plane and camera lens
 Principal point (up, vp): intersection of optic axis
with image plane in measurement reference
frame located on image plane
 Geometric distortion characteristics of the lens;
assuming isotropic around the principal point

DC & CV Lab.
NTU CSIE
Stereo
 Optical axes parallel to one another and
perpendicular to baseline simple camera
geometry for stereo photography

DC & CV Lab.
NTU CSIE
Stereo (cont.)
 Parallax: deplacement in perspective
projection by position translation
(x, y, z): 3D point position
(uL, vL): perspective projection on left image
of stereo pair
(uR, vR): perspective projection on right image
of stereo pair
bx: baseline length in x-axis

DC & CV Lab.
NTU CSIE
Stereo (cont.)

DC & CV Lab.
NTU CSIE
Stereo (cont.)
 Relation is close to being useless in
real-world, because
 Observed perspective projections are subject to
measurement errors so that vL ≠ vR for corresponding
points
 Left and right camera frames may have slightly different
orientations
 When two cameras used, almost always fR ≠ fL

DC & CV Lab.
NTU CSIE
Relationship Between Coordinate
System
 The relationship between two coordinate
systems is easy to find if we can measure the
coordinates of a number of points in both
systems

DC & CV Lab.
NTU CSIE
Relationship Between Coordinate
System(cont.)
 It takes three measurements to tie two
coordinate systems together uniquely
 A single measurement leaves three degrees of
freedom motion
 A second measurement removes all but one
degree of freedom
 Third measurement rigidly attaches two
coordinate systems to each other

DC & CV Lab.
NTU CSIE
2D-2D Pose Detection Problem
 Determine from matched points more precise
estimate of rotation matrix R and translation t
such that yn = Rxn + t, n = 1, …, N
 Determine R and t that minimize weighted
sum of residual errors:

DC & CV Lab.
NTU CSIE
3D-3D Absolute Orientation
 We must determine rotation matrix R and
translation vector t satisfying
 Constrained least-squares problem to
minimize

DC & CV Lab.
NTU CSIE
3D-3D Absolute Orientation
(cont.)
 The least-square problem can be modeled by
a mechanical system in which corresponding
points in the two coordinate systems are
attached to each other by means of springs
 The solution to the least-squares problem
corresponds to the equilibrium position of the
system, which minimizes the energy stored in
the springs

DC & CV Lab.
NTU CSIE
Robust M-Estimation
 Least-squares techniques are ideal when
random data perturbations or measurement
errors are Gaussian distribution
 We need some robust techniques for
nonlinear regression

DC & CV Lab.
NTU CSIE
Robust M-Estimation (cont.)
 M-Estimator:

DC & CV Lab.
NTU CSIE
or

DC & CV Lab.
NTU CSIE
 ρ:
 Symmetric
 Positive-defined function
 Has unique minimum at zero
 Chosen to be less increasing than square

DC & CV Lab.
NTU CSIE

DC & CV Lab.
NTU CSIE
Error Propagation
 If we have the input parameter x1, …, xN ,
and random errors Δx1, …, ΔxN , the quantity
y depends on input parameters through
known function f: y = f(x1, …, xN ) will become
y + Δy= f(x1 +Δx1, …, xN +ΔxN )

DC & CV Lab.
NTU CSIE
Error Propagation Analysis
 Determines expected value and variance of
y + Δy
 Known information about Δx1, …, ΔxN :
mean and variance

DC & CV Lab.
NTU CSIE
Implicit Form
 A known function f has the form:
f(x1, …, xN, y) = 0
The quantities (x1 +Δx1, …, xN +ΔxN ) are
observed, and the quantity y + Δy is
determined to satisfy
f(x1 +Δx1, …, xN +ΔxN , y + Δy ) =0

DC & CV Lab.
NTU CSIE
Implicit Form: General Case
 General case: y is not a scalar but a L × 1
vector β
 x1, …, xN : are K N × 1 vectors representing
true values
 x1 +Δx1, …, xK +ΔxK : are K N × 1 vectors
representing noisy observed values
 Δx1, …, ΔxK : random perturbations
 β: a L × 1 vector representing unknown true
parameters

DC & CV Lab.
NTU CSIE
Implicit Form: General Case
 Noiseless model:
 With noisy observations, the idealized model:

DC & CV Lab.
NTU CSIE
Summary
 We have shown how to:
 Take a nonlinear least-squares problem
 Linearize it
 Solve by iteratively solving successive linearized
least-squares problems

Ch14.ppt

Recommended

Recommended

More Related Content

Similar to Ch14.ppt

Similar to Ch14.ppt (20)

Recently uploaded

Recently uploaded (20)

Ch14.ppt