1. Trifocal tensor Three-View Geometry
Encapsulates the projective geometry relations between 3 views.
Independent of scene structure.
C OMPUTER V ISION : T HREE -V IEW G EOMETRY Analogous to fundamental matrix.
Depends only on the relative pose between the three cameras
and the internal parameters of the cameras.
IIT Kharagpur Can be uniquely determined by
� Camera matrices
Computer Science and Engineering, OR
� Point correspondences between the images.
Indian Institute of Technology
Kharagpur.
U SAGE OF T RIFOCAL T ENSOR
Transfer points from a correspondence in two views to the
corresponding point in a third view.
Transfer lines .....
1 / 85 2 / 85
Trifocal tensor Three-View Geometry Trifocal tensor Three-View Geometry
W HAT WE ARE INTERESTED IN ?
Homography between two of the views induced by a plane
back-projected from a line in the other view.
Relations between image correspondences between points and
lines.
Retrieval of the fundamental matrices.
Retrieval of the camera matrices.
Consider the set of corresponding lines l ↔ l� ↔ l�� .
The planes back-projected from l� l� � l�� are incident on the space
line L.
This is the GEOMETRIC INCIDENCE RELATION for corresponding
lines.
3 / 85 4 / 85
Trifocal tensor Three-View Geometry Trifocal tensor Three-View Geometry
� �� I NTERSECTION OF PLANES :
P = [ I | 0] P = [A | a4 ] P = [B | b4 ]
� �� �
B� l��
� � � �
l A l
The camera matrices for the 3 views are taken as P� P� � P�� . � = P� l = �� = P�� l� = ��� = P��� l�� =
0 a � l�
4
b� l��
4
a4 and b4 are the epipoles in views 2n� and 3r� , arising from the 1st
camera.
All the 3 planes intersect in a common line.
These epipoles are denoted as e� and e�� .
Algebraically it means that the 4 × 3 matrix M = [�� �� � ��� ] has
e� = P� C� e�� = P�� C rank 2.
C is the center of the 1st camera. Points on a line can be represented as: X = αX� + βX2 where X�
The back-projected planes can be written as: and X2 are linearly independent.
� � � �� � � � �� � For the line of intersection L of the 3 planes we have
l A l B l �� X = ��� X = ���� X = 0
� = P� l = �� = P�� l� = ��� = P��� l�� =
0 � l�
a4 b� l��
4 Given M = [�� �� � ��� ] , we have M� X = 0.
Hence M� X� = M� X2 = 0
5 / 85 6 / 85

2. Trifocal tensor Three-View Geometry Trifocal tensor Three-View Geometry
I NTERSECTION OF PLANES : I NTERSECTION OF PLANES :
� �� � � �� �
B� l�� B� l��
� � � � � � � �
l A l l A l
� = P� l = �� = P�� l� = ��� = P��� l�� = � = P� l = �� = P�� l� = ��� = P��� l�� =
0 a � l�
4
b� l��
4 0 a � l�
4
b� l��
4
l A� l� B� l�� l A� l� B� l��
� � � �
� �
M4×3 = [m� � m2 � m3 ] = M X� = M X2 = 0 M4×3 = [m� � m2 � m3 ] = M� X� = M� X2 = 0
0 a� l� b� l��
4 4 0 a� l� b� l��
4 4
Applying values of α = k�b� l�� ) and β = −k �a� l� )
The condition M� X� = M� X2 = 0 for two linearly independent 4 4
vectors X� and X2 implies that M has a two dimensional null
l = αA� l� + βB� l�� = �b� l�� )A� l� − �a� l� )B� l��
4 4
space.
= �l��� b4 )A� l� − �l�� a4 )B� l��
This implies there is linear dependence on the columns of M, i.e.
m� = αm2 + βm3 li = l��� �b4 a� )l� − l�� �a4 b� )l��
i i
Applying this to M gives: 0 = αa� l�
4
+ βb� l�� .
4
Thus α = k �b� l�� ) and β = −k �a� l� )
4 4
= l�� �ai b� )l�� − l�� �a4 b� )l��
4 i
7 / 85 8 / 85
Trifocal tensor Three-View Geometry Trifocal tensor Three-View Geometry
I NTERSECTION OF PLANES : � �� � ��
li = l�� �a i b4 )l − l�� �a 4 bi )l
� �� �
B� l��
� � � �
l A l
� = P� l = �� = P�� l� = ��� = P��� l�� = Ti = ai b� − a4 b�
0 a � l�
4
b� l��
4
4 i
li = l�� Ti l��
l A� l� B� l��
� �
� �
M4×3 = [m� � m2 � m3 ] = M X� = M X2 = 0 The set of 3 matrices {T1 � T2 � T3 } constitute the trifocal tensor.
0 a� l� b� l��
4 4
The ensemble of matrices [T1 � T2 � T3 ] can be denoted as [Ti ] .
li = l�� �ai b� )l�� − l�� �a4 b� )l��
4 i l� = l�� [T1 � T2 � T3 ] l��
Ti = ai b� − a4 b�
4 i where
li = l�� T i l�� l�� [T1 � T2 � T3 ] l�� represents �l�� T1 l�� � l�� T2 l�� � l�� T3 l�� )
The set of 3 matrices {T1 � T2 � T3 } constitute the trifocal tensor.
9 / 85 10 / 85
Trifocal tensor Three-View Geometry
l� = l�� [Ti ] l��
� ��
l = l [T1 � T2 � T3 ] l ��
l�� = l� [T� ] l��
i
l��� = l� [T�� ] l�
i
The three tensors [Ti ] [T� ] [T�� ] exist, but are distinct.
i i
All three tensors can be computed from any one of them.
Matrix elements [Ti ] are independent of the form of cameras.
The simple formula for computing the trifocal tensor
Ti = ai b� − a4 b�
4 i
is valid only for chosen canonical cameras:
P = [ I | 0] P� = [A | a4 ] P�� = [B | b4 ]
11 / 85 12 / 85

3. Trifocal tensor Three-View Geometry Homographies induced by a plane Three-View Geometry
D EGREES OF F REEDOM
l� = l�� [Ti ] l��
� ��
l = l [T1 � T2 � T3 ] l ��
l�� = l� [T� ] l��
i
l��� = l� [T�� ] l�
i
The trifocal tensor consists of three 3 × 3 matrices: [Ti ] [T� ] [T�� ] .
i i
Three 3 × 3 matrices have 27 dofs =⇒ 26 independent ratios.
Three camera matrices have 11 dofs each. Hence 33 dofs.
The projective world frame is not to be specified for trifocal tensor.
Hence 15 dofs can be subtracted from 33.
We are left with 33-15 = 18 dofs
Number of independent algebraic constraints satisfied by the
trifocal tensor: 26-18 = 8.
13 / 85 14 / 85
Homographies induced by a plane 3-View Geometry Homographies induced by a plane 3-View Geometry
Using li = l�� Ti l�� and l = H� l� we get
H = [h1 � h2 � h3 ] with hi = T� l�
i
This H is the homography H13 between the 1st and the 3r� views
Consider a 3D line L and its projection as image plane lines
induced by the line l� in the 2n� image.
l� l� � l�� . The trifocal tensor satisfies the line incidence relation:
� �
H13 �l� ) = T� � T� � T� l�
li = l�� Ti l�� 1 2 3
A line in the 2n� view can be back-projected to a plane in 3-space.
Likewise the homography between the 1st and the 2n� view,
This plane induces a homography between the 1st and the 3r�
induced by a line in the 3r� view is given as:
views.
H12 �l�� ) = [T1 � T2 � T3 ] l��
x�� = Hx l�� = H−� l l = H� l��
15 / 85 16 / 85
Point and line Incidence 3-View Geometry Point-line-line Relationship 3-View Geometry
The trifocal tensor relation l� = l�� [T1 � T2 � T3 ] l�� involves
homogeneous quantities and holds only up to scale.
To make this relation independent of the scale factor we can take
the cross product:
� �
l�� [T1 � T2 � T3 ] l�� [l]× = 0�
Likewise we can have:
� � � �
l��� T� � T� � T� l� [l]× = 0�
1 2 3
�
We shall discuss 3 types of I NCIDENCE R ELATIONS : �
Point-line-line correspondence A 3D line L maps to l� and l�� in the 2n� and 3r� images and to a
Point-line-point correspondence line passing through x in the 1st image.
� �
The point x on the line must satisfy x� l = i x i li = 0
�
3-point correspondence
17 / 85 18 / 85

4. Point-line-line Relationship 3-View Geometry Point-line-point Relationship 3-View Geometry
x� l i
�
The point x on the line must satisfy = =0 i x li
Since li = l�� Ti l�� , we have i x i l�� Ti l�� = 0, i.e.
�
�� �
l�� iT l�� = 0
ix i
iT
�
where � ix i ) is simply a 3 × 3 matrix.
There exists a 3D
point X which maps
to x in the 1st image
and to points on the
lines l� and l�� in the
2n� and 3r� images. The 3D point X maps to points x and x�� on 1st and 3r� images and
to a point on the line l� in the 2n� image.
19 / 85 20 / 85
Point-line-point Relationship 3-View Geometry Point-point-point Relationship 3-View Geometry
The 3D point X maps to points x and x�� on 1st and 3r� images and to a
point on the line l� in the 2n� image.
� 
�� �
�
� � � � � �
� � i �  �
x = H13 �l ) x = T1 l � T2 l � T3 l x =  x Ti  l

 
 

i
This is valid for any line l� passing through x� in the 2n� image.
The homogeneous scale factor may be eliminated by post
multiplying the transpose of both sides by [x�� ]×
�� �
x��� [x�� ]× = l�� iT [x�� ]× = 0�
ix i
� 
� i  ��
[x� ]×  x Ti  [x ]× = 03�3

 

 
i
21 / 85 22 / 85
Point-point-point Relationship 3-View Geometry Summary of relations 3-View Geometry
��
iT
� Line-line-line: l ↔ l� ↔ l��
[x� ]× ix i [x�� ]× = 03�3 How?
l�� [T1 � T2 � T3 ] l�� = l�
Any line l� passing through x� can be written as:
� �
l� = x� × y� = [x� ]× y� for some point y� on l� l�� [T1 � T2 � T3 ] l�� [l]× = 0�
By the point-line-point relation we have:
�  �  Point-line-line: x ↔ l� ↔ l��
� i  ��
�� 
� i  ��
l    [x ] = y�� [x� ] 
x Ti 
  x Ti  [x ] = 0�
 ��
iT
�
× × × l�� l�� = 0
ix
   
 i
i i
This is true for all lines l� through x� , hence independent of y� .
�� �
[x� ]× iT [x�� ]× = 03�3
ix i
23 / 85 24 / 85

5. Summary of relations 3-View Geometry Point to note Caution
Point-line-point: x ↔ l� ↔ x�� The lines l� l� � l�� are projections of 3D line L.
The points x� x� � x�� are projections of 3D point X.
�� �
l�� iT [x�� ]× = 0�
ix i
l ↔ l� ↔ l��
Point-point-line: x ↔ x� ↔ l�� Implies that there exists a 3D line L which projects to l� l� � l�� in the
�� � 1st , 2n� , 3r� views respectively.
[x� ]× iT l�� = 0
ix i x ↔ x� ↔ x��
Implies that there exists a 3D line X which projects to x� x� � x�� in
the 1st , 2n� , 3r� views respectively.
Point-point-point: x ↔ x� ↔ x��
�� �
[x� ]× iT [x�� ]× = 03�3
ix i
25 / 85 26 / 85
Point to note Caution Incidence of X and L Caution
The lines l� l� � l�� are projections of 3D line L.
The points x� x� � x�� are projections of 3D point X.
x ↔ l� ↔ l��
Implies that there exists a 3D line L which projects to l� � l�� in the
2n� , 3r� views, and to a line passing through x in the 1st view. The
3D point X corresponding to x may or may not lie on the 3D line L.
x ↔ l� ↔ x��
Implies that there exists a 3D point X which projects to x� x�� in the
1st and 3r� views and to a point lying on line l� in the 1st view. The
line l� is a projection of some 3D line L.
We cannot say whether X lies on L Entities satisfying a tensor relation do not guarantee incidence in
3-space.
Incidence of L and X is not guaranteed for x ↔ l� ↔ x�� relation
27 / 85 28 / 85
Three-view Geometry Epipolar Geometry 3-View Geometry
Next �
−→ �
Extracting Epipolar Lines �♣�
Extracting Fundamental matrix
Retrieving Camera matrices
� �
Consider the plane �� back-projected from l� . If this passes
through the 1st camera center C then it is the epipolar plane for the
1st and 2n� views.
Suppose X is a point on �� . The image of this point is x� x� in the
two views.
29 / 85 30 / 85

6. Epipolar Geometry 3-View Geometry Epipolar Geometry 3-View Geometry
Point-line-line: x ↔ l� ↔ l�� �� �
l�� iT l�� = 0
ix i
If the 3D line L corresponding to back-projection of l� and l�� lies
on the epipolar plane �� for the 1st and 2n� views, then the above
relation will be satisfied for any line l�� . Hence:
�� �
l�� iT = 0�
ix i
This is valid even when the roles of l� and l�� are reversed.
A plane ��� back-projected from a line l�� in the 3r� image will
intersect the plane �� in a line L. ��
i
�
�� �
i x Ti l = 0
The ray back-projected from point x must intersect this 3D line L
We shall use the correspondence x ↔ l� ↔ l��
31 / 85 32 / 85
Epipolar Geometry 3-View Geometry Epipolar Geometry 3-View Geometry
�� � �� �
l�� iT = 0� iT l�� = 0�
ix i ix i
The two relations indicate that the epipolar lines can be computed
�� �
as the left and right null vectors of the matrix i
i x Ti .
The epipole can be computed as the intersection of 3 different The epipole e� The epipole e��
epipolar lines. Choose 3 points x
x
� i The common intersection of The common intersection of
i x Ti
�1� 0� 0) � T1 lines represented by left null lines represented by the right
�0� 1� 0)� T2 vectors of the Ti ’s null vectors of the Ti ’s.
�0� 0� 1)� T3
The left null spaces of T1 � T2 � T3 would give the 3 epipolar lines.
The epipole e� in the 2n� image is the common intersection of
these epipolar lines
33 / 85 34 / 85
Algebraic Properties Ti matrices Algebraic Properties Ti matrices
The left null vector of Ti is l� = e� × ai
i Each matrix Ti has rank 2. This is because Ti = ai e��� − e� b� is
i
This gives the epipolar line in the 2n� view for the points the sum of two outer products.
x = �1� 0� 0)� � �0� 1� 0)� � �0� 0� 1)� as i = 1� 2� 3
The epipole e� is the common intersection of the epipolar lines l�
�� �
The sum of the matrices iT also has rank 2.
i ix i
for i = 1� 2� 3
�� �
The left null vector of the sum iT is the epipolar line l� of x
ix i
The right-null vector of Ti is = l�� e��
× bi
i in the 2n� view.
This gives the epipolar line in the 3r� view for the points ��
iT
�
x = �1� 0� 0)� � �0� 1� 0)� � �0� 0� 1)� as i = 1� 2� 3 The right null vector of the sum ix i is the epipolar line l�� of
The epipole e�� is the common intersection of the epipolar lines l�� x in the 3r� view.
i
for i = 1� 2� 3
35 / 85 36 / 85

7. Three-view Geometry Extracting Fundamental matrices Ti matrices
Consider a point x in the 1st view.
A line l�� in the 3r� view induces a homography H12 from the 1st to
the 2n� view as given by: refer slide 16
Next �
−→ � x� = �[T1 � T2 � T3 ] l�� ) x
Extracting Epipolar Lines The epipolar line corresponding to x is the line joining x� to the
Extracting Fundamental matrix �♣� epipole e� .
Retrieving Camera matrices l� = [e� ]× �[T1 � T2 � T3 ] l�� ) x
� � Hence
F21 = [e� ]× [T1 � T2 � T3 ] l��
F21 is the F matrix between 1st and 2n� views.
F31 is the F matrix between the 1st and 3r� views.
37 / 85 38 / 85
Extracting Fundamental matrices Ti matrices Three-view Geometry
� ��
F21 = [e ]× [T1 � T2 � T3 ] l
This formula for F21 is valid for any choice of l�� . However we must
avoid the degenerate case when Ti l�� = 0, i.e. l�� lies in the null Next �
−→ �
space of any of the Ti .
The right-null vector of each Ti is the epipolar line l�� = e�� × bi . Extracting Epipolar Lines
i
Extracting Fundamental matrix
Hence if we choose the vector e�� for l�� , then we are guaranteed Retrieving Camera matrices �♣�
� �
that l�� will be perpendicular to the right null space of each Ti .
� ��
F21 = [e ]× [T1 � T2 � T3 ] e
Likewise we have the formula for F31
� �
F31 = [e�� ]× T� � T� � T� e�
1 2 3
39 / 85 40 / 85
Retrieving camera matrices P� P� � P�� matrices Retrieving camera matrices P� P� � P�� matrices
Trifocal tensor is independent of the 3-D projective
P = [ I | 0] P� = [[T1 � T2 � T3 ] e�� | e� ]
transformations.
Hence the camera matrices can be retrieved only up to a
projective ambiguity. We choose P� P� to be consistent with fundamental matrix F21 .
The first camera can be chosen as P = [ I | 0] Having chosen P� P� , the two cameras P� P� have now established
Since F21 = [e� ]× [T1 � T2 � T3 ] e�� is known, the second camera can some projective world frame.
be taken as: The third camera P�� must be consistent with this projective frame.
P� = [[T1 � T2 � T3 ] e�� | e� ]
P = [ I | 0] P� = [A | a4 ] P�� = [B | b4 ]
R ECALL
If the F matrix can be written as F = [[m]× M] then the cameras can be Since we have chosen P� , each ai = Ti e�� and a4 = e�
� �
chosen as P = [ I | 0] and P� = [M | m] Also, since F31 = [e�� ]× T� � T� � T� e� , we know b4 = e��
1 2 3
How do we choose P�� = [B | b4 ] ?
41 / 85 42 / 85

8. Retrieving camera matrices P� P� � P�� matrices Retrieving camera matrices P� P� � P�� matrices
P = [ I | 0] P� = [A | a4 ] P�� = [B | b4 ] P = [ I | 0] P� = [A | a4 ] P�� = [B | b4 ]
We found � �
We know that: ai = Ti e�� , a4 = e� and b4 = e�� what is bi ? bi = e�� e��� − I T� e�
i and b4 = e��
Substituting this in the trifocal tensor relation Hence �� � � �
P�� = e�� e��� − I T� e� � e��
�
i
Ti = ai b�
4 − a 4 b�
i
� � �
we get Ti = Ti e�� e��� − e� b� P = [ I | 0] P� = [T1 � T2 � T3 ] e�� � e�
�
i
This gives e� b� = Ti �e�� e��� − I )
i
Multiplying on the left by e� � we get: e� � e� b� = e� � Ti �e�� e��� − I )
i
Normalizing such that e� � e� = 1 gives b� = e� � Ti �e�� e��� − I )
i
All these 3 cameras are projectively equivalent, i.e. they are consistent
Taking the transpose of both sides: bi = �e�� e��� − I ) T� e� in terms of the projective world frame.
i
43 / 85 44 / 85
Tensor Notation Introduction
Representation of geometric entities (like points, lines, etc.)
depends on the basis vectors.
The way we represent a quantity with a tensor depends on the
Next �
−→ � way it gets transformed when the basis gets transformed�
Tensor Notation �♣� B ASIS VECTORS ei
� � Consider a set of basis vectors ei , i = 1� . . . � 3 for a 2-dimensional
projective space IP2 .
P OINTS x
With respect to this basis, a point in IP2 is represented by a set of
coordinates x i ,
x = 3 x i ei
�
i�1
� ��
We represent this as x = x1 � x2 � x3 [superscript index]
45 / 85 46 / 85
T RANSFORMATION OF B ASIS Tensor Notation Introduction
We transform ei to a new basis ˆj = i Hi ei where H is the basis
�
e j
transformation matrix with entries Hi .
j Superscript/ Subscript/ convention
Indices which transform according to H−1 are written as
T RANSFORMATION OF P OINT
super-scripts. They are the contravariant indices.
With respect to the new basis the point x transforms to
� �� Indices which transform according to H or H� are written as
� = ˆ1 � ˆ2 � ˆ3
x x x x sub-scripts. They are covariant indices.
� = H−1 x
x
Hence points transform according to H−1 . Tensor Summation Convention
An index repeated in upper and lower positions in a product
T RANSFORMATION OF L INE represents summation over a range of the index.
A line in IP2 is represented by l = �l1 � l2 � l3 ) in the original basis. � �i
x i = H−1 x j
ˆ ˆi = H j lj
l
The transformed line ˆ l j i
ˆ = H� l
l
Coordinates of line transform as H� .
47 / 85 48 / 85

9. Tensor Notation Introduction Tensor Notation Introduction
The number of indices of a tensor are called as the valency of the
tensor.
Transformation of Projective mapping Example j
The sum over an index, e.g. Hi lj is referred to as contraction. In
Let P be a matrix representing a mapping between projective (or j
this case the tensor Hi is contracted with the line lj .
vector) spaces.
Let G and H represent basis transformations in the domain and
range spaces.
With respect to the new bases, the new mapping is represented
as:
ˆ
P = H−1 PG
I N TENSOR NOTATION :
�i
ˆi
�
Pj = H−1 Gl Pk
j lk
49 / 85 50 / 85
The tensor �rst Introduction The tensor �rst Introduction
The tensor �rst is defined for r� s� t = {1� 2� 3} as follows: The skew symmetric matrix [a]× is written in tensor notation as:

 0

 unless r� s� and t are distinct �[a]× ) ik = �ijk a j if a is contravariant
�rst =  +1 if r� s� t have even permutation of 123

�[a]× ) ik = �ijk aj

 −1 if r� s� t have odd permutation of 123 if a is covariant

The tensor �ijk (or its contravariant counterpart �ijk ) is connected R ECALL :
with the cross-product of two vectors. Cross product matrix: e = �e1 � e2 � e3 )
If a and b are 2 vectors, and c = a × b, then  
 0 −e3 e2 
 
ci = �a × b) i = �ijk a j bk [e]× =  3 0 −e1 

 e 
 

 
−e2 e1 0
 
The tensor �ijk is related to determinants:
For 3 contravariant tensors ai � bj � ck , it can be verified that Any skew symmetric 3 × 3 matrix may be written in the form [e]× for a
ai bj ck �ijk is the determinant of the 3 × 3 matrix with rows ai � bj suitable vector e.
and ck
51 / 85 52 / 85
Tensor Notation
Image points are represented by column 3-vectors.
� 1 
 x 
 
x =  x2 
Next � �

 


 3 
−→ 
x


Using the Tensor Notation for Trifocal Tensor
� �
Lines are represented by homogeneous row 3-vectors.
l = �l1 � l2 � l3 )
The i� j-th entry of a matrix A is denoted by ai
j
i is the row (contra-variant) index
j is the column (covariant) index
53 / 85 54 / 85

10. Tensor Notation Tensor Notation
The equation x� = Ax is equivalent to: The trifocal tensor �
jk
has one covariant and two contravariant
i
� indices.
x� i = ai x j
j written as x� i = ai x j
j The arrangement of indices for the trifocal tensor implies the
j
transformation rule:
An index repeated in the contra-variant and covariant positions � �j� �k
ˆ jk
� = Fr G−1 H−1 � st
would mean summation over that index. i i r s t
The trifocal tensor Ti = ai b� − a4 b�
4 i where F� G� H indicate the basis transformations in the 3 images.
jk j j
�i = ai bk − a4 bk
4 i
55 / 85 56 / 85
Tensor Notation
The line-line-line relation l�� [T1 � T2 � T3 ] l�� = l� is written in tensor
notation as:
jk jk
li = l�j l�� � = l�j � l��
k i i k
� �
jk jk jk jk
since l�j l�� � =
k i l�j l�� � =
k i l�j � l�� = l�j � l��
i k i k
j�k j�k
This relation is used for line transfer from one view to another H OMOGRAPHY BETWEEN 1st AND 3r� VIEWS Point transfer
The plane defined by back-projecting the line l � induces a
homography between the 1st and the 3r� views.
jk
� jk
� jk
li = l�j l�� � = l�� l�j �
k i k i = l�� hk where hk = l�j �
k i i i
Here hk are the elements of the homography matrix H13 .
i
jk
x�� k = hk x i = x i l�j �
i i
57 / 85 58 / 85
Trifocal Tensor Incidence relations Trifocal tensor
P ROPERTIES
A homography is obtained from the trifocal tensor by contraction
with a line, i.e. l� extracts a 3 × 3 matrix from the line. Line-line-line: l ↔ l� ↔ l�� Point-line-line: x ↔ l� ↔ l��
A pair of important tensors are �ijk and �ijk l�� [T1 � T2 � T3 ] l�� = l�
�
� i

 ��
��  l =0
l  x Ti

This tensor is used to represent the vector product.
 
 
i
The skew-symmetric matrix [x]× is written as x i �irs � �
l�� [T1 � T2 � T3 ] l�� [l]× = 0�
The line joining two points x i and y j is equal to the cross product x i l�j l�� �
jk
=0
k i
i j
x y �ijk = lk
� � jk
lr �ris l�j l�� �
k i = 0s
59 / 85 60 / 85

11. Incidence relations Trifocal tensor Incidence relations Trifocal tensor
Point-line-point: Point-point-line: Point-line-point: Point-point-line:
x ↔ l� ↔ x�� x ↔ x� ↔ l�� x ↔ l� ↔ x�� x ↔ x� ↔ l��
�  �  �  � 
� i  ��
�� 
� i  �� � i  �� � i  ��
l  x Ti  [x ]× = 0� � 
[x ]×  x Ti l =0 �� 
l  x Ti  [x ]× = 0� � 
[x ]×  x Ti l =0
     
 
 
 
  
 
 

i i i i
� � jq � � pk
� � jq � � pk
x i l�j x�� k �kqs � = 0s
i x i x� j �jpr l�� � = 0r
k i x i l�j x�� k �kqs � = 0s
i x i x� j �jpr l�� � = 0r
k i
Point-point-point: x ↔ x� ↔ x�� Point-point-point: x ↔ x� ↔ x��
�� � �� �
[x� ]× iT [x�� ]× = 03�3 [x� ]× iT [x�� ]× = 03�3
ix i ix i
� �� � pq � �� � pq
x i x� j �jpr x�� k �kqs � = 0rs
i x i x� j �jpr x�� k �kqs � = 0rs
i
61 / 85 62 / 85
Incidence relations Trifocal tensor Incidence relations Trifocal tensor
Consider the point-line-line relation T RILINEARITIES FOR POINT- POINT- POINT
jk
� �� � pq
x i l�j l�� �
k i =0 x i x� j �jpr x�� k �kqs � = 0rs
i
This relation is a contraction of the tensor over all 3 of its indices.
This relation is linear in the 3 image entities involved: x i � l�j � l�� .
k Clearly there are 9 different equations possible for different
Likewise all other tensor relations are also linear in the image choices of r and s. We examine 1 equation with r = 1 and s = 2.
entities involved. Choosing r = 1 in the 2n� view and expanding x� j �jpr results in
For example, in the point-point-point relation � �
� �� � pq l�p = x� j �jp1 = 0� −x� 3 � x� �
x i x� j �jpr x�� k �kqs � = 0rs
i
which is a horizontal line in the 2n� view through x� .
if x1 and x2 satisfy this relation, then so does x = αx1 + βx2 . Choosing s = 2 in the 3r� view and expanding x�� k �kqs results in
The right hand side of the above equation is 0rs which is a 3 × 3 � �
matrix with indices r� s. i.e. r = {1� 2� 3} � s = {1� 2� 3} . l�� = x�� k �kq2 = x�� 3 � 0� −x�� 1
q
63 / 85 64 / 85
Incidence relations Trifocal tensor Incidence relations Trifocal tensor
T RILINEARITIES FOR POINT- POINT- POINT
pq
� �� � pq 0 = x i x� j x�� k �jp1 �kq2 �i
x i x� j �jpr x�� k �kqs � = 0rs
i
� � � � ��
= x i −x� 3 x�� 3 � 21 − x�� 1 � 23 + x� � x�� 3 � 31 − x�� 1 � 33
i i i i
� �
l�p = x� j �jp1 = 0� −x� 3 � x� �
W HY DO WE CALL IT T RILINEARITY
� �
l�� = x�� k �kq2 = x�� 3 � 0� −x�� 1
q Prefix tri indicates that every monomial in the relation involves a
coordinate from each of the 3 image elements involved. (in the
Trilinear point relation now reduces to:
above case a p-p-p relation has 3 points, x� x� � x�� )
pq The relations are linear in each of the algebraic entities involved.
0 = x i x� j x�� k �jp1 �kq2 �i
� � � � �� This is just one of the 9 tri-linearities. Why? There are 9
= x i −x� 3 x�� 3 � 21 − x�� 1 � 23 + x� � x�� 3 � 31 − x�� 1 � 33
i i i i
trilinearities for 3 choices of r and 3 choices of s.
This is just one of the 9 tri-linearities. Further 3 equations for each i for the � xx entries.
i
65 / 85 66 / 85

12. 3-view geometry The Transfer problem
Given a pair of matched points in two views, what will be its
Next �
−→ � position in the 3r� view?
Transfer of geometric entities from one/two views How to transfer the lines ?
to another
Point Transfer �♣�
This problem can be solved...
Line Transfer Transfer
� �
If the camera matrices P� P� � P�� are known. Reconstruct the 3D
point X by back-projecting rays from two views, and then project X
on to the 3r� view.
By using fundamental matrices: F21 � F31 � F32 .
By using trifocal tensor [T1 � T2 � T3 ] .
67 / 85 68 / 85
Epipolar Transfer Using F matrices Epipolar Transfer Using F matrices
Let x� x� be matched points in the two images. D EGENERATE C ONDITIONS
We know F31 , hence we compute the epipolar line in the 3r� view One can define a plane passing through the 3 camera centers
corresponding to x. C� C� � C�� . This called as the trifocal plane.
We know F32 , hence we compute the epipolar line in the 3r� view What if the 3D point X lies on this plane?
corresponding to x� . The epipoles e32 � e31 and the point x�� will lie on the same line�
Intersection of the two epipolar lines gives x�� If the C� C� � C�� are collinear, then the trifocal plane does not exist.
In this case e32 = e31
x�� = �F31 x) × �F32 x� )
However there are certain degenerate conditions.
69 / 85 70 / 85
Using Trifocal Tensor Points Transfer Using Trifocal Tensor Points Transfer
We are given a point correspondence x ↔ x� S TEPS : (How to choose the line l� )
We choose a line l� passing through point x� in the 2n� view. Compute F21 from the trifocal tensor and correct x ↔ x� to the
As shown in slide 58, the point x�� may be computed as: exact correspondence � ↔ �� .
x x
jk
Compute the line l� through �� and perpendicular to l�e = F21�. If
x x
x�� k = x i l�j �i l�e = �l1 � l2 � l3 )� and �� = �x1 � x2 � 1) then
x ˆ ˆ
�
ˆ ˆ
l� = �l2 � −l1 � −x1 l2 + x2 l1 )
This transfer is not degenerate for general points X lying on the
trifocal plane. D EGENERATE C ONFIGURATION
D EGENERATE C ONDITIONS What if the 3D point X lies on the base-line joining the 1st and 2n�
l�
If is the epipolar line corresponding to x, then x i l�j �
jk
= 0k . cameras?
i
Hence the point x�� is undefined. �see slide 36) The points x and x� correspond with the epipoles in the two
A good choice for l� is the line perpendicular to F21 x images.
It is not possible to identify a line passing through x� which is not
the epipolar line.
71 / 85 72 / 85