Tracking faces using PCA
Stephen Amar & Xuyan Yan
Do we use markers ?
• NO !
• The whole point is to have something that do
not require any physical markers.
How is this working ?
• OpenCV implementation of Principal
• 2 steps:
– Real-time tracking (webcam, movie …)
• We have a training set:
– A bunch of pictures and a list of coordinates
• We apply PCA on this train set.
– The coordinates are written in the last line of the
– Each picture is transformed into a giant vector
(128x128 pix => 16384 dimensions vector).
– The Average Image and the Covariance Matrix of
these pictures-vectors is computed .
– We find the eigenvectors and the eigenvalues of
the covariance matrix (OpenCV uses the Jacobi
eigenvalue algorithm but we could also use
• Once we have some training data:
– Get a new image to track
– Decompose this new image using the
eigenvectors. From this operation, we get
[α1,…, αn] where n is the number of train pictures -
– Now, we are going to compute:
Output Avg 1 I 1 ... 1 I n
– Using the output, we just need to read the last
line to get our new coordinates.
• A train set can work for only one face.
• Very sensitive to the lighting conditions.
• Very sensitive to the background.
• The “actor” must remain still.
• The OpenCV implementation is quite limited
(support only 8 bits grayscale pictures).
• Instead of using the OpenCV implementation,
do it ourselves.
– OpenCV offers all the basic functions
(Computation of the Covariance Matrix, SVD and
other Matrix multiplication algorithms)
– We could do this using 32 bits float pictures.