Deformable Facial Model Construction for non-rigid motion tracking, 3D Face Reconstruction Methods, Geometry-Based Methods , Stereo methods, Shape from Motion models, Face Models, Cylindrical Model, Ellipsoidal Model, Planar Model , Facial deformable models, Holistic models, Part based models, Eigenfaces, Active Shape Models, Combined Appearance Models, comparison of 3D facial features,list of 3d face databases containing 3D static expressions

1. Deformable Facial Model
Construction for non-rigid motion
tracking

2. Workflow

3. 3D Face Reconstruction Methods
1. Geometry-Based Methods
Assumptions: Faces have symmetrical shape and texture
Input: two face images which are orthogonal to each other
• The frontal photo provides the x and y coordinates .
• Side photo provides the y and z coordinates
• Reconstructed face is texture mapped using the blended texture form the
orthogonal photos.
Advantage:
1. Does not require 3D face database
Disadvantage
1. Depends on quality of the acquisition
2.Fail to produce accurate results for asymmetrical face geometry and
appearance

4. 2. Stereo methods
• Pixel correspondences are established between the two images to create the
disparity map.
• The disparity map and distance between the two cameras are used to
compute the depth map.
Disadvantage
• Performance is often affected by the environment conditions.
3. Shape from Motion models
• Manually annotate 44 facial points on a face as input
• Mark manually same feature points on a generic 3D model.
• Cylindrical projection to map all 3D generic mesh points to 2D
• Triangulate 2D feature points
• Texture map the face onto 3D generic model
• Morph the generic model to match the original.
DISADVANTAGE:
• Requires more source information and the operation is relatively complex.

5. Face Models:
Features of Generic 3D face mesh: Candide v3.1.6 :
• 113 vertices
• All coordinates are between -1.0 to 1.0
• 184 faces/triangles and for each triangle, 3 vertices.
• Each action is implemented as a list of vertex displacements,
describing the change in face geometry.
Advantages:
• Well-defined features
• Efficient Triangulation
Why we require Face Model?
• To interpret images of faces, it is important to have a model of how the
face can appear.
• Changes can be broken down into two parts: changes in shape and
changes in texture (patterns of pixel values) across the face.

6. Cylindrical Model
Advantages:
• Includes both circular and
elliptical cylinder.
• Copes up with large out of plane
rotation
• Robustness to initialization error
• Copes with self occlusions and
pose variations generated by large
head rotations.
• Simple
• Less computational load of a
fitting process
Disadvantages:
• Non-rigid motions cannot be
calculated as the vertices of the
model do not displace.
• Cannot generate actual shape and
texture

7. Ellipsoidal Model
• Ellipsoidal considers horizontally and vertical curved surfaces.
• Accurately captures the 3D motion parameters of the head.
• Is robust to small variations in the initial fit, enabling
the automation of the model initialization.
• It considers the entire 3D aspect of the head, the
tracking is very stable over a large number of frames. This
robustness extends even to sequences with very low frame
rates and noisy camera images.
Planar Model
• Plane model does not represent curved surfaces and is not
robust to out-of-plane rotations.
Figures taken from [12]

8. Facial deformable models
• Holistic models
uses holistic texture based facial representation
Ex: AAM, 3D deformable models
# Discriminative
# Generative
• Part based models
uses local image patches around landmark points
Ex: ASM, CLMs and Tree-based pictorial structures.
Slide Taken from:http://www.robots.ox.ac.uk/~minhhoai/papers/learn2align_CVPR08.pdf

9. Appearance Models
• Eigenfaces (Turk and Pentland, 1991)
– Not robust to shape changes
– Not robust to changes in pose and expression
• Ezzat and Poggio approach (1996)
– Synthesize new views of face from set of example
views
– Does not generalize to unseen faces
Slide taken from: http://www.ai.mit.edu/courses/6.899/lectures/lecture17aam.ppt

10. Active Shape Models
• Point Distribution Model
Training: Apply PCA to labeled images
New image
– Project mean shape
– Iteratively modify model points to fit local neighborhood
Advantages and Disadvantage
• ASM is relatively fast
• ASM too simplistic; not robust when new images
are introduced
• May not converge to good solution
• Key insight: ASM does not incorporate all gray-
level information in parameters
Slide taken from: http://www.ai.mit.edu/courses/6.899/lectures/lecture17aam.ppt

11. Slide Taken from :pages.cpsc.ucalgary.ca/~marina/601/Week6_Face_tracking.ppt
Example of ASM failing
The figure demonstrates the Active Shape Model (ASM) failing. The
main facial features have been found, but the local models searching
for the edges of the face have failed to locate their correct positions,
perhaps because they are too far away. The ASM is a local method and
prone to local minima.
Example of ASM search failure. The search profiles are not long enough to
locate the edges of the face.

12. Combined Appearance Models
• Combine shape and gray-level variation in
single statistical appearance model
• Advantages and Disadvantage
– Inherits appearance model benefits
• Able to represent any face within bounds of the
training set
• Robust interpretation
– Model parameters characterize facial features
Slide taken from: http://www.ai.mit.edu/courses/6.899/lectures/lecture17aam.ppt

13. Parts Based Models
• Each part explains the image data underneath it.
• Model is represented as a graph.
• Vertices represents parts
• Edges represent connection between parts
• If we calculate best location for each part- we get
connections for free.
Deformable model considers each object as a
deformed version of a template leading to compact
representation

14. Refer [9]

16. References
[1]Leung, W., Tseng, B., Shae, Z., Hendriks, F., and Chen, T. 2010. Realistic video avatar. Multimedia and Expo. IEEE.
[2] CANDIDE – a parameterized face. http://www.bk.isy.liu.se/candide/main.html
[3] Narendra Patel, Mukesh Zaveri," 3D Facial Model Construction and Expression Synthesis using a Single Frontal Face
Image”, International Journal of Graphics, November 2010
[4] R. Valenti, N. Sebe, and T. Gevers, "Facial expression recognition: A fully integrated approach," in Int. Workshop on
Visual and Multimedia Digital Libraries, 2007
[5] Iain Matthews, Jing Xiao, Simon Baker. “2D vs 3D Deformable Face Models: Representational Power, Construction
and Real-Time Fitting” International Journal of Computer Vision, Springer 2007.
[6] P. Viola and M. Jones. Robust real-time object detection. International Journal of Computer Vision, 57(2):137–154,
2004.
[7]M.Turk and A.Pentland. Eigen faces for recognition. Journal Cognitive Neuroscience, 3(1):71-86,1991
[8] I. Matthews and S.Baker. Active appearance models revisited. International Journal of Computer Vision, 60(2):135-
164, Nov. 2004
[9] Hamimah Ujir, “3D facial expresion classification using a statistical model of surface normals and modular approach,
theisus university of bBirmingham, 2012
[10] K. H. An and M. Chung "3D head tracking and pose-robust 2D texture map-based face recognition using a simple
ellipsoid model", Proc. Intell. Robots Syst., pp.307 -312 2008
[11] Jung, Sung-Uk, and Mark S. Nixon. "On using gait biometrics to enhance face pose estimation." Biometrics: Theory
Applications and Systems (BTAS), 2010 Fourth IEEE International Conference on. IEEE, 2010.
[12] S. Basu , I. A. Essa and A. P. Pentland "Motion regularization for model-based head tracking", International
Conference on Pattern Recognition, 1996
[13] La Cascia, M.; Sclaroff, S.; Athitsos, V., "Fast, reliable head tracking under varying illumination: an approach based
on registration of texture-mapped 3D models," Pattern Analysis and Machine Intelligence, IEEE Transactions on ,
vol.22, no.4, pp.322,336, Apr 2000