Automated Face Detection and Recognition


Published on

Published in: Technology
1 Comment
  • sir can u send me ur code for this whole process on
    because i tried to make face recognition but the eigen faces are not coming.
    Thank you!
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Automated Face Detection and Recognition

  1. 1. Automated Face Detection and Recognition A Survey Waldir Pimenta [email_address] Universidade do Minho Mestrado em Informática MI-STAR 2010
  2. 2. Face Detection Locating generic faces in images ©2009 Angelo State University
  3. 3. Face Detection: applications <ul><ul><li>Web cams that track the user </li></ul></ul><ul><ul><li>Cameras that shoot automatically when they detect smiles </li></ul></ul><ul><ul><li>Blurring of faces in public image databases </li></ul></ul>©2009 Google <ul><ul><li>Counting of people in a room (e.g. for temperature adjustment) </li></ul></ul>
  4. 4. Face Recognition Distinguishing a specific face from other faces ©2009
  5. 5. Face Recognition: applications <ul><ul><li>Biometrics / access control </li></ul></ul>&quot;&quot;Minority Report&quot; ©2002 20th Century Fox Superbad&quot; ©2007 Columbia Pictures <ul><ul><li>Searching mugshot databases </li></ul></ul><ul><ul><li>Tagging photo albums </li></ul></ul><ul><ul><li>Detecting fake ID cards </li></ul></ul><ul><ul><li>  </li></ul></ul><ul><ul><ul><li>no action required </li></ul></ul></ul><ul><ul><ul><li>scan many people at once </li></ul></ul></ul><ul><ul><ul><li>places: airports, banks, safes </li></ul></ul></ul><ul><ul><ul><li>data: laptops, medical info </li></ul></ul></ul>
  6. 6. Humans vs. Computers <ul><ul><li>&quot;Built-in&quot; face detection / recognition ability </li></ul></ul><ul><ul><li>detection & recognition in different areas of the brain </li></ul></ul><ul><ul><li>can be fooled by look-alikes </li></ul></ul>© <ul><ul><li>Algorithms must be built from scratch </li></ul></ul><ul><ul><li>Virtually perfect memory </li></ul></ul><ul><ul><li>Can work 24/7 without degrading performance </li></ul></ul><ul><ul><li>  Can apply stricter matching criteria </li></ul></ul>
  7. 7. Computer representation of faces <ul><ul><li>Faces vary across many attributes — they're multidimensional </li></ul></ul><ul><ul><li>Plotted in spaces with more than 3 dimensions </li></ul></ul><ul><ul><ul><li>in fact, it's commonly one dimension per pixel </li></ul></ul></ul><ul><ul><ul><li>on a 20×20px image, that's 400 dimensions! </li></ul></ul></ul><ul><ul><li>Humans can't visualize or compute distances intuitively in >3D space. Computers can. But... </li></ul></ul><ul><ul><li>It is computationally intensive. Dimensionality reduction is applied to enhance efficiency </li></ul></ul>
  8. 8. PCA: Principal component analysis <ul><ul><li>Data is projected into a lower dimensional space </li></ul></ul><ul><ul><li>preserving the directions that are most significant </li></ul></ul><ul><ul><li>not necessarily orthogonal to the original ones! </li></ul></ul>cc-by  Lydia E. Kavraki <>
  9. 9. What defines a &quot;match&quot;? <ul><ul><li>Ideally, distance in &quot;facespace&quot; should be: </li></ul></ul><ul><ul><ul><li>zero, for a specific match in face recognition </li></ul></ul></ul><ul><ul><ul><li>small, for a generic face </li></ul></ul></ul><ul><ul><ul><li>large, otherwise </li></ul></ul></ul><ul><ul><li>But there are variations due to: </li></ul></ul><ul><ul><ul><li>facial expressions </li></ul></ul></ul><ul><ul><ul><li>illumination variance </li></ul></ul></ul><ul><ul><ul><li>pose (orientation) </li></ul></ul></ul><ul><ul><ul><li>dimensionality reduction </li></ul></ul></ul>
  10. 10. The distance theshold <ul><ul><li>faces closer to each other than a given limit (threshold) are considered matches. </li></ul></ul><ul><ul><li>  A looser threshold can be used for face detection. </li></ul></ul>©  1991 M. Turk and A. Pentland
  11. 11. The ROC curve <ul><ul><li>Too low threshold = more false negatives </li></ul></ul><ul><ul><li>Too high threshold = more false positives </li></ul></ul><ul><ul><li>EER = Equal error rate </li></ul></ul>© 2007 Y. Du and C.-I. Chang &quot;Handbook of Fingerprint Recognition&quot; © 2004 D. Maltoni et al.
  12. 12. Some history... Francis Galton (1888) Designed a biometric system for description and identification of faces © 2007 University of Texas at Austin Public Domain Woody Bledsoe (1964) First implementation of automatic facial recognition in a mug shot database. <ul><ul><li>Michael D. Kelly (1970) </li></ul></ul><ul><ul><ul><li>Visual identification of people by computer </li></ul></ul></ul><ul><ul><li>Takeo Kanade (1973) </li></ul></ul><ul><ul><ul><li>Computer recognition of human faces </li></ul></ul></ul>
  13. 13. Classification Zhao et al. , 2003: “ [The facial recognition problem has] attracted researchers from very diverse backgrounds: psychology, pattern recognition, neural networks, computer vision, and computer graphics. ”   geometric (feature based) × photometric (image based) detection × recognition pre-processing 3D Video
  14. 14. Pre-processing <ul><ul><li>Face location / normalization </li></ul></ul><ul><ul><li>Later processing doesn't need to scan the whole image </li></ul></ul><ul><ul><li>Morphological operators (very fast) </li></ul></ul><ul><ul><li>Rough operators to detect heads </li></ul></ul><ul><ul><li>Finer confirmation operators to detect prominent features </li></ul></ul>© Brunelli and Poggio 1993 © Reisfeld et al. , 1995
  15. 15. Eigenfaces <ul><ul><li>Sirovich and Kirby 1987; Turk and Pentland 1991 </li></ul></ul><ul><ul><li>Uses PCA to discover principal components (eigenvectors) </li></ul></ul><ul><ul><li>Each face is described as a linear combination of the main eigenvectors </li></ul></ul><ul><ul><li>Image-based approach (features might not be intuitive) </li></ul></ul><ul><ul><li>eigenvectors can be translated back to the original pixel–based representation, many producing face-like images (hence the name eigenfaces ) </li></ul></ul>© AT&T Laboratories
  16. 16. Fisherfaces <ul><ul><li>Instead of PCA, it uses Linear disciminant analysis (LDA), developed by Robert Fisher in 1936 </li></ul></ul><ul><ul><li>Variation can be greater due to lighting than due to different faces (Moses el al. 1994) </li></ul></ul>©1997 Belhumeur et al. <ul><ul><li>Shashua [1994] demonstrated that images from same face but under different illumination conditions lie close to each other in the high- dimensional facespace </li></ul></ul><ul><ul><li>LDA can grasp these similarities better than PCA, which makes Fisherfaces more illumination independent than eigenfaces </li></ul></ul>
  17. 17. Neural networks <ul><ul><li>Based on the natural brain structure of simple, interconnected neurons </li></ul></ul><ul><ul><li>Good at approximating complex prob- lems without deterministic solutions </li></ul></ul><ul><ul><li>Each pixel of the face image is mapped to an input neuron </li></ul></ul><ul><ul><li>The intermediate (hidden-layer) neurons are as many as the number of reduced dimensions that are intended. </li></ul></ul><ul><ul><li>The network “learns” what patterns are likely faces or not </li></ul></ul><ul><ul><li>Initially promising, but Cottrell and Fleming [1990] showed that they can at best match an eigenface approach.  </li></ul></ul>cc-by-sa Cburnett <>
  18. 18. Gabor wavelets <ul><ul><li>First proposed in 1968 by Dennis Gabor </li></ul></ul><ul><ul><li>Analog to Fourier series: images are decomposed in a series of wavelets applied in different points </li></ul></ul><ul><ul><li>Further developed to flexible models: elastic grid matching.  </li></ul></ul>GFDL Wikimedia Commons © Wiskott et al. 1997
  19. 19. Active Shape/Appearance Models <ul><ul><li>Original concept by  Kass et al. , 1987: “snakes”, deformable curves that adjust to edges </li></ul></ul><ul><ul><li>Yuille [1987] extended the concept to flexible sets of geometrically related points (not necessarily on a curve) </li></ul></ul><ul><ul><li>Cootes [2001] applies statistical analysis to model and restrict the variation (flexibility) of model points </li></ul></ul>©2001 Cootes et al.
  20. 20. 3D <ul><ul><li>2D deal poorly with varying poses (orientation) of the head </li></ul></ul><ul><ul><li>Many have attempted to compensate by storing several views per face </li></ul></ul><ul><ul><ul><li>obviously resource-consuming </li></ul></ul></ul><ul><ul><li>  3D attempts to solve this issue, using: </li></ul></ul>©2006 Bowyer et al. <ul><ul><li>active range sensors (laser scanners, ultrasound) </li></ul></ul><ul><ul><li>passive sensors (structured light: grid projected on face) </li></ul></ul><ul><ul><li>New poses can be matched by deforming the 3D model </li></ul></ul>
  21. 21. Video <ul><ul><li>Lower quality images (frames), due to compression. Reconstructed models will have low accuracy. </li></ul></ul><ul><ul><li>Advantage: temporal coherence, optical flow </li></ul></ul><ul><ul><li>Simplest approach: use frame difference to detect moving foreground objects and match their shapes (blobs) to heads </li></ul></ul><ul><ul><li>Locate faces, then track them </li></ul></ul><ul><ul><li>Reconstruct 3D shape from relative movement of tracked points. This is called Structure from Motion (SfM) </li></ul></ul>©2010 Christian Rakete <>
  22. 22. Comparison <ul><li>Standard tests needed for valid results comparison </li></ul><ul><li>Databases: FERET, MIT, Yale, and many smaller ones </li></ul><ul><li>  </li></ul><ul><li>Evaluations: </li></ul><ul><ul><li>Face Recognition Vendor Test (FVRT) </li></ul></ul><ul><ul><li>Face Recognition Grand Challenge </li></ul></ul><ul><ul><li>XM2VTS </li></ul></ul><ul><li>Conferences: </li></ul><ul><ul><li>International Conference in Audio- and Video-Based Person Authentication (AVBPA) </li></ul></ul><ul><ul><li>International Conference in Automatic Face and Gesture Recognition (AFGR) </li></ul></ul>
  23. 23. Questions?