< Xin Wang > Face-ID recognition module for OPASCA   Karlsruhe, < 2nd Sep. 2008 >
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Task und system Requirement ◆ Task :  Implementation of an optical face-ID recognition module using MATLAB with steering c...
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Project Overview ◆ System diagram : Face  Detection Filtering Subspace  Projection Recognition System ID
Implementation steps <ul><li>◆ System implementation diagram </li></ul>Face  Data-base Creation   Eigenface Fisherface Fil...
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Implementation steps <ul><li>◆ IITB research group face database  </li></ul><ul><li>5 places , each with 2 situations ( wi...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Algorithm ? </li></ul><ul><li>Near real-time p...
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Feature extraction </li></ul><ul><li>Images mu...
Implementation steps <ul><li>◆ Core Algorithm - Eigenface </li></ul><ul><li>Computation problem   </li></ul><ul><li>Exampl...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Solutions </li></ul><ul><li>They could be desc...
Implementation steps <ul><li>◆ Core Algorithm -Eigenface </li></ul><ul><li>Eigenface Initialization </li></ul><ul><li>Acqu...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Center and create data matrix </li></ul><ul><l...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Create covariance matrix , compute </li></ul><...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>120 Eigenvectors(eigenface) </li></ul><ul><li>...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Project training images and test </li></ul><ul...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Recognition(Similarity measurement) </li></ul>...
Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Experiment results: </li></ul>Remark: the test...
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Main problems faced by Eigenface : robustness...
Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Solution: find a subspace which is insensitiv...
Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Fisher’s Linear Discriminant </li></ul><ul><l...
Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Difference with Eigenface in  </li></ul><ul><...
Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Keep first C-1 eigenvectors to form </li></ul...
Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Experiment results </li></ul>Eigen:Test2 Room...
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Improvement steps <ul><li>◆ Gabor-wavelet filter for preprocessing </li></ul><ul><li>Main   problem : poor robustness in c...
Improvement steps <ul><li>◆ Gabor-wavelet filter for preprocessing </li></ul><ul><li>Complex Gabor function in space domai...
Improvement steps <ul><li>◆ Gabor-wavelet filter for preprocessing </li></ul><ul><li>Too high dimension output? </li></ul>...
Improvement steps ◆ Gabor-wavelet filter for preprocessing Testing from Room5
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Improvement steps <ul><li>◆  Composition training set </li></ul><ul><li>Initial idea : more information for system to “lea...
Improvement steps <ul><li>◆  Composition training set </li></ul><ul><li>Why composition training achieve best? </li></ul><...
Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Imple...
Improvement steps <ul><li>◆  N-nearest Classifier / Distance </li></ul>Test13, Room1 Result: 3%-5% improvement for avg and...
Project Overview <ul><li>◆   System performance overview </li></ul><ul><li>Within-database Test </li></ul><ul><li>Both Eig...
Future work <ul><li>Video based recognition system </li></ul><ul><li>Main problem we still faced : Robustness, computation...
Q&A <ul><li>Thank you for your attention! </li></ul><ul><li>Questions? </li></ul>
Additional <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>“ Information theory approach of coding and decoding fac...
Additional <ul><li>◆ How many eigenvectors we need in Eigenface </li></ul><ul><li>From non-zero eigenvalue -> P-1 is upbou...
Additional <ul><li>◆ Solve SSS problem in Fisherface  </li></ul>?
Additional <ul><li>◆ Adaboost feature extraction for Gabor </li></ul>
Additional <ul><li>◆ Gabor filter vision  </li></ul>
Additional <ul><li>◆ EBGM : </li></ul><ul><li>◆ FERET DATABASE:  http://face.nist.gov/colorferet/   </li></ul>
Additional <ul><li>ORL Facedatabase test: </li></ul><ul><li>5 persons selected and 5 pic/ per person used in both training...
<Subtopic> <Title> (P-C) × 1 Vector (C-1) × 1 Vector PCA LDA N×1 Vector X matrix
Upcoming SlideShare
Loading in …5
×

Face Identification for Humanoid Robot

9,338 views

Published on

Published in: Technology, News & Politics
  • Be the first to comment

Face Identification for Humanoid Robot

  1. 1. < Xin Wang > Face-ID recognition module for OPASCA Karlsruhe, < 2nd Sep. 2008 >
  2. 2. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  3. 3. Task und system Requirement ◆ Task : Implementation of an optical face-ID recognition module using MATLAB with steering camera in real-time application ◆ Requirement : Good recognition rate with robustness property Multi-person , Multi-places , Multi-illumination , Cross-poses , Cross-expressions , Cross-view of angle As the real situations the social robot will encounter ! ?
  4. 4. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  5. 5. Project Overview ◆ System diagram : Face Detection Filtering Subspace Projection Recognition System ID
  6. 6. Implementation steps <ul><li>◆ System implementation diagram </li></ul>Face Data-base Creation Eigenface Fisherface Filters (Gabor, Histogram) Classifier (N-Nearest, Mahalanobis) Composition Training Set Core Algorithm Improvement Solution Train &Test Image
  7. 7. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  8. 8. Implementation steps <ul><li>◆ IITB research group face database </li></ul><ul><li>5 places , each with 2 situations ( with or without light) </li></ul><ul><li>50 * 50 pixel image size </li></ul><ul><li>6-9 persons with 50 pic / person in each situation </li></ul><ul><li>Multiple illumination , cross-pose , cross-expression </li></ul><ul><li>and various view of angle. </li></ul><ul><li>For robustness test </li></ul><ul><li>purpose </li></ul>
  9. 9. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Algorithm ? </li></ul><ul><li>Near real-time performance and accuracy with robustness </li></ul><ul><li>Problem for feature-based face recognition? </li></ul>
  10. 10. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  11. 11. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Feature extraction </li></ul><ul><li>Images must share a number of similarities (as T-F domain </li></ul><ul><li>transform , exp. Fourier transform  find basis) </li></ul><ul><li>Capture the variation in a set of images ,independent of any </li></ul><ul><li>judgment of features (Information theory based features) </li></ul><ul><li>How to do it? </li></ul><ul><li>A face image in our database is a point in 50*50=2500 high </li></ul><ul><li>dimensional image space , how to describe similarities </li></ul><ul><li>between images ? ( Image encoding, computation model) </li></ul>
  12. 12. Implementation steps <ul><li>◆ Core Algorithm - Eigenface </li></ul><ul><li>Computation problem </li></ul><ul><li>Example for “ How far apart “ between two points (image) in 2D </li></ul><ul><li>d 12 = sqrt ( △x 2 + △y 2 ) , </li></ul><ul><li>where △x = x 2 - x 1 , △y = y 2 - y 1 </li></ul><ul><li>Point in 2500 dimensions ? </li></ul><ul><li>Distance calculation means: </li></ul><ul><li>d = sqrt ( sum ( △x 2 + △y 2 + …. )) </li></ul>
  13. 13. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Solutions </li></ul><ul><li>They could be described by a relatively low dimensional </li></ul><ul><li>subspace (PCA , dimension reduction) </li></ul><ul><li>Project the face images into an appropriately </li></ul><ul><li>chosen subspace and perform classification </li></ul><ul><li>by similarity computation </li></ul><ul><li>( distance, angle) </li></ul>
  14. 14. Implementation steps <ul><li>◆ Core Algorithm -Eigenface </li></ul><ul><li>Eigenface Initialization </li></ul><ul><li>Acquire an initial set of face </li></ul><ul><li>images (Training Set) </li></ul><ul><li>Each image is stored in a </li></ul><ul><li>vector of Size N </li></ul>
  15. 15. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Center and create data matrix </li></ul><ul><li>Each image as vector of size N </li></ul><ul><li>Data Matrix as size of N × P </li></ul>M=
  16. 16. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Create covariance matrix , compute </li></ul><ul><li>eigenvalues and eigenvectors </li></ul><ul><li>(eigenfaces) </li></ul><ul><li>Covariance Matrix with size N × N </li></ul><ul><li>V (N×1)is set of eigenvectors ,with eigenvalues ∧ </li></ul>M eigenvectors correspoding to M largest non-zero eigenvalues
  17. 17. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>120 Eigenvectors(eigenface) </li></ul><ul><li>They are the orthogonal </li></ul><ul><li>basis to build a dimensional </li></ul><ul><li>reduction subspace </li></ul><ul><li>(facespace) </li></ul><ul><li>Facespace with 120 </li></ul><ul><li>dimensions , eigenvector-> </li></ul><ul><li>weight </li></ul>
  18. 18. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Project training images and test </li></ul><ul><li>image </li></ul><ul><li>Projected train image and test </li></ul><ul><li>image in “facespace”,their </li></ul><ul><li>dimension are M × 1 </li></ul>
  19. 19. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Recognition(Similarity measurement) </li></ul><ul><li>use Euclidian Distance , tested </li></ul><ul><li>image will be classified to nearest </li></ul><ul><li>euclidian distance picture in train </li></ul>Test Matched Image:
  20. 20. Implementation steps <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>Experiment results: </li></ul>Remark: the testing is based on the following condition , with 6-9 persons in each test for both training set and testing pictures, and 50 pictures for each person in the test and training set is assumed
  21. 21. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  22. 22. Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Main problems faced by Eigenface : robustness </li></ul><ul><li>Experiment in 5 rooms against each other : cross-database </li></ul><ul><li>Mean recognition rate: 20%-30% </li></ul><ul><li>Standard deviation : over 40% </li></ul><ul><li>Why? </li></ul><ul><li>The variations between the images of the same face due </li></ul><ul><li>to illumination and view direction are almost always larger </li></ul><ul><li>than image variations due to change in face ID. </li></ul><ul><li>--Moses,Adini,Ullman , ECCV’94 </li></ul>
  23. 23. Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Solution: find a subspace which is insensitive </li></ul><ul><li>to illumination change (Fisher Space) </li></ul><ul><li>Fisher subspace </li></ul>?
  24. 24. Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Fisher’s Linear Discriminant </li></ul><ul><li>Preserves separability of classes </li></ul><ul><li>Maximizes ratio of projected between-classes </li></ul><ul><li>to projected within-class scatter </li></ul><ul><li>Between-class scatter </li></ul><ul><li>Within-class scatter </li></ul>Where: S w and S B are Both N × N size and c :number of class, m :mean of all image m i : mean of class i th n i : No. of images in i th class
  25. 25. Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Difference with Eigenface in </li></ul><ul><li>: how to solve eigenvector pro </li></ul>Finding an optimal linear mapping W
  26. 26. Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Keep first C-1 eigenvectors to form </li></ul><ul><li>Fisherface subspace </li></ul>
  27. 27. Implementation steps <ul><li>◆ Core Algorithm – Fisherface </li></ul><ul><li>Experiment results </li></ul>Eigen:Test2 Room1,Fisher Test6 Room1
  28. 28. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters , Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  29. 29. Improvement steps <ul><li>◆ Gabor-wavelet filter for preprocessing </li></ul><ul><li>Main problem : poor robustness in cross-database test </li></ul><ul><li>Even Fisherface only achieve around 40-50% rate, </li></ul><ul><li>10%-20% improvement against Eigenface method </li></ul><ul><li>(Maybe because our face database, quite various situations) </li></ul><ul><li>Possible Solution : choose one representation of image </li></ul><ul><li>that is insensitive to illumination </li></ul><ul><li>change ( Filtering step pre-step) </li></ul><ul><li> Gabor-Wavelet Filter </li></ul>Exploit salient visual properties, spatial localization,orientation selectivity as retina
  30. 30. Improvement steps <ul><li>◆ Gabor-wavelet filter for preprocessing </li></ul><ul><li>Complex Gabor function in space domain </li></ul><ul><li>Complex sinusoidal carrier </li></ul><ul><li>The Gaussian envelop </li></ul>5 scales,8 Phase , Real part
  31. 31. Improvement steps <ul><li>◆ Gabor-wavelet filter for preprocessing </li></ul><ul><li>Too high dimension output? </li></ul><ul><li>2500*5*8=100000 dimension-> take real, </li></ul><ul><li>then mean them as output </li></ul><ul><li>Experiment results (Test 16, use room5 worst case) </li></ul>After Gabor After Histeq
  32. 32. Improvement steps ◆ Gabor-wavelet filter for preprocessing Testing from Room5
  33. 33. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  34. 34. Improvement steps <ul><li>◆ Composition training set </li></ul><ul><li>Initial idea : more information for system to “learn” </li></ul><ul><li>Implementation: training pic – 6 pic/ person / room </li></ul><ul><li>4 other rooms used, 5 person </li></ul><ul><li>thus 6 × 5 × 4=120 pic totally </li></ul><ul><li>testing pic – </li></ul><ul><li>from other 1 room , </li></ul><ul><li>24 random/ person , 5 person </li></ul>TEST 13, ROOM1
  35. 35. Improvement steps <ul><li>◆ Composition training set </li></ul><ul><li>Why composition training achieve best? </li></ul><ul><li>Lambertian surface model : </li></ul><ul><li>given 3 images of a lambertian surface -> </li></ul><ul><li>albedo and surface normal can be </li></ul><ul><li>recovered </li></ul><ul><li>Further Idea? </li></ul><ul><li>Active learning method for robot to </li></ul><ul><li>build its training database </li></ul><ul><li>automatically </li></ul>
  36. 36. Outline for the presentation <ul><li>Task und system requirement </li></ul><ul><li>System overview </li></ul><ul><li>Implementation steps with experiment results </li></ul><ul><li>Face detection + Face recognition (Basic System) </li></ul><ul><li>Eigenface , Fisherface </li></ul><ul><li>Improvement solutions </li></ul><ul><li>Filters, Composition training set , Classifier </li></ul><ul><li>System performance overview </li></ul><ul><li>Future work </li></ul><ul><li>Questions </li></ul>
  37. 37. Improvement steps <ul><li>◆ N-nearest Classifier / Distance </li></ul>Test13, Room1 Result: 3%-5% improvement for avg and std
  38. 38. Project Overview <ul><li>◆ System performance overview </li></ul><ul><li>Within-database Test </li></ul><ul><li>Both Eigenface and Fisherface algorithm achieve very good recognition rate around 90%-99% with low Std around 1%-10% </li></ul><ul><li>Cross-database Test </li></ul><ul><li>Eigenface Algorithm rate drops a lot (in different illumination condition )to around 20% with high Std around 40% </li></ul><ul><li>Fisherface Algorithm shows better robustness , get rate around 40%-50% with Std around 30% </li></ul><ul><li>Best Result after improvement in Cross-Database Test </li></ul><ul><li>By using GaborFilter+Fisherface+4-Nearest Classifier </li></ul><ul><li>get rate around 92.5% with Std 10.5% (Test 13 Room1 Condition) </li></ul>
  39. 39. Future work <ul><li>Video based recognition system </li></ul><ul><li>Main problem we still faced : Robustness, computational </li></ul><ul><li>complexity (efficiency) </li></ul><ul><li>Ideas? </li></ul><ul><li>Facedetection+Gabor-wavelet+Adaboost+Fisher+ </li></ul><ul><li>Bayesian classifier (or use SVM) </li></ul><ul><li>Active learning model for building robot’s training </li></ul><ul><li>database automatically (face/not face? know/ unknow?) </li></ul>
  40. 40. Q&A <ul><li>Thank you for your attention! </li></ul><ul><li>Questions? </li></ul>
  41. 41. Additional <ul><li>◆ Core Algorithm – Eigenface </li></ul><ul><li>“ Information theory approach of coding and decoding face </li></ul><ul><li>images may give insight into the information content of face </li></ul><ul><li>images, emphasizing the significant local and global </li></ul><ul><li>“ features“ . Such features may or not be directly related to </li></ul><ul><li>our intuitive notion of face features , such as eyes, nose, </li></ul><ul><li>lips, or hair. </li></ul><ul><li>M. Turk and A. Pentland, “Eigenfaces for recognition” , </li></ul><ul><li>Journal of Cognitive Science , pp, 73, 1991 </li></ul>
  42. 42. Additional <ul><li>◆ How many eigenvectors we need in Eigenface </li></ul><ul><li>From non-zero eigenvalue -> P-1 is upboundary </li></ul><ul><li>From practical use -> ½ of image number </li></ul><ul><li>other improvement-> get rid of first 3 eigenvectors </li></ul><ul><li>Experiment result: </li></ul>Test15, 120 pic in training
  43. 43. Additional <ul><li>◆ Solve SSS problem in Fisherface </li></ul>?
  44. 44. Additional <ul><li>◆ Adaboost feature extraction for Gabor </li></ul>
  45. 45. Additional <ul><li>◆ Gabor filter vision </li></ul>
  46. 46. Additional <ul><li>◆ EBGM : </li></ul><ul><li>◆ FERET DATABASE: http://face.nist.gov/colorferet/ </li></ul>
  47. 47. Additional <ul><li>ORL Facedatabase test: </li></ul><ul><li>5 persons selected and 5 pic/ per person used in both training and testing set, </li></ul><ul><li>The results show all 100% rate for Eigen , Fisher and Fisher+Gabor method, </li></ul><ul><li>Then I reduce the training set picture for only 1 pic / per person ,and 9 pic/ person in testing set , for 5 persons </li></ul><ul><li>Even for Eigen algorithm , it achieves: </li></ul><ul><li>Eigen’s Mean 0.8889 , Std 0.1361 </li></ul><ul><li>So I think there is no doubt that other research paper states they get over 98% recognition rate , it must depend on the face database they used. ORL facedatabase include very little illumination and view of angle changing . </li></ul><ul><li>ORL face database: http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html </li></ul>
  48. 48. <Subtopic> <Title> (P-C) × 1 Vector (C-1) × 1 Vector PCA LDA N×1 Vector X matrix

×