Computer vision:models, learning and inference            Chapter 18    Models for style and identity       Please send er...
Identity and Style                                                              Identity differs,                         ...
Structure•   Factor analysis review•   Subspace identity model•   Linear discriminant analysis•   Non-linear models•   Asy...
Factor analysis reviewGenerative equation:Probabilistic form: Marginal density:            Computer vision: models, learni...
Factor analysisComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   5
Factor analysis reviewE-Step:M-Step:          Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   6
Factor analysis vs. Identity model• Each color is a different identity• multiple images lie in similar part of subspace   ...
Subspace identity modelGenerative equation:Probabilistic form: Marginal density:            Computer vision: models, learn...
Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   9
Factor analysis vs. subspace identity        Factor analysis                                     Subspace identity        ...
Learning subspace identity modelE-Step:Extract moments:           Computer vision: models, learning and inference. ©2011 S...
Learning subspace identity modelE-Step:M-Step:          Computer vision: models, learning and inference. ©2011 Simon J.D. ...
Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   13
Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   14
Inference by comparing modelsModel 1 – Faces match (identity shared):Model 2 – Faces dont match (identities differ):Both m...
Inference by comparing modelsCompute likelihood (e.g. for model zero)whereCompute posterior probability using Bayes rule  ...
Face Recognition Tasks    GALLERY                                         PROBE                …                          ...
Inference by comparing models    Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   18
Relation between models Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   19
Structure•   Factor analysis review•   Subspace identity model•   Linear discriminant analysis•   Non-linear models•   Asy...
Probabilistic linear                 discriminant analysisGenerative equation:Probabilistic form:            Computer visi...
Probabilistic linear discriminant analysis       Computer vision: models, learning and inference. ©2011 Simon J.D. Prince ...
LearningE-Step   – write out all images of same person as system of equations   – Has standard form of factor analyzer   –...
Probabilistic linear discriminant analyis       Computer vision: models, learning and inference. ©2011 Simon J.D. Prince  ...
InferenceModel 1 – Faces match (identity shared):Model 2 – Faces dont match (identities differ):Both models have standard ...
Example results (XM2VTS database)     Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   26
Structure•   Factor analysis review•   Subspace identity model•   Linear discriminant analysis•   Non-linear models•   Asy...
Non-linear models (mixture)Mixture model can describe non-linear manifold.Introduce variable ci whichrepresents which clus...
Non-linear models (kernel)•   Pass hidden variable through non-linear function f[ ].•   Leads to kernelized algorithm•   I...
Structure•   Factor analysis review•   Subspace identity model•   Linear discriminant analysis•   Non-linear models•   Asy...
Asymmetric bilinear model• Introduce style variable sij• indicates conditions in which data was observed• Example: lightin...
Asymmetric bilinear model  Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   32
Asymmetric bilinear modelGenerative equation:Probabilistic form:Marginal density:           Computer vision: models, learn...
LearningE-Step:M-Step:          Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   34
Asymmetric bilinear model  Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   35
Inference – inferring styleLikelihood of stylePrior over styleCompute posterior over style using Bayes’ rule           Com...
Inference – inferring identityLikelihood of identityPrior over identityCompute posterior over identity using Bayes’ rule  ...
Inference – comparing identitiesModel 1 – Faces match (identity shared):Model 2 – Faces dont match (identities differ):Bot...
Inference – Style translation• Compute distribution over identity• Generate in new style           Computer vision: models...
Structure•   Factor analysis review•   Subspace identity model•   Linear discriminant analysis•   Non-linear models•   Asy...
Symmetric bilinear modelGenerative equation:Probabilistic form:Mean can also depend on style...           Computer vision:...
Symmetric bilinear model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince   42
Inference – translating style or identity       Computer vision: models, learning and inference. ©2011 Simon J.D. Prince  ...
Multilinear modelsExtension of symmetric bilinear model to more  than two factorse.g.,        Computer vision: models, lea...
Structure•   Factor analysis review•   Subspace identity model•   Linear discriminant analysis•   Non-linear models•   Asy...
Face recognitionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   46
TensortexturesComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   47
Synthesizing animationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince   48
Discussion• Generative models• Explain data as combination of identity and  style factors• In identity recognition, we bui...
Upcoming SlideShare
Loading in …5
×

18 cv mil_style_and_identity

401 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
401
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

18 cv mil_style_and_identity

  1. 1. Computer vision:models, learning and inference Chapter 18 Models for style and identity Please send errata to s.prince@cs.ucl.ac.uk
  2. 2. Identity and Style Identity differs, but images similar Identity same, but images quite differentComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 2
  3. 3. Structure• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3
  4. 4. Factor analysis reviewGenerative equation:Probabilistic form: Marginal density: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4
  5. 5. Factor analysisComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 5
  6. 6. Factor analysis reviewE-Step:M-Step: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6
  7. 7. Factor analysis vs. Identity model• Each color is a different identity• multiple images lie in similar part of subspace Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 7
  8. 8. Subspace identity modelGenerative equation:Probabilistic form: Marginal density: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8
  9. 9. Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9
  10. 10. Factor analysis vs. subspace identity Factor analysis Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10
  11. 11. Learning subspace identity modelE-Step:Extract moments: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11
  12. 12. Learning subspace identity modelE-Step:M-Step: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12
  13. 13. Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13
  14. 14. Subspace identity model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 14
  15. 15. Inference by comparing modelsModel 1 – Faces match (identity shared):Model 2 – Faces dont match (identities differ):Both models have standard form of factor analyzer Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15
  16. 16. Inference by comparing modelsCompute likelihood (e.g. for model zero)whereCompute posterior probability using Bayes rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16
  17. 17. Face Recognition Tasks GALLERY PROBE … ? 1. CLOSED SET FACE IDENTIFICATION GALLERY PROBE … NO ? 2. OPEN SET MATCH FACE IDENTIFICATION PROBE NOMATCH ? 3. FACE VERIFICATION ? 4. FACE CLUSTERING Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 17
  18. 18. Inference by comparing models Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18
  19. 19. Relation between models Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19
  20. 20. Structure• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20
  21. 21. Probabilistic linear discriminant analysisGenerative equation:Probabilistic form: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21
  22. 22. Probabilistic linear discriminant analysis Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22
  23. 23. LearningE-Step – write out all images of same person as system of equations – Has standard form of factor analyzer – Use standard EM equationM-Step – write equation for each individual data point – Has standard form of factor analyzer – Use standard EM equation Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23
  24. 24. Probabilistic linear discriminant analyis Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24
  25. 25. InferenceModel 1 – Faces match (identity shared):Model 2 – Faces dont match (identities differ):Both models have standard form of factor analyzerCompute likelihood in standard way Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25
  26. 26. Example results (XM2VTS database) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26
  27. 27. Structure• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27
  28. 28. Non-linear models (mixture)Mixture model can describe non-linear manifold.Introduce variable ci whichrepresents which clusterTo be the same identity, must alsobelong to the same cluster Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28
  29. 29. Non-linear models (kernel)• Pass hidden variable through non-linear function f[ ].• Leads to kernelized algorithm• Identity equivalent of GPLVM Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29
  30. 30. Structure• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30
  31. 31. Asymmetric bilinear model• Introduce style variable sij• indicates conditions in which data was observed• Example: lighting, pose, expression face recognitionAsymmetric bilinear model• Introduce style variable sij• indicates conditions in which data was observed• Example: lighting, pose, expression face recognition Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31
  32. 32. Asymmetric bilinear model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32
  33. 33. Asymmetric bilinear modelGenerative equation:Probabilistic form:Marginal density: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33
  34. 34. LearningE-Step:M-Step: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34
  35. 35. Asymmetric bilinear model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35
  36. 36. Inference – inferring styleLikelihood of stylePrior over styleCompute posterior over style using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36
  37. 37. Inference – inferring identityLikelihood of identityPrior over identityCompute posterior over identity using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37
  38. 38. Inference – comparing identitiesModel 1 – Faces match (identity shared):Model 2 – Faces dont match (identities differ):Both models have standard form of factor analyzerCompute likelihood in standard way, combine with prior in Bayes rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 38
  39. 39. Inference – Style translation• Compute distribution over identity• Generate in new style Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 39
  40. 40. Structure• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40
  41. 41. Symmetric bilinear modelGenerative equation:Probabilistic form:Mean can also depend on style... Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41
  42. 42. Symmetric bilinear model Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42
  43. 43. Inference – translating style or identity Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43
  44. 44. Multilinear modelsExtension of symmetric bilinear model to more than two factorse.g., Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44
  45. 45. Structure• Factor analysis review• Subspace identity model• Linear discriminant analysis• Non-linear models• Asymmetric bilinear model• Symmetric bilinear model• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45
  46. 46. Face recognitionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 46
  47. 47. TensortexturesComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 47
  48. 48. Synthesizing animationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 48
  49. 49. Discussion• Generative models• Explain data as combination of identity and style factors• In identity recognition, we build models where identity was same or different• Other forms of inference such as style translation also possible Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 49

×