The document describes several computer vision models for representing identity and style in images, including factor analysis, subspace identity models, linear discriminant analysis, nonlinear models, and bilinear models. It discusses how each model is defined mathematically, how they are learned using EM algorithms, and how inferences can be made regarding identity and style when comparing images. Applications mentioned include face recognition under different lighting and pose conditions.

1. Computer vision:
models, learning and inference
Chapter 18
Models for style and identity
Please send errata to s.prince@cs.ucl.ac.uk

2. Identity and Style
Identity differs,
but images similar
Identity same, but
images quite
different
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2

3. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3

4. Factor analysis review
Generative equation:
Probabilistic form:
Marginal density:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4

5. Factor analysis
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5

6. Factor analysis review
E-Step:
M-Step:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6

7. Factor analysis vs. Identity model
• Each color is a different identity
• multiple images lie in similar part of subspace
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 7

8. Subspace identity model
Generative equation:
Probabilistic form:
Marginal density:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8

9. Subspace identity model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9

10. Factor analysis vs. subspace identity
Factor analysis Subspace identity
model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10

11. Learning subspace identity model
E-Step:
Extract moments:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11

12. Learning subspace identity model
E-Step:
M-Step:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12

13. Subspace identity model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13

14. Subspace identity model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 14

15. Inference by comparing models
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15

16. Inference by comparing models
Compute likelihood (e.g. for model zero)
where
Compute posterior probability using Bayes rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 16

17. Face Recognition Tasks
GALLERY PROBE
… ? 1. CLOSED SET
FACE IDENTIFICATION
GALLERY PROBE
… NO ? 2. OPEN SET
MATCH FACE IDENTIFICATION
PROBE
NO
MATCH
? 3. FACE VERIFICATION
? 4. FACE CLUSTERING
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 17

18. Inference by comparing models
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 18

19. Relation between models
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19

20. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20

21. Probabilistic linear
discriminant analysis
Generative equation:
Probabilistic form:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21

22. Probabilistic linear discriminant analysis
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22

23. Learning
E-Step
– write out all images of same person as system of equations
– Has standard form of factor analyzer
– Use standard EM equation
M-Step
– write equation for each individual data point
– Has standard form of factor analyzer
– Use standard EM equation
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23

24. Probabilistic linear discriminant analyis
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24

25. Inference
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Compute likelihood in standard way
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25

26. Example results (XM2VTS database)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26

27. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27

28. Non-linear models (mixture)
Mixture model can describe non-
linear manifold.
Introduce variable ci which
represents which cluster
To be the same identity, must also
belong to the same cluster
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28

29. Non-linear models (kernel)
• Pass hidden variable through non-linear function f[ ].
• Leads to kernelized algorithm
• Identity equivalent of GPLVM
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29

30. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30

31. Asymmetric bilinear model
• Introduce style variable sij
• indicates conditions in which data was observed
• Example: lighting, pose, expression face recognition
Asymmetric bilinear model
• Introduce style variable sij
• indicates conditions in which data was observed
• Example: lighting, pose, expression face recognition
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31

32. Asymmetric bilinear model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32

33. Asymmetric bilinear model
Generative equation:
Probabilistic form:
Marginal density:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33

34. Learning
E-Step:
M-Step:
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34

35. Asymmetric bilinear model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35

36. Inference – inferring style
Likelihood of style
Prior over style
Compute posterior over style using Bayes’ rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 36

37. Inference – inferring identity
Likelihood of identity
Prior over identity
Compute posterior over identity using Bayes’ rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 37

38. Inference – comparing identities
Model 1 – Faces match (identity shared):
Model 2 – Faces dont match (identities differ):
Both models have standard form of factor analyzer
Compute likelihood in standard way, combine with prior in Bayes rule
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 38

39. Inference – Style translation
• Compute distribution over identity
• Generate in new style
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 39

40. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40

41. Symmetric bilinear model
Generative equation:
Probabilistic form:
Mean can also depend on style...
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41

42. Symmetric bilinear model
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42

43. Inference – translating style or identity
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43

44. Multilinear models
Extension of symmetric bilinear model to more
than two factors
e.g.,
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44

45. Structure
• Factor analysis review
• Subspace identity model
• Linear discriminant analysis
• Non-linear models
• Asymmetric bilinear model
• Symmetric bilinear model
• Applications
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45

46. Face recognition
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46

47. Tensortextures
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47

48. Synthesizing animation
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 48

49. Discussion
• Generative models
• Explain data as combination of identity and
style factors
• In identity recognition, we build models where
identity was same or different
• Other forms of inference such as style
translation also possible
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 49