Upcoming SlideShare
×

# 06 cv mil_learning_and_inference

340 views
263 views

Published on

1 Like
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
340
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
2
0
Likes
1
Embeds 0
No embeds

No notes for slide

### 06 cv mil_learning_and_inference

1. 1. Computer vision: models, learning and inference Chapter 6 Learning and inference in vision Please send errata to s.prince@cs.ucl.ac.uk
2. 2. Structure• Computer vision models – Three types of model• Worked example 1: Regression• Worked example 2: Classification• Which type should we choose?• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 2
3. 3. Computer vision models• Observe measured data, x• Draw inferences from it about state of world, wExamples: – Observe adjacent frames in video sequence – Infer camera motion – Observe image of face – Infer identity – Observe images from two displaced cameras – Infer 3d structure of scene Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 3
4. 4. Regression vs. Classification• Observe measured data, x• Draw inferences from it about world, wWhen the world state w is continuous we’ll call this regressionWhen the world state w is discrete we call this classification Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 4
5. 5. Ambiguity of visual world• Unfortunately visual measurements may be compatible with more than one world state w – Measurement process is noisy – Inherent ambiguity in visual data• Conclusion: the best we can do is compute a probability distribution Pr(w|x) over possible states of world Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 5
6. 6. Refined goal of computer vision• Take observations x• Return probability distribution Pr(w|x) over possible worlds compatible with data(not always tractable – might have to settle for an approximation to this distribution, samples from it, or the best (MAP) solution for w) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 6
7. 7. Components of solutionWe need• A model that mathematically relates the visual data x to the world state y. Model specifies family of relationships, particular relationship depends on parameters q• A learning algorithm: fits parameters q from paired training examples xi,yi• An inference algorithm: uses model to return Pr(y|x) given new observed data x. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 7
8. 8. Types of ModelThe model mathematically relates the visual data x to the world state w. Every model falls into one of three categories1. Model contingency of the world on the data Pr(w|x)2. Model joint occurrence of world and data Pr(x,w)3. Model contingency of data on world Pr(x|w) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 8
9. 9. Generative vs. Discriminative1. Model contingency of the world on the data Pr(w|x) (DISCRIMINATIVE MODEL)2. Model joint occurrence of world and data Pr(x,w)3. Model contingency of data on world Pr(x|w) (GENERATIVE MODELS)Generative as probability model over data and so when we draw samples from model, we GENERATE new data Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 9
10. 10. Type 1: Model Pr(w|x) - DiscriminativeHow to model Pr(w|x)? 1. Choose an appropriate form for Pr(w) 2. Make parameters a function of x 3. Function takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: just evaluate Pr(w|x) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 10
11. 11. Type 2: Model Pr(x,w) - GenerativeHow to model Pr(x,w)? 1. Concatenate x and w to make z= [xT wT]T 2. Model the pdf of z 3. Pdf takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: compute Pr(w|x) using Bayes rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 11
12. 12. Type 3: Pr(x|w) - GenerativeHow to model Pr(x|w)? 1. Choose an appropriate form for Pr(x) 2. Make parameters a function of w 3. Function takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: Define prior Pr(w) and then compute Pr(w|x) using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 12
13. 13. SummaryThree different types of model depend on the quantity of interest: 1. Pr(w|x) Discriminative 2. Pr(w,y) Generative 3. Pr(w|y) GenerativeInference in discriminative models easy as we directly model posterior Pr(w|x). Generative models require more complex inference process using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 13
14. 14. Structure• Computer vision models – Three types of model• Worked example 1: Regression• Worked example 2: Classification• Which type should we choose?• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 14
15. 15. Worked example 1: RegressionConsider simple case where• we make a univariate continuous measurement x• use this to predict a univariate continuous state w(regression as world state is continuous) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 15
16. 16. Regression application 1: Pose from SilhouetteComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 16
17. 17. Regression application 2: Changing face poseComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 17
18. 18. Regression application 3: Head pose estimationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 18
19. 19. Worked example 1: RegressionConsider simple case where• we make a univariate continuous measurement x• use this to predict a univariate continuous state w(regression as world state is continuous) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 19
20. 20. Type 1: Model Pr(w|x) - DiscriminativeHow to model Pr(w|x)? 1. Choose an appropriate form for Pr(w) 2. Make parameters a function of x 3. Function takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: just evaluate Pr(w|x) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 20
21. 21. Type 1: Model Pr(w|x) - DiscriminativeHow to model Pr(w|x)? 1. Choose an appropriate form for Pr(w) 2. Make parameters a function of x 3. Function takes parameters q that define its shape 1. Choose normal distribution over w 2. Make mean m linear function of x (variance constant) 3. Parameter are f0, f1, s2. This model is called linear regression. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 21
22. 22. Parameters are y-offset, slope and variance Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 22
23. 23. Learning algorithm: learn q from training data x,y. E.g. MAP Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 23
24. 24. Inference algorithm: just evaluate Pr(w|x) for new data x Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 24
25. 25. Type 2: Model Pr(x,y) - GenerativeHow to model Pr(x,w)? 1. Concatenate x and w to make z= [xT wT]T 2. Model the pdf of z 3. Pdf takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: compute Pr(w|x) using Bayes rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 25
26. 26. Type 2: Model Pr(x,y) - GenerativeHow to model Pr(x,w)? 1. Concatenate x and w to make z= [xT wT]T 2. Model the pdf of z 3. Pdf takes parameters q that define its shape 1. Concatenate x and w to make z= [xT wT]T 2. Model the pdf of z as normal distribution. 3. Pdf takes parameters m and S that define its shape Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 26
27. 27. Learning algorithm: learn parameters q from training data x,w Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 27
28. 28. Inference algorithm: compute Pr(w|x) using Bayes rule (normalize each column in figure) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 28
29. 29. Type 3: Pr(x|w) - GenerativeHow to model Pr(x|w)? 1. Choose an appropriate form for Pr(x) 2. Make parameters a function of w 3. Function takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: Define prior Pr(w) and then compute Pr(w|x) using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 29
30. 30. Type 3: Pr(x|w) - GenerativeHow to model Pr(x|w)? 1. Choose an appropriate form for Pr(x) 2. Make parameters a function of w 3. Function takes parameters q that define its shape 1. Choose normal distribution over x 2. Make mean m linear function of w (variance constant) 3. Parameter are f0, f1, s2. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 30
31. 31. Learning algorithm: learn q from training data x,w. e.g. MAP Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 31
32. 32. Pr(x|w) x Pr(w) = Pr(x,w) Can get back to joint probability Pr(x,y) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 32
33. 33. Inference algorithm: compute Pr(w|x) using Bayes rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 33
34. 34. Structure• Computer vision models – Three types of model• Worked example 1: Regression• Worked example 2: Classification• Which type should we choose?• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 34
35. 35. Worked example 2: ClassificationConsider simple case where• we make a univariate continuous measurement x• use this to predict a discrete binary world w(classification as world state is continuous) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 35
36. 36. Classification Example 1: Face DetectionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 36
37. 37. Classification Example 2: Pedestrian DetectionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 37
38. 38. Classification Example 3: Face RecognitionComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 38
39. 39. Classification Example 4:Semantic SegmentationComputer vision: models, learning and inference. ©2011 Simon J.D. Prince 39
40. 40. Worked example 2: ClassificationConsider simple case where• we make a univariate continuous measurement x• use this to predict a discrete binary world w(classification as world state is continuous) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 40
41. 41. Type 1: Model Pr(w|x) - DiscriminativeHow to model Pr(w|x)? – Choose an appropriate form for Pr(w) – Make parameters a function of x – Function takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: just evaluate Pr(w|x) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 41
42. 42. Type 1: Model Pr(w|x) - DiscriminativeHow to model Pr(w|x)? 1. Choose an appropriate form for Pr(w) 2. Make parameters a function of x 3. Function takes parameters q that define its shape 1. Choose Bernoulli dist. for Pr(w) 2. Make parameters a function of x 3. Function takes parameters f0 and f1 This model is called logistic regression. Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 42
43. 43. Two parametersLearning by standard methods (ML,MAP, Bayesian)Inference: Just evaluate Pr(w|x) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 43
44. 44. Type 2: Model Pr(x,y) - GenerativeHow to model Pr(x,w)? 1. Concatenate x and w to make z= [xT wT]T 2. Model the pdf of z 3. Pdf takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: compute Pr(w|x) using Bayes rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 44
45. 45. Type 2 ModelCan’t build this model very easily:• Concatenate continuous vector x and discrete w to make z• No obvious probability distribution to model joint probability of discrete and continuous Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 45
46. 46. Type 3: Pr(x|w) - GenerativeHow to model Pr(x|w)? 1. Choose an appropriate form for Pr(x) 2. Make parameters a function of w 3. Function takes parameters q that define its shapeLearning algorithm: learn parameters q from training data x,wInference algorithm: Define prior Pr(w) and then compute Pr(w|x) using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 46
47. 47. Type 3: Pr(x|w) - GenerativeHow to model Pr(x|w)? 1. Choose an appropriate form for Pr(x) 2. Make parameters a function of w 3. Function takes parameters q that define its shape 1. Choose a Gaussian distribution for Pr(x) 2. Make parameters a function of discrete binary w 3. Function takes parameters m0, m0, s20, s21 that define its shape Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 47
48. 48. Learn parameters m0, m0, s20, s21 that define its shape Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 48
49. 49. Inference algorithm: Define prior Pr(w) and then compute Pr(w|x) using Bayes’ rule Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 49
50. 50. Structure• Computer vision models – Three types of model• Worked example 1: Regression• Worked example 2: Classification• Which type should we choose?• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 50
51. 51. Which type of model to use?1. Generative methods model data – costly and many aspects of data may have no influence on world state Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 51
52. 52. Which type of model to use?2. Inference simple in discriminative models3. Data really is generated from world – generative matches this4. If missing data, then generative preferred5. Generative allows imposition of prior knowledge specified by user Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 52
53. 53. Structure• Computer vision models – Three types of model• Worked example 1: Regression• Worked example 2: Classification• Which type should we choose?• Applications Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 53
54. 54. Application: Skin Detection Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 54
55. 55. Application: Background subtraction Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 55
56. 56. Application: Background subtractionBut consider this scene in which the foliage is blowing in the wind. A normal distribution is not good enough! Need a way to make more complex distributions Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 56
57. 57. Future Plan• Seen three types of model – Probability density function – Linear regression – Logistic regression• Next three chapters concern these models Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 57
58. 58. Conclusion• To do computer vision we build a model relating the image data x to the world state that we wish to estimate w• Three types of model • Model Pr(w|x) -- discriminative • Model Pr(w,x) -- generative • Model Pr(w|x) – generative Computer vision: models, learning and inference. ©2011 Simon J.D. Prince 58