Upcoming SlideShare
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

# Saving this for later?

### Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Standard text messaging rates apply

# Common Probability Distibution

103
views

Published on

Presentation for reading session of Computer Vision: Models, Learning, and Inference

Presentation for reading session of Computer Vision: Models, Learning, and Inference

Published in: Technology

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total Views
103
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
1
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Transcript

• 1. CHAPTER 3:COMMON PROBABILITYDISTRIBUTIONSCOMPUTER VISION: MODELS, LEARNING ANDINFERENCELukas Tencer
• 2. 2 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 3. Why model these complicated quantities?3 Because we need probability distributions over model parameters as well as over data and world state. Hence, some of the distributions describe the parameters of the others: Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 4. Why model these complicated quantities?4 Because we need probability distributions over model parameters as well as over data and world state. Hence, some of the distributions describe the parameters of the others: Example: Parameters modelled by: Models variance Models mean Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 5. Bernoulli Distribution5 or For short we write:Bernoulli distribution describes situation where only twopossible outcomes y=0/y=1 or failure/successTakes a single parameter Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 6. Beta Distribution6 Defined over data (i.e. parameter of Bernoulli) • Two parameters both > 0 For short we write: • Mean depends on relative values E[ ] = . • Concentration depends on magnitude Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 7. Categorical Distribution7 or can think of data as vector with all elements zero except kth e.g. e4 = [0,0,0,1,0] For short we write:Categorical distribution describes situation where K possibleoutcomes y=1… y=k.Takes K parameters where Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 8. Dirichlet Distribution8Defined over K values where Or for short: Has k parameters k>0 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 9. Univariate Normal Distribution9 For short we write: Univariate normal distribution describes single continuous variable. Takes 2 parameters and Computer vision: models, learning2and inference. ©2011 >0Simon J.D. Prince
• 10. Normal Inverse Gamma10 Distribution Defined on 2 variables and 2>0 or for short Four parameters and Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 11. Multivariate Normal Distribution11 For short we write: Multivariate normal distribution describes multiple continuous variables. Takes 2 parameters • a vector containing mean position, • a symmetric “positive definite” covariance matrix Positive definite: is positive for any real Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 12. Types of covariance12 Covariance matrix has three forms, termed spherical, diagonal and full Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 13. Normal Inverse Wishart13 Defined on two variables: a mean vector and a symmetric positive definite matrix, . or for short: Has four parameters • a positive scalar, • a positive definite matrix • a positive scalar, • a vector Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 14. Samples from Normal Inverse14 Wishart (dispersion) (ave. Covar) (disper of means) (ave. of means) Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 15. Conjugate Distributions15 The pairs of distributions discussed have a special relationship: they are conjugate distributions  Beta is conjugate to Bernouilli  Dirichlet is conjugate to categorical  Normal inverse gamma is conjugate to univariate normal  Normal inverse Wishart is conjugate to multivariate normal Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 16. Conjugate Distributions16 When we take product of distribution and it’s conjugate, the result has the same form as the conjugate. For example, consider the case where then a constant A new Beta distribution Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 17. Example proof17 When we take product of distribution and it’s conjugate, the result has the same form as the conjugate. Computer vision: models, learning and inference. ©2011 Simon J.D. 17 Prince
• 18. Bayes’ Rule Terminology18 Likelihood – propensity Prior – what we know for observing a certain about y before seeing value of x given a certain x value of y Posterior – what we Evidence – a constant to know about y after ensure that the left hand seeing x side is a valid distribution Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 19. Importance of the Conjugate19 Relation 1 1. Choose prior  Learning parameters: that is conjugate to likelihood 2. Implies that posterior 3. Posterior must be a distribution must have same form as which implies that evidence must conjugate prior equal constant from conjugate distribution relation Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• 20. Importance of the Conjugate20 Relation 2  Marginalizing over parameters2. Integral becomes easy --the product 1. Chosen sobecomes a constant times a distribution conjugate to othe termIntegral of constant times probabilitydistribution= constant times integral of probabilitydistribution = constant vision: models, learning and inference. Computer x 1 = constant ©2011 Simon J.D. Prince
• 21. Conclusions21 • Presented four distributions which model useful quantities • Presented four other distributions which model the parameters of the first four • They are paired in a special way – the second set is conjugate to the other • In the following material we’ll see that this relationship is verymodels, learning and inference. ©2011 Computer vision: useful Simon J.D. Prince
• 22. 22 Thank You for you attentionBased on:Computer vision: models, learning and inference. ©2011 Simon J.D. Princehttp://www.computervisionmodels.com/