Successfully reported this slideshow.
Upcoming SlideShare
×

# Doing Bayesian Data Analysis, Chapter 5

3,661 views

Published on

03/AUG/2013 @Matsuo lab, the University of Tokyo.

Published in: Education, Technology
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

### Doing Bayesian Data Analysis, Chapter 5

1. 1. www.***.com Part2: Chapter 5 Inferring a Binominal Proportion via Exact Mathematical Analysis Haru Negami 03/08/2013
2. 2. summary ! binomial proportion " the likelihood function ! the Bernoulli likelihood function " the prior/posterior distribution ! beta distribution " estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+z) " comparison ---- best model <-p(D|M)
3. 3. Binomial Proportion !  ! binomial or dichotomous !  "
4. 4. p(θ| ) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability
5. 5. Likelihood function ! Example : coin flipping "  y = {0,1} ex) (0: , 1: ) " p(y=1|θ)=f(θ)=θ θ [0,1] " p(y=0|θ) = 1-θ " the Bernoulli distribution " p(y|θ) = θy(1-θ)(1-y) !  θ y !  (Σy p(y|θ) = 1)
6. 6. Likelihood function ! Bernoulli likelihood function " p(y|θ) = θy(1-θ)(1-y) " y θ !  θ !  y "  ! p(y|θ) = θy(1-θ)(1-y)(y=i) θ
7. 7. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability done
8. 8. belief (to make a model) !  " p(θ|y) = p(y|θ)p(θ)/Σyp(y|θ) " p(θ) p(y|θ)p(θ) !  !  " denominator Σyp(y|θ) "  p(θ) a conjugate prior for p(y|θ)
9. 9. belief (to make a model) ! p(θ) = θa(1-θ)b p(y|θ)×p(θ) = θy(1-θ)(1-y) × θa(1-θ)b = θy+a(1-θ)(1-y+b) !  beta distribution
10. 10. belief (to make a model) ! beta distribution " a, b 2 (a,b > 0) " p(θ|a,b) = beta(θ|a,b) = θ(a-1)(1-θ)(b-1)/B(a,b) ! B(a,b) beta function " beta distribution normalizer ! B(a,b) = ∫0 1dθ θ(a-1)(1-θ)(b-1)
11. 11. belief (to make a model) ! Beta Distribution b a
12. 12. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability done
13. 13. belief in detail (prior) ! beta distribution : beta(θ|a,b) " two parameters ! mean : ! standard deviation :
14. 14. belief in detail (prior) ! beta distribution : beta(θ|a,b) " guess the values of a and b ! from (observed) data " ex) a=b=1, a=b=4, etc… ! m = a/(a+b), n=(a+b) a = mn, b = (1-m)n ! from mean and standard deviation a 1 & b 1 a<1 &/or b<1
15. 15. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability done done
16. 16. belief in detail(posterior) ! supposition : N flips, z heads ! prior distribution : beta(θ|a,b) ! posterior distribution : beta(θ|z+a, N-z+b)
17. 17. belief in detail(posterior) ! supposition : N flips, z heads ! prior distribution : beta(θ|a,b) ! posterior distribution : beta(θ|z+a, N-z+b) one of the beauties of mathematical approach to Bayesian inference!
18. 18. belief in detail (updated parameters) ! probability distribution : " prior : beta(θ|a,b) N flips, z heads " posterior : beta(θ|z+a, N-z+b) ! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)] = (z+a)/(N+a+b) !!!
19. 19. belief in detail (updated parameters) ! probability distribution : " prior : beta(θ|a,b) N flips, z heads " posterior : beta(θ|z+a, N-z+b) ! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)] = (z+a)/(N+a+b) !!!
20. 20. belief in detail (updated parameters) ! probability distribution : " prior : beta(θ|a,b) N flips, z heads " posterior : beta(θ|z+a, N-z+b) ! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)] = (z+a)/(N+a+b) !!! 0 z/N a/(a+b) 1-α α α = N/(N+a+b)
21. 21. Discussion(?) Three inferential goals ! from chapter 4 " estimating the binominal proportion " predicting Data " comparing models
22. 22. estimation ! uncertainty of the prior distribution " From the posterior distribution ! HDI : the highest density interval (chapter 3) HDI L : broad R : narrow prior dist. L : more uncertain
23. 23. estimation !  reasonable credibility of a value concerned " From the posterior distribution ! ROPE : region of practical equivalence coin flipping θ = 0.5 credible? ROPE = [0.48,0.52] if 95% HDI ∩ ROPE = then θ is incredible
24. 24. estimation !  reasonable credibility of a value concerned " From the posterior distribution ! ROPE : region of practical equivalence coin flipping θ = 0.5 credible? ROPE = [0.48,0.52] if 95% HDI ∩ ROPE = then θ is incredible includes many extra assumptions!
25. 25. prediction ! p(y) = ∫dθp(y|θ)p(θ) <-posterior
26. 26. prediction ! p(y) = ∫dθp(y|θ)p(θ) <-posterior 0 z/N a/(a+b) 1-α α α = N/(N+a+b)
27. 27. prediction (example 1) ! 1st beta(θ|1,1) mean 1/2 (= p(y)) observation : head (N=1,z=1) ! 2nd beta(θ|2,1) mean 2/3 (= p(y)) observation : head (N=1,z=1) ! 3rd beta(θ|3,1) mean 3/4 (= p(y))
28. 28. prediction (example 2) ! 1st beta(θ|100,100) : 1/2 (= p(y)) observation : head (N=1,z=1) observation : head (N=1,z=1) ! 3rd beta(θ|102,100) : 102/202( 50%)
29. 29. comparison to compare the models, p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability
30. 30. comparison ! Calculation of evidence " p(D|M) = p(z,N)
31. 31. comparison uniform strongly peaked uniform strongly peaked N = 14, z = 11 N = 14, z = 7 p(D|M)=0.000183>p(D|M)=6.86×10-5 p(D|M)=1.94×10-5<p(D|M)=5.9×10-5
32. 32. comparison ! both are important " the prior distribution " the likelihood function " in detail, see chapter 4 The best model (so far) is not a good model.
33. 33. summary ! binomial proportion " the likelihood function ! the Bernoulli likelihood function " the prior/posterior distribution ! beta distribution " estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+b) " comparison ---- best model <-p(D|M)