Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Doing Bayesian Data Analysis, Chapter 5

3,601 views

Published on

03/AUG/2013 @Matsuo lab, the University of Tokyo.

Published in: Education, Technology
  • Be the first to comment

Doing Bayesian Data Analysis, Chapter 5

  1. 1. www.***.com Part2: Chapter 5 Inferring a Binominal Proportion via Exact Mathematical Analysis Haru Negami 03/08/2013
  2. 2. summary ! binomial proportion " the likelihood function ! the Bernoulli likelihood function " the prior/posterior distribution ! beta distribution " estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+z) " comparison ---- best model <-p(D|M)
  3. 3. Binomial Proportion !  ! binomial or dichotomous !  " 
  4. 4. p(θ| ) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability
  5. 5. Likelihood function ! Example : coin flipping "  y = {0,1} ex) (0: , 1: ) " p(y=1|θ)=f(θ)=θ θ [0,1] " p(y=0|θ) = 1-θ " the Bernoulli distribution " p(y|θ) = θy(1-θ)(1-y) !  θ y !  (Σy p(y|θ) = 1)
  6. 6. Likelihood function ! Bernoulli likelihood function " p(y|θ) = θy(1-θ)(1-y) " y θ !  θ !  y "  ! p(y|θ) = θy(1-θ)(1-y)(y=i) θ
  7. 7. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability done
  8. 8. belief (to make a model) !  " p(θ|y) = p(y|θ)p(θ)/Σyp(y|θ) " p(θ) p(y|θ)p(θ) !  !  " denominator Σyp(y|θ) "  p(θ) a conjugate prior for p(y|θ)
  9. 9. belief (to make a model) ! p(θ) = θa(1-θ)b p(y|θ)×p(θ) = θy(1-θ)(1-y) × θa(1-θ)b = θy+a(1-θ)(1-y+b) !  beta distribution
  10. 10. belief (to make a model) ! beta distribution " a, b 2 (a,b > 0) " p(θ|a,b) = beta(θ|a,b) = θ(a-1)(1-θ)(b-1)/B(a,b) ! B(a,b) beta function " beta distribution normalizer ! B(a,b) = ∫0 1dθ θ(a-1)(1-θ)(b-1)
  11. 11. belief (to make a model) ! Beta Distribution b a
  12. 12. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability done
  13. 13. belief in detail (prior) ! beta distribution : beta(θ|a,b) " two parameters ! mean : ! standard deviation :
  14. 14. belief in detail (prior) ! beta distribution : beta(θ|a,b) " guess the values of a and b ! from (observed) data " ex) a=b=1, a=b=4, etc… ! m = a/(a+b), n=(a+b) a = mn, b = (1-m)n ! from mean and standard deviation a 1 & b 1 a<1 &/or b<1
  15. 15. p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability done done
  16. 16. belief in detail(posterior) ! supposition : N flips, z heads ! prior distribution : beta(θ|a,b) ! posterior distribution : beta(θ|z+a, N-z+b)
  17. 17. belief in detail(posterior) ! supposition : N flips, z heads ! prior distribution : beta(θ|a,b) ! posterior distribution : beta(θ|z+a, N-z+b) one of the beauties of mathematical approach to Bayesian inference!
  18. 18. belief in detail (updated parameters) ! probability distribution : " prior : beta(θ|a,b) N flips, z heads " posterior : beta(θ|z+a, N-z+b) ! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)] = (z+a)/(N+a+b) !!!
  19. 19. belief in detail (updated parameters) ! probability distribution : " prior : beta(θ|a,b) N flips, z heads " posterior : beta(θ|z+a, N-z+b) ! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)] = (z+a)/(N+a+b) !!!
  20. 20. belief in detail (updated parameters) ! probability distribution : " prior : beta(θ|a,b) N flips, z heads " posterior : beta(θ|z+a, N-z+b) ! mean : " prior : a/(a+b) " posterior : (z+a)/[(z+a)+(N-z+b)] = (z+a)/(N+a+b) !!! 0 z/N a/(a+b) 1-α α α = N/(N+a+b)
  21. 21. Discussion(?) Three inferential goals ! from chapter 4 " estimating the binominal proportion " predicting Data " comparing models
  22. 22. estimation ! uncertainty of the prior distribution " From the posterior distribution ! HDI : the highest density interval (chapter 3) HDI L : broad R : narrow prior dist. L : more uncertain
  23. 23. estimation !  reasonable credibility of a value concerned " From the posterior distribution ! ROPE : region of practical equivalence coin flipping θ = 0.5 credible? ROPE = [0.48,0.52] if 95% HDI ∩ ROPE = then θ is incredible
  24. 24. estimation !  reasonable credibility of a value concerned " From the posterior distribution ! ROPE : region of practical equivalence coin flipping θ = 0.5 credible? ROPE = [0.48,0.52] if 95% HDI ∩ ROPE = then θ is incredible includes many extra assumptions!
  25. 25. prediction ! p(y) = ∫dθp(y|θ)p(θ) <-posterior
  26. 26. prediction ! p(y) = ∫dθp(y|θ)p(θ) <-posterior 0 z/N a/(a+b) 1-α α α = N/(N+a+b)
  27. 27. prediction (example 1) ! 1st beta(θ|1,1) mean 1/2 (= p(y)) observation : head (N=1,z=1) ! 2nd beta(θ|2,1) mean 2/3 (= p(y)) observation : head (N=1,z=1) ! 3rd beta(θ|3,1) mean 3/4 (= p(y))
  28. 28. prediction (example 2) ! 1st beta(θ|100,100) : 1/2 (= p(y)) observation : head (N=1,z=1) observation : head (N=1,z=1) ! 3rd beta(θ|102,100) : 102/202( 50%)
  29. 29. comparison to compare the models, p(θ|D,M) = p(D|θ,M)p(θ|M)/p(D|M) posterior likelihood prior evidence prior probability observation posterior probability
  30. 30. comparison ! Calculation of evidence " p(D|M) = p(z,N)
  31. 31. comparison uniform strongly peaked uniform strongly peaked N = 14, z = 11 N = 14, z = 7 p(D|M)=0.000183>p(D|M)=6.86×10-5 p(D|M)=1.94×10-5<p(D|M)=5.9×10-5
  32. 32. comparison ! both are important " the prior distribution " the likelihood function " in detail, see chapter 4 The best model (so far) is not a good model.
  33. 33. summary ! binomial proportion " the likelihood function ! the Bernoulli likelihood function " the prior/posterior distribution ! beta distribution " estimation ---- uncertainty <-HDI " prediction ---- p(y)<-(z+a)/(N+a+b) " comparison ---- best model <-p(D|M)

×