Download as PDF, PPTX
Uploaded byYuki Soma
PRML 10.1節 ~ 10.3節 - 変分ベイズ法
PRML（パターン認識と機械学習 by Christopher M. Bishop）の10.1節 ~ 10.3節の解説スライド
![[PRML 3.1~3.2] Linear Regression / Bias-Variance Decomposition](https://cdn.slidesharecdn.com/ss_thumbnails/bayesianlinearregressionpart1-170117134816-thumbnail.jpg?width=640&height=640&fit=bounds)
PRML 10.1節 ~ 10.3節 - 変分ベイズ法
- 1. PRML 10.1 ~10.3 7/31 Yuki Soma
- 2. 10.1 VariationalInference . . . . . . . . . . . . . . . . . . . . . . 462 ◦ 10.1.1 Factorized distributions . . . . . . . . . . . . . . . . . . . . 464 ◦ 10.1.2 Properties of factorized approximations . . . . . . . . . . . 466 ◦ 10.1.3 Example: The univariate Gaussian . . . . . . . . . . . . . . 470 ◦ 10.1.4 Model comparison . . . . . . . . . . . . . . . . . . . . . . 473 10.2 Illustration: Variational Mixture of Gaussians . . . 474 ◦ 10.2.1 Variational distribution . . . . . . . . . . . . . . . . . . . . 475 ◦ 10.2.2 Variational lower bound . . . . . . . . . . . . . . . . . . . 481 ◦ 10.2.3 Predictive density . . . . . . . . . . . . . . . . . . . . . . . 482 ◦ 10.2.4 Determining the number of components . . . . . . . . . . . 483 ◦ 10.2.5 Induced factorizations . . . . . . . . . . . . . . . . . . . . 485 10.3 Variational Linear Regression . . . . . . . . .. . . . . . 486 ◦ 10.3.1 Variational distribution . . . . . . . . . . . . . . . . . . . . 486 ◦ 10.3.2 Predictive distribution . . . . . . . . . . . . . . . . . . . . 488 ◦ 10.3.3 Lower bound . . . . . . . . . . . . . . . . . . . . . . . . . 489
- 3. 𝐗 𝐙𝑝(𝐙|𝐗) 𝐙 ◦ ◦ 11 MCMC
- 4.
- 5. 𝐙 1. ◦ 𝑞𝐙 = 𝑞(𝐙|𝛚) 𝛚 2. 𝐙 ◦ ◦ 𝑞𝑖 𝐙𝑖 = 𝑞𝑖
- 6. ◦ ◦ 𝐙 𝐗 𝑝(𝐗, 𝐙) 𝑝(𝐙|𝐗) 𝑝(𝐗) ◦
- 7. ℒ(𝑞) 𝐾𝐿(𝑞| 𝑝= 0 𝑞 𝐙 = 𝑝(𝐙|𝐗)
- 8.
- 9. 10.5 ℒ(𝑞) KL 𝑞𝑗 = 𝑝 𝐗, 𝐙𝐣
- 10. ℒ(𝑞) 𝑞𝑗 ∗ 𝑗 = 1, … , 𝑀 𝑞𝑖 𝑖 ≠ 𝑗
- 11. 1. 𝑞 𝑗 2. foreach 𝑞𝑖: 𝑞𝑖 𝑞 𝑗 𝑞𝑖 𝑞𝑖 ◦ ℒ(𝑞) 𝑞𝑖
- 12.
- 13. KL KL𝑝(𝐙) 0 𝑞(𝐙) 0 KL 𝑞(𝐙) 𝑝(𝐙)
- 14.
- 15.
- 16. 𝑝(𝑚) 𝑚 ◦𝑝(𝑚|𝐗) 𝑞 𝐙, 𝑚 = 𝑞 𝐙 𝑞(𝑚) ◦ 𝐙 ⇒ 𝑞 𝐙, 𝑚 = 𝑞 𝐙|𝑚 𝑞(𝑚)
- 17. 10.10 ln𝑝 𝐗 = ℒ − 𝑞 𝑍 𝑚 𝑞 𝑚 ln 𝑝 𝑍,𝑚 𝑋 𝑞 𝑍 𝑚 𝑞 𝑚𝐙𝑚 ◦ ℒ = 𝑞 𝑍 𝑚 𝑞 𝑚 ln 𝑝(𝑍,𝑋,𝑚) 𝑞 𝑍 𝑚 𝑞 𝑚𝐙𝑚 ℒ 𝑞(𝑚) ◦ 𝑞(𝑚) ∝ 𝑝(𝑚)𝑒ℒ 𝑚 (10.36) ℒ 𝑚 = 𝑞 𝑍 𝑚 𝑞 𝑚 ln 𝑝(𝑍,𝑋|𝑚) 𝑞 𝑍 𝑚𝐙 ℒ 𝑚 𝑞(𝑍|𝑚) (10.36) 𝑞(𝑚)
- 18. ◦ 𝐾 𝐾 𝛑 1 ◦ 𝐳𝑖 1-of-𝐾 𝐾
- 19.
- 20. 𝜋 𝜇,Λ ◦ 𝐦0 = 𝟎
- 21. ◦ 𝑝 𝑞 ◦PRML
- 22. 1. E 𝑧𝑛𝑘 = 𝑟𝑛𝑘 2. 𝑞∗ (𝜋, 𝜇, Λ) 3. 𝑞∗ 𝐙 ◦ 𝑟𝑛𝑘 4. 2.
- 23.
- 24. 𝛼0 <1 ◦ 𝛼0 = 10−3
- 25.
- 26.
- 27. t 10.81 ◦ 𝑁 𝑞𝜋 𝑞(𝜇, Λ)
- 28. 𝐾 𝐾! K ln 𝐾! 10.22
- 29. 𝜋 𝜋𝑞 … 0 ◦ RVM 7.22
- 30.
- 31. 3.3 𝛼, 𝛽 ◦𝛽 10.6
- 32.
- 33. … 10.9𝛼
- 35.
- 36.
- 37.