Your SlideShare is downloading. ×
0
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Prml
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Prml

2,414

Published on

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,414
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
83
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. PRML 2.4 (id:syou6162) June 13, 2009 (id:syou6162) PRML 2.4
  • 2. 2.4 2 x η p(x|η) = h(x)g(η) exp {ηT µ(x)} x η (natural parameter) u(x) x (id:syou6162) PRML 2.4
  • 3. Figure: Figure: Figure: Figure: t (id:syou6162) PRML 2.4
  • 4. ? hoge ← (id:syou6162) PRML 2.4
  • 5. hoge ( ) η ηML ( ) (id:syou6162) PRML 2.4
  • 6. ∈ (1/2) : p(x|µ) = Bern(x|µ) = µ x (1 − µ)1−x → p(x|η) = h(x)g(η) exp {ηT µ(x)} p(x|µ) = (1 − µ) exp {log( 1−µ )x} µ natural parameter η η= log( 1−µ ) µ µ µ = σ(η) 1 σ(η) = 1+exp(−η) (id:syou6162) PRML 2.4
  • 7. ∈ (2/2) (2.194) p(x|η) = σ(−η) exp(ηx) p(x|η) = h(x)g(η) exp {ηT µ(x)} η = log( 1−µ ) µ µ(x) = x h(x) = 1 g(η) = σ(−η) (id:syou6162) PRML 2.4
  • 8. ∈ : M x M p(x|µ) = k=1 µk k = exp { k=1 xk log µk } (2.194) p(x|µ) = exp(ηT x) ηk = log uk η = (η1 , · · · , ηM )T p(x|η) = h(x)g(η) exp {ηT µ(x)} µ(x) = x h(x) = 1 g(η) = 1 (id:syou6162) PRML 2.4
  • 9. uk (k = 1, · · · , M) M k=1 uk =1 → uk M−1 M−1 (id:syou6162) PRML 2.4
  • 10. M     exp  xk log uk        k=1    M−1    = exp  xk log uk + x M log u M        k=1    M−1  M−1    M−1      = exp  xk log uk + 1 − xk  log 1 −      µk                     k=1 k=1 k=1   M−1 M−1  M−1   M−1       = exp  xk log uk − xk log 1 −  + log 1 −      µk  µk                   k=1 k=1 k=1 k=1   M−1    M−1   µk   = exp  xk log   + log 1 −        µk      M−1   1 − j=1 µ j            k=1 k=1  M−1    M−1      µk  = 1 − µk  exp  xk log            M−1   1 − j=1 µ j             k=1 k=1  (id:syou6162) PRML 2.4
  • 11. (1/2) M−1 M−1 1− k=1 µk exp k=1 xk log 1− µk M−1 µj j=1 log 1− µk M−1 µj = ηk j=1 k exp(ηk ) µk = 1+ M−1 exp(η j ) j=1 4 (id:syou6162) PRML 2.4
  • 12. (2/2) M−1 −1 p(x|η) = 1 + j=1 exp(η j ) exp(ηT x) natural parameter η = (η1 , · · · , ηM−1 )T p(x|η) = h(x)g(η) exp {ηT µ(x)} µ(x) = x h(x) = 1 M−1 −1 g(η) = 1 + j=1 exp(η j ) (id:syou6162) PRML 2.4
  • 13. ∈ : 1 p(x|µ, σ) = 1 exp {− 1 σ2(x − µ)2} 2 (2πσ2 ) 2 (2.194) 1 1 1 2 p(x|µ, σ) = 1 exp {− 2σ2 x2 + µ σ2 x − 2σ2 µ} (2πσ2 ) 2 p(x|η) = h(x)g(η) exp {ηT µ(x)} µ/σ2 η= −1/2σ2 x µ(x) = x2 1 h(x) = (2π)− 2 1 η2 g(η) = (−2η2 ) 2 exp ( 4η12 ) (id:syou6162) PRML 2.4
  • 14. 2.4.1 η p(x|η) = h(x)g(η) exp {ηT µ(x)} → g(η) h(x) exp {ηT u(x)}dx + g(η) h(x) exp {ηT u(x)}u(x)dx = 0 − log g(η) = E[u(x)] − log g(η) = cor[u(x)] (id:syou6162) PRML 2.4
  • 15. & i.i.d. X = (x1, · · · , xn ) : p(X|η) = L(η; X) = N N n=1 h(xn ) g(η)N exp ηT n=1 u(xn ) : 1 N − g(ηML ) = N n=1 u(xn ) N n=1 u(xn ) (sufficient statistic) (id:syou6162) PRML 2.4
  • 16. u(x) = x N n=1 xn u(x) = (x, x2 )T N N ( n=1 xn, n=1 x2 )T n ( ) 8 (id:syou6162) PRML 2.4
  • 17. 2.4.2 : ( ) ( ) (id:syou6162) PRML 2.4
  • 18. p(η|χ) = f (χ, ν)g(η)ν exp {νηT χ} = × = p(η|χ) × p(X|η) = f (χ, ν)g(η)ν exp {νηT χ}  N   N   g(η)N exp ηT   h(xn) u(xn )      ×           n=1 n=1     N   T    ∝ g(η) µ+N exp η     u(xn) + νχ       n=1  (id:syou6162) PRML 2.4
  • 19. ν µ(x) χ (id:syou6162) PRML 2.4
  • 20. 2.4.3 ν ν → (id:syou6162) PRML 2.4
  • 21. 2.4.3 λ K 1 p(λ) = K x 1 p(x) = b−a constant 1 1 2 (id:syou6162) PRML 2.4
  • 22. 1: 1 → (improper prior) ( ) (id:syou6162) PRML 2.4
  • 23. (2.3.6 ) p(µ) = N(µ0 , σ2) 0 → µ0 = 0 → σ0 → ∞ → σ2 Nσ20 σ2 /σ2 0 N µN = µ Nσ2 +σ 0 + µ Nσ2 +σ ML = N+σ/σ2 0 µ + µ N+σ/σ2 ML → 0 0 0 0 µML 1 1 N N σ2 = σ2 + σ2 → σ2 N 0 (id:syou6162) PRML 2.4
  • 24. (1/2) h(λ) λ = η2 η = h(η2) ˆ pλ (λ) λ = η2 pη (η) = pλ (λ)| dλ | = pλ (η2 )2η ∝ η dη pλ (λ) pη (η) (id:syou6162) PRML 2.4
  • 25. (2/2) ? (translation invariance) (scale invariance) (id:syou6162) PRML 2.4
  • 26. p(x|µ) = f (x − µ) x x = x+c ˆ µ =µ+c ˆ p(x|µ) = f ( x − µ) = f ((x + c) − (µ + c)) = p( x|µ) ˆ ˆ ˆˆ (id:syou6162) PRML 2.4
  • 27. A≤µ≤B A−c ≤µ≤ B−c B B−c B A p(µ)dµ = p(µ)dµ = A−c A p(µ − c)dµ A B → p(µ − c) = p(µ) p(µ) ? (id:syou6162) PRML 2.4
  • 28. µ µ0 = 0 σ2 Nσ20 µN = µ Nσ2 +σ 0 + µ Nσ2 +σ ML = 0 0 σ2 /σ2 0 N N+σ/σ2 0 µ + µ N+σ/σ2 ML → µML 0 0 σ2 → 0 ∞ µ (id:syou6162) PRML 2.4
  • 29. 1 x σ>0 p(x|σ) = σ f (σ) x x = cx ˆ σ = cσ ˆ 1 x 1 cx 1 cx p(x|σ) = σ f ( σ ) = σ f ( cσ ) = σ f(σ) = ˆ 1 x ˆ 1 x ˆ cσ f ( σ ) = σ f ( σ )p( x|σ) ˆ ˆ ˆ ˆˆ (id:syou6162) PRML 2.4
  • 30. A≤σ≤B A/c ≤ µ ≤ B/c B B/c B A p(σ)dσ = p(σ)dσ = A/c A p( 1 σ) 1 dσ c c A B → p(σ) = p( 1 σ) 1 c c 1 p(σ) ∝ σ ? ? f (x) = 1/x x = 10 f (10/2) × 2 = f (5) × 1 = 0.2 × 1 2 1 2 = 0.1 = f (10) 1/x (id:syou6162) PRML 2.4
  • 31. µ σ N(x|µ, σ2 ) ∝ σ−1 exp{−( x/σ)2 } ˆ x= x−µ ˆ λ = 1/σ2 p(σ) ∝ 1/σ p(λ) ∝ 1/λ 3 3 p(λ) = p(σ) × | dσ | = p(σ) × | − λ− 2 /2| ∝ 1/σ × λ− 2 = dλ 1/2 −3 2 = 1/λ λ ×λ √ σ = 1/λ 3 dσ/dλ = −λ− 2 /2 λ (2.146) Gam(λ|a0 , b0 ) (id:syou6162) PRML 2.4
  • 32. Figure: (2.146) Gam(λ|a, b) a b a=b=0 (id:syou6162) PRML 2.4
  • 33. a0 = 0 b0 = 0 a N = a0 + N2 N bN = 1 n=1 (xn − µ)2 = b0 + N σ2 2 2 ML a0 = 0 b0 = 0 (id:syou6162) PRML 2.4

×