Like this presentation? Why not share!

# Ml4nlp04 1

## on Oct 11, 2010

• 2,238 views

### Views

Total Views
2,238
Views on SlideShare
1,102
Embed Views
1,136

Likes
1
11
0

### Report content

• Comment goes here.
Are you sure you want to

## Ml4nlp04 1Presentation Transcript

• 4.1 4.2 . . 2010/10/12 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• 4 1 2 . . 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• (classiﬁcation categorization) 4.1 4.2
• 2 SVM 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• d P(c|d) c∈C P(c|d) 1 P(c)P(d|c) P(c|d) = P(d) . 2 P(d) P(c)P(d|c) c max . P(c)P(d|c) c max = arg max c P(d) = arg max P(c)P(d|c) c . 4.1 4.2
• P(d|c) d d d P(d|c) d - - 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• - P(d|c) d ∏ δω,d P(d|c) = pω,c (1 − pω,c )1−δω,d ω∈V V ω pc P(c) pω,c pc 2 δω,d pω,c (1 − pω,c )1−δω,d c ω d 4.1 4.2
• - ∏ δω,d P(c)P(d|c) = pc pω,c (1 − pω,c )1−δω,d ω∈V c pω,c P(d|c) P(d|c) pω,c 1 4.1 4.2
• D D = {(d(1) , c(1) ), (d(2) , c(2) ), ..., (d|D| , c|D| )} ∑ log P(D) = log P(d, c) (d,c)∈D   ∑  ∏ δω,d      = log  pc    pω,c (1 − pω,c )1−δω,d     (d,c)∈D ω∈V   ∑   ∑    log pc +  =    (δω,d log pω,c + (1 − ωω,d ) log(1 − pω,c ))    (d,c)∈D ω∈V ∑ ∑∑ ∑∑ = N c log pc + Nω,c log pω,c + (N c − Nω,c ) log(1 − pω,c ) c c ω∈V c ω∈V Nc : c Nω,c : c ω 4.1 4.2
• pc max . log P(D) ∑ s.t. pc = 1. c L(θ, λ) ∑        L(θ, λ) = log P(D) + λ   pc − 1    c θ: { pomega,c }ωinV,c∈C , {pc } c∈C 4.1 4.2
• ∂L(θ, λ) = 0 ∂ pω,c ∂L(θ, λ) = 0 ∂ pc ∂L(θ, λ) = 0 ∂λ 4.1 4.2
•  ∂L(θ, λ) ∂ ∑   ∑∑ =    N c log pc + Nω,c log pω,c ∂ pω,c ∂ pω,c  c c ω∈V ∑∑ ∑      + (N c − Nω,c ) log(1 − pω,c ) + λ      pc − 1    c ω∈V c ∂(1−pω,c ) Nω,c ∂pω,c = + (N c − Nω,c ) pω,c (1 − pω,c ) Nω,c (N c − Nω,c ) = − pω,c 1 − pω,c  ∂L(θ, λ) ∂ ∑  ∑∑ =    N c log pc + Nω,c log pω,c ∂pc  ∂ pc c c ω∈V ∑∑ ∑      +   (N c − Nω,c ) log(1 − pω,c ) + λ    pc − 1    c ω∈V c Nc = +λ pc 4.1 4.2
• pω,c Nω,c (N c − Nω,c ) − = 0 pω,c 1 − pω,c (1 − pω,c )Nω,c − pω,c (N c − Nω,c ) = 0 pω,c (N c − Nω,c + Nω,c ) = Nω,c Nω,c pω,c = Nc 4.1 4.2
• pc Nc +λ = 0 pc Nc pc = − λ ∑ pc = 1 c 1∑ − Nc = 1 λ c ∑ λ = − Nc c Nc Nc pc = − = ∑ λ c Nc 4.1 4.2
• c ω pω,c = c c pc = 4.1 4.2
• 4.1 P 3 d(1) = ”good bad good good” d(2) = ”exciting exciting” d(3) = ”good good exciting boring” N 3 d(4) = ”bad boring boring boring” d(5) = ”bad good bad” d(6) = ”bad bad boring exciting” P N 4.1 4.2
• 4.1 V = {bad, boring, exciting, good} N P = 3, N N = 3, N bad,P = 1, N bad,N = 3, N boring,P = 1, N boring,N = 2, Nexciting,P = 2, Nexciting,N = 1, N good,P = 2, N good,N = 1, NP NN pP = N P +N N = 3+3 = 0.50 3 pN = N p+NN = 3+3 = 0.50 3 N bad,P N bad,N pbad,P = N P = 1 = 0.33 3 pbad,N = NN = 3 = 1.00 3 N boring,P N bof ing,N pboring,P = N P = 3 = 0.33 1 pbof ing,N = NN = 2 = 3 0.67 Nexciting,P pexciting,P = N P = 2 = 0.67 3 Nexciting,N 1 pexciting,N = = 3 = 0.33 N good,P N good,N pgood,P = N P = 2 = 0.67 3 pgood,N = NN = 1 = 0.33 3 4.1 4.2
• 4.2 4.1 d d = ”good good bad boring” pP pd|P pN pd|N pP pd|P = pP × pbad,P × pboring,P × (1 − pexciting,P ) × pgood,P = 0.5 × 0.33 × 0.33 × (1 − 0.67) × 0.67 = 0.012 pN pd|N = pN × pbad,N × pboring,N × (1 − pexciting,N ) × pgood,N = 0.5 × 1.00 × 0.67times(1 − 0.33) × 0.33 = 0.074 4.1 d N 4.1 4.2
• 4.2 4.1 d d = ”good good bad boring” pP pd|P pN pd|N pP pd|P = pP × pbad,P × pboring,P × (1 − pexciting,P ) × pgood,P = 0.5 × 0.33 × 0.33 × (1 − 0.67) × 0.67 = 0.012 pN pd|N = pN × pbad,N × pboring,N × (1 − pexciting,N ) × pgood,N = 0.5 × 1.00 × 0.67times(1 − 0.33) × 0.33 = 0.074 4.1 d N 4.1 4.2
• 4.3 4.1 d(1) d(1) = ”good bad good good ﬁne” d d = ”bad bad boring boring ﬁne” 4.1 4.2
• 4.3 “ﬁne” ﬁne N f ine,P N f ine,N p f ine,P = NP = 1 3 = 0.33 p f ine,N = NN = 0 3 = 0.00 pP pd|P = pP × pbad,P × pboring,P × (1 − pexciting,P ) × p f ine,P × (1 − pgood,P ) = 0.5 × 0.33 × 0.33 × (1 − 0.67) × 0.33 × (1 − 0.67) = 0.002 pN pd|N = pN × pbad,N × pboring,N × (1 − pexciting,N ) × p f ine,N × (1 − pgood,N ) = 0.5 × 1.00 × 0.67 × (1 − 0.33) × 0.00 × 0.67 = 0.00 P 4.1 4.2
• 4.3 d “bad” ”boring” ”good” ”exciting” P p f ine,N = 0.00 N pN pd|N = 0.00 0 MAP 4.1 4.2
• MAP 0.00 MAP ∏  ∏  ∑      × α−1    α−1    log P(θ) + log P(D) ∝ log  pc      pω,c  +       log P(d, c) c ω,c (d,c)∈ D ∑ ∑ = (α − 1) log pc + (α − 1) log pω,c c ω,c   ∑  ∏ δ     1−δω,d   + log  pc   ω,d ( pω,c (1 − pω,c ) )    (d,c)∈ D ω∈V ∑ c p(c) = 1 4.1 4.2
• MAP ∑        L(θ, λ) = log P(θ) + log P( D) + λ   pc − 1    c ∂L(θ, λ) (α − 1) Nω,c N c − Nω,c = + − ∂ pω,c pω,c pω,c 1 − pω,c ∂L(θ, λ) (α − 1) N c = + +λ ∂ pc pc pc 4.1 4.2
• MAP ∑ 0 c pc = 1 Nω,c + (α − 1) pω,c = Nc + 2 Nc + 1 pc = ∑ c N c + |C| α 4.1 4.2
• 4.4 4.3 MAP α=1 P 3 d(1) = ”good bad good good ﬁne” d(2) = ”exciting exciting” d(3) = ”good good exciting boring” N 3 d(4) = ”bad boring boring boring” d(5) = ”bad good bad” d(6) = ”bad bad boring exciting” 4.1 4.2
• 4.4 Table: MAP MAP pP 0.50 0.50 pN 0.50 0.50 pbad,P 0.33 0.40 pbad,N 1.00 0.80 pboring,P 0.33 0.40 pboring,N 0.67 0.60 pexciting,P 0.67 0.60 pexciting,N 0.33 0.40 p f ine,P 0.33 0.40 p f ine,N 0.00 0.20 pgood,P 0.67 0.60 pgood,N 0.33 0.40 MAP smoothing MAP 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• V 1 |d| P(d|c) d ω nω,d  ∑  (∑ n )! ∏     ω ω,d P(d|c) = P  K =  nω,d  ∏  nω,d     qω,c ω ω∈V nω,d ! ω∈V K: ( ∑ ) ∑ P K = ω nω,d : ω nω,d 4.1 4.2
• c ∑  (∑ n )! ∏     ω ω,d pc P   nω,d  ∏  nω,d P(c)P(d|c) =     qω,c ω ω∈V nω,d ! ω∈V ∑  (∑ n )! ∏     ω ω,d arg max P(c)P(d|c) = arg max pc P   nω,d  ∏  n     q ω,d c c ω ω∈V nω,d ! ω∈V ω,c ∏ nω = arg max pc qω,c c ω∈V ∏ nω c pc ω∈V qω,c 4.1 4.2
• 2 4.1 4.2
• ∑ log P( D) = log P(d, c) (d,c)∈ D   ∑  p(|d|)|d|!  ∏ n     ω,d  = log  ∏   pc qω,c    (d,c)∈ D ω∈Vn ! ω,d ω∈V ∑ P(|d|)|d|! ∑ ∑ ∑ = log ∏ + log pc + nω,d log qω,c (d,c)∈ D ω∈V nω,d ! (d,c)∈ D (d,c)∈ D ω∈V ∑ P(|d|)|d|! ∑ ∑∑ = log ∏ + log nc pc + nω,c log qω,c (d,c)∈ D ω∈V nω,d ! c c ω∈V max. log P(D) ∑ s.t. pc = 1. c∈C ∑ qω,c = 1; ∀c ∈ C ω∈V 4.1 4.2
•     ∑ ∑    ∑         L(θ, β, γ) = log P(D) + βc    qω,c − 1 + γ        pc − 1    c∈C ω∈V c∈C ∂L(θ, β, γ) = 0 ∂qω,c ∂L(θ, β, γ) = 0 ∂ pc ∂L(θ, β, γ) = 0 ∂β ∂L(θ, β, γ) = 0 ∂γ 4.1 4.2
•  ∂L(θ, β, γ) ∂  ∑   P(|d|)|d|! ∑ ∑∑ =    log ∏ + nc log pc +   nω,c log qω,c ∂qω,c ∂qω,c (d,c)∈D ω∈V nω,d ! c c ω∈V  ∑ ∑ ∑     βc ( −1) + γ( pc − 1)  c∈C ω∈V c∈C nω,c = + βc = 0 qω,c nω,c qω,c = βc 4.1 4.2
• βc ∑ qω,c = 1 ω∈V 1 ∑ nω,c = 1 β c ω∈V 1 βc = ∑ ω∈V nω,c nω,c qω,c = ∑ ω nω,c pc 4.1 4.2
• c ω qω,c = c c ω pω,c = c 4.1 4.2
• MAP 0.00 MAP MAP ∏  ∏  ∑        α−1  log P(θ) + log P(D) ∝ log       pα−1  ×      qω,c  +   c    log P(d, c) c ω,c (d,c)∈D     ∑   ∑    ∑  P(|d|)|d|!   ∏ n  ω,d   =  (α − 1)   log pc + log qω,c  +   log  ∏   pc qω,c      n !  c ω,c (d,c)∈D ω∈V ω,d ω∈V ∑ ∑ c p(c) = 1 ω qω,c = 1 4.1 4.2
• MAP L(θ, β, γ) = log P(θ) + log P(D)     ∑ ∑    ∑        + βc     pω,c − 1 + γ        pc − 1    c∈C ω∈V c∈C ∂L(θ, β, γ) (α − 1) nω,c = + + βc ∂qω,c qω,c qω,c ∑ 0 ω∈V qω,c = 1 nω,c + (α − 1) qω,c = ∑ ω nω,c + |W|(α − 1) 4.1 4.2
• AGENDA 1 2 . 3 . 4 . . 4.1 4.2
• d MAP 4.1 4.2
• ( ) Ml for nlp chapter 4 4.1 4.2
• 4.1 4.2