A neural attention model for sentence summarization, EMNLP'15 の論文紹介．http://aclweb.org/anthology/D15-1044

7. x M x1, ..., xM
y1, ..., yNy N
y (N < M)
argmax s(x, y)
y 2
s

8. argmax s(x, y)
y 2
argmax s(x, x[m1,...,mN ])
m 2 {1, ..., M}N
m 2 {1, ..., M}N
, mi 1 < mi
argmax s(x, x[m1,...,mN ])
x[i,j,k]

9. s(x, y) ⇡
N 1X
i=0
g(yi+1, x, yc)
log p(y|x; ✓) ⇡
N 1X
i=0
log p(yi+1|x, yc; ✓)
s(x, y) = log p(y|x; ✓)
C
yc

10. Cs(x, y) ⇡
N 1X
i=0
g(yi+1, x, yc)
log p(y|x; ✓) ⇡
N 1X
i=0
log p(yi+1|x, yc; ✓)
s(x, y) = log p(y|x; ✓)
C
yc

11. s(x, y) ⇡
N 1X
i=0
g(yi+1, x, yc)
log p(y|x; ✓) ⇡
N 1X
i=0
log p(yi+1|x, yc; ✓)
s(x, y) = log p(y|x; ✓)
C
yc

12. x y y
argmax s(x, y)
y 2
argmax s(x, y)
y 2
argmax s(x, x[m1,...,mN ])
m 2 {1, ..., M}N
m 2 {1, ..., M}N
, mi 1 < mi
argmax s(x, x[m1,...,mN ])
s(x, y) = log p(y|x; ✓) ⇡
N 1X
i=0
log p(yi+1|x, yc; ✓)

14. log p(yi+1|x, yc; ✓)

18. p(yi+1|yc, x; ✓) / exp(Vh + Wenc(x, yc))
˜yc = [Eyi C+1, ..., Eyi]
h = tanh(U˜yc)
✓ = (E, U, V, W)

19. p(yi+1|yc, x; ✓) / exp(Vh + Wenc(x, yc))
˜yc = [Eyi C+1, ..., Eyi]
h = tanh(U˜yc)
✓ = (E, U, V, W)
E =
Eyi

20. p(yi+1|yc, x; ✓) / exp(Vh + Wenc(x, yc))
˜yc = [Eyi C+1, ..., Eyi]
h = tanh(U˜yc)
✓ = (E, U, V, W)
˜ycU h
tanh
CV
H H

21. p(yi+1|yc, x; ✓) / exp(Vh + Wenc(x, yc))
˜yc = [Eyi C+1, ..., Eyi]
h = tanh(U˜yc)
✓ = (E, U, V, W)
H H
V
Vh +
+ V V
W enc(x, yc)

22. p(yi+1|yc, x; ✓) / exp(Vh + Wenc(x, yc))
˜yc = [Eyi C+1, ..., Eyi]
h = tanh(U˜yc)
✓ = (E, U, V, W)

23. enc1(x, yc) = pT
˜x p = [1/M, ..., 1/M] ˜x = [Fx1, ..., FxM ]

24. 8i, l 2 {1, ..., L}, ˜xl
j = tanh(max{¯xl
2i 1, ¯xl
2i})
˜x0
= [Fx1, ..., FxM ]
8j, enc2(x, yc)j = max ˜xL
i,j
i
8i, l 2 {1, ..., L}, ¯xl
i = Ql
˜xl 1
[1 Q,...,1+Q]

25. enc3(x, yc) = pT
¯x
p / exp(˜xP˜y
0
c)
˜y
0
c = [Gyi C+1, ..., Gyi]
˜x = [Fx1, ..., FxM ]
i + Q
q = i Q
8i ¯xi =
X
˜xq/Q

27. (x(1)
, y(1)
), ..., (x(J)
, y(J)
)
J

28. y⇤
= argmax
X
g(yi+1, x, yc)
y 2
N 1
i = 0
y⇤

31. s(y, x) =
N 1X
i=0
↵T
f(yi+1, x, yc)

32. f(yi+1, x, yc)
↵ =< 1, 0, ..., 0 >