「知識」のDeep Learning

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Deep Learning
1 Preferred Infrastructure
(@unnonouno)
2015/06/04 PFI

!  1
! 
! 
NLP 12014-

NLP 1YANS
!  YANS 19
!  140
! 
! 
!  YANS 13
! 
!  "

Knowledge Representation

1.
3.
4.
2.

1.
3.
4.
2.

1.
3.
4.
2.
RNN

Recurrent Neural Network Language Model
(RNNLM) [Mikolov+10]
!  RNN
! 
! 
!  LSTM

Long Short-Term Memory (LSTM)
! 
! 
1
!

RNN1LSTM
! 
!

! 
! 
argmaxy P(y|x) = argmaxy P(x|y) P(y)

1.
3.
4.
2.

[ 15]
! 
1
!  -‐‑‒

-‐‑‒ [ 15]
!  Pydata Tokyo

1.
3.
4.
2.

-‐‑‒
! 
! 
!

Knowledge Representation

!  EMNLP2014 Bordes Weston
1 part 2 [Bordes&Weston14]

!  [
15]

Q: Deep Learning
Representation Learning
-‐‑‒
A: Deep

推論を導けるような知識の表現、およびその方法
を開発する人工知能研究の領域である。

! 
! 
!  1

1
!  2 t t 3
!  t t
!  RDF
(x, r, y)
x yr

1Knowledge Base

!  New York NY

! 
!  t t
!  t t
!  t t

-‐‑‒
! 
-‐‑‒
NLP

!  {(xi, ri, yi)}:
!  x, y: t t
!  r: t t
!  x, y
!  r

! 
! 
1
argmax ∑i f(xi, ri, yi)

Distance model (Structured Embedding) [Bordes
+11]
!  e
!  Rleft, Rright
! 
!  f
f(x, r, y) = || Rleft(r) e(x) – Rright(r)...

TransE model [Brodes+13]
!  r r
! 
f(x, r, y) = || e(x) + r – e(y) ||2
2

TransE model
x
y
r

TransE model
1 TransE
!  TransM:
!  TransH:

TransM model [Fan+14]
!  r
!  wr r x, y
f(x, r, y) = wr|| e(x) + r – e(y) ||2
2

TransH model [Wang+14]
!  TransE

Bilinear model
!  r
! 
f(x, r, y) = e(x)T Wr e(y)

Neural Tensor Network (NTN) [Socher+13]
f(x, r, y) = ur tanh(e(x)Wre(y)
+ V1
r e(x) + V2
r e(y) + br)
!  r
!

[Yang+15]
!  2
! 
Bilinear

Q:
-‐‑‒
A: [Nickel+11]

Link prediction
!  t t 1
!  QA
!  -‐‑‒
!  t t
(e1, r, e2) (e1, r, ? )

TransH
TransE
[Bordes&Weston14]

! 
!

[Weston+13]
!  x y r
! 
[Bordes&Weston14]

[Weston+13]
! 
! 
-‐‑‒
[Bordes&Weston14]

[Bordes&Weston14]

Link prediction 1QA
!  Link prediction
!

QA [Bordes+14]
! 
!  q t
f(q) g(t)
!  f(q), g(t) q, t

Memory networks:
[Weston+15]
!  I: I(x)
!  G: mi = G(mi, I(x), m)
!  O: o = O(I(x), m)
!  R: r = R(o)

I:
O:
R:
G:

! 
! 
! 
!

!  Deep
Learning -‐‑‒

-‐‑‒
! 
!  -‐‑‒
!  -‐‑‒
!  -‐‑‒
! 
!  factoid
!  -‐‑‒
!

RNN [Peng&Yao15]
!  RNN
!

! 
! 
! 
! 
! 
! 
! 
!

1/4
!  [Mikolov+10] T. Mikolov, M. Karafiat, L. Burget, J. H. Cernocky, S.
Khudanpur.
Recurrent neural network based langu...

2/4
!  [Bordes+11] A. Bordes, J. Weston, R. Collobert, Y. Bengio.
Learning structured embeddings of knowledge bases.
AAAI2...

3/4
!  [Socher+13] R. Socher, D. Chen, C. D. Manning, A. Y. Ng.
Reasoning With Neural Tensor Networks for Knowledge Base
C...

4/4
!  [Bordes+14] A. Bordes, J. Weston, N. Usunier.
Open Question Answering with Weakly Supervised
Embedding Models.
ECML...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

言語と知識の深層学習@認知科学会サマースクール by Yuya Unno 26917 views
[Dl輪読会]A simple neural network modu... by Deep Learning JP 2346 views
最先端NLP勉強会“Learning Language Games ... by Yuya Unno 5503 views
自然言語処理@春の情報処理祭 by Yuya Unno 8441 views
ﾋﾟｰFIの研究開発現場 by Yuya Unno 7986 views
Chainerのテスト環境とDockerでのCUDAの利用 by Yuya Unno 16576 views

View on Slideshare

1 of 59 Ad

View on Slideshare

1 of 59 Ad