2017/11/27 Deep Learning JP: http://deeplearning.jp/hacks/

1. A simple neural network module
for relational reasoning
2017/11/20
Ken Maeda
Yamaguchi Lab.
The University of Tokyo

2. 2

3. •
A simple neural network module for relational reasoning
(Relational Networks / RNs)
•
Adam Santoro , David Raposo , David G.T. Barrett, Mateusz Malinowski, Razvan
Pascanu, Peter Battaglia, Timothy Lillicrap
•
2017/6/5 on arXiv (https://arxiv.org/pdf/1706.01427.pdf)
3

4. • Relational Reasoning ( )
•
• Neural Network
• RN
• DL
• 4
• CLEVR, Sort-of-CLEVR, bAbI, Dynamic physical systems
4

5. Relational Reasoning
•
•
5
• Non-Relational Question:
• Relational Question:

6. Relational Reasoning
•
• Symbol Grounding problem
•
•
• CNN MLP
6

7. Relational Network
• NN
•
• NN
7

8. Relational Network
• NN
•
• NN
8
MLP
relation
2
end-to-end

9. RN 3
• (Learn to infer relations)
•
→RN
→RN
• (Data efficient)
• !"
• !" =
• #2
• (Operate on a set of objects)
• (summation Σ )
→
9

10. (1/4)
1. CLEVR
• 3D QA
• querry attribute, compare attribute, …
• 2
1) pixel version
2D pixel form
2) state description version
3D …
10
CLEVR( )

11. (2/4)
2. Sort-of-CLEVR
• CLEVR
• 6
• relational/non-relational 10
11
Sort-of-CLEVR
2

12. (3/4)
3. bAbI
• QA
•
20
• Ex: “Sandra picked up the football” and
“Sandra went to the office”
Q: “Where is the football?”
A: “office”
12

13. (4/4)
4. Dynamic physical systems
• MuJoCo ( )
•
•
•
( )
13

14. (1/4)
• RN CNN LSTM
•
14
CLEVR
RN
CLEVR

15. (2/4)
• pixel( ) - CNN -
• 128 128
• 4 d d k
• d d k
RN ( )
•
•
• RN
15

16. (3/4)
• RN - LSTM -
• ! RN
• Ex.
•
16
LSTM

17. (4/4)
• - LSTM -
• 20
LSTM
•
• LSTM RN
17
RN
bAbI
bAbI

18. (CLEVR-from-pixels)
• Image: 4 CNNs (24 kernels each), ReLU, batch normalization
• Question: 128 unit LSTM, 32 unit look-up embeddings
• !: 4-layer MLP (256 units each), ReLU
• $: 3-layer MLP (256, 256[50% dropout], 29), ReLU
• final layer: softmax over the answer vocab
• Learning rate: 2.5e-4, Adam, cross-entropy
• Training: 64 mini-batches, 10 computers
CLEVR
18Appendix

19. (CLEVR-from-pixels)
• 95.5%
• 27%
19
SA : Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked attention networks for
image question answering. In CVPR, 2016. 16

20. (CLEVR-from-pixels)
• compare count
•
•
20

21. • ( )
•
21

22. ( )
• CLEVR(from state descriptions): 96.4%
• RN
• Sort-of-CLEVR: 94% ( )
• RN MLP (63%)
• ”closest-to” ”furthest-to” (52.3%)
→
RN
22

23. ( )
• bAbI: 18/20( 95%)
• Dynamic physical systems
1) : 93%
2) : 95%
• MLP
23

24. • RN
24
• RN CNN
• ResNet
• RN
• RN
• RN
•
• RN

25. 25
https://github.com/whiteking64/Relational_Networks

26. • Sort-of-CLEVR p.12
• DL Pytorch
• CNN_MLP
26

27. • 10000 200
• 75x75
• Relational/Non-relational 10
• 3
• Relational Question
• Non-relational Question /
• 11
27

28. (RN )
•
• CNN, 4 ( 32, 64, 128, 256)
• ReLU
• Batch Normalization
•
• RN ( embedding )
28

29. (RN )
• MLP -!-
• 4 ( 2000 units)
• ReLU
• MLP -$-
• 4 (2000, 1000, 500, 100 units)
• ReLU
•
• 10 units(answer vector )
• softmax
•
•
• Adam
• : 1e-4
• : 64
29

30. iPython notebook
30
https://github.com/whiteking64/Relational_Networks/tree/master/Python_ver

31. (1/2)
• 20epoch
2epoch …!
•
•
• Epoch1 48%, 50%
• Epoch2 63%, 63%
31

32. (2/2)
• CLEVR (20 epochs)
• Relation accuracy: 88%
• Non-relation accuracy: 99%
accuracy
• [1]
• Relation accuracy: 88%
• Non-relation accuracy: 99%
32

33. [1]
https://github.com/kimhc6028/relational-networks
33

34. 34