20160203_마인즈랩_딥러닝세미나_02 deep and wide analytics 최대우교수님

1. 최 대 우 교수 / 센터장
한국외대 데이터시각화연구센터
DEEP & WIDE ANALYTICS
AdviceAnalytic Data Visualization Research Center

2. AdviceAnalytic Data Visualization Research Center
Neural Network Revisited – History of NN
2

3. AdviceAnalytic Data Visualization Research Center
History of Neural Network - 1943
3
Warren McCulloch Walter Pitts
• Input : Binary
• weights : designed
• Output : 0 or 1

4. AdviceAnalytic Data Visualization Research Center
History of Neural Network - 1957
4
• Input : real-valued
• weights : learning
• Output : 0 or 1
Frank Rosenblatt

5. AdviceAnalytic Data Visualization Research Center
History of Neural Network - 1969
5
• Perceptron can’t do XOR!
• Need multi-layer perceptron
Minsky & Papert

6. AdviceAnalytic Data Visualization Research Center
History of Neural Network - 1986
6
• Backpropagation of errors
• Methods for training multilayer network
David Rumelhart

7. AdviceAnalytic Data Visualization Research Center
Theoretical Background of Neural Network
7
• 수학 문제 23개로, 독일의 수학자인 David Hilbert 가 1900년 프랑스 파리에서 열
린 세계 수학자 대회에서 20세기에 풀어야 할 가장 중요한 문제로 제안한 것임
• 세계 수학자 대회에서 Hilbert는 10문제(1, 2, 6, 7, 8, 13, 16, 19, 21, 22)를 공개했
고, 나중에 모든 문제가 출판되었음
Hilbert’s Problem
David Hilbert

8. AdviceAnalytic Data Visualization Research Center
Theoretical Background of Neural Network
8
Andrey Kolmogorov

9. AdviceAnalytic Data Visualization Research Center
Neural Network – Universal Approximator
9
Kolmogorov (1963)

10. AdviceAnalytic Data Visualization Research Center
Neural Network – Universal Approximator
10
6 + A - 2B +3C
A
B
C
Kolmogorov (1963)

11. AdviceAnalytic Data Visualization Research Center
Dark Age of NN
11
Warren McCulloch Walter Pitts Frank Rosenblatt
Minsky & Papert
David Rumelhart
1900 1943 1957 1963 1969 1986
Andrey Kolmogorov
 Overfitting
 Local minima
 Heavy computing
 Too many tuning parameters…

12. AdviceAnalytic Data Visualization Research Center
KSF in Deep Learning
12
 Overfitting
 Local minima
 Heavy computing
 Too many tuning parameters…
 Autoencoder
 Drop-out
 GPU
 Big Data

13. AdviceAnalytic Data Visualization Research Center
KSF in Deep Learning
13
J.H. Friedman
Projection Pursuit
Regression & Classification
Leo Breiman
Radom Forest

14. AdviceAnalytic Data Visualization Research Center
Big Change in Analytics – Deep and Wide
14
Wide
(more features)
Wide
(more cases)
Deep
(more thinking)

15. AdviceAnalytic Data Visualization Research Center
Big Change in Analytics – Deep and Wide with ensemble
15

16. AdviceAnalytic Data Visualization Research Center
Future: Big Data = Zero Optimism
16
• Test error = Training error + Optimism
• = # of parameters
• = # training cases
• = # models considered
N
p
2Optimism 
N
Mlog
~
p
N
M

17. AdviceAnalytic Data Visualization Research Center
Future: Reinforcement Learning
17
Agent
Environment
reward
State
Action
Policy
Value

18. 감사합니다