This document provides an overview of deep learning. It discusses the motivation and history of machine learning, including pattern recognition, machine learning algorithms based on linear models, and neural networks. It then introduces deep learning, noting that deep neural networks combined with GPUs and large datasets have led to significant performance gains compared to other machine learning techniques.

1. DEEP LEARNING
A movie for everyone

2. Content
• Motivation
• Machines that learn
• Machine learning history
• Binary classifier
• Linear models and tricks
• Neural Network
• Deep learning

4. Motivation / Google Trends

5. Motivation / Google Trends
Jan 2012 - Big Data

6. Motivation / Google Trends
Oct 2012 - DL

7. Motivation / Google Trends
Dec 2014 - DL vs NN

8. Motivation / Google Trends
Ago 2015 - ML vs AI

9. Motivation / AI landscape

10. Motivation / AI landscape

11. Machines that learn
A computer program is said to learn from experience E
with respect to some class of tasks T and performance measure P
if its performance at tasks in T, as measured by P,
improves with experience E.
Tom M. Mitchell

12. Machines that learn
MACHINE
PROGRAM
INPUT OUTPUT

13. Machines that learn
MACHINE OUTPUT
INPUT
( FEATURE )
PROGRAM
Experience == Accumulated knowledge ( labelled data )

14. Machines that learn
MACHINE
OUTPUT
( Prediction )
INPUT
( Any )
PROGRAM

15. Machine Learning History
Pattern Recognition Machine Learning Deep Learning …
(*)
It is a set of algorithms mainly based on linear models.

16. Binary Classifier

17. Binary Classifier
Vegetable
(1,4,3,1,4,3,5,4,6,7,4,5)

18. Binary Classifier
Animal
(0,1,0,2,3,4,0,6,7,8,1,0)

19. Binary Classifier

20. Binary Classifier

21. Binary Classifier

22. Binary Classifier

23. Binary Classifier

24. Binary Classifier

25. Binary Classifier

26. Binary Classifier

27. Binary Classifier

28. Binary Classifier

29. Binary Classifier

30. Binary Classifier

31. Binary Classifier
( 0, …. )
( 1, …. )
(1,1,2,3,5,3,2,1 )

33. Linear Models
XOR problem
Y X1 X2
Blue 1 1
Red 0 1
Blue 0 0
Red
1 0
(0,1)
(0,0)
(1,1)
(1,0)

34. Linear Models
XOR problem
(0,1)
(0,0)
(1,1)
(1,0)

35. Linear Models

36. Linear Models
Y X1 X2 X3
Blue 1 1 0
Red 0 1 1
Blue 0 0 1
Red 1 0 0 Human did classification!

37. Linear Models

38. Linear Models
SUPPORT VECTOR MACHINES

39. Neural Network
Let's mimic what Mother Nature did.

40. Neural Network

41. Neural Network
Feed forward is required
to make things simpler.

42. Neural Network

43. Neural Network
+1
x1
x2
A
B
C
+1
20
-30
20
20
10
-20
-20
-10
20
x1 x2 A B C
0 0 0 1 1
0 1 0 0 0
1 0 0 0 0
1 1 1 0 1

44. Neural Network
x1 x2 A B C
0 0 0 1 1
0 1 0 0 0
1 0 0 0 0
1 1 1 0 1
(0,1)
(0,0)
(1,1)
(1,0)

45. Neural Network
x1 x2 A B C
0 0 0 1 Blue
E
0 1 0 0 Red
1 0 0 0 Red
1 1 1 0 Blue
(0,1)
(0,0)
(1,1)
(1,0)

46. Deep Learning

47. Deep Learning
Deep NN, GPUs and
large amount of data

48. Deep Learning
Performance
Amount of data
Deep learning
Most ML algorithms

49. Deep Learning
Big Data
Data Science
Data Mining
AI
Machine Learning
Deep Learning

50. Deep Learning