25-min talk about Machine Learning and a little bit of Deep Learning. Starts with some basic definitions (Supervised and Unsupervised Learning). Then, neural networks basic functionality is explained, ending up in Deep Learning and Convolutional Neural Networks. Machine Learning Meetup that happened in Porto Alegre, Brazil.

1. An introduction to
Machine Learning
Thomas Paula
#1 Porto Alegre ML
Meetup
and a little bit of Deep Learning

2. whoami
Thomas Paula
Software Engineer
Machine Learning Engineer
(On going) Master’s Degree
in Machine Learning

3. How could we teach
machines in a way that they
could learn from
experience?

4. “A computer program is said to learn
from experience E with respect to some
class of tasks T and performance
measure P, if its performance at tasks in
T, as measured by P, improves with
experience E.
Tom M. Mitchell

5. “A computer program is said to learn
from experience E with respect to some
class of tasks T and performance
measure P, if its performance at tasks in
T, as measured by P, improves with
experience E.
Tom M. Mitchell

6. “A computer program is said to learn
from experience E with respect to some
class of tasks T and performance
measure P, if its performance at tasks in
T, as measured by P, improves with
experience E.
Tom M. Mitchell

7. “A computer program is said to learn
from experience E with respect to some
class of tasks T and performance
measure P, if its performance at tasks in
T, as measured by P, improves with
experience E.
Tom M. Mitchell

8. Example
Task T: Classify an e-mail as spam or not spam.
Performance measure P: Percentage of e-mails correctly
classified.
Training experience E: E-mails manually labeled by
humans.

9. Types of
Learning
Supervised Learning
Unsupervised Learning

10. Supervised Learning

11. This is a pawn! This is a knight!
This is a king!
Supervised Learning

12. Unsupervised Learning

13. Smaller than
others, with a
ball on the top:
group 0!
Not conic, nothing
on the top: group 1!
Higher than others,
with cross on the top:
group 2!
Unsupervised Learning

14. Group 0 Group 1
Unsupervised Learning

15. Task
Categories
Classification
Regression
Clustering

16. Neural
Network

17. Neural Network
"Convolutional Neural Networks for Visual Recognition" (http://cs231n.github.io/convolutional-
networks/)

18. Neuron Model
+1
a1
Input Layer Output Layer

19. Neuron Model
+1
a1
Weights
Input Layer Output Layer

20. Neuron Model
+1
a1
Inputs
Input Layer Output Layer

21. Neuron Model
+1
a1
Bias
Input Layer Output Layer

22. Neuron Model
+1
a1
Output
Input Layer Output Layer

23. Sigmoid

24. Neural Network Example
+1
a1
Create a neural
network that
behaves as an
Logic OR.

25. Neural Network Example
+1
a1
Input Output
0 0 0
0 1 1
1 0 1
1 1 1

26. Neural Network Example
+1
a1
Input Output
0 0 0
0 1 1
1 0 1
1 1 1

27. Neural Network Example
+1
a1
Input Output
0 0 0
0 1 1
1 0 1
1 1 1
a = -10*1 + 20*0 + 20*0 = -101

28. Neural Network Example
Input Output
0 0 0
0 1 1
1 0 1
1 1 1

29. Neural Network Example
Input Output
0 0 0
0 1 1
1 0 1
1 1 1

30. Deep Learning

31. “A machine learning approach
that attempts to extract high-
level and hierarchical features
from raw data.

32. Deep Neural Network

33. Where is it used?
•Speech Recognition;
•Music Classification;
•Recommendation;
•Image Classification;
•...

34. Some of the precursors...
Yann LeCun Geoffrey Hinton
Yoshua
Bengio

35. AI Winter
and
the Canadian Mafia

36. What happened?
• Major breakthrough in 2006 in the way deep architectures
were handled
• Unsupervised learning of representations is used to pre-train each
layer
• Unsupervised training of each layer at a time, on top of the previous
ones. The representation learned at each level is the input to the
next one.
• Use supervised learning to fine-tune all the layers
http://www.slideshare.net/perone/deep-learning-convolutional-neural-
networks

37. Besides that...
• Better hardware
• GPUs enable ~9x faster training compared to CPUs.
• High-quality datasets;
• New activation functions;
• Regularization methods.

38. Different implementations
• Convolutional Neural Networks (CNN);
• Deep Belief Networks;
• Autoencoders;
• Long Short Term Memory (LSTM)
• ...

39. Convolutional Neural
Networks

40. CNNs
"Convolutional networks and applications in vision" (LeCun et al., 2010)

41. Convolution
• In convolutional layers, all units are organized in feature maps;
• Mathematical term discrete convolution, which describes the
filtering operation performed by the feature map.

42. Convolution
"Deep Learning in a Nutshell" (https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-
concepts/)

43. Pooling
• Usually used after convolution
layers;
• Reduces spatial size of the
representation
• Reducing the quantity of needed
parameters and computation;
• Controlling overfitting.
"Convolutional Neural Networks for Visual Recognition" (http://cs231n.github.io/convolutional-
networks/)

44. Example – Max Pooling
"Convolutional Neural Networks for Visual Recognition" (http://cs231n.github.io/convolutional-
networks/)

45. Example of Application

46. Image Classification Problem
"Convolutional Neural Networks for Visual Recognition" (http://cs231n.github.io/convolutional-
networks/)

47. Image Classification - Challenges

48. Traditional way
"Deep Learning Methods for Vision" (Honglak Lee, 2012)

49. Hand-crafted features
•Time consuming;
•Demand expert knowledge;
•Sometimes meaningful for a very specific scenario.

50. But what if we could learn feature extractors
instead?
Traditional
Deep
Learning
Hand-crafted
feature extractor
Trainable Classifier
Trainable feature
extractor
Trainable Classifier

51. Architectures Overview and Results

52. Thank you!
Thomas Paula - @tsp_thomas