A PRACTICAL GUIDE TO DEEP LEARNING
Tess Ferrandez – Microsoft - @TessFerrandez

from @teenybiscuit

FROM ML TO DEEP LEARNING
Predicting the price of a house

int EstimatePrice(...){
price = 10000 +
6700 * area_in_sqm +
20000 * has_pool +
10000 * new_kitchen +
5000 * neighborhood_quality;
return price;
}
Price = b + w1*area_in_sqm + w2*has_pool + ...

Price = b + w1*area_in_sqm
[LINEAR REGRESSION]
[GRADIENT DESCENT]

int EstimatePrice(...){
price = 10000 +
6700 * area_in_sqm +
20000 * has_pool +
10000 * new_kitchen +
5000 * neighborhood_quality;
return price;
}
Price = b + w1*area_in_sqm + w2*has_pool + ...
[LINEAR REGRESSION]

[NEURAL NET]

[NEURAL NET]

UNSTRUCTURED DATA
Machine Learning on Images

[HISTOGRAMS]

[PURE PIXELS]

F. Chollet – Creator of Keras

[DENSLEY CONECTED] [CONVOLUTIONAL] [RECURRENT]
NETWORK ARCHITECTURES

CONVOLUTIONAL NEURAL NETWORKS
The basics

http://deeplearning.stanford.edu/wiki/index.php/Feature_extraction_using_convolution

https://en.wikipedia.org/wiki/Kernel_(image_processing)

0*1+0*1+0*1 + 0*0+0*0+0*0 + 0*-1+0*-1+0*-1 = 01*1+1*1+1*1 + 1*0+1*0+1*0 + 0*-1+0*-1+0*-1 = 3

https://www.quora.com/How-can-l-explain-the-dimensionality-reduction-in-convolutional-neural-network-CNN-from-this-image

0 0
2 3

layer 1 layer 2 layer 3 layer 4 layer 5
https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf

CNNs IN PRACTICE
Finally time for some code

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 EPOCH = 1 pass through the training data

Time for the Epoch
Training data
Validation data

MODEL LOSS ACCURACY
BASIC 0.2507 91.05%

OOPSIE DOOPSIE!
We’re overfitting

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

[DATA AUGMENATION]

Chihuahua the movie

[DROPOUT]
http://jmlr.org/papers/v15/srivastava14a.html

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

PREDICTED
Chihuahua Muffin
TRUE
ChihuahuaMuffin

MODEL LOSS ACCURACY
BASIC 0.2507 91.05%
AUGMENTATION 0.1988 93.68%

1 PREPARE DATA
CREATE MODEL
TRAIN MODEL (UNTIL OVERFIT)
GET MORE DATA OR ADD DROPOUT
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

TRAINING ON PRETRAINED MODELS
Feature Extraction and Transfer Learning

F. Chollet – Deep Learning with Python

1 EXTRACT FEATURES FROM A PRE-TRAINED
MODEL
CREATE A SHALLOW NETWORK TO PREDICT2

1 EXTRACT FEATURES FROM A PRE-TRAINED
MODEL
CREATE A SHALLOW NETWORK TO PREDICT2

1 EXTRACT FEATURES FROM A PRE-TRAINED
MODEL
CREATE A SHALLOW NETWORK TO PREDICT2

1 EXTRACT FEATURES FROM A PRE-TRAINED
MODEL
CREATE A SHALLOW NETWORK TO PREDICT2

MODEL LOSS ACCURACY
BASIC 0.2507 91.05%
AUGMENTATION 0.1988 93.68%
FEATURE EXTR. 0.01253 99.47%

1 ADD DENSE LAYERS ON TOP OF CONV. BASE
FREEZE THE CONV. BASE
TRAIN MODEL
UNFREEZE SOME LAYERS
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 ADD DENSE LAYERS ON TOP OF CONV. BASE
FREEZE THE CONV. BASE
TRAIN MODEL
UNFREEZE SOME LAYERS
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

1 ADD DENSE LAYERS ON TOP OF CONV. BASE
FREEZE THE CONV. BASE
TRAIN MODEL
UNFREEZE SOME LAYERS
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

MODEL LOSS ACCURACY
BASIC 0.2507 91.05%
AUGMENTATION 0.1988 93.68%
FEATURE EXTR. 0.01253 99.47%
TRANSFER LEARNING 0.01842 100%

1 ADD DENSE LAYERS ON TOP OF CONV. BASE
FREEZE THE CONV. BASE
TRAIN MODEL
UNFREEZE SOME LAYERS
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

MODEL LOSS ACCURACY
BASIC 0.2507 91.05%
AUGMENTATION 0.1988 93.68%
FEATURE EXTR. 0.01253 99.47%
TRANSFER LEARNING 0.01842 100%
TRANSFER UNFREEEZE 0.01081 99.47%

1 ADD DENSE LAYERS ON TOP OF CONV. BASE
FREEZE THE CONV. BASE
TRAIN MODEL
UNFREEZE SOME LAYERS
TRAIN MODEL
PREDICT ON TEST DATA
2
3
4
5
6

VISUALIZE THE NETWORK
Understanding what it learns

[Grad-CAM Heatmaps]

layer 1 layer 2 layer 3 layer 4 layer 5
https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf

ADVANCED TOPICS
Extra Extra!

GrabCut
OpenCV
http://www.australiandoglover.com/2016/05/chihuahua-breed-profile.html

[GENERATOR (forger)]
NETWORK ARCHITECTURES
[ORIGINALS]
[DETECTOR (detective)]GENERATIVE ADVERSARIAL NETWORK
Generating images

Alec Radford – DCGAN paper - https://arxiv.org/pdf/1511.06434.pdf

https://deepart.io/

Ian Goodfellow: Adversarial Examples

DeepFace from Facebook
Image: Daily Mirror
ONE SHOT LEARNING
DeepFace and FaceNet

FaceNet from Google
Image: https://omoindrot.github.io/triplet-loss

http://slideshare.net/Tess
@TessFerrandez

QUICK, PRE-FAB AND EASY
Cognitive Services

COMPUTER VISION
Azure Cognitive Services

CUSTOM VISION
Azure Cognitive Services

A PRACTICAL GUIDE TO DEEP LEARNING
Tess Ferrandez – Microsoft - @TessFerrandez