1.
DEEP LEARNING IN PRACTICE
Tess Ferrandez – Microsoft - @TessFerrandez

2.
TIM
MITYA
YANA
VITO
CLAUS
TESS

3.
SHOTS ON GOAL

4.
DETECTING CANCER

5.
SHOPLIFTING

6.
DEEP LEARNING
What’s so magical about it

7.
int EstimatePrice(...){
price = 100000 +
67000 * area_in_sqm +
200000 * has_pool +
100000 * new_kitchen +
5000 * neighborhood_quality;
return price;
}
Price = b + w1*area_in_sqm + w2*has_pool + ...

8.
[LINEAR REGRESSION]

9.
GIVEN ENOUGH SAMPLES,
A NEURAL NETWORK WILL
FIND THE PATTERN

10.
GIVEN ENOUGH SAMPLES,
A NEURAL NETWORK WILL
FIND THE PATTERN

11.
SEQUENCESPACE
CONVOLUTIONAL NN RECURRENT NNDENSE NN

12.
[ 0.01949719, 0.09399229, -0.01618082, -0.00876935, 0.03146157, 0.06853894, 0.00096175, -0.06854118, -0.04771797, -0.05296798, 0.02119147, 0.00511259, 0.13726683
INTERMEDIATE REPRESENTATIONEMBEDDINGSECRET CODE

13.
RECOMMEND A BOOK

14.
-1.0 1.0
ADULTYOUTH

15.
ADULT
FICTION
(-0.6, 0.4) *
(0.7, 0.9) *

16.
FICTION ADULT
MATH REFERENCES
US-CENTRIC
CHICK LIT
FUNNY
SCI-FI
LAWYERS
WOULD BRAD PITT PLAY A
CHARACTER IN THE MOVIE?
EMBEDDING
[ 0.01949719, 0.09399229, -0.01618082, -0.00876935, 0.03146157, 0.06853894, 0.00096175, -0.06854118, -0.04771797, -0.05296798, 0.02119147, 0.00511259, 0.1372668

17.
WORD
EMBEDDINGS

18.
[ 0.01949719, 0.09399229, -0.01618082, -0.00876935, 0.03146157, 0.06853894, 0.00096175, -0.06854118, -0.04771797, -0.05296798, 0.02119147, 0.00511259, 0.1372668

19.
[FACENET]
T-SNE
Projection of
128D to 2D

20.
FACE RECOGNITION DEMO

21.
SEGMENTATION
encoder decoder

22.
encoder decoder

23.
encoder decoder

24.
IN PRACTICE
The secrets behind the magic

25.
Time for the Epoch
Training data
Validation data

26.
MODEL LOSS ACCURACY
BASIC 0.2507 91.05%

27.
OOPSIE DOOPSIE!
We’re overfitting

28.
Chihuahua the movie
[DATA AUGMENTATION]

29.
[DROPOUT]

30.
MODEL LOSS ACCURACY
BASIC 0.2507 91.05%
AUGMENTATION 0.1988 93.68%

31.
MODEL LOSS ACCURACY
BASIC 0.2507 91.05%
AUGMENTATION 0.1988 93.68%
TRANSFER LEARN 0.01253 99.47%

32.
APPLIED MACHINE LEARNING
When the magic is gone, and we’re left with Software Engineering

33.
UNDERSTAND THE BUSINESS NEEDS

34.
WHAT IS THE PROBLEM?
HOW WILL THE MODEL BE
USED? / REQUIREMENTS
HOW IS IT DONE TODAY?
IS IT FEASIBLE?
ETHICAL CONCERNS
UNDERSTAND THE BUSINESS NEEDS

35.
UNDERSTAND THE
BUSINESS NEEDS
MINE CLEAN EXPLORE

36.
UNDERSTAND THE
BUSINESS NEEDS
MINE CLEAN EXPLORE
ENGINEER MODEL DEPLOY

37.
LOTS OF LABLED SAMPLES and
NO CONSEQUENTIAL DECISIONS
SHOTS ON GOAL

38.
LOUD CROWD
GOAL VISIBLE
SPEED/DIRECTION PLAYER DENSITY
PLAYER POSES

39.
SCENE CHANGES
GOAL IN VIEW
NEGATIVE SAMPLING

40.
5S VIDEOS - AROUND ACTION
NEGATIVE SAMPLES FROM ATTACKS

41.
VGG EMBEDDINGS

42.
90+%
MODEL
ACCURACY
GRASS?
GOAL?
SCENE CHANGE

43.
ShotNoShot
https://github.com/tyiannak/pyAudioAnalysis
AUDIO

44.
PEOPLE CLUSTERS - SIZES

45.
https://github.com/fizyr/keras-retinanet
model_path =
'c:/Tess/source/vision_samples/models/resnet50_coco_best_v2.1.0.h5'
model = models.load_model(model_path, backbone_name='resnet50’)
image_path = 'C:/Tess/source/vision_samples/data/images/basket_image.jpg'
image = read_image_bgr(image_path)
image = preprocess_image(image)
image, scale = resize_image(image)
# process image
boxes, scores, labels = model.predict_on_batch(np.expand_dims(image,
axis=0))
from keras_retinanet import models
from keras_retinanet.utils.image import read_image_bgr, preprocess_image,
resize_image

46.
GOAL / NO GOAL

47.
SCENE CHANGE DETECTION

48.
FOCUSED OPTICAL FLOW ON PLAYERS

49.
DETECTING CANCER
VERY FEW POSITIVE SAMPLES
EXTREME ACCURACY NEEDS
POTENTIAL FOR BIAS

50.
HARD TO DIFFERENTIATE
ONLY PARTIALLY LABLED
EXTREMELY LARGE IMAGES

51.
COLOR SEGMENTATION

52.
CONVEX HULL

53.
SHOPLIFTING
VERY FEW POSITIVE SAMPLES
VERY FEW SAMPLES PER ACTION TYPE
VERY SENSITIVE TO BIAS

54.
COVERED FACES
ALONE
MEN 20-40 HOODIES
SHOPLIFTING POSES
MEN 20-40 HOODIESCOVERED FACES
ALONE

55.
12:32:00CHRISTMAS
FISH EYE
DETECT not PREDICT HUD ARTIFACTS

56.
NEGATIVE SAMPLES FROM SAME VIDEOS
PEOPLE SHOPPING

57.
POSE DETECTION

58.
BACKGROUND SUBTRACTION

59.
CLASSIFICATION AT THE BOX LEVEL

60.
A LITTLE
DOMAIN KNOWLEDGE
GOES A LONG WAY

61.
KISSKeep it Simple …

62.
DEEP LEARNING IN PRACTICE
Tess Ferrandez – Microsoft - @TessFerrandez