Paper review of the FaceNet paper for the Machine Learning at Microsoft Youtube channel. https://www.youtube.com/watch?v=w--c0qG9MCc&t=2392s

1. FaceNet:
A Unified Embedding for Face Recognition and Clustering
Florian Schroff
Dmitry Kalenichenko
James Philbin
Google [2015]
[ 0.01949719, 0.09399229, -0.01618082, -0.00876935, 0.03146157, 0.06853894, 0.00096175, -0.06854118, -0.04771797, -0.05296798, 0.02119147, 0.00511259, 0.1372668

2. [ 0.01949719,
0.09399229, -
0.01618082, -
0.00876935,
0.03146157,
0.06853894,
0.00096175, -
0.06854118, -
0.04771797, -
0.05296798,
0.02119147,
0.00511259,
0.13726683, -
0.05780432, -
0.00541799,
0.01251621, -
0.08900651, -
0.15897971, -
0.07564467,
0.16574059,
…]

3. EMBEDDING

4. MOTIVATION
PROBLEMS WITH OLD
APPROACHES

5. Is this Tess? This is Anders and Tess? Who diz?
VERIFICATION IDENTIFICATION CLUSTERING

6. TIMTESS TEMPEST YANA
TESS UNKNOWN?
SIMON CHRIS

7. We have lots and lots of
people…

8. We need lots and lots of
selfies…

9. FRANCOIS
Let’s not hire him,
I don’t want to retrain the network
TIMTESS TEMPEST YANASIMON CHRIS

10. This Photo by Unknown Author is licensed under CC BY-NC
LET’S TRAIN A SIMILARITY FUNCTION INSTEAD

11. EMBEDDINGS

12. APRICOT PEACH BEACH

13. MOVIE RECOMMENDATION

14. AdultChild
Blockbuster
Arthouse
* (-1,-0.95)
* (0.65, 0.2)

17. * (-1,-0.95)
* (0.65, 0.2)
DISTANCE
d(P,Q) = f(P)−f(Q) 2
2
𝐿2 𝑁𝑜𝑟𝑚

18. * (-1,-0.95)
* (0.65, 0.2)
DISTANCE
Embedding on the
d-dimensional hypher-sphere
f(X) 2
2
= 1
0 = Identical
4 = As different as can be
d(P,Q) = f(P)−f(Q) 2
2
𝐿2 𝑁𝑜𝑟𝑚

19. USING EMBEDDINGS
TO SOLVE FACE TASKS

20. Is this Tess?
VERIFICATION
Verification
=
Distance
Threshold
𝑑(𝑇𝑒𝑠𝑠, 𝑇𝑒𝑠𝑡) ≤ 𝜏

21. This is <unknown> and Tess?
IDENTIFICATION
Identification
=
search lowest
distance
or
K-nn/SVM
classification
0.3
1.2
1.4
2.3
1.7
1.1
1.6
0.8

22. Who diz?
CLUSTERING

23. DEEPFACE

24. DeepFace by Facebook
1
0

25. FACENET
TRIPLET LOSS AND
TRIPLET SELECTION

26. TRIPLET LOSS
ANCHOR (A) POSITIVE (P) ANCHOR (A) NEGATIVE (N)
d(A,P) ≤ d(A,N)d(A,P) - d(A,N) ≤ 0𝑓 𝐴 − 𝑓(𝑃) - 𝑓 𝐴 − 𝑓(𝑁) ≤ 0𝑓 𝐴 − 𝑓(𝑃) - 𝑓 𝐴 − 𝑓(𝑁) + α ≤ 0
0.3 0.32

27. COST FUNCTION
𝑓 𝑎 − 𝑓(𝑝) - 𝑓 𝑎 − 𝑓(𝑛) + α ≤ 0

28. FINDING THE RIGHT TRIPLETS
We need triplets that violate the equation to ensure fast convergance
𝑓 𝑎 − 𝑓(𝑝) - 𝑓 𝑎 − 𝑓(𝑛) + α ≤ 0
find P where argmax( 𝑓 𝑎 − 𝑓(𝑝) )
find N where argmin( 𝑓 𝑎 − 𝑓(𝑛) )

29. Triplet selection options
A) Generate triplets offline every N steps from a subset of data
B) Generate triplets online from a minibatch
Paper uses option B, with large minibatches with 1000s of examples

30. CNN ARCHITECTURE

31. OpenFace – Open Source FaceNet implementation

34. GoogLeNet Inception Network

36. DATASETS
PREPARATION AND RESULTS

38. http://vis-www.cs.umass.edu/lfw/
13233 images
5749 people
1680 people with two or more images
Labeled Faces in the Wild (LFW)
77.5 % Male
83.5 % White (Han & Jain, 2014)

41. BEYOND THE PAPER

42. https://www.researchgate.net/publication/317649493_Analysis_and_
Automation_of_Deep_Face_Recognition

43. DATA PREPRATION with DLIB