http://ieeexplore.ieee.org/document/6712732/

1. Using Nasal Curves Matching for Expression
Robust 3D Nose Recognition
Authors:
Mehryar Emambakhsha
, Adrian N Evansa
and Melvyn Smithb
a
University of Bath, UK
b
University of the West of England, UK

2. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

3. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

4. Introduction and background
• Why the nose region?
• Easy to detect
• Hair occlusion robustness
• Difficult to conceal
• Relatively expression invariant ....
… but not all parts!

5. Introduction and background
ICP one-to-one
matching on
multiple nasal
regions
K. Chang, W. Bowyer, and P. Flynn. Multiple nose region matching for 3D face
recognition under varying facial expression. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 28(10):1695–1700, 2006.

6. ICP one-to-one matching on
the cropped nasal region using
curvature maps
H. Dibeklio lu, B. Gökberk, and L. Akarun. Nasal region based 3D face recognition under pose and expressionǧ
variations. In Proceedings of the Third International Conference on Advances in Biometrics, pages 309–318, 2009.
Introduction and background

7. Distances
between the
geodesic
contours
H. Drira, , B. Amor, M. Daoudi, and A. Srivastava. Nasal region contribution in 3D face biometrics using shape analysis
framework. In Proceedings of the Third International Conference on Advances in Biometrics, pages 357–366, 2009.
Introduction and background

8. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

9. Why curves are better than direct 3D surface?
• Lower complexity than direct 3D
• Potentially more robust against expression variation and
spike noise
• Can correspond to more invariant regions
Preprocessing
Feature
selection
Landmarking
and curves
finding
Raw
3D
data
Expression
robust
curves
Nasal curves matching

10. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

11. Preprocessing
Resampling and denoising
• Resampling to remove noise from the X and Y maps
• Denoising:
• Holes  conditional morphological filling
• Missing data  Cubic interpolation
• High frequency and spike noise  Median filtering with 2.31mm
2.31mm maskᵡ
Nasal curves matching

12. Nasal curves matching
Facial region cropping and alignment
• Sphere (r=80mm) centred on the nose tip, intersected
with the face surface
• PCA iteratively applied to the cropped face
• Depth map resampled in every iteration
• The nose tip is iteratively re-detected using the shape
index thresholding
A. Mian, M. Bennamoun, and R. Owens. An efficient multimodal 2D-3D
hybrid approach to automatic face recognition. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 29(11):1927–1943, 2007.
Preprocessing

13. Nasal region cropping
• Three cylinders are intersected with the face surface
• Convex hull of AND found
Preprocessing
AND Convex hull
Cropped nose
Cylinders’ intersections with the face surface
Nasal curves matching

14. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

15. Landmarking and curves finding
16 landmarks are detected on the nasal region
Nasal curves matching

16. • Detecting L9 (nose tip)
• The filled nose region is
multiplied by thresholded
shape index
• The result is dilated
• Point with the highest
depth value is detected as
L9
M. Emambakhsh and A. Evans. Self-dependent 3D face rotational alignment
using the nose region. In 4th IET International Conference on Imaging for Crime
Detection and Prevention 2011 (ICDP 2011), pages 1–6, 2011.
Landmarking and curves finding
Filling
Convex
regions
Multiplication Dilation
Nasal curves matching

17. Landmarking and curves finding
• Detecting L1
• Set of orthogonal planes,
passing L9 are
intersected with the nose
surface
• The intersections’ minima
are detected (green
curve)
• The maximum of the
minima is chosen as L9
Nasal curves matching

18. • Detecting L5 and L13
(alar)
• Second set of orthogonal
planes, passing L9 are
intersected with nose surface
• The intersections’ gradient is
computed and its maxima are
detected (green points)
• Outliers are removed using K-
means (K=2) clustering
• Those whose x and y values
are closest to L9’s are
selected as L5 and L13
Landmarking and curves finding
Nasal curves matching

19. • Other landmarks are
selected by dividing the
lines connecting L1 to L5,
L5 to L9, L9 to L13 and
L13 to L1 into 4 sections.
Landmarking and curves finding
Nasal curves matching

20. Nasal curves matching
• Curves are found by intersecting planes with the nose
surface, passing different pairs of the landmarks.
• 75 curves are created.
Landmarking and curves finding

21. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

22. Nasal curves matching
• Forward sequential feature selection (FSFS) is used to
detect the most robust curves in different expressions.
• Those curves which produce the highest rank-one
recognition, based on the leave-one-out method are
kept.
• Genetic algorithm-based feature selection is also used.
• The rank-one recognition rate is the cost function, while
the binary weights are the parameters:
Feature selection

23. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

24. Experimental results
• Datasets
• Face Recognition Grand Challenge (FRGC) v2.0
• 557 unique subjects
• Images obtained in three different time periods: Spring
2003, Fall 2003 and Spring 2004
• Bosphorus dataset
• 105 unique subjects
• Various expressions: neutral, happy, surprise, fear,
sadness, anger and disgust
• Bosphorus  Feature selection
• FRGC v2.0  Nose recognition evaluation

25. Experimental results
• Landmarking accuracy and consistency
• The nose tip is translated to the origin.
• The Euclidean distance between different landmarks are
computed per class.
• The overall mean and standard
deviation of the distances
L1: 3.3620 ± 1.6798 mm
L5: 2.3557 ± 1.1822 mm
L13: 2.4259 ± 1.2691 mm
C. Creusot, N. Pears, and J. Austin. A Machine-Learning Approach to Keypoint
Detection and Landmarking on 3D Meshes. International Journal of Computer
Vision, pages 1–34, 2013.

26. Experimental results
• FSFS results
• 28 curves produce the highest rank-one recognition rate.
• These curves are the least sensitive ones to expression
variation.

27. Experimental results
• Comparison with GA’s result
• FSFS outperforms GA in terms of convergence and computational
cost.
• FSFS selects fewer curves, with higher rank-one recognition rate.

28. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

29. Experimental results
• Subspace projection methods  projection to a 277-
dimensional space
• Principal Component Analysis (PCA)
• Linear Discriminant Analysis (LDA)
• Kernel Fisher’s Analysis (KFA)  polynomial (Poly)
and fractional power polynomial (FPP) kernels are
used
• Multi-class Support Vector Machines (Multi-SVM)
• One-vs.-all scenario for multi-classification
• Linear kernels
• Bootstrap aggregation decision trees (TreeBagger)
• 119 trees are aggregated
• Direct city-block (CB) distance
6
methods
Classification algorithms

30. Experimental results
• The number of training samples are varied per class.
• KFA-Poly outperforms all the classification methods.
Classification algorithms

31. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

32. Experimental results
• Spring 2003  Training
• Fall 2003 + Spring 2004  Test
FRGC v2.0 Experiment III results

33. Experimental results
• Comparison of equal error rate (EER) for the best
performing KFA-Poly curve
K. Chang, W. Bowyer, and P. Flynn. Multiple nose region matching for 3D face
recognition under varying facial expression. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 28(10):1695–1700, 2006.
FRGC v2.0 Experiment III results

34. Experimental results
• Rank-one recognition rate comparison
FRGC v2.0 Experiment III results

35. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

36. Conclusion: Summary
• In this work the potential of the nasal curves for 3D nose
recognition is evaluated.
• A robust landmarking algorithm is proposed and curves
are created by planes intersections with the nose
surface.
• The most invariant curves are selected by applying
FSFS over the Bosphorus dataset.
• Various classifiers are used on the feature space.
• KFA-Poly generated the highest rank-one recognition
rate and lowest EER.

37. Outline
• Introduction and background
• Nasal Curves Matching
• Preprocessing
• Landmarking and curves finding
• Feature selection
• Experimental results
• Classification algorithms results
• FRGC v2.0 Experiment III results
• Conclusion
• Summary
• Future work

38. Conclusion: Future work
• Curves extension to other facial parts
• Application in pattern rejection
• Robustness
• Against partial/self- occlusion
• Training-free matching
• Other feature selection methods
• Applying rank-level fusion techniques on the curves
• Other feature spaces, not just depth!
• Gabor-wavelets
• Local binary patterns (LBP)
• …

39. Thank you!