This document discusses face detection, analysis, and recognition using different techniques. It begins by introducing Matteo Valoriani and Luigi Oliveto. It then discusses doing face analysis at home using OpenCV/EmguCV. It covers using cloud services like Betaface and Microsoft Project Oxford. It also discusses using special cameras like Kinect and RealSense for face analysis. It concludes with discussing common problems and limits of face analysis techniques.

1. Template designed by
Face Recognition using C#
Matteo Valoriani
mvaloriani@gmail.com
@matteovaloriani
Luigi Oliveto
luigi.oliveto@gmail.com
@luigioliveto

2. Matteo Valoriani
CEO of Fifth Element
Speaker and Consultant
PhD at Politecnico of Milano
Microsoft MVP
Intel Software Innovator
email: mvaloriani@gmail.com
twitter: @MatteoValoriani
linkedin: https://it.linkedin.com/in/matteovaloriani
Nice to Meet You
2

3. Luigi Oliveto
Developer
Co-Speaker
Master of Science at Politecnico of Milano
email: luigi.oliveto@gmail.com
twitter: @LuigiOliveto
linkedin: https://it.linkedin.com/in/luigioliveto
Nice to Meet You
3

4. Face Detection vs Face Recognition vs Face Identification
Face Analysis HomeMade
• OpenCV/Emgu
Face Analysis with Cloud Services
• BetaFace
• Microsoft Face API
Face Analysis with Special Camera
• Kinect
• RealSense
Conclusions
• Common Problems and Limits
Agenda

5. Detection vs Analysis vs Recognition

6. Face detection is a computer technology that
identifies human faces in digital images.
Face Detection
Facial Point DetectionFace Detection/Tracking

7. Face Analysis is a computer technology that
analyze human faces in digital images and
elaborate physical and emotional characteristics.
Face Analysis
Gender/Age/Race AnalysisEmotion Analysis

8. Facial recognition system is a computer
application for automatically identifying or
verifying a person from a digital image or a video
frame from a video source.
Face Recognition
Face Similarity/GroupingFace Verification Face Identification

9. Face Analysis Home Made
OpenCV/Emgu

10. Emgu CV is a cross platform .Net wrapper to the OpenCV image
processing library.
What is EMGU

11. The basic layer (layer 1)
contains function, structure
and enumeration mappings
which directly reflect those in
OpenCV.
The second layer (layer 2)
contains classes that mix in
advantanges from the .NET
world.
EMGU Architecture

12. To start with you need to reference 3 EMGU DLL’s.
• Emgu.CV.dll
• Emgu.CV.UI.dll
• Emgu.Util.dll
using Emgu.CV;
using Emgu.Util;
using Emgu.CV.Structure;
Create a project with EMGU

13. cudart64_32_16.dll
cufft64_32_16.dll
cvextern.dll
npp64_32_16.dll
opencv_calib3d220.dll
opencv_contrib220.dll
opencv_core220.dll
opencv_features2d220.dll
opencv_flann220.dll
opencv_gpu220.dll
opencv_highgui220.dll
opencv_imgproc220.dll
opencv_legacy220.dll
opencv_ml220.dll
opencv_objdetect220.dll
opencv_video220.dll
Add > Existing Item

14. The goal of statistical classification is to use an object's
characteristics to identify which class (or group) it belongs to.
An object's characteristics are also known as feature values and
are typically presented to the machine in a vector called a feature
vector.
Machine Learning Classifier

15. Supervised Learning Model

16. OpenCV/EmguCV uses a type of face
detector called a Haar Cascade.
The Haar Cascade is a classifier (detector)
trained on thousands of human faces.
This training data is stored in an XML file,
and is later used by the classifier during
detection.
It’s the easiest ready to use face detection
method which is supported by
OpenCV/EmguCV and has great results.
Haar Feature-based Cascade Classifier

17. The Fisher Classifier is a linear classifier.
A linear classifier achieves this by
making a classification decision based
on the value of a linear combination of
the characteristics.
The Fisher Classifier

18. demo
Smile 

19. Face Analysis with Cloud Services
BetaFace
http://betaface.com/

20. SDK for Private Cloud Configuration
WebAPI for Public Cloud
BetaFace APIs

21. General face info:
- faces (positions, sizes, angles)
- face landmarks locations (22 basic, 101 pro)
- cropped face images
- gender, age, ethnicity, smile, glasses, mustache and beard detection
Extended measurements:
- face and facial features shapes description
- hair and skin color
- facial hair detection
- approximate hairstyle shape
- background color and clothes color.
BetaFace - Metadata

22. Following functions supported:
- upload image file or submit image url
- retrieve image and face metadata, including cropped face image
- compare single faces or groups of faces and receive similarity confidence along with match
decision.
- transform face image(s) - generate averages from two or more faces, change face
expression or otherwise modify them.
- add user defined metadata tags, store user-adjusted points and face info.
BetaFace - Metadata

23. JSON/XML response

24. JSON/XML response
<FaceInfo>
<angle>3.3149</angle>
<height>78.05</height>
<image_uid>2bdcd1ad-47a6-45f8-ba74-
86c765272422</image_uid>
<points>
<PointInfo>
<name>basic eye left</name>
<type>512</type>
<x>313.82</x>
<y>48.88</y>
</PointInfo>
…
</points>
<score>4.81</score>
<tags>
<TagInfo>
<confidence>0.07</confidence>
<name>age</name>
<value>31</value>
</TagInfo>
….
< /tags >
22 Points (Basic mode)
101 Points (Advanced Mode)
Basic:
Age - approximate age value
beard - yes, no
gender - male, female
glasses - yes, no
mustache - yes, no
smile - yes, no
race - asian-middle-eastern, asian,
african-american, hispanic, white, middle
eastern, other

25. 1. Get XML string
2. Generate XSD
• https://devutilsonline.com/xsd-xml/generate-xsd-from-xml
3. Generate C# classes
• XML Schema Definition Tool (Xsd.exe)
Create C# Classes from XML

26. From XML to XSD

27. https://msdn.microsoft.com/en-us/library/x6c1kb0s(v=vs.110).aspx
From XSD to Class

28. FREE:
Current public API key limits: faces search/recognition requests - no limits; new images - 500 images per day (15000 images per month);
Same image with different set of processing flags counts as new image; images in processing queue - 500; transform requests - no limits.
Freemium: 0 EUR/month 500 IMAGE /day, 0.035 EUR extra
Basic: 199 EUR/month 40000 IMAGE/month 0.025 EUR extra
Premium: 399 EUR/month 100000 IMAGE/month 0.02 EUR extra.
IMAGE – Each new image processed via UploadImage, UploadNewImage_File or UploadNewImage_Url functions.
- uploading the same image with different detection_flags counts as IMAGE.
- uploading the same image with the same set of detection_flags while previous processing results are still in cache does not count as
IMAGE.
- no restrictions on recognize, GetRecognizeInfo or GetImageInfo requests; no restrictions on number of namespaces or their size
If you like to subscribe to one of those plans send email to info@betaface.com with your details for invoice and plan you selected. We will
send you your personal API key.
Current data storage policy: Source images are removed from cache shortly after processing. Faces that have no person/namespace
assigned and corresponding image metadata usually cleaned up after 10 days (face IDs and image IDs will be invalidated).
Licensing: Free VS PRO

29. Online test: http://www.betafaceapi.com/demo.html#
Documentation:
http://www.betafaceapi.com/service_json.svc/help
Links

30. Face Analysis with Cloud Services
Project Oxford
http://www.projectoxford.ai/

31. Face Detection
Face Recognition
• Face Verification
• Similar Face Searching
• Automatic Face Grouping
• Person Identification
Project Oxford Services

32. Powered by Azure

33. 1. Access the Project Oxford Portal https://www.projectoxford.ai, and then click
on the "Sign up" button.
2. Sign in with your Microsoft account, or Sign up for a new Azure subscription if
you don't already have one.
3. Go down the list to select an offered service such as "Face APIs" from the list,
and then click through the various windows in order to make a purchase.
4. Click on the item to view the dashboard, and at the bottom of the page, click
on the 'Manage' button to go to the 'Developer Manage Keys' page.
5. Finally, Copy or regenerate subscription keys in the page.
Get Start

34. demo
Smile 

35. Face Analysis with Special Cameras
Kinect ONE

36. Face Analysis with Special Cameras
Real Sense F200

37. demo
Smile 

38. Q&A

39. Grazie a tutti per la partecipazione
Riceverete il link per il download a slide e demo via email nei
prossimi giorni
Per contattarmi
mvaloriani@gmail.com
luigi.oliveto@gmail.com
Grazie