Machine Learning in Computer Vision

RECOGNISING FACES
MACHINE LEARNING & COMPUTER VISION: 101
SAKSHAM GAUTAM

08.06.2016 Saksham Gautam
LET’S START WITH A QUICK SHOW OF HANDS
HOW MANY OF YOU…
▸ have heard about machine learning?
▸ have used machine learning in your projects?
▸ have implemented any ML algorithm from scratch?
▸ have done Andrew Ng’s (or other) courses on ML?
▸ understand that deep learning uses neural network?
▸ still remember what the kernel trick is?

SHOW OF HANDS ON YOUR FAMILIARITY WITH COMPUTER VISION
HOW ABOUT…
▸ know how an image can be represented as a matrix?
▸ have used openCV or MATLAB?
▸ understand how convolution can be used to detect edges?
▸ know the role of scale space in computer vision?
▸ remember how eigenvectors can be used for face
recognition?

WHAT DO WE WANT TO ACHIEVE?
FACE DETECTION & RECOGNITION
http://docs.opencv.org/master/d7/d8b/tutorial_py_face_detection.html#gsc.tab=0
FACE
MONA LISA
NOT A FACE!

TO RECOGNISE AND LABEL OBJECTS IN AN IMAGE
OBJECT RECOGNITION

BASIC STEPS FOR FACE RECOGNITION
BUT HOW EXACTLY?
1. Capture image
2. Filter out noise
3. Find face in the image
4. Create a similarity metric and a model (Training)
5. Match any given face to one from the database
6. Return the closest match with the probability

FACE RECOGNITION CAN BE BROKEN DOWN INTO SIMPLE STEPS
BUILDING BLOCKS
RAW IMAGE
PROCESSED
IMAGE
FEATURES
MACHINE LEARNING
ALGORITHM
Training
Validation
MODEL
RAW IMAGE
DECISION
How can I capture image?
Remove any noise?
What’s the information 
in the image?
Can we match patterns?

HUMAN SEES BEAUTY BUT COMPUTER SEES NUMBERS
WHAT IS THIS?
=

PIXEL VALUES CAN BE THOUGHT OF AS ‘ELEVATION’ IN IMAGE PLANE
MONALISA IN 3D?

FILTER - I, MEAN FILTER CAN REMOVE RANDOM NOISE
MEAN FILTER

FILTERS - II, MEDIAN FILTER CAN REMOVE SALT & PEPPER NOISE
MEDIAN FILTER

FILTERS-III, ADVANCED FILTERS CAN REMOVE DIFFERENT TYPES OF NOISE
ADVANCED FILTERS
▸ Bilateral ﬁlters
▸ Adaptive bilateral ﬁlters
▸ …

38 53 38 70 84
5 56 69 82 98
34 87 85 5 40
25 99 43 69 76
11 61 86 94 59
-1 0 1
-1 0 1
-1 0 1
CONVOLUTION CAN BE APPLIED VERY QUICKLY ON AN IMAGE
CONVOLUTION
*
38 0 -38
5 0 -69
34 0 -85
-115
-115 56 16
-69 112 12
-93 3 6

CONVOLUTION CAN BE USED FOR COMPUTING IMAGE GRADIENT
IMAGE GRADIENT
0 0 0
100 100 100
0 50 100
100 50 0
-1 0 1
*
0
0
100
-100
=

EDGES AND CORNERS ARE FEATURES IN AN IMAGE
SOBEL FILTER FOR DETECTING EDGES
HARRIS CORNER DETECTOR
-1 0 1
-2 0 2
-1 0 1
Gx =
-1 -2 -1
0 0 0
1 2 1
Gy =
$python sobel-filter.py
$python harris-corner.py

MORE ROBUST FEATURES CAN BE USED FOR OBJECT RECOGNITION
SIFT, SURF, HOG
▸ More advanced features can be used for scale invariance
▸ Some are robust even under varying lighting conditions
▸ These serve as the starting point for the ML part

CASCADES OF FILTERS ON AN IMAGE CAN BE USED FOR DETECTING FACES
DETECTING FACES
http://siret.ms.mff.cuni.cz/facereco/method/
$python viola-jones.py

FEATURES FRO THE FACE CAN BE FED TO AN ML ALGORITHM
BUILDING BLOCKS
RAW IMAGE
PROCESSED
IMAGE
FEATURES
MACHINE LEARNING
ALGORITHM
Training
Validation
MODEL
RAW IMAGE
DECISION

PERFORMANCE (P) OF A METHOD
FOR A TASK (T) INCREASES WITH
EXPERIENCE (E)
Tom Mitchell
BTW, WHO LEARNS? THE MACHINE, REALLY?

PROBABILITY AND STATISTICS CAN HELP ANSWER MANY QUESTIONS
T-SHIRT SIZE FOR THE SUMMIT MAYBE I SHOULD
HAVE PICKED ‘M’
INSTEAD OF ’S'

EXACT SOLUTION IS NOT ALWAYS POSSIBLE
T-SHIRT SIZE ~ MY BODY’S MEASUREMENTS

MAXIMUM LIKELIHOOD ESTIMATE HELPS ON THE FACE OF UNCERTAINTY
CLASSIFICATION PROBLEM?
Length #2
Length #1
S
M
Length #1
Length #2
S

MEMBERSHIP TO A CLUSTER CAN BE USED FOR CLASSIFICATION
CLUSTERING
Length #2
Length #1
S
M

MARGINS CAN BE USED FOR SEPARATING CLASSES
LARGE MARGIN CLASSIFIER
Length #2
Length #1

LOG(DISTANCE FROM THE HYPERPLANE) = PROBABILITY
LOGISTIC REGRESSION
Length #2
Length #1

LARGE MARGIN CLASSIFIER CAN BE USED FOR DETECTING FACES
FEEDING VALUES FROM VIOLA JONES FILTER
X1
X2
FACE
NOT A FACE

FEATURES SHOULD MAXIMISE VARIANCE (SCATTER)
EIGENVECTORS
V1
V2
X1
X2

MATRIX DECOMPOSITION ON FACES CAN BE USED TO FIND EIGENFACES
FACIAL FEATURES AS EIGENVECTORS
V1
V2
Projection

ACCURACY ITSELF IS NOT ENOUGH
CROSS VALIDATION AND F SCORE
DUMMY CANCER
DETECTOR
Data
No
ACCURACY = 98%

IMAGE CAN BE DIRECTLY FED INTO NEURAL NETWORK
NEURAL NETWORK

GO WITH THE HYPE, BUT WITH CARE
DEEP LEARNING ~ MASSIVE NEURAL NETWORK
▸ Learning algorithm is the same, i.e. back propagation
▸ Has the same problem with overﬁtting
▸ Can be used for feature extraction and selection
▸ Mathematical foundations for neural network still not
“perfect”
▸ Pointer: https://www.tensorﬂow.org from Google

MACHINE LEARNING PIPELINE
SUMMARY
RAW IMAGE
PROCESSED
IMAGE
FEATURES
MACHINE LEARNING
ALGORITHM
Training
Validation
MODEL
RAW IMAGE
DECISION

REFERENCES
• OpenCV Documentation. http://docs.opencv.org/3.1.0/#gsc.tab=0
• Andrew Ng. Machine Learning Courser on Coursera. http://www.coursera.org/learn/machine-learning
• Christopher Bishop. Machines that Learn. https://www.youtube.com/watch?v=icaA7gVxqSs
• Video Lecture on Face Detection and Tracking. https://www.youtube.com/watch?v=WfdYYNamHZ8
• Adam Harvey explains Viola-Jones Face Detection. http://www.makematics.com/research/viola-jones/
• Christopher, M. Bishop. "Pattern recognition and machine learning." Company New York 16.4 (2006):
049901.
• Bradski, Gary, and Adrian Kaehler. Learning OpenCV: Computer vision with the OpenCV library. "
O'Reilly Media, Inc .", 2008
• Solem, Jan Erik. Programming Computer Vision with Python: Tools and algorithms for analyzing
images. " O'Reilly Me dia, Inc.", 2012.
• Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge
university press, 2003.

IMAGE GRADIENT IS THE BASIS OF FEATURE EXTRACTION
GRADIENT IMAGE
-1 0 1gx =
-1
0
1
gy =* A * A

EVERY SIGNAL CAN BE DECOMPOSED TO SINES AND COSINES
FOURIER TRANSFORM
▸ Frequency can be thought of as information in the image
▸ Fourier Transform can be used to decompose a signal into
these components
▸ Signal can be multiplied with ﬁlter in frequency domain
▸ Multiplication in frequency domain is convolution in time
domain

Machine Learning in Computer Vision

More Related Content

Similar to Machine Learning in Computer Vision

Recently uploaded

Machine Learning in Computer Vision