2. Aim of the project
To detect all the faces in a given RGB image
Match them one by one with the faces present in the
database
If the detected face matches closely with a face present
in the database then it is classified as a recognized face
If a face can not be recognized , then add the face to
the existing database so that it can be recognized next
time
3. Face recognition using Eigenfaces
Suppose Γ is a 𝑁2 X 1 vector, corresponding to an NxN face
image I
Problems arise when performing recognition in a high-
dimensional space
The idea is to represent G into a low-dimensional space
4. Computation of the eigenfaces
1- Obtain face images I1, I2, ..., IM (training faces)
the face images must be centered and of the same size
2- Represent every image Ii(NXN)as a vector Γi(𝑁2X1)
3- Compute the average face vector Ψ
Ψ= 1/𝑀( Γ𝑖 )∞
𝑖=1
4-Subtract the mean face: Φi = Γi -Ψ
5- Compute the covariance matrix C
5. 6-Compute the eigenvectors ui of 𝐴𝐴 𝑇
The matrix 𝐴𝐴 𝑇
is very large(𝑁2
X𝑁2
) so it is not
practical to compute its eigenvectors
6.1- Consider the matrix 𝐴 𝑇
𝐴 (MxM matrix)
6.2- Compute the eigenvectors vi of 𝐴 𝑇
𝐴
𝐴 𝑇 𝐴 vi =μi vi
6.3- 𝐴𝐴 𝑇 and 𝐴 𝑇 𝐴 have the same eigenvalues and their
eigenvectors are related as follows: ui = Avi
6.4-Compute the M best eigenvectors of 𝐴𝐴 𝑇
: ui = Avi
(normalize ui such that ||ui|| = 1)
7-Keep only K eigenvectors (corresponding to the K
largest eigenvalues)
7. Representing faces onto this basis
Each face (minus the mean) Φi=Γi-Ψ in the training
set can be represented as a linear combination of the
best K eigenvectors:
Φi= 𝑤𝑗 𝑢𝑗𝑘
𝑗=1 , (wj = 𝑢𝑗 𝑇Φi )
uj ’s are the ‘eigenfaces’
Each normalized training face Φi is represented in this
basis by a vector:
8. Face Recognition Using Eigenfaces
Given an unknown face image Γ
(centered and of the same size like the training faces)
1: Normalize Γ: Φ = Γ-Ψ
2: Project on the eigenspace
Φ= 𝑤𝑗 𝑢𝑗𝑘
𝑗=1 , (wj = 𝑢𝑗 𝑇Φ)
3: Represent Φ as:
9. 4: Find er = minl ||W-Wl||
er is the minimum euclidean distance between
weights vector of unknown image and its best
matching counterpart in image database
5: If er < Tr ( threshold ) , then G is recognized as face l
from the training set
13. Face detection from RGB image
Approach
Color Segmentation
False hit removal
Template matching
Recognition of Detected faces with training
images
14. Color Segmentation
The aim of color segmentation is to separate out skin
regions from non skin regions in RGB image
Use Gray World Assumption to get color balanced RGB
image
Convert RGB image into YCbCr image
The equation for transforming from RGB to YCbCr
15. Gray World Assumption
It states that for a typical scene, the average intensity
of the red, green, and blue channels should be equal
Let an image I(x,y) has size M x N, where x and y
denote the indices of the pixel position.
Ir(x,y), Ig(x,y) and Ib(x,y) denote the red,green, and
blue channels of the image respectively
Compute
𝑹 𝒂𝒗𝒈 =
𝟏
𝑴𝑵
𝑰 𝒓(𝒙, 𝒚)
𝑵
𝒚=𝟏
𝑴
𝒙=𝟏
𝑮 𝒂𝒗𝒈 =
𝟏
𝑴𝑵
𝑰 𝒈(𝒙, 𝒚)
𝑵
𝒚=𝟏
𝑴
𝒙=𝟏
𝑩 𝒂𝒗𝒈 =
𝟏
𝑴𝑵
𝑰 𝒃(𝒙, 𝒚)
𝑵
𝒚=𝟏
𝑴
𝒙=𝟏
16. If the three values are identical, the image already
satisfies the gray world assumption
If the three values are not identical we keep the green
channel unchanged, and define the gain for the red
and blue channels
α= 𝐺 𝑎𝑣𝑔/𝑅 𝑎𝑣𝑔 and β = 𝐺 𝑎𝑣𝑔/𝐵𝑎𝑣𝑔
Adjust the Red and Blue pixels by
𝐼𝑟(𝑥, 𝑦)= α𝐼𝑟(𝑥, 𝑦) and 𝐼 𝑏(𝑥, 𝑦)= β𝐼 𝑏 𝑥, 𝑦
18. Skin segmentation using YCbCr
Convert color balanced RGB image into YCbCr color
space
The pixels belonging to following range can be
classified as skin pixels
77<=Cb<=127 and
133<=Cr<=173
The skin segmentation gives a binary image
White pixels denote skin region
Black pixels denote non-skin region
20. False Hit Removal
In the binary image fill the black holes
present inside the white skin regions
Then check the aspect ratio for the regions
aspect ratio=width/height
For face (0.4<=aspect ratio <= 1 )
Regions not satisfying above conditions
are not considered
Original image
21. False Hit Removal
Skin segmented
image
Filled Image False detections Result after false
hit removal
23. Work to be done in next semester
Implement template matching for face detection after
skin segmentation
Study Viola–Jones object detection framework
- Slow but accurate algorithm
- Speed can be increased if used in the skin segmented
image
Implement face recognition via sparse representation
and compare its accuracy with current algorithm used
