Robust Face Recognition under Varying Illumination and Occlusion Considering Structured Sparsity, The International Conference on Digital Image Computing: Techniques and Applications (DICTA’12), December 2012, Australia

1. Xingjie Wei, Chang-Tsun Li and Yongjian Hu
Department of Computer Science, University of Warwick
x.wei@warwick.ac.uk
http://warwick.ac.uk/xwei
Robust Face Recognition under Varying
Illumination and Occlusion Considering
Structured Sparsity

2. Face
• People love faces !
– Biological nature
– Sensitive to the face pattern
2
A house with a
Hitler face

3. Face Recognition
• Uncontrolled conditions: large changes in
pose, illumination, expression and occlusion,
aging… Still challenging
3

4. 4
Motivation
• Face recognition in real-world environments
often has to confront with uncontrolled and
uncooperative conditions
– illumination changes, occlusion
• Uncontrolled variations are usually coupled
• Less work focuses on simultaneously
handling them

5. 5
Our Method
• Our work deals with the illumination changes
and occlusion simultaneously considering
structured sparsity
Sparse Representation
flat sparsity
represents a test image using minimal number of training images
from all classes
represents a test image using the minimal number of clusters

6. 6
Our Method
• Our work deals with the illumination changes
and occlusion simultaneously considering
structured sparsity aided with:
– Structural occlusion dictionary: better modelling
contiguous occlusion
contiguous occlusion also forms a cluster structure

7. 7
Our Method
• Our work deals with the illumination changes
and occlusion simultaneously considering
structured sparsity aided with:
– Structural occlusion dictionary: better modelling
contiguous occlusion
– WLD feature: robust to illumination changes,
remove shadows
Inspired by the psychophysical
Weber’s Law

8. Sparse Representation
8
• Models a test image as a linear
combination of training images
– Using minimal number of training images
y X α
= ×
1
0
0
1
.
.
.
…
sparse

9. Sparse Representation
• Involves training images from all classes
– Optimal for reconstruction but not
necessary for classification
9
Using the same
number of base
vectors

10. 10
Our Method
• Structured Sparsity
– Each class form a cluster
cluster structure

11. 11
Our Method
• Structured Sparsity
– Represents a test image using the
minimum number of clusters
y X α
= ×
1
1
0
0
0
.
.
…
…
cluster1 cluster2
…

12. Sparse Representation
• Occlusion modelling: identity matrix
– limitation: is able to represent any
image of size m 12
×=
1
0
0
1
.
.
1
0
… …
I
sparse
size: m X
m base
vectors

13. Our method
• Contiguous occlusion: the nonzeros entries
are likely to be spatially continuous, are
aligned to clusters
13
index of occlusion base vectors
size: 83*60=4980

14. Our method
• Structural occlusion dictionary
– uses the cluster occlusion dictionary to
replace the identity matrix I
14
×=
1
1
0
0
…
0
1
1
… …
D
cluster2 cluster
(s+1)
…cluster1 … cluster
s
X cluster
structure

15. 15
Our Method
• Extreme illumination + occlusion:
– coupled occlusion takes up a large ratio of
the image
– not “sparse” error

16. 16
Our Method
• A different view: extract relevant features
that reduce the difference
• Using WLD feature +1 +1 +1
+1 -8 +1
+1 +1 +1
0 0 0
0 +1 0
0 0 0
Filtering
Original image WLD feature
 Maintain most salient
facial features
 Insensitive to
illumination changes
 Can correct shadow
effects
Chen et al, Wld: A robust local image descriptor, PAMI, 2010

17. Illustrative Example
17
Test image
Estimated
occlusion
Reconstruction
Sparse coefficients
Residuals
Reference
image
Sparse coefficients
Residuals
Test image
Estimated
occlusion
Reconstruction
Reference
image
belongs
to class 1
class 1

18. Experiments
• Synthetic Occlusion with Extreme
Illumination
– Extended Yale B database
– Occlusion levels: 0% ~ 50% of the image
18
Subset 3
Subset 4
Subset 5
Training set Testing set

19. Experiments
• Synthetic Occlusion with Extreme
Illumination
– using only the raw pixel intensity as feature
19[15] Wright et al, TPAMI, 2009. [17] Zhang et al, ICCV, 2011

20. Experiments
• Synthetic Occlusion with Extreme
Illumination
– using WLD feature
20
[15] Wright et al, TPAMI, 2009. [16] Yang et al, ECCV, 2010

21. Experiments
• Synthetic Occlusion with Extreme
Illumination
– using WLD feature
21
[15] Wright et al, TPAMI, 2009. [16] Yang et al, ECCV, 2010

22. Experiments
• Disguise with Non-uniform Illumination
– The AR Database
– Real occlusion, 2 sessions
22
Training set Testing set

23. Experiments
• Disguise with Non-uniform Illumination
23

24. 24
Thank you
• Questions ?
• Xingjie Wei
• x.wei@warwick.ac.uk
• http://warwick.ac.uk/xwei
• Department of Computer Science, University
of Warwick