This document describes a facial expression recognition system with two parts: face recognition and facial expression recognition. It discusses using principal component analysis (PCA) and linear discriminative analysis (LDA) for face recognition, and PCA to extract eigenfaces for facial expression recognition. The system first performs face detection, then extracts facial expression data and classifies the expression. MATLAB is used as the tool for its faster programming capabilities.

1. Predicting Emotions through
Facial Expressions
MADE BY-
TWINKLE SINGH
URJA THAKUR

2. Introduction about the minor project
Developing a computational model of face recognition is
quite difficult because faces are complex ,multidimensional
and meaningful visual stimuli. To make it easier and more
natural to interact with computers, people put forward new
demands to human computer interaction.
Project aim :A facial expression emotion
recognition system is proposed which consist of
two parts face recognition and facial expression
recognition .

3. Problem Statement
• Three basic problems need to be solved:
▫ Face detection
▫ Facial expression data extraction
▫ Facial expression classification
• Both static images and image sequences have
been used in studies surveyed in the PROJECT

4. Solution Statement that are proposed :
• Given an image, to identify it as a face and/or
extract face images from it.
• To retrieve the similar images (based on a
heuristic) from the given database of face images.
• And extract the facial expression.

5. Algorithm used:
for face recognition
The Principal Component Analysis (PCA) and
Linear Discriminative Analysis (LDA)
face expression recognition .
Eigen space and vector (based on PCA and
LDA)

6. Face recognition : Most Basic algorithm for face
recognition (Pattern Matching) is Principal Component
Analysis(PCA). One another pattern matching algorithm,
Linear Discriminative Analysis (LDA) is also being
applied.

7. Facial expression recogniton :
o The task of facial recogniton is discriminating input
signals (image data) into several classes (persons).
o The input signals are highly noisy yet the input images
are not completely random and in spite of their differences
there are patterns which occur in any input signal.
o These characteristic features are called eigenfaces in the
facial recognition domain (or principal components
generally).
o They can be extracted out of original image data by
means of a mathematical tool called Principal Component
Analysis (PCA).

8. Software:
MATLAB is being used as tool.
WHY MATLAB? NOT OpenCV !!
• Matlab --> Faster programming, less efficient
• OpenCV -->Slower programming, more efficient
MATLAB provide very nice platform for
computational problems.
Matlab, is more recommended since we will be able
to program faster. If you already had a result of your
research, and want to make an application with it,
OpenCV recommended, it will take longer to
program.

9. Algorithm
• FACE RECOGNITION
Principal Component Analysis (PCA)
 An image can be considered as 2D array or Matrix.
 Each pixel can be classified as variable(one axis in co-
ordinate system in which image is represented) for
image. The idea is that a high-dimensional dataset is
often described by correlated variables and therefore only
a few meaningful dimensions account for most of the
information.
 The PCA method finds the directions with the greatest
variance in the data, called principal components.

10. • Example
In this particular example, data set is given in 2
dimension space(x1,x2).

11. • Linear discriminant analysis (LDA)
• The LDA perform dimensionality reduction
while preserving as much of the class
discriminatory information as possible.
• . It seeks to find directions along which the
classes are best separated.
• The LDA takes into consideration the scatter
within-classes but also the scatter between-
classes.

16. • It maximize between classes scattering.
• It minimizes within class scattering.
• So maximize ratio of between classes scattering
and within class ratio. This can be done by
finding Eigen Values.

18. ▫ FACIAL EXPRESSION RECOGNITION
Eigen faces
• Most of the studies perform an emotion
classification and use the following
6 basic categories: happiness,
sadness, surprise,
fear, anger, and disgust.
Eigen Space and Eigen Faces:
 Face Images are projected into
a feature space (“Face Space”)
that best encodes the variation
among known face images.

19.  The face space is defined by the “eigenfaces”, which
are the eigenvectors of the set of faces.
STEPS
 Initialization :
 Acquire the training set and calculate eigenfaces
(using PCA projections) which define eigenspace.
 When a new face is encountered, calculate its weight.
 Determine if the image is face.
 If yes, classify the weight pattern as known or
unknown.
 (Learning) If the same unknown face is seen several
times incorporate it into known faces.

20. PCA
Main assumption of PCA approach:
• Face space forms a cluster in image space.
• PCA gives suitable representation.

21. Eigenfaces (1)
• Calculation of Eigenfaces
(1) Calculate average face : v.
(2) Collect difference between training images and
average face in matrix A (M by N), where M is the number of
pixels and N is the number of images.
(3) The eigenvectors of covariance matrix C (M by M)
give the eigenfaces.
▫ M is usually big, so this process would be time consuming.
▫ What to do?

22. Eigenfaces (2)
• Calculation of Eigenvectors of C
If the number of data points is smaller than the dimension
(N<M), then there will be only N-1 meaningful eigenvectors.
Instead of directly calculating the eigenvectors of C, we can
calculate the eigenvalues and the corresponding
eigenvectors of a much smaller matrix L (N by N).
if λi are the eigenvectors of L then A λi are the eigenvectors for
C.
▫ The eigenvectors are in the descent order of the corresponding
eigenvalues.

23. Eigenfaces (3)
• Representation of Face Images using Eigenfaces
• The training face images and new face images
can be represented as linear combination of the
eigenfaces.
• When we have a face image u :
Since the eigenvectors are orthogonal :

24. Eigenfaces (4)
• Experiment and Results
Data used here are from the ORL database of
faces. Facial images of 16 persons are used. - Tra
iTest set contains 16×3 images.

25. REFERENCES
• Automatic analysis of facial actions, IEEE paper, vol 8 ,2017
• A facial expression emotion recognition based human robot interaction
system ,IEEE journal , october 2017
• http://www.cs.ucsb.edu/%7Emturk/Papers/mturk-CVPR91.pdf
• http://openbio.sourceforge.net/resources/eigenfaces/eigenfaces-
html/facesOptions.html
• http://face-rec.org/ - Facial Recognition homepage
• https://in.mathworks.com/matlabcentral/fileexchange/33325-
eigenface-based-facial-expression-classification
• Cambridge University Computer Laboratory The Digital Technology
Group orl databse.
• Ekman P, Friesen WV. The Facial Action Coding System A Technique
for the Measurement of Facial Movement. San Francisco: Consulting
Psychologists Press; 1978. http://face-and-
emotion.com/dataface/facs/manual/TitlePage.html