This document presents a methodology for facial emotion recognition. It involves capturing an image, preprocessing by converting to grayscale and cropping regions of interest around facial features. Edge detection is then performed on the regions of interest. Emotions are classified by calculating displacement of edge points, averaging values, standard deviation, and comparing to a threshold. The methodology achieved 83.33% accuracy in classifying six basic emotions from facial images. Future work could improve the automated cropping of regions of interest and simplify the edge detection process.

1. (Roll no- 91/MCA/090021)
Project under the guidance of
DR. AMLAN CHAKRABARTI
1

2. What is Emotion?
 Emotions are reflected in voice, hand and body gestures,
and mainly through facial expressions.
2

3. Facial Emotion
 There are six types of facial emotions.
Anger Fear Disgust Happy Sad Surprise
3

4. Importance of emotion recognition
 Human beings express emotions in day to day
interactions.
 Understanding emotions and knowing how to react to
people’s expressions greatly enriches the interaction.
4

5. Review Of Previous Work
 Image pre-processing
 Facial feature extraction
-select some point on feature region
-consider a matrix surrounding that point
-calculate distances
 Facial expression detection
Reff.: [“Optical flow based analyses to detect emotion from human facial image data”- Axel Besinger ,
Tamara Sztynda, Sara Lal , Carmen Duthoit, Johnson Agbinya, Budi Jap ,David Eager, Gamini
Dissanayake]
5

6. Proposed Methodology
 Capture Image
 Image Pre-processing
-RGB to gray scale conversion
-Scale Normalization
 Crop Feature Regions
 Edge Detection
-Prewitt edge detection
 Facial Emotion Classification
6

7. Methodology(Cont…)
7

8. Capture Image
 Digital camera, web-cam
 Resolution must be fixed
 Distance between the web-cam and the face has to be
fixed.
 Position of face must be fixed
 Must not wear glasses or have facial hair.
8

9. Capture Image(Cont…)
Normal Facial Image Emotional Facial Image
9

10. RGB to gray scale conversion
Color Facial Image Grayscale Facial Image
 Reduce complexity.
10

11. Scale Normalization
 Select only the face portion within a rectangular
frame
 Remove unnecessary information
 By defining region.
11

12. Crop Feature Regions
 Region of Interest: two Eyes, two Eye brows, Lip
 By defining the particular regions.
12

13. Edge detection
ROI ROI after edge detection
13

14. Algorithm:
Input: A Sample Image
Output: Detected Edges
Step 1: Accept the input image
Step 2: Apply mask h1 and h3 (prewitt mask) to the input
image.
Step 3: Masks manipulation of h1 and h3 separately on
the input image
Edge Detection (Cont…)
14

15. Edge detection(Cont…)
Step 4: Results combined to find the absolute magnitude of the
gradient
Where
Step 5: The absolute magnitude is the output edges.
15

16. Facial Emotion Classification
 Calculate displacement of the points.
 Calculate averages of those values.
 Calculate standard deviation (SD) of those averages.
 Compare this SD with a threshold.
 Get Emotion.
16

17. Algorithm
 Step 1: Take a still image of a normal expression pic1
(say) of a human face.
 Step 2: Converts the color image to grayscale.
 Step 3: Crop the five facial image region of interest (ROI)
(eyes, eye brows and lip) from the image by defining
region.
 Step 4: Find edges of all image region.
 Step 5: Take a still image of a emotional face (angry or
happy) pic2 (say) of same person and repeat step 2, 3
and 4.
17

18. Algorithm(Cont…)
 Step 7: Comparing the deviation of edges of the
specified region of pic1 with pic2 by finding the
Euclidian distances of coordinate of each pixel.
 Step 8: Put the Euclidian distances in a array k (say).
 Step 9: Find the standard deviation (SD) from the
elements of array.
 Step 10: Comparing the SD with pre-define threshold
and get the emotions.
18

19. Experiment & Result
 30 images were used to create the template.
 60 were tested.
Statistical Values of Three categories

19

20. Experiment & Result
 Total 83.33% classify Facial Emotion.
63.33% were correct
20

21. Experiment & Result
 80% happy Emotion classification were correct.
 50% Angry Emotion classification were correct.
 Almost 100% null classification were correct.
21

22. Applications
 E-Learning system
 Robotics System
 Human-Computer interaction System
22

23. Conclusion and Future Work
 Implementation through this process is quite easy.
 Have to improve over the capturing Process.
 ROI extracted manually
-That should be automatically.
 Edge detection procedure should be less complex.
23

24. THANK YOU
24