SlideShare a Scribd company logo
Traditional Approaches
EigenFace FisherFace Technique
Convolution Neural Network with
Inception V3
Simon Fraser University
PROBLEM
Facial expression classification:
• Capture live stream from a video camera attached to a
laptop for our experiments.
• Apply and Benchmark different machine learning models
for facial expression recognition
• Classify three different facial expressions: Neutral, Happy
and Surprise.
• Final predicted facial expression is displayed via a live
feed using the laptop camera.
Applications of facial expression classification:
• Customer Engagement
• Virtual Reality Avatar
Countenance Classifier : How are you feeling today?
Liam Bui, Alexandre Lopes
Simple Convolution Neural Network
EXPERIMENTS
Summary of Results
Validation Accuracy by Epoch
DATASET SOURCES
• Initial models trained on the Kaggle Dataset with 35,000
facial expression images
• Limited computation resources makes it infeasible to
experiment with Kaggle Dataset
• CK Dataset supplemented with our self-created images is
used in our final experiments
• Class such as Neutral and Sad appeared to be very similar
to each other.
LIVE DEMO
• Happy Expression
• Neutral Expression
• Surprise Expression
Method
Accuracy without
data augmentation
Accuracy with Data
Augmentation
EigenFace 66% 68%
FisherFace 87% 88.2%
Simple CNN 94% 94%
Inception V3
(Inception training
disabled)
91% 91%
Inception V3
(Inception training
enabled)
99% 99%
CONCLUSIONS
• Out of the two traditional approaches, FisherFace
technique outperforms EigenFace technique.
• Convolutional Neural Network outperforms traditional
techniques.
• Enabling Inception V3 training improves model
performance significantly.
Input Layer Convolution
Layer 1
Convolution
Layer 2
Pooling
Layer
Dense Layer
Output
Layer
Dropout
Layer
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Validation%
Inception V3 vs Simple CNN(with Data Augmentation)
Validation Accuracy - CNN Validation Accuracy - Inception V3
LIMITATIONS
• Due to computing resource constraints, training the
Inception V3 model on CPU’S is time consuming.
• Hence, small data set is used in our experiments
• Live demo is more challenging because of different
conditions (lightning, blur, angles, glasses)
Input Layer Inception V3 Dense Layer
Output
Layer
Dropout
Layer

More Related Content

Similar to Real-time Emotion Recognition

Deep Representation: Building a Semantic Image Search Engine
Deep Representation: Building a Semantic Image Search EngineDeep Representation: Building a Semantic Image Search Engine
Deep Representation: Building a Semantic Image Search Engine
C4Media
 
Face recognition technology
Face recognition technologyFace recognition technology
Face recognition technology
SARATHGOVINDKK
 
ppt.pdf
ppt.pdfppt.pdf
Emotion recognition using image processing in deep learning
Emotion recognition using image     processing in deep learningEmotion recognition using image     processing in deep learning
Emotion recognition using image processing in deep learning
vishnuv43
 
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Alok Singh
 
ppt 20BET1024.pptx
ppt 20BET1024.pptxppt 20BET1024.pptx
ppt 20BET1024.pptx
ManeetBali
 
cvpresentation-190812154654 (1).pptx
cvpresentation-190812154654 (1).pptxcvpresentation-190812154654 (1).pptx
cvpresentation-190812154654 (1).pptx
PyariMohanJena
 
An Enhanced Independent Component-Based Human Facial Expression Recognition ...
An Enhanced Independent Component-Based Human Facial Expression Recognition  ...An Enhanced Independent Component-Based Human Facial Expression Recognition  ...
An Enhanced Independent Component-Based Human Facial Expression Recognition ...
أحلام انصارى
 
Machine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine LearningMachine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine Learning
Arshad Ahmed
 
FACE RECOGNITION ACROSS NON-UNIFORM MOTION BLUR
FACE RECOGNITION ACROSS  NON-UNIFORM MOTION BLUR FACE RECOGNITION ACROSS  NON-UNIFORM MOTION BLUR
FACE RECOGNITION ACROSS NON-UNIFORM MOTION BLUR
Koduru KrisHna
 
A study on face recognition technique based on eigenface
A study on face recognition technique based on eigenfaceA study on face recognition technique based on eigenface
A study on face recognition technique based on eigenface
sadique_ghitm
 
Computer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and PythonComputer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and Python
Akash Satamkar
 
Face Recognition
Face RecognitionFace Recognition
Face Recognition
Saraj Sadanand
 
Facial recognition locker for android
Facial recognition locker for androidFacial recognition locker for android
Facial recognition locker for android
konark jain
 
A DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGES
A DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGESA DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGES
A DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGES
PNandaSai
 
Online exam software
Online exam softwareOnline exam software
Online exam software
Pondurai Singh
 
ppt 20BET1024.pptx
ppt 20BET1024.pptxppt 20BET1024.pptx
ppt 20BET1024.pptx
ManeetBali
 
neuralAC
neuralACneuralAC
neuralAC
Dr Rupesh Shet
 
KorraAI - a probabilistic virtual agent framework
KorraAI - a probabilistic virtual agent frameworkKorraAI - a probabilistic virtual agent framework
KorraAI - a probabilistic virtual agent framework
AntonAndreev13
 
Face recognition: A Comparison of Appearance Based Approaches
Face recognition: A Comparison of Appearance Based ApproachesFace recognition: A Comparison of Appearance Based Approaches
Face recognition: A Comparison of Appearance Based Approaches
sadique_ghitm
 

Similar to Real-time Emotion Recognition (20)

Deep Representation: Building a Semantic Image Search Engine
Deep Representation: Building a Semantic Image Search EngineDeep Representation: Building a Semantic Image Search Engine
Deep Representation: Building a Semantic Image Search Engine
 
Face recognition technology
Face recognition technologyFace recognition technology
Face recognition technology
 
ppt.pdf
ppt.pdfppt.pdf
ppt.pdf
 
Emotion recognition using image processing in deep learning
Emotion recognition using image     processing in deep learningEmotion recognition using image     processing in deep learning
Emotion recognition using image processing in deep learning
 
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...
 
ppt 20BET1024.pptx
ppt 20BET1024.pptxppt 20BET1024.pptx
ppt 20BET1024.pptx
 
cvpresentation-190812154654 (1).pptx
cvpresentation-190812154654 (1).pptxcvpresentation-190812154654 (1).pptx
cvpresentation-190812154654 (1).pptx
 
An Enhanced Independent Component-Based Human Facial Expression Recognition ...
An Enhanced Independent Component-Based Human Facial Expression Recognition  ...An Enhanced Independent Component-Based Human Facial Expression Recognition  ...
An Enhanced Independent Component-Based Human Facial Expression Recognition ...
 
Machine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine LearningMachine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine Learning
 
FACE RECOGNITION ACROSS NON-UNIFORM MOTION BLUR
FACE RECOGNITION ACROSS  NON-UNIFORM MOTION BLUR FACE RECOGNITION ACROSS  NON-UNIFORM MOTION BLUR
FACE RECOGNITION ACROSS NON-UNIFORM MOTION BLUR
 
A study on face recognition technique based on eigenface
A study on face recognition technique based on eigenfaceA study on face recognition technique based on eigenface
A study on face recognition technique based on eigenface
 
Computer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and PythonComputer Vision - Real Time Face Recognition using Open CV and Python
Computer Vision - Real Time Face Recognition using Open CV and Python
 
Face Recognition
Face RecognitionFace Recognition
Face Recognition
 
Facial recognition locker for android
Facial recognition locker for androidFacial recognition locker for android
Facial recognition locker for android
 
A DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGES
A DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGESA DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGES
A DEEP LEARNING APPROACH FOR SEMANTIC SEGMENTATION IN BRAIN TUMOR IMAGES
 
Online exam software
Online exam softwareOnline exam software
Online exam software
 
ppt 20BET1024.pptx
ppt 20BET1024.pptxppt 20BET1024.pptx
ppt 20BET1024.pptx
 
neuralAC
neuralACneuralAC
neuralAC
 
KorraAI - a probabilistic virtual agent framework
KorraAI - a probabilistic virtual agent frameworkKorraAI - a probabilistic virtual agent framework
KorraAI - a probabilistic virtual agent framework
 
Face recognition: A Comparison of Appearance Based Approaches
Face recognition: A Comparison of Appearance Based ApproachesFace recognition: A Comparison of Appearance Based Approaches
Face recognition: A Comparison of Appearance Based Approaches
 

Recently uploaded

GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 

Recently uploaded (20)

GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 

Real-time Emotion Recognition

  • 1. Traditional Approaches EigenFace FisherFace Technique Convolution Neural Network with Inception V3 Simon Fraser University PROBLEM Facial expression classification: • Capture live stream from a video camera attached to a laptop for our experiments. • Apply and Benchmark different machine learning models for facial expression recognition • Classify three different facial expressions: Neutral, Happy and Surprise. • Final predicted facial expression is displayed via a live feed using the laptop camera. Applications of facial expression classification: • Customer Engagement • Virtual Reality Avatar Countenance Classifier : How are you feeling today? Liam Bui, Alexandre Lopes Simple Convolution Neural Network EXPERIMENTS Summary of Results Validation Accuracy by Epoch DATASET SOURCES • Initial models trained on the Kaggle Dataset with 35,000 facial expression images • Limited computation resources makes it infeasible to experiment with Kaggle Dataset • CK Dataset supplemented with our self-created images is used in our final experiments • Class such as Neutral and Sad appeared to be very similar to each other. LIVE DEMO • Happy Expression • Neutral Expression • Surprise Expression Method Accuracy without data augmentation Accuracy with Data Augmentation EigenFace 66% 68% FisherFace 87% 88.2% Simple CNN 94% 94% Inception V3 (Inception training disabled) 91% 91% Inception V3 (Inception training enabled) 99% 99% CONCLUSIONS • Out of the two traditional approaches, FisherFace technique outperforms EigenFace technique. • Convolutional Neural Network outperforms traditional techniques. • Enabling Inception V3 training improves model performance significantly. Input Layer Convolution Layer 1 Convolution Layer 2 Pooling Layer Dense Layer Output Layer Dropout Layer 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Validation% Inception V3 vs Simple CNN(with Data Augmentation) Validation Accuracy - CNN Validation Accuracy - Inception V3 LIMITATIONS • Due to computing resource constraints, training the Inception V3 model on CPU’S is time consuming. • Hence, small data set is used in our experiments • Live demo is more challenging because of different conditions (lightning, blur, angles, glasses) Input Layer Inception V3 Dense Layer Output Layer Dropout Layer