Synopsis of Facial Emotion Recognition to Emoji Conversion

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 10 Issue: 12 | Dec 2023 www.irjet.net p-ISSN: 2395-0072
© 2023, IRJET | Impact Factor value: 8.226 | ISO 9001:2008 Certified Journal | Page 206
Synopsis of Facial Emotion Recognition to Emoji Conversion
Rohit Karmakar1, Subhadip Das2, Angshuman Rana3, Kunal Roy4
123 Students, Dept. of Computer Science and Engineering, Durgapur Institute of Advanced Technology and
Management, Durgapur, West Bengal, India
4 Assistant Professor, Durgapur Institute of Advanced Technology and Management, Durgapur, West Bengal, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - This project presents a real-time Facial Emotion
Recognition (FER)toEmojiConversionsystemdevelopedusing
OpenCV, TensorFlow, and NumPy. The system aims to identify
and classify human emotions from facial expressionscaptured
in real-time through a webcam. The FER system leverages the
powerful image processing capabilities of OpenCV, the
computational efficiency of NumPy, and the machine learning
techniques provided by TensorFlow. A model is trained on a
large dataset of facial imageslabeledwiththeircorresponding
emotions. The trained model is then used to predict emotions
from facial expressions capturedbyawebcam inreal-timeand
the conversion of recognized emotions into corresponding
emojis, providing a visual representation of the detected
emotion.
Key Words: Facial Emotion Recognition (FER), OpenCV,
TensorFlow, NumPy, Computer Vision
1.INTRODUCTION
Emotion recognition constitutesa captivatingfacetofhuman
behaviour and assumes a pivotal role in our daily
interpersonal exchanges. With the progression of
technology, the capability to discern emotions extends
beyond the realm of humans, encompassing the training of
computers for such recognition. Facial Emotion Recognition
(FER) emerges as a subset within computer vision and AI,
dedicated to formulating algorithms and methodologies for
identifying human emotions through facial expressions and
transformation of recognized emotions into corresponding
emojis. This will amplify user engagement and interaction,
making the system more user-friendly and appealing.
This undertaking centers on the creation of a FER system
utilizing OpenCV, TensorFlow, and NumPy. OpenCV
furnishes a robust framework for analysing images and
videos, TensorFlow provides a comprehensive platform for
machine learning, and NumPy facilitates efficient
computation with expansive, multi-dimensional arrays and
matrices. The selection of these tools is grounded in their
suitability for the project's objectives.
2. Motivation
The ability to decipher human emotions from facial
expressions to emoji conversion provides a window into
human behavior. FER, a convergence of computervisionand
machine learning, enables computers to interpret and
classify human emotions from facial cues.
This project dives into the creationofa real-timeFERsystem
utilizing OpenCV,TensorFlow,andNumPy.OpenCVprovides
a robust foundation for image and video analysis, while
TensorFlow offers a comprehensive platform for machine
learning. NumPy, with its efficient handling of large, multi-
dimensional arrays, proves to be an ideal complement to
these tools.
The project’s target is to develop a system capable of
accurately identifying and categorizing human emotions in
real-time based on facial expressions. Employing machine
learning techniques and leveraging OpenCV’s adept image
processing capabilities, a model will be trained to recognize
diverse emotions from a vast dataset of labelled facial
images. Once trained, the model will be deployed to analyze
facial expressions captured by a webcam, providing real-
time emotion predictions.
This initiative marks a significant step toward empowering
machines with a deeper understanding of human emotions,
paving the way for the realization of truly intelligent and
empathetic artificial intelligence.
3. Project Objectives
The primary objectives of this project on Facial Emotion
Recognition to Emoji Conversion usingOpenCV,TensorFlow,
and NumPy are as follows:
 Develop a Real-Time Facial Emotion Recognition to
Emoji Conversion System:
The main objective is to create a system that can
accurately identify and classify human emotions from
facial expressions in real-time and will display the
emotion. This involves developing an algorithmthatcan
process facial images, extract relevant features, and
classify the emotion expressed.
 Utilize OpenCV for Robust Image and Video
Analysis:
OpenCV will serve as the cornerstone for image and
video analysis, providing a robust framework to handle
real-time video streams from the webcam. The system

will employ OpenCV functionalities for face detection,
tracking, and feature extraction to ensure precise and
efficient processing of facial expressions.
 Employ TensorFlow for Machine Learning:
The project aims to integrate TensorFlow, a
comprehensive machine learning platform, todevelopa
model capableof recognizing diverseemotions.Through
machine learning techniques, the model will be trained
on a substantial dataset of labeled facial images,
enabling it to make accurate predictions in real-time.
 Use NumPy for Efficient Computation:
NumPy will have a crucial role in optimizing
computational efficiency, particularly in handling large,
multi-dimensional arrays and matrices involved in the
machine learning process. This integration will ensure
that the Facial Emotion Recognition system operates
seamlessly, even with the complexities of real-time
video data.
 Train the Model on a Diverse Dataset:
To enhance the system’s accuracy and versatility, the
model will undergo training using a diverse dataset of
facial images labeled with correspondingemotions.This
comprehensive training approach will equip the model
to recognize a wide spectrum of human emotions under
various conditions.
 Train the Model on a Diverse Dataset:
To enhancethesystem’saccuracyandversatility,the
model will undergo training using a diverse dataset
of facial images labeled with corresponding
emotions. This comprehensive training approach
will equip the model torecognizeawidespectrumof
human emotions under various conditions.
 Implement Emotion to Emoji Conversion:
An integral part of this project is to convert the
recognized emotions into corresponding emojis. This
feature aims to provide a more intuitive and engaging
way for users to understand the output of the emotion
recognition system.
 Real-time Emotion Detection using webcam input:
The system will leverage the webcam as the input
device, capturing real-time video data forinstantaneous
emotion recognition. The integration of OpenCV and
TensorFlow will enable the system to process each
frame efficiently, providing a continuous and dynamic
assessment of the user’s emotional state.
 Creating a User-Friendly Interface:
In addition to robust functionality, the project aims to
deliver a user-friendly interface. The system should
provide a seamless and intuitive experience for users
interacting with the real-time Facial Emotion
Recognition, promoting accessibility and ease of use.
 Evaluate System Performance:
Thorough testing and evaluation will be conducted to
assess the performance,accuracy,andresponsivenessof
the Facial Emotion Recognition system. Real-world
scenarios and a range of emotions will be simulated to
ensure the system’s reliability and effectiveness.
 Enhance User Interaction and Experience:
By recognizing the user’s emotions, the system can
potentially enhance user interaction and experience in
various applications. This could include adapting the
system’s responses based on the user’s emotional state
or providing feedback to the user about their emotional
state.
Lastly, implementing the above-mentioned objectives
will help seamlessly integrate
OpenCV, TensorFlow, and NumPy to developa real-time
Facial Emotion Recognition system. Utilizing webcam
input, the system will accurately identify and classify
human emotions, enhancing user experience through a
user-friendly interface. Thorough testing and
documentation will ensure the system’s reliability,
contributing to the advancement ofemotionrecognition
technology.
4. Literature Review
This list is not exhaustive but represents a selection of key
sources that informed our understanding of the topic.
 “Computer Vision and Image Processing” by Victor
Wiley, Thomas Lucas [1] gives an overview of
computer vision and its relation to image
processing and pattern. It also describes the basic
stages of image analysis and the common
frameworks used in computer vision. It highlights
the benefits and limitations of computer vision and
image processing, and suggests some future
directions for research and development. It also
emphasizes the importance ofmachinelearningand
artificial intelligenceforimprovingtheperformance
and accuracy of computer vision systems.
 “A Brief Review of Facial Emotion Recognition
Based on Visual Information” by Myoung Chul Ko
[2] reviews the conventional and deep-learning-

based approaches for FER, which is an important
topic in computer vision andartificial intelligence.It
introduces the terminology, databases, evaluation
metrics, and challenges related to FER, and
compares the advantages and disadvantages of
different methods. It also focuses on the recent
advances in FER using CNN and LSTM models,
which can learn spatial and temporal features from
facial images or videos. The article also suggests
some future directions and applications for FER,
such as using multi-modal data, improving
robustness, and enhancing human-computer
interaction.
 “Face Detection and Recognition Using OpenCV” by
Ramadan TH. Hasan, Amira Bibo [3] is about Face
Detection and RecognitionUsingOpenCV,a freeand
open source library for computer vision
applications. The article introduces the main
OpenCV modules, features, and algorithms that are
used for face detection and recognition, such as
Haar Cascade, LBP, EigenFaces, FisherFaces, LBPH,
YOLO, Faster R-CNN, and SSD. This journal also
explains how to use OpenCV based on Python, a
popular and easy-to-use programminglanguagefor
computer vision projects. It also reviews some
recent literature that use OpenCV for various face
detection and recognition tasks, such as emotion
recognition, attendance system, security system,
mask detection, and personal identifier.Itcompares
the accuracy, performance, and results of different
techniques and classifiers of OpenCV used in the
reviewed studies. The article concludes that
OpenCV can be used in differentfieldsandscenarios
for face detection and recognition, and thatitcan be
combined with other libraries and frameworks to
improve its efficiency and functionality.
 “REALTIME FACIAL EMOTIONS RECOGNITION
SYSTEM (RFERS)” by Mohit Verma, Arpit Sharma,
Mayank Verma, Tushar Rawat,andTeena Verma [4]
aims to design a Python-based system that can
detect and recognize the facial expressions of six
universal emotions plus neutral using a
Convolutional Neural Network (CNN) model and
TensorFlow library. The system has two
components: a pre-train program that uses a
dataset of labeled facial images to train the CNN
model, and a recognizer that uses a camera to
capture real-time video and apply facial detection
and recognition on the extracted facial images. The
journal reports that the system achieved
73.8percent accuracy on fixed user and was able to
train images, detect faces, and recognize facial
expressions in real time. The journal also discusses
the challenges and limitations ofthe system,suchas
the size and quality of the training images, the error
rate of the facial detector, and the processing speed
of the facial expression recognizer. The journal
suggests some futureimprovements,suchasusinga
larger and more diverse dataset, making the CNN
model deeper and wider, adding eye and mouth
detectors, and making the system more compatible
for different platforms and applications.
 “Dynamic Emotion Recognition and Emoji
Generation” by Pradnya Bohr, Ashwini Raut, Sachin
Kate, Anurag Salve, Chandan Prasad [5] proposes a
system that can detect human emotions from facial
expressions and generate corresponding emojis in
real time. The system uses machine learning
techniques to process the input from the camera
and create emojis that can be shared or saved
across different platforms. The paper also reviews
the existing systems and the literature onemojiand
emotion recognition. Emoji are pictographic forms
of facial expressions, objects, and symbols that
originated fromJapanandhaveevolvedintovarious
forms and styles with different colors and
meanings. Emoji became popular worldwide and
are used in various mobile messaging applications
and social media platforms. Emoji can express the
feelings and emotions of the sender and help the
receiver to understand the message better. Emoji
can also demonstrate interpersonal functions such
as personal expressions and mood boost.
 “Emoji: A Deep Learning Approach For Custom
Emoji Creation And Recognition” by Venkata Ravi,
Kiran Kolla [6] the paper describes a software
program that can createandrecognize personalized
emojis based on human facial expressions. This is
the main method used in the paper, which involves
building and training a convolutional neural
network (CNN) to classify facial emotions into five
categories: sad, surprised, scared, happy, and
neutral. The process of creating a graphical user
interface (GUI) that can capture the user’sfacefrom
a webcam and display the corresponding emoji or
avatar based on the CNN’s prediction. The paper
concludes that the project is a novel and fun way of
communicating with customizable emoticons.
 A Systematic Review of Emoji: Current Research
and Future Perspectives” by Qiyu Bai, Qi Dan, Zhe
Mu and Maokun Yang [7] the article delves into the
intricate world of emojis and their impact on
computer-mediated communication. Emojis have
evolved from humble beginnings as smileys and
emoticons into a diverse set of symbols capable of
representing facial expressions, emotions, objects,
concepts, and activities. Their popularity has led to
widespread integration into onlinecommunication,
but their usage is subject to the influence of

individual preferences, cultural nuances, and
platform-specific factors. Emojis function as non-
verbal cues, facilitatingtheconveyanceofmeanings,
fostering interactions, and expressing emotions.
Their significance extends across various fields,
attracting the attention of researchers in computer
science,communication,marketing, psychology,and
education. Research endeavors span a spectrum of
topics, reflecting the multidisciplinary nature of
emoji studies.
5. Methodology
Fig. 1. Flowchart of proposed Methodology
The project consists of seven chapters, and the organization
of the project is as follows:
1. Project Objectives:
The project’s scope is to create a Facial Emotion
Recognition toEmojiConversionsystemthatoperatesin
real-time. It will use OpenCV for image processing,
TensorFlow for machine learning, and NumPy for
efficient computation. The system will capturereal-time
video data through a webcam, identify and classify
human emotions from facial expressions, and enhance
user interaction and experience in various applications.
2. Research:
Gathering information for this project involves a
thorough exploration of the strengths and weaknesses
of OpenCV, TensorFlow, and NumPy, adopting effective
integration strategies, and delving into the realm of
accessible full-stack technologies. Image Processing
Techniques. Various algorithms are utilized to develop
software and hardware that recognizes the human face.
[8] The algorithm will compare the various pictures to
pre-defined or learnt images, as well as real video
images.
3. Data Collection:
This chapter details the process of collecting and
preparing the dataset used for training the emotion
recognition model. It includes the sources of the data,
the types of data collected, and the preprocessing steps
undertaken to prepare the data for model training.
4. Model Development:
It mentions all the intricate details of constructing the
emotion recognition model. It meticulously outlines the
machine learning techniques employed, unravels the
intricate architecture of the model. Also, the advanced
deep expression recognition algorithms in the field, the
performance of thesealgorithmsoncommonexpression
databases is compared, and the strengths and
weaknesses of these algorithms are analyzed in detail.
[9] This comprehensiveexploration providesa profound
understanding of the model’s inner workings, enabling
readers to replicate its success and expand upon its
capabilities [10]
5. Model Training:
The Facial Emotion Recognition (FER) project utilizes a
pretrained model provided by the FER2013 dataset to
streamline the model training process. [11] This robust
TensorFlow-based model hasbeenmeticulouslytrained
on a vast collection of facial images and their
corresponding emotion labels, empowering it to
accurately detect emotions in real-time scenarios.
During the training phase, the model is systematically
fed with a diverse range of facial images,eachassociated
with its corresponding emotion label. This process
enables the model to establish intricate associations
between facial features and the emotions they convey,
laying the foundation for its real-time emotion
recognition capabilities.This approachnot12onlysaves

time and computational resources but also ensures the
model’s robustness and generalizability across various
facial expressions and individuals. [12]
6. Implementation:
The project implements real-time Facial Emotion
Recognition (FER) using Open-CV. [13] By harnessing
the capabilities of Open-CV, the project establishes a
real-time facial emotion recognition system, effectively
capturing live video from a webcam and accurately
identifying the emotions of each detected face. These
emotions are then instantaneously superimposed onto
the video frames, creating an engaging and responsive
interface.
7. System Testing and Evaluation:
The important step in ensuring the accuracy and
reliability of the facial emotion recognition (FER)
system. To evaluate the system’s performance,a dataset
of facial images with corresponding emotion labels can
be employed. The system’s ability to correctly classify
emotions should be assessed using metrics such as
precision, recall [14]. Additionally, real-time
performance should be evaluated by measuring the
system’s latency and frame rate.
8. Final Deployment:
The project’s final deployment marks the end of the
development journey and its transitionintothehandsof
analysts. This critical phase ensures that the project
meets its scalability and performance requirements.
Streamlining the deployment process and integrating
continuous integration facilitates seamless updatesand
adjustments in response to user feedback and evolving
requirements.
6. Conclusion
One of the most innovative aspects of our project is the FER
to Emoji Conversion. This feature takes the Facial Emotion
Recognition (FER) technology to the next level by not only
recognizing emotions but also translating them into
universally understood emojis. This conversion process
involves mapping each recognized emotion to a
corresponding emoji. For instance, happiness could be
represented by a smiling face emoji, sadness by a cryingface
emoji, and so on. This provides a more intuitive and
engaging way for users to understand the output of the FER
system. Moreover, this feature significantly enhances the
user experience, especially in applications like social media,
gaming, and virtual meetings, where emojis are widely used
to express emotions. It’s like having a personal emotion
translator that speaks the universal language of emojis! Our
project has not only achieved its goal of real-time emotion
recognition but has also opened up new possibilities for
emotion-based human-computer interaction. We believe
that the FER to emoji conversion feature of our project will
pave the way for more empathetic and interactive AI
systems in the future.
7. Future Work
Facial emotion recognition (FER) is a rapidly growing field
with a wide range of potential applications. As FER
technology continues to develop, it is important to consider
the ethical implications of this technology and to develop
safeguards to prevent its misuse.
1. Improved accuracy and robustness:Oneofthemost
important areas of future work is to improve the
accuracy and robustness of FER systems. This
includes developing algorithms that can better
recognize emotions in real-world conditions, such
as in low-light or noisy environments.
2. Increased diversity: Another important area of
future work is to increase the diversity of FER
datasets. Current datasets are often biased towards
certain demographics, such as young, white males.
This can lead to FER systems that are less accurate
for people from other demographics.
3. Multimodal FER: Multimodal FER systems combine
information from multiple sources, such as facial
expressions, voice, and body language, to improve
emotion recognition accuracy. Multimodal FER is a
promising area of future research, as it can
potentially overcome some of the limitations of
unimodal FER systems.
4. Ethical considerations: The use of FER technology
raises a number of ethical concerns. For example,
FER could be used to unfairly discriminate against
individuals or to invade their privacy. It is
important to develop ethical guidelines for the use
of FER technology and to ensure that it is used
responsibly.
5. Commercial applications: FER technology has the
potential to be used in a wide range of commercial
applications. For example, it could be used to
improve customer service by detecting customer
emotions and tailoring responses accordingly. FER
could also be used to develop personalizedlearning
systems that adapt to the student’s emotional state.
6. Mental health applications: FER technology could
also be used to develop mental health applications.
For example, it could be used to detect signs of
depression or anxiety and to provide early
intervention. FER could also be used to develop
personalized mental health treatments.

7. Educational applications: FER technology could be
used to develop educational applicationsthatadapt
to the student’s emotional state. For example, an
FER system could detect when a student is
frustrated and provide additional support.
8. Human-computerinteraction:FER technologycould
be used to improve human-computer interaction.
For example, an FER system could detect when a
user is confused and provide additional help.
9. Safety and security: FER technology could be used
to improve safety and security.Forexample,itcould
be used to detect when a person is angry or upset
and to take steps to prevent violence.
These are just a few of the exciting areas of future work in
FER. As FER technology continues to develop, we can expect
to see even more innovative andimpactapplications emerge.
REFERENCES
[1] Victor Wiley and Thomas Lucas. Computer vision and
image processing: a paper review. International Journal
of Artificial Intelligence Research, 2(1):29–36, 2018.
[2] Byoung Ko. A brief review of facial emotion recognition
based on visual information. Sensors,18(2):401, January
2018.
[3] Ramadan TH Hasan and Amira Bibo Sallow. Face
detection and recognition using opencv. Journal of Soft
Computing and Data Mining, 2(2):86–97, 2021.
[4] Mohit Verma, Arpit Sharma, Mayank Verma, Tushar
Rawat, and Ms Teena Verma. Realtime facial emotions
recognition system (rfers).
[5] Pradnya Bhor, Ashwini Raut, Sachin Kate, Anurag Salve,
and Chandan Prasad. Dynamic emotion recognition and
emoji generation. International Research Journal of
Engineering and Technology, 6(9):1891–1894, 2019.
[6] Venkata Ravi Kiran Kolla. Emojify: A deep learning
approach for custom emoji creation and recognition.
International Journal of Creative Research Thoughts,
2021.
[7] Qiyu Bai, Qi Dan, Zhe Mu, and Maokun Yang. A systematic
review of emoji: Current research and future
perspectives. Frontiers in Psychology, 10:2221, 2019.
[8] Heera Mohan KV, Pranav PV, Swetha Pai, et al. A
synopsis on intelligent face discovery frameworks.
Journal of Applied Information Science, 10(1), 2022.
[9] Heechul Jung, Sihaeng Lee, Sunjeong Park, Byungju Kim,
Junmo Kim, Injae Lee, and Chunghyun Ahn.
Development of deep learning-based facial expression
recognition system. In 2015 21st Korea-Japan Joint
Workshop on Frontiers of ComputerVision(FCV),pages
1–4. IEEE, 2015.
[10] D Mary Prasanna and Ch Ganapathy Reddy.
Development of real time face recogni tion system
using opencv. Development, 4(12):791, 2017.
[11] Panagiotis Giannopoulos, Isidoros Perikos, and Ioannis
Hatzilygeroudis. Deep learning approaches for facial
emotion recognition: A case study on fer-2013.
Advances in 18 Hybridization of Intelligent Methods:
Models, Systems and Applications, pages 1–16, 2018.
[12] VS Amal, Sanjay Suresh, and G Deepa. Real-time
emotion recognition from facial expressions using
convolutional neural network with fer2013 dataset. In
Ubiquitous Intelligent Systems: Proceedings of ICUIS
2021, pages 541–551. Springer, 2022.
[13] Lyron Foster. Recognizing human emotions with ai.
(tensorflow, keras, opencv), Mar 2023.
[14] Daniel E Ho, Emily Black, Maneesh Agrawala,andLiFei-
Fei. Evaluating facial recognition technology: a protocol
for performance assessment in new domains. Denv. L.
Rev., 98:753, 2020.

Synopsis of Facial Emotion Recognition to Emoji Conversion

Recommended

Recommended

More Related Content

Similar to Synopsis of Facial Emotion Recognition to Emoji Conversion

Similar to Synopsis of Facial Emotion Recognition to Emoji Conversion (20)

More from IRJET Journal

More from IRJET Journal (20)

Recently uploaded

Recently uploaded (20)

Synopsis of Facial Emotion Recognition to Emoji Conversion