The article discusses facial expression recognition (FER) using deep learning, highlighting the limitations of conventional approaches that require extensive manual feature engineering. It introduces three deep learning methods for FER: Convolutional Neural Networks (CNN), Long Short Term Memory (LSTM), and Generative Adversarial Networks (GAN), which are more data-driven and robust. The performance of these models is analyzed across various datasets, indicating ongoing challenges and opportunities in the field.

1. Facial Expression Recognition
(FER) using Deep Learning
Emmeline Tsen
Medium article: https://medium.com/@emmelinetsen/facial-expression-recognition-using-deep-learning-
3ec1d7426604?sk=72c845275071df13dccc97bd07a73830

2. Conventional FER Approaches
● Lots of manual feature engineering
● Images need to be preprocessed
● Need to select appropriate feature extraction and classification method for
output
○ Both separated into two components
Conventional FER Approach Workflow

3. Deep Learning Approaches
● More dependent on data and hardware
● More robust to its environments
● Three proposed ways for FER
○ Convolutional Neural Network (CNN)
○ Long Short Term Memory (LSTM)
○ Generative Adversarial Network (GAN)

4. Convolutional Neural Network (CNN)

5. Long Short Term Memory (LSTM)
● Recurrent neural network (RNN)
● Extracts temporal features within consecutive frames
● Recommended for video sequences

6. Generative Adversarial Network (GAN)
● Consists of a generative network and discriminative network
● Generates images
● Synthesizes facial images to make them look realistic
● Able to increase FER training dataset
○ Automatically generating images

7. Metrics
● Numerous testing and training datasets for FER
● Best represent advantages of using deep learning based FER models

8. Performance of FER using CNN & LSTM on MMI and CK+ Datasets
Performance of FER using CNN & GAN on Oulu-CASIA, BU-3DFE, and
Multi-PIE Datasets

9. Final Thoughts
● FER has been receiving a lot of attention
● Different approaches of FER
● There are still challenges & lots of opportunities to evaluate different
algorithms

10. References
● Huang, Y., Chen, F., Lv, S., & Wang, X. (2019, September 20). Facial Expression Recognition: A Survey. Retrieved May 8, 2020,
from https://www.mdpi.com/2073-8994/11/10/1189
● Divya, M., Reddy, R. O., & Raghavendra, C. (2019). Effective Facial Emotion Recognition using Convolutional Neural Network
Algorithm. International Journal of Recent Technology and Engineering Regular Issue, 8(4), 4351–4354.
doi:10.35940/ijrte.d8275.118419
● Amidi, A., & Amidi, S. (n.d.). Convolutional Neural Networks cheatsheet Star. Retrieved May 8, 2020, from
https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks