Emotion Recognition

11,003 views

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
11,003
On SlideShare
0
From Embeds
0
Number of Embeds
32
Actions
Shares
0
Downloads
357
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Emotion Recognition

  1. 1. Multimodal emotion recognition and expressivity
  2. 2. Reference <ul><li>S. Kollias, K. Karpouzis, “Multimodal emotion recognition and expressivity,” Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on , 6-8 July 2005, p.p. 779- 783 </li></ul>
  3. 3. Introduction <ul><li>People express their emotions through multiple modalities </li></ul><ul><ul><li>Humans’ speech </li></ul></ul><ul><ul><li>Facial expressions </li></ul></ul><ul><ul><li>Body pose </li></ul></ul><ul><li>Emotional feature and signs </li></ul>
  4. 4. Recognition of the user’s emotional state <ul><li>Emotion analysis and recognition </li></ul><ul><ul><li>Audio </li></ul></ul><ul><ul><li>Visual </li></ul></ul><ul><ul><li>Physiological signal </li></ul></ul><ul><li>Emotional psychological background </li></ul><ul><li>Human computer interaction (HCI) </li></ul>
  5. 5. Emotional speech analysis <ul><li>Speech is a major channel for communicating emotion </li></ul><ul><li>Speech signal conveys </li></ul><ul><ul><li>Textual, lexical, emotional and gestural information </li></ul></ul><ul><li>The set of features in the speech signal </li></ul><ul><li>Classification algorithm </li></ul>
  6. 6. Emotion recognition system
  7. 7. Paralinguistic speech analysis <ul><li>Prosody is composed of </li></ul><ul><ul><li>Intonation </li></ul></ul><ul><ul><li>Duration </li></ul></ul><ul><ul><li>Intensity </li></ul></ul><ul><ul><li>Speech quality </li></ul></ul><ul><li>Voice quality is influenced by physiological factors </li></ul>
  8. 8. Feature extraction <ul><li>Extracting information from </li></ul><ul><ul><li>Pitch contour, range, variance, mean, jitter, intensity, shimmer </li></ul></ul><ul><ul><li>Voice quality </li></ul></ul><ul><ul><li>Duration : pauses, speaking rate </li></ul></ul><ul><ul><li>Background information on the speaker </li></ul></ul>
  9. 9. Emotional facial analysis <ul><li>Facial action coding system (FACS) </li></ul><ul><li>Facial definition parameter (FDP) </li></ul><ul><li>Facial animation parameter (FAP) </li></ul><ul><li>MPEG-4 standard </li></ul>
  10. 10. Facial animation
  11. 11. Emotional Gesture Analysis <ul><li>Hand tracking systems </li></ul><ul><li>Tracking the centroid of skin masks </li></ul><ul><li>Estimates of user’s movements </li></ul>
  12. 12. Gesture recognition
  13. 13. Targeting Emotion Recognition <ul><li>Facial animation parameter from the user’s face </li></ul><ul><li>Future merging of different emotional representations </li></ul>
  14. 14. Targeting Expressivity <ul><li>Facial Expressivity </li></ul><ul><li>Time-varying facial movements </li></ul><ul><ul><li>Quantity and quality of movement </li></ul></ul><ul><ul><li>Interaction </li></ul></ul><ul><ul><li>Transition </li></ul></ul><ul><li>Gesture Expressivity </li></ul><ul><ul><li>Speed, acceleration, direction variation </li></ul></ul>
  15. 15. Physiologocal signal analysis <ul><li>Visceral differences between emotional states </li></ul><ul><ul><li>Heart rate </li></ul></ul><ul><ul><li>Skin conductance level </li></ul></ul><ul><ul><li>Finger temperature </li></ul></ul><ul><ul><li>Muscle activity </li></ul></ul>
  16. 16. Measurement with physiological information <ul><li>Biosensor </li></ul><ul><li>The value of skin conductivity </li></ul><ul><li>Electromyography (EMG) sensors for muscle-activity </li></ul>
  17. 17. Multimodal emotion recognition <ul><li>Define the processes and functions </li></ul><ul><ul><li>Visual, auditory and physiological modalities </li></ul></ul><ul><li>Identify different emotions in the recognition processes </li></ul><ul><li>Synchronization and temporal sequence in different modalities </li></ul>
  18. 18. Conclusions <ul><li>Multimodal emotion recognition and expressivity analysis </li></ul><ul><li>Human computer interaction (HCI) </li></ul><ul><li>Pattern recognition in combination with different techniques </li></ul>

×