Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Automated Prediction of
Preferences Using Facial
Expressions
David Masip, Michael S. North, Alexander Todorov, Daniel Oshe...
Introduction: Reading minds
Detailed problem: subtle facial expressions
• State-of-the-art
• Example
Methods
• Target phas...
1. Reading minds
Can we read the mind? The answer is NO. Not completely at least…
Our face reflects somehow our feelings, ...
4
Facial expression is a powerful non verbal communication cue beyond natural language.
“The expression of the Emotions in...
5
FACS 46 atomic movements of facial muscles.
Action Units = Building blocks of the emotions
Facial emotions = combination...
6
Which cartoon this student enjoyed the most?
Automated Prediction of Preferences Using Facial Expressions
Methods: target phase
Target phase: Students evaluate two visual stimuli. [People,
Cartoons, Animals, Paintings]
Visualiza...
Methods: perceiver phase
Students infer the preferences of other students from covertly
recorded videos of target’s faces....
Methods: Algorithm
Facial landmarks detection per frame
Dispersion descriptor based on temporal landmarks displacements
Mo...
10
Results
Hypothesis one: Maximum – minimum displacement
JESP is human accuracy
Automated Prediction of Preferences Using...
11
Results
Hypothesis two: apply SVM to the 66 differences between the
maximum and minimum centered displacement.
SVM: Sta...
Conclusions
Complex subtle emotion reading technology
Can we read minds from facial expression?
Yes, if we lower the expec...
Automated Prediction of Preferences Using Facial Expressions
Upcoming SlideShare
Loading in …5
×

Automated Prediction of Preferences Using Facial Expressions

676 views

Published on

In this presentation the authors (David Masip, Michael S. North, Alexander Todorov, Daniel N. Osherson) introduce a computer vision problem from social cognition, namely, the automated detection of attitudes from a person’s spontaneous facial expressions. To illustrate the challenges, they introduce two simple algorithms designed to predict observers’ preferences between images (eg, of celebrities) based on covert videos of the observers’ faces. The two algorithms are almost as accurate as human judges performing the same task but nonetheless far from perfect.

Published in: Devices & Hardware
  • Be the first to comment

  • Be the first to like this

Automated Prediction of Preferences Using Facial Expressions

  1. 1. Automated Prediction of Preferences Using Facial Expressions David Masip, Michael S. North, Alexander Todorov, Daniel Osherson, PLOSOne 1 Automated Prediction of Preferences Using Facial Expressions
  2. 2. Introduction: Reading minds Detailed problem: subtle facial expressions • State-of-the-art • Example Methods • Target phase • Perceiver phase • Algorithm Results • Hypothesis one: simple classifier • Hypothesis two: machine learning classifier Conclusions Índex 2 Automated Prediction of Preferences Using Facial Expressions
  3. 3. 1. Reading minds Can we read the mind? The answer is NO. Not completely at least… Our face reflects somehow our feelings, emotions, … We propose to read/predict what people thinks about two stimuli in a very particular setting: Preferences We explore whether computers can read and infer what humans believe in a binary choosing scenario, and we use only subtle facial expressions and a laptop. 3 Automated Prediction of Preferences Using Facial Expressions
  4. 4. 4 Facial expression is a powerful non verbal communication cue beyond natural language. “The expression of the Emotions in Man and Animals”. Charles Darwin Paul Ekman. Universality of Facial expression. 6 Universal human emotions. “We need to give computers the capacity to read our feelings and react, in ways that have come to seem startlingly human.” Rana El Kaliouby 1.1. Detailed problem Automated Prediction of Preferences Using Facial Expressions
  5. 5. 5 FACS 46 atomic movements of facial muscles. Action Units = Building blocks of the emotions Facial emotions = combinations of AUs 1.1. Detailed problem Applications: We transmit more data with our expressions than with what we say. Facial expressions predict: the result of a negotiation, the winner of a congress elections, the election of a partner,… GOAL: apply Computer Vision algorithms to model user preferences in non-posed scenarios. Automated Prediction of Preferences Using Facial Expressions
  6. 6. 6 Which cartoon this student enjoyed the most? Automated Prediction of Preferences Using Facial Expressions
  7. 7. Methods: target phase Target phase: Students evaluate two visual stimuli. [People, Cartoons, Animals, Paintings] Visualization for 3 seconds in a screen (E-Prime SW) 7 Automated Prediction of Preferences Using Facial Expressions
  8. 8. Methods: perceiver phase Students infer the preferences of other students from covertly recorded videos of target’s faces. They only see the target’s face, and guess the preference 8 Automated Prediction of Preferences Using Facial Expressions
  9. 9. Methods: Algorithm Facial landmarks detection per frame Dispersion descriptor based on temporal landmarks displacements Model: 9 Face Detection Mesh Fitting Dynamic Landmark Descriptor Classifier Automated Prediction of Preferences Using Facial Expressions
  10. 10. 10 Results Hypothesis one: Maximum – minimum displacement JESP is human accuracy Automated Prediction of Preferences Using Facial Expressions
  11. 11. 11 Results Hypothesis two: apply SVM to the 66 differences between the maximum and minimum centered displacement. SVM: Standard Machine Learning classifier on a 132 dimensional space Automated Prediction of Preferences Using Facial Expressions
  12. 12. Conclusions Complex subtle emotion reading technology Can we read minds from facial expression? Yes, if we lower the expectations [Binary preference selection problem] Applications: - Depression monitorization - Pain detection - Deception detection - Dynamically price advertising depending on how people responded to it, - Autism - Sony, Yahoo!, Motorola, Verizon, … “Research in this area provides a rare point of convergence between Computer Science and Social Psychology” 12 Automated Prediction of Preferences Using Facial Expressions

×