This document discusses integrating facial expression analysis with survey tools to better understand emotional responses to advertising. It summarizes: 1) Facial coding technology from Affectiva enables automatically coding facial reactions to marketing stimuli for emotional and cognitive states like enjoyment, confusion, and attention without relying on verbal reports. 2) The system works by identifying panelists' facial features, recognizing expressions, aggregating and analyzing emotions experienced as they view ads as part of an online interview. 3) Integrating these "funky data" insights with standard survey responses provides a more holistic understanding of what drives reactions, but challenges remain to ensure the approach is responsible, reliable and can be repeated by others.

1. Face To Face:
Integrating facial expressions
and survey tools
SARAH WALKER
MILLWARD BROWN
1

2. MILLWARD BROWN NEUROSCIENCE PRACTICE
8-strong team across the globe, with network of local experts
Over 500 projects to date, using a wide variety of methods, in a range of
countries
Continual program of development
and assessment of methods
GOAL:
TO INTEGRATE NEUROSCIENCE AND
PSYCHOLOGY-BASED METHODS INTO THE
RESEARCH WE DO FOR CLIENTS

3. ‘NEUROMARKETING’ IS A BROAD FIELD
Implicit
Brain Brainwave Autonomic Eye
Facial coding association
Scanning measurement arousal Tracking
measurement
Neuroscience Physiology Cognitive
Psychology
ALL INDIRECT MEASURES OF RESPONSES
TO MARKETING AND BRANDS

4. WHAT IS FACIAL CODING?
Based on the idea of many facial expressions are instinctively shown in
people‟s faces
Darwin: Expressions are universal and
shared with animals
Ekman (1972) confirmed universality of 6
core expressions
Ekman & Friesen (1978) developed
Facial Action Coding System to
consistently code facial expressions &
movements

5. FACIAL CODING HELPS US UNDERSTAND EMOTIONAL RESPONSES
TO ADVERTISING IN GREATER DETAIL
Survey metrics are extremely good at capturing
overall response to a piece of advertising
Adding facial coding allows us to investigate
what is driving these emotional responses
on a moment-by-moment basis

6. AUTOMATED FACIAL CODING
Technology from Affectiva enables us to:
• Record respondents‟ facial reactions to
marketing stimulus
• Automatically code them for emotional
and cognitive states
• Diagnose emotional reaction, moment
by moment, without relying on verbal
report
MAKES NON-VERBAL EMOTIONAL DIAGNOSIS AFFORDABLE AND
SCALABLE FOR THE FIRST TIME

7. HOW THE SYSTEM WORKS
Panelist turns on Facial features Facial expression Expression Emotions
webcam identified recognition aggregation and experienced
analysis
THE SYSTEM WORKS SIMPLY AND QUICKLY AS PART OF A
STANDARD COMPUTER INTERVIEW

8. THE SYSTEM CODES FOR AN ARRAY OF EXPRESSIONS
DIMENSIONS OF
EMOTIONS
RESPONSE
Enjoyment Expressiveness
(Engagement)
Confusion / Dislike
Valence
(Net Positivity)
Surprise
Attention
Disgust (Concentration on task)

9. THE REAL POWER COMES FROM THE INTEGRATION WITH SURVEY
RESPONSES
Results from facial coding
are interpreted alongside
Link™ findings into an
interactive dashboard

10. FACIAL CODING HAS ADDED INSIGHT IN MANY DIFFERENT
SITUATIONS
Highlighting the power of key
Optimising Ad length
moments
Understanding wear-in and comprehension
First
Second
view
view

11. CREATING A SCALABLE TOOL LETS US BUILD UNDERSTANDING
ACROSS A LARGE NUMBER OF ADS
ACTIVELY ENGAGING ads generate
stronger expressions
POSITIVE EXPRESSIONS relate to
increased enjoyment and likelihood of
short term sales
SURPRISE & DISGUST can indicate
disbelief & distrust of ideas

12. TESTING MORE ADS HELPS US EVOLVE OUR
UNDERSTANDING AND INTERPRETATION
Variation can be more informative than magnitude
Context is key to interpreting what is driving expressions

13. HOWEVER...
Integrating „funky data‟ and survey responses has not been without its
challenges!
For the approach to succeed it needed to be:
RESPONSIBLE: Making sure what we are measuring is
meaningful
RELIABLE: Making sure it works in the real world, with real
respondents
REPEATABLE: Making sure our teams are equipped to sell and
interpret projects on their own

14. “This is
the future…
right?”