Presented by Pankaj Jha, Director of Global Innovation, Millward Brown at Market Research in the Mobile World Asia-Pacific 30-31 January 2013, Kuala Lumpur, Malaysia This event is proudly organised by Merlien Institute Check out our upcoming events by visiting http://www.mrmw.net

1. TM
Proudly supported by Kantar, the Leader in Mobile Marketing Research
January 30-31, 2013
Kuala Lumpur, Malaysia
Organized by
Asia-Pacific Edition 2013 WWW.MRMW.NET

2. Thank you to our sponsors!
Title Sponsor Platinum Sponsor Gold Sponsors
Silver Sponsor Exhibitor Networking Evening Sponsor Networking Break Sponsor
Association Partners
Media Partners

3. Using facial coding technology to
capture emotions on mobile
Pankaj Jha
1

4. Agenda
Measuring emotions thru facial expressions
• Using mobile in field
• Getting desired outcomes
Adding insights to Ad testing
Possibilities
2

5. What did we do
• Pilot study in India
• 300 participants shown 6 beverage ads each in two cities: Hyderabad and Delhi
• Interviewers showed ads to respondents on a phone
• Recorded the face video of respondents synched with ad playing
• Analysis of face expressions to provide moment by moment emotional feedback

6. Objectives
• Feasibility of using mobile to capture emotions
• Feasibility of obtaining sensible data using this technology
• Feasibility of providing additional insights on advertising

7. Feasibility of using mobile to capture emotions
Lighting and
overexposure
Full face in frame
not visible
Non-frontal,
multiple faces
Less than 50% of the videos were usable initially.
Iterative training of interviewers to ensure that we got better quality videos.

8. Feasibility of using mobile to capture emotions
Iterative training of interviewers ensured that we finally got good quality videos.
1. Simple rules: Place the device at arm’s length from respondent
2. Higher resolution/wider angle cameras for better results.
3. Define a minimal acceptable threshold: 40% of frames must be discernible.

9. Feasibility of obtaining sensible data
• Validated that the participants where “emoting” while watching the ads on a
mobile device, and we are able to accurately capture these facial expressions.
• Range of facial expressions observed in response to the ads - enjoyment to
surprise & confusion

10. Providing additional insights on advertising
We compared the results from the facial expressions with
our advertising testing outcomes for ads.
We were able to find out insights which helped sharpen
our understanding of how the ads were working.

11. Sprite – Delhi
Facial expressions confirmed the key aspects liked in the ad were causing most smiles, the
extent of emotions displayed can help prioritise between different parts.
Smile Confusion
The scene in the middle did not evoke a strong response in Consumers fail to understand the role of
facial expressions as the last scene of ‘boy being successful professor/theme of the Ad in Link – this is a point of
in selling his excuse for staring at the other girl’. confusion captured from facial expressions

12. Sprite – Hyderabad
Valence Confusion
Again, in Hyderabad we could see that it is the Here too, the appearance of the Professor of
scene where the boy is successful in selling his Freshology is clearly creating confusion for
excuse for ogling at the other girl in front of his the consumers
girlfriend which gets the highest emotional payoff

13. Coke - Delhi
Smile Confusion
In Delhi, respondents smile the most when the teacher is Sequences in quick succession create confusion
shown holding the Coke bottle that Imran Khan meant to pass
on to the girl in the classroom
The Sardar & Parrot scenes also evokes smiles

14. Coke - Delhi
Facial expressions helped pinpoint the exact moments which created positive payoff – both
through evoking smiles and resolving confusion
Peak valence for Delhi
respondents watching the
Valence
Coke ad occurs when the
parrot also shakes in a
manner similar to the rest
of the characters in the ad,
this not only evokes smiles,
but also resolves confusion
One of the reasons why we
felt the brand had a good
recall in Link was due to
the product being right
there when the most
emotionally positive
moment in the ad occurs

15. Coke - Hyderabad
We can see the sharpness with which the key moments which cause a response come
through compared to the more aggregate responses in Link
Valence Confusion
In Hyderabad, valence peaks at the The parrot scene Multiple sequences shown in quick succession
culmination of the teacher sequence continues to evoke a create confusion among the consumers
sharp positive response
in Hyderabad as well

16. 7 Up - Delhi
Confusion
The role of the brand in the second half of the
ad is unclear leading to confusion

17. 7 Up
Smile Smile
Hyderabad Delhi
The emotions in the two markets peak at different points of the narrative

18. RECAP: Using Emotions to Add Value to Advertising testing
Lumi Technology: Affectiva Facial Coding Analysis Facial coding dashboards
Showing Ads & Capture
Face Video on Mobile
Facial Coding data Analysed with Link to provide additional insights

19. POSSIBILITIES
• The technology enables us to use it with advertising
testing at various stages:
• Capturing consumers’ emotions when they are watching an ad
remotely on their mobile devices
• Capture emotions in tracking studies & evaluate response over time

20. Thank You
18

21. Thank you to our sponsors!
Title Sponsor Platinum Sponsor Gold Sponsors
Silver Sponsor Exhibitor Networking Evening Sponsor Networking Break Sponsor
Association Partners
Media Partners

22. TM
Proudly supported by Kantar, the Leader in Mobile Marketing Research
January 30-31, 2013
Kuala Lumpur, Malaysia
Organized by
Asia-Pacific Edition 2013 WWW.MRMW.NET