Tammy Everts (@tameverts)
eMetrics Chicago – June 16-19, 2014
Emotional Engagement
and Brand Perception
How we used EEG technology to measure
the neurological impact of slow web
pages on mobile device users
Slide 2
Have you ever done any of the following
when a site takes too long to load?
a. Cursed at your phone
b. Screamed at your phone
c. Thrown your phone
Slide 3
Tealeaf/Harris Interactive, 2011@tameverts
Slide 4
1 Why do we care so much about mobile web
performance?
2 Why neuroscientific mobile testing?
3 What is emotional engagement research?
4 How did we perform our study?
5 What kinds of insights did we gain?
@tameverts
It’s a mobile-first world.
Slide 6
55% of all time spent
on retail sites takes
place on a mobile device.
comScore, October 2013
@tameverts
Slide 7
@tameverts
Four all-too-common
mobile assumptions
Slide 9
Assumption #1
My site isn’t slow on mobile.
Radware, 2013 State of the Union: Mobile Ecommerce Performance
Slide 10
@tameverts
Radware, 2013 State of the Union: Mobile Ecommerce Performance
Slide 11
@tameverts
Assumption #2
Mobile users expect pages to be slow.
Slide 12
Keynote, 2012 Mobile User Survey
Slide 13
@tameverts
Slide 14
Assumption #3
Mobile users want to browse, not buy.
Slide 15
By 2017,
retail mcommerce
is expected to hit
$113 billion –
26% of total
ecommerce sales.
eMarketer, September 2013
Mobile shopping cart
abandonment rate
is 39% greater
than desktop rate.
2013 Google I/O
Slide 16
@tameverts
Slide 17
Assumption #4
Users will stick around, even if pages
are slow, if they really want to buy.
Skava/Harris Interactive, 2013
Slide 18
@tameverts
Slide 19
Case study: The impact of HTML delay on mobile business metrics@tameverts
Slide 20
1 Mobile usage (time on site) for retail has overtaken
desktop.
2 People expect sites to be at least as fast on their
mobile devices as on their PCs.
3 Most mobile sites are far too slow.
4 This slowness has a significant impact on
abandonment rate and other business metrics.
@tameverts
Why neuroscientific mobile testing?
Slide 21
Slide 22
• 2010 EEG study of desktop
users
• Throttled connection from 5MB
to 2MB
• Found that participants had to
concentrate up to 50% harder
• Afterward, participants reported
negative brand associations
@tameverts
Slide 23
Slide 24
What is emotional
engagement research?
“95% of the consumer’s
decisions are made at the
subconscious level.”
Dr. Gerald Zaltman, Harvard University
Executive Committee of Harvard University’s
Mind, Brain and Behavior Interfaculty Initiative
Slide 26
@tameverts
Slide 27
Slide 28
Patients with damage to
emotional parts of the brain
cannot make decisions,
despite having no change in IQ.
Antonio Damasio, Descartes’ Error
Slide 29
@tameverts
The problem with surveys…
Traditional research relies on eliciting post-cognitive
responses.
But thinking and talking about emotions changes and
distorts them.
Slide 30
@tameverts
Five benefits of neuroscientific testing
1 Evaluates think/feel (not say)
2 Quantified data
3 Moment-by-moment interaction
4 Cause-and-effect triggers
5 Fresh, deeper insights
Slide 31
@tameverts
Simplified cognitive timeline
Slide 32
@tameverts
EEG Emotional Engagement Study:
How We Did It
Our research team
• Seren – leaders in customer experience & service design
• NeuroStrata – expert consultants in blending neuromarketing
applications
• Neurosense – global leader in implicit methodologies
Slide 34
@tameverts
The brands we tested
Slide 35
Our test participants
• 24 participants (12 male and 12 female)
• Pre-screened to ensure normal cognitive
functioning
• Experienced mobile device users
• Did not know they were part of a performance
study
Slide 36
Jakob Nielsen, Why You Only Need to Test with 5 Users, 2000
Slide 37
@tameverts
Methodology
• Standardized set of shopping tasks (browsing and
checkout)
• Testers served sites over one of two speeds:
– normal Wifi
– artificial 500ms delay
• Using EEG headset and eyetracker, measured moment-
by-moment responses
Slide 38
@tameverts
Slide 39
Slide 40
Why test a 500ms delay?
Case study: The impact of HTML delay on mobile business metrics
@tameverts
Slide 41
We focused on the metrics most affected by the 500ms delay:
Emotional engagement
Frustration
Slide 42
Normal speed
Slide 43
2.66s 2.92s 2.83s 4.24s
Frustration levels across sites (normal speed)
Slide 44
@tameverts
Engagement levels across sites (normal speed)
Slide 45
@tameverts
500ms delay: Peak frustration results
Slide 46
@tameverts
500ms delay: Average engagement results
Slide 47
@tameverts
EEG test summary
• A mere 500ms delay results in significant increase in
frustration levels.
• Faster pages result in higher levels of engagement.
• Different sites trigger emotional shifts at different phases
of the experience (browsing vs. checkout).
• Important: These tests happened under ideal browsing
conditions.
Slide 48
@tameverts
Slide 49
Impact of site speed on post-test brand association
Slide 50
@tameverts
If pages aren’t fast, everything suffers.
Content
“boring”
Visual design
“tacky” and “confusing”
Navigation
“frustrating” and “hard-to-navigate”
Slide 51
@tameverts
Slide 52
@tameverts
Takeaways
1 People feel “web stress” even when shopping under ideal
conditions.
2 Slower web performance has a clear and measurable
impact on people at a neurological level.
3 Slow sites can seriously undermine overall brand health.
4 The nature and scale of impact varies, depending on a
number of factors (e.g. inherent strength/weakness of
brand).
5 This presents great opportunities to strengthen overall
brand by investing in performance optimization.
Slide 54
@tameverts
Slide 55
http://www.radware.com/mobile-eeg2013/
Slide 56
@tameverts
Sources
Web Stress: A Wake-Up Call for European Business (Foviance, 2010)
http://www.ca.com/us/~/media/files/supportingpieces/final_webstress_survey_report_229296.aspx
2013 Social & Mobile Commerce Consumer Report (Shop.org / comScore)
http://shop.org/research/original/2013-social-mobile-commerce-consumer-report
2012 Mobile User Survey (Keynote)
http://www.keynote.com/docs/reports/Keynote-2012-Mobile-User-Survey.pdf
2013 State of the Union: Mobile Ecommerce Performance (Radware)
http://www.radware.com/mobile-sotu2013/
The Danger of a Poor Mobile Shopping Experience [INFOGRAPHIC]
http://www.getelastic.com/the-danger-of-a-poor-mobile-shopping-experience-infographic/
Case study: The impact of HTML delay on mobile business metrics (Web Performance Today, November 2011)
http://www.webperformancetoday.com/2011/11/23/case-study-slow-page-load-mobile-business-metrics/
Slide 57
webperformancetoday.com
twitter.com/tameverts
linkedin.com/in/tammyeverts
plus.google.com/+TammyEverts/
Slide 58
Questions?

Emotional Engagement and Brand Perception

  • 1.
    Tammy Everts (@tameverts) eMetricsChicago – June 16-19, 2014 Emotional Engagement and Brand Perception How we used EEG technology to measure the neurological impact of slow web pages on mobile device users
  • 2.
    Slide 2 Have youever done any of the following when a site takes too long to load? a. Cursed at your phone b. Screamed at your phone c. Thrown your phone
  • 3.
  • 4.
    Slide 4 1 Whydo we care so much about mobile web performance? 2 Why neuroscientific mobile testing? 3 What is emotional engagement research? 4 How did we perform our study? 5 What kinds of insights did we gain? @tameverts
  • 5.
  • 6.
    Slide 6 55% ofall time spent on retail sites takes place on a mobile device. comScore, October 2013 @tameverts
  • 7.
  • 8.
  • 9.
    Slide 9 Assumption #1 Mysite isn’t slow on mobile.
  • 10.
    Radware, 2013 Stateof the Union: Mobile Ecommerce Performance Slide 10 @tameverts
  • 11.
    Radware, 2013 Stateof the Union: Mobile Ecommerce Performance Slide 11 @tameverts
  • 12.
    Assumption #2 Mobile usersexpect pages to be slow. Slide 12
  • 13.
    Keynote, 2012 MobileUser Survey Slide 13 @tameverts
  • 14.
    Slide 14 Assumption #3 Mobileusers want to browse, not buy.
  • 15.
    Slide 15 By 2017, retailmcommerce is expected to hit $113 billion – 26% of total ecommerce sales. eMarketer, September 2013
  • 16.
    Mobile shopping cart abandonmentrate is 39% greater than desktop rate. 2013 Google I/O Slide 16 @tameverts
  • 17.
    Slide 17 Assumption #4 Userswill stick around, even if pages are slow, if they really want to buy.
  • 18.
  • 19.
    Slide 19 Case study:The impact of HTML delay on mobile business metrics@tameverts
  • 20.
    Slide 20 1 Mobileusage (time on site) for retail has overtaken desktop. 2 People expect sites to be at least as fast on their mobile devices as on their PCs. 3 Most mobile sites are far too slow. 4 This slowness has a significant impact on abandonment rate and other business metrics. @tameverts
  • 21.
    Why neuroscientific mobiletesting? Slide 21
  • 22.
    Slide 22 • 2010EEG study of desktop users • Throttled connection from 5MB to 2MB • Found that participants had to concentrate up to 50% harder • Afterward, participants reported negative brand associations @tameverts
  • 23.
  • 24.
  • 25.
  • 26.
    “95% of theconsumer’s decisions are made at the subconscious level.” Dr. Gerald Zaltman, Harvard University
Executive Committee of Harvard University’s Mind, Brain and Behavior Interfaculty Initiative Slide 26 @tameverts
  • 27.
  • 28.
  • 29.
    Patients with damageto emotional parts of the brain cannot make decisions, despite having no change in IQ. Antonio Damasio, Descartes’ Error Slide 29 @tameverts
  • 30.
    The problem withsurveys… Traditional research relies on eliciting post-cognitive responses. But thinking and talking about emotions changes and distorts them. Slide 30 @tameverts
  • 31.
    Five benefits ofneuroscientific testing 1 Evaluates think/feel (not say) 2 Quantified data 3 Moment-by-moment interaction 4 Cause-and-effect triggers 5 Fresh, deeper insights Slide 31 @tameverts
  • 32.
  • 33.
    EEG Emotional EngagementStudy: How We Did It
  • 34.
    Our research team •Seren – leaders in customer experience & service design • NeuroStrata – expert consultants in blending neuromarketing applications • Neurosense – global leader in implicit methodologies Slide 34 @tameverts
  • 35.
    The brands wetested Slide 35
  • 36.
    Our test participants •24 participants (12 male and 12 female) • Pre-screened to ensure normal cognitive functioning • Experienced mobile device users • Did not know they were part of a performance study Slide 36
  • 37.
    Jakob Nielsen, WhyYou Only Need to Test with 5 Users, 2000 Slide 37 @tameverts
  • 38.
    Methodology • Standardized setof shopping tasks (browsing and checkout) • Testers served sites over one of two speeds: – normal Wifi – artificial 500ms delay • Using EEG headset and eyetracker, measured moment- by-moment responses Slide 38 @tameverts
  • 39.
  • 40.
    Slide 40 Why testa 500ms delay? Case study: The impact of HTML delay on mobile business metrics @tameverts
  • 41.
    Slide 41 We focusedon the metrics most affected by the 500ms delay: Emotional engagement Frustration
  • 42.
  • 43.
    Normal speed Slide 43 2.66s2.92s 2.83s 4.24s
  • 44.
    Frustration levels acrosssites (normal speed) Slide 44 @tameverts
  • 45.
    Engagement levels acrosssites (normal speed) Slide 45 @tameverts
  • 46.
    500ms delay: Peakfrustration results Slide 46 @tameverts
  • 47.
    500ms delay: Averageengagement results Slide 47 @tameverts
  • 48.
    EEG test summary •A mere 500ms delay results in significant increase in frustration levels. • Faster pages result in higher levels of engagement. • Different sites trigger emotional shifts at different phases of the experience (browsing vs. checkout). • Important: These tests happened under ideal browsing conditions. Slide 48 @tameverts
  • 49.
  • 50.
    Impact of sitespeed on post-test brand association Slide 50 @tameverts
  • 51.
    If pages aren’tfast, everything suffers. Content “boring” Visual design “tacky” and “confusing” Navigation “frustrating” and “hard-to-navigate” Slide 51 @tameverts
  • 52.
  • 53.
  • 54.
    1 People feel“web stress” even when shopping under ideal conditions. 2 Slower web performance has a clear and measurable impact on people at a neurological level. 3 Slow sites can seriously undermine overall brand health. 4 The nature and scale of impact varies, depending on a number of factors (e.g. inherent strength/weakness of brand). 5 This presents great opportunities to strengthen overall brand by investing in performance optimization. Slide 54 @tameverts
  • 55.
  • 56.
  • 57.
    Sources Web Stress: AWake-Up Call for European Business (Foviance, 2010) http://www.ca.com/us/~/media/files/supportingpieces/final_webstress_survey_report_229296.aspx 2013 Social & Mobile Commerce Consumer Report (Shop.org / comScore) http://shop.org/research/original/2013-social-mobile-commerce-consumer-report 2012 Mobile User Survey (Keynote) http://www.keynote.com/docs/reports/Keynote-2012-Mobile-User-Survey.pdf 2013 State of the Union: Mobile Ecommerce Performance (Radware) http://www.radware.com/mobile-sotu2013/ The Danger of a Poor Mobile Shopping Experience [INFOGRAPHIC] http://www.getelastic.com/the-danger-of-a-poor-mobile-shopping-experience-infographic/ Case study: The impact of HTML delay on mobile business metrics (Web Performance Today, November 2011) http://www.webperformancetoday.com/2011/11/23/case-study-slow-page-load-mobile-business-metrics/ Slide 57
  • 58.