User Experience vs. Conversion Optimization 
@jeroentjepkema 
Shoppingtoday 2014
Twinkle100: How fast are we?
1,49Mb 
average 
pagesize 
Twinkle100 data collected with webpagetest.org with Chrome, IE11, Firefox & Safari (10Mbs down/1,5Mbs up) 
4.59s 
average 
page load time 
130 
requests on 
average 
86% 
slower than 
3 sec. 
Desktop Performance
1,2Mb 
average 
pagesize 
4.27s 
via wifi 
8.57s 
via 3G 
Mobile Performance 
41% of 
Twinkle100 have 
optimized site 
Twinkle100 data colelcted with webpagetest.org on both Android & iPhone (WiFi (10Mbs down/1,5Mbs up) & 3G (2000Kbps Down, 764Kbps Up))
Year over year performance (baseline = 2011) 
-3% 
25% 
3% 5% 
30% 
17% 
10% 
Data collected with webpagetest.org on Crome, Firefox, IE11, Android & iPhone (WiFi (10Mbs down/1,5Mbs up) & 3G (2000Kbps Down, 764Kbps Up)) 
8% 
4% 
2012 2013 2014 
2012 2013 2014 
2012 2013 2014 
0 
Avg. Speed Avg. Size Avg. # Request
Retail100 Travel30 
Rank Brand Speed 
1 Autobandencollectief 1,35 
2 Worldticketshop 1,75 
3 NRClux 2,29 
4 Kinderkleding-tekoop 2,30 
5 Toplenzen.nl 2,46 
Average 4,69 
96 Score.nl 7,27 
97 Vente Exclusive 7,35 
98 Ticketmaster 7,74 
99 Jack&Jones 7.97 
100 Blokker 8.30 
Rank Brand Speed 
1 Sunweb 2,31 
2 Sunnycars 2,64 
3 Elmarreizen 3,49 
4 Bizztravel 3,72 
5 DeJongintra 3,80 
Average 5,49 
96 D-reizen 7,17 
97 Roompot 7,54 
98 Centerparcs 7,71 
99 Stedentrips.nl 8,24 
100 Travelbird 11,35
Why is this important?
“Both offline and online consumers 
associate long wait times with 
poor customer service”
Click away slide Performance tolerance
Complaints Top 5 topics 
21% 
6% 
41% 
32% 
Delivery Ordering UX Other 
Downtime 
Mobile 
Speed 
Login 
Forms 5% 
6% 
21% 
19% 
36% 
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014
40 
30 
20 
10 
0 
0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 22.00 24.00 
Time of day (hours) 
# of tweets) 
Complaints per branche/day 
Retail100 
Travel30 
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014
Purpose vs. Context
Online users often lack context for delays...
...and see no other option than to click away
User Experience vs. Conversion
100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 
Bouncerate per pagetype/session 
0 1 2 3 4 5 6 7 8 9 10 
Bouncerate (%) 
Page load time (sec.) 
Median Campaign Product search 
Real User Monitoring data collected by MeasureWorks for several eCommerce webshops, anonymized and grouped per pagetype
100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 
Bouncerate per pagetype/session 
0 1 2 3 4 5 6 7 8 9 10 
Bouncerate (%) 
Page load time (sec.) 
Median Campaign Product search 
Real User Monitoring data collected by MeasureWorks for several eCommerce webshops, anonymized and grouped per pagetype
Back in a second....
Navigate Payment 
Home Product/ 
Search Detail Basket Details Payment Thank you
Navigate Payment 
Home Product/ 
Search Detail Basket Details Payment Thank you 
Detail 
Detail 
Product 
Detail 
Product 
Product
Navigate Payment 
Home Product/ 
Search Detail Basket Details Payment Thank you 
Detail 
Detail 
Product 
Detail 
Product 
Product 
“Make the return feel fast 
enough to stimulate 
pageviews”
“Speed is at it’s best when it creates the feeling 
that you don’t have to work to achieve your goal”
The waiting experience
0 0,5 1,5 3 10 
What happens in a page view? 
Source: Jakob Nielsen
Visual 
confirmation 
0 0,5 1,5 3 10 
What happens in a page view? 
Source: Jakob Nielsen
Visual 
confirmation 
Understand 
Navigation 
0 0,5 1,5 3 10 
What happens in a page view? 
Source: Jakob Nielsen
Visual 
confirmation 
Understand 
Navigation 
Relevant 
Content? 
0 0,5 1,5 3 10 
What happens in a page view? 
Source: Jakob Nielsen
Visual 
confirmation 
Understand 
Navigation 
Relevant 
Content? 
Bye 
Bye! 
0 0,5 1,5 3 10 
What happens in a page view? 
Source: Jakob Nielsen
Purpose vs. Context 
Standing in line for iPhone6 launch - London 2014
5 insights for speed(y) experiments
1. Speed-up your perception
Which is faster?
Which is faster?
Which is faster?
Which is faster?
Fast response = Fast Experience
http://www.fastcodesign.com/1669788/the-3-white-lies-behind-instagrams-lightning-speed
“The faster you can visualize your priority content, 
the more engaged your visitors will become”
2. Increase perceived value
Navigate Payment 
Home Product/ 
Search Detail Basket Details Payment Thank you 
Detail 
Detail 
Product 
Detail 
Product 
Product
Flow
Navigate Payment 
Home Product/ 
Search Detail Basket Details Payment Thank you 
4 
3 
2 
1 
0 
Home Product Detail Basket Details Payment Thank you 
Start Render Document Complete Tolerance 
Speed (sec.)
People prefer to wait for up 
to a minute to get what they 
want from an app rather 
than get it instantly – if, and 
it’s an important if, they 
believe the app is working 
The kayak effect: http://bit.ly/UgTneD 
for them
Show effort, 
provide specific feedback, 
build trust
Navigate Payment 
Product/ 
Search Payment 
Home Detail Basket Details Thank you 
4 
3 
2 
1 
0 
Home Product Detail Basket Details Payment 
Thank you 
Start Render Document Complete Tolerance 
Speed (sec.)
3. Do I need that carousel?
83% of all Twinkle100 sites loads menu 
before content...
< Call to action > 
Item 1 Item 2 Item 3 Item 4 
Descriptive labels 
What’s on next page? 
Designing your carousel 
Arrows for navigation 
confirmation
Loading your carousel 
< Call to action > 
Item 1 Item 2 Item 3 Item 4 
Descriptive labels 
Call to action first 
Load other items 
seperately, in the 
background 
Load only primary 
image second
4. Recognizable navigation
Performance influence cycle 
Daily surfing 
Search/Orientation phase 
1 
Perception of experience 
http://benchmark.twinkle100.nl
Delivered 
experience 
3 
Your website 
5 
2 
Stimulate content/ 
conversion 
4 
Performance influence cycle 
Daily surfing 
Search/Orientation phase 
1 
Perception of experience 
http://benchmark.twinkle100.nl
Don’t interrupt the flow...
Homepage Search Product 
overview 
Product 
Detail Basket
Homepage Search Product 
overview 
Product 
Detail Basket
Homepage Search Product 
overview 
Product 
Detail Basket 
First 
navigation 
(1-2 sec.) 
Consistent 
navigation 
(0,5-1 sec.) 
Renew your 
navigation 
(1-2 sec.) 
renew 
your navigation 
(1-2 sec.)
Consistent, simple & recognizable navigation
5. Let the device work...
Search
Touch gestures
Native gestures
Scrolling
Questions so far?
Good Design 
+ 
Fast Experience 
= 
Great User Experience
Paint your #UX canvas
3 (user experience) design questions to ask...
Above the fold?
Which priority?
Improving which metrics?
Optimize your website...
Homepage 
Above the fold 
Below the fold 
Logo Menu 
Carousel 
Tekst 1 Tekst 2 Tekst 3 
Advert 1 Advert 2 Advert 3 
1 
Top 10 
Nieuw 
Overig 
3 
2 
4 4 4 
Logo Menu 
Banner(s) 
2 3 
Advert 
1 
Tekst 1 Tekst 2 
Content Advert 3 
1 
Top 10 
1 
5 5 
Advert 3 
Advert 2 Advert 3 Advert 3 
Advert 2 Advert 3 Advert 3 
Top 10 
5 
2 
2 
Eerder 
bekeken 
Product 
beschrijving 
Logo 1 
Menu 
Product images 
1 
2 3 
Tekst 1 Tekst 2 
Ook 
gekocht 
1 
Ook 
gekocht 
2 
Ook 
gekocht 
3 
4 
5 
Eerder 
bekeken 
Eerder 
bekeken 
Product 
Overview 
Product 
detail
Will it work?
Bol.com: Speed vs. Engagement per month 
Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum 
100 
87,5 
75 
62,5 
50 
37,5 
25 
12,5 
0 
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10 
Non-Bounce rate (%) 
Categorienaam 
Pageviews (#) 
Bouncerate # Pageviews
100 
87,5 
75 
62,5 
50 
37,5 
25 
12,5 
0 
Bol.com: Speed vs. Engagement per month 
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10 
Non-Bounce rate (%) 
Categorienaam 
Pageviews (#) 
Bouncerate # Pageviews 
Non-Bounce Mobile 
LD50: 4.85 sec. 
Baseline: 2,7 sec. 
Poverty Line: 10,6 sec. 
Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum
Metrics to target for: 
Perception 
Actionable 
Flow 
Time to first paint 0.5 - 1 sec. 
Time to interact: 1-3 sec 
Variation < 20% between pages
Deliver before it 
turns green!
Expert review...
View slides: bit.ly/MW-shopping14 
Thanks! More questions? 
M: jtjepkema@measureworks.nl 
T: @jeroentjepkema 
W: www.measureworks.nl

MeasureWorks - 5 insights for speed(y) experiments

  • 1.
    User Experience vs.Conversion Optimization @jeroentjepkema Shoppingtoday 2014
  • 2.
  • 3.
    1,49Mb average pagesize Twinkle100 data collected with webpagetest.org with Chrome, IE11, Firefox & Safari (10Mbs down/1,5Mbs up) 4.59s average page load time 130 requests on average 86% slower than 3 sec. Desktop Performance
  • 4.
    1,2Mb average pagesize 4.27s via wifi 8.57s via 3G Mobile Performance 41% of Twinkle100 have optimized site Twinkle100 data colelcted with webpagetest.org on both Android & iPhone (WiFi (10Mbs down/1,5Mbs up) & 3G (2000Kbps Down, 764Kbps Up))
  • 5.
    Year over yearperformance (baseline = 2011) -3% 25% 3% 5% 30% 17% 10% Data collected with webpagetest.org on Crome, Firefox, IE11, Android & iPhone (WiFi (10Mbs down/1,5Mbs up) & 3G (2000Kbps Down, 764Kbps Up)) 8% 4% 2012 2013 2014 2012 2013 2014 2012 2013 2014 0 Avg. Speed Avg. Size Avg. # Request
  • 6.
    Retail100 Travel30 RankBrand Speed 1 Autobandencollectief 1,35 2 Worldticketshop 1,75 3 NRClux 2,29 4 Kinderkleding-tekoop 2,30 5 Toplenzen.nl 2,46 Average 4,69 96 Score.nl 7,27 97 Vente Exclusive 7,35 98 Ticketmaster 7,74 99 Jack&Jones 7.97 100 Blokker 8.30 Rank Brand Speed 1 Sunweb 2,31 2 Sunnycars 2,64 3 Elmarreizen 3,49 4 Bizztravel 3,72 5 DeJongintra 3,80 Average 5,49 96 D-reizen 7,17 97 Roompot 7,54 98 Centerparcs 7,71 99 Stedentrips.nl 8,24 100 Travelbird 11,35
  • 7.
    Why is thisimportant?
  • 15.
    “Both offline andonline consumers associate long wait times with poor customer service”
  • 16.
    Click away slidePerformance tolerance
  • 17.
    Complaints Top 5topics 21% 6% 41% 32% Delivery Ordering UX Other Downtime Mobile Speed Login Forms 5% 6% 21% 19% 36% Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014
  • 18.
    40 30 20 10 0 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 22.00 24.00 Time of day (hours) # of tweets) Complaints per branche/day Retail100 Travel30 Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014
  • 19.
  • 20.
    Online users oftenlack context for delays...
  • 21.
    ...and see noother option than to click away
  • 22.
  • 23.
    100 90 80 70 60 50 40 30 20 10 0 Bouncerate per pagetype/session 0 1 2 3 4 5 6 7 8 9 10 Bouncerate (%) Page load time (sec.) Median Campaign Product search Real User Monitoring data collected by MeasureWorks for several eCommerce webshops, anonymized and grouped per pagetype
  • 24.
    100 90 80 70 60 50 40 30 20 10 0 Bouncerate per pagetype/session 0 1 2 3 4 5 6 7 8 9 10 Bouncerate (%) Page load time (sec.) Median Campaign Product search Real User Monitoring data collected by MeasureWorks for several eCommerce webshops, anonymized and grouped per pagetype
  • 25.
    Back in asecond....
  • 26.
    Navigate Payment HomeProduct/ Search Detail Basket Details Payment Thank you
  • 27.
    Navigate Payment HomeProduct/ Search Detail Basket Details Payment Thank you Detail Detail Product Detail Product Product
  • 28.
    Navigate Payment HomeProduct/ Search Detail Basket Details Payment Thank you Detail Detail Product Detail Product Product “Make the return feel fast enough to stimulate pageviews”
  • 29.
    “Speed is atit’s best when it creates the feeling that you don’t have to work to achieve your goal”
  • 30.
  • 31.
    0 0,5 1,53 10 What happens in a page view? Source: Jakob Nielsen
  • 32.
    Visual confirmation 00,5 1,5 3 10 What happens in a page view? Source: Jakob Nielsen
  • 33.
    Visual confirmation Understand Navigation 0 0,5 1,5 3 10 What happens in a page view? Source: Jakob Nielsen
  • 34.
    Visual confirmation Understand Navigation Relevant Content? 0 0,5 1,5 3 10 What happens in a page view? Source: Jakob Nielsen
  • 35.
    Visual confirmation Understand Navigation Relevant Content? Bye Bye! 0 0,5 1,5 3 10 What happens in a page view? Source: Jakob Nielsen
  • 36.
    Purpose vs. Context Standing in line for iPhone6 launch - London 2014
  • 37.
    5 insights forspeed(y) experiments
  • 38.
    1. Speed-up yourperception
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
    Fast response =Fast Experience
  • 44.
  • 45.
    “The faster youcan visualize your priority content, the more engaged your visitors will become”
  • 46.
  • 47.
    Navigate Payment HomeProduct/ Search Detail Basket Details Payment Thank you Detail Detail Product Detail Product Product
  • 48.
  • 49.
    Navigate Payment HomeProduct/ Search Detail Basket Details Payment Thank you 4 3 2 1 0 Home Product Detail Basket Details Payment Thank you Start Render Document Complete Tolerance Speed (sec.)
  • 50.
    People prefer towait for up to a minute to get what they want from an app rather than get it instantly – if, and it’s an important if, they believe the app is working The kayak effect: http://bit.ly/UgTneD for them
  • 51.
    Show effort, providespecific feedback, build trust
  • 52.
    Navigate Payment Product/ Search Payment Home Detail Basket Details Thank you 4 3 2 1 0 Home Product Detail Basket Details Payment Thank you Start Render Document Complete Tolerance Speed (sec.)
  • 53.
    3. Do Ineed that carousel?
  • 56.
    83% of allTwinkle100 sites loads menu before content...
  • 57.
    < Call toaction > Item 1 Item 2 Item 3 Item 4 Descriptive labels What’s on next page? Designing your carousel Arrows for navigation confirmation
  • 58.
    Loading your carousel < Call to action > Item 1 Item 2 Item 3 Item 4 Descriptive labels Call to action first Load other items seperately, in the background Load only primary image second
  • 59.
  • 60.
    Performance influence cycle Daily surfing Search/Orientation phase 1 Perception of experience http://benchmark.twinkle100.nl
  • 61.
    Delivered experience 3 Your website 5 2 Stimulate content/ conversion 4 Performance influence cycle Daily surfing Search/Orientation phase 1 Perception of experience http://benchmark.twinkle100.nl
  • 62.
  • 63.
    Homepage Search Product overview Product Detail Basket
  • 64.
    Homepage Search Product overview Product Detail Basket
  • 65.
    Homepage Search Product overview Product Detail Basket First navigation (1-2 sec.) Consistent navigation (0,5-1 sec.) Renew your navigation (1-2 sec.) renew your navigation (1-2 sec.)
  • 66.
    Consistent, simple &recognizable navigation
  • 68.
    5. Let thedevice work...
  • 70.
  • 71.
  • 72.
  • 73.
  • 74.
  • 75.
    Good Design + Fast Experience = Great User Experience
  • 76.
  • 77.
    3 (user experience)design questions to ask...
  • 78.
  • 79.
  • 80.
  • 81.
  • 82.
    Homepage Above thefold Below the fold Logo Menu Carousel Tekst 1 Tekst 2 Tekst 3 Advert 1 Advert 2 Advert 3 1 Top 10 Nieuw Overig 3 2 4 4 4 Logo Menu Banner(s) 2 3 Advert 1 Tekst 1 Tekst 2 Content Advert 3 1 Top 10 1 5 5 Advert 3 Advert 2 Advert 3 Advert 3 Advert 2 Advert 3 Advert 3 Top 10 5 2 2 Eerder bekeken Product beschrijving Logo 1 Menu Product images 1 2 3 Tekst 1 Tekst 2 Ook gekocht 1 Ook gekocht 2 Ook gekocht 3 4 5 Eerder bekeken Eerder bekeken Product Overview Product detail
  • 83.
  • 84.
    Bol.com: Speed vs.Engagement per month Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum 100 87,5 75 62,5 50 37,5 25 12,5 0 0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10 Non-Bounce rate (%) Categorienaam Pageviews (#) Bouncerate # Pageviews
  • 85.
    100 87,5 75 62,5 50 37,5 25 12,5 0 Bol.com: Speed vs. Engagement per month 0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10 Non-Bounce rate (%) Categorienaam Pageviews (#) Bouncerate # Pageviews Non-Bounce Mobile LD50: 4.85 sec. Baseline: 2,7 sec. Poverty Line: 10,6 sec. Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum
  • 86.
    Metrics to targetfor: Perception Actionable Flow Time to first paint 0.5 - 1 sec. Time to interact: 1-3 sec Variation < 20% between pages
  • 87.
    Deliver before it turns green!
  • 88.
  • 89.
    View slides: bit.ly/MW-shopping14 Thanks! More questions? M: jtjepkema@measureworks.nl T: @jeroentjepkema W: www.measureworks.nl