1. Selling Performance
@AndyDavies · NCC Group · May 2017
https://www.flickr.com/photos/chrisjohnbeckett/10446312055

2. Performance often isn’t at the top of people’s concerns
• ~~~~~~~~~~~~~~~~~~~~~
• ~~~~~~~~~~~~~~~~~~~~~
• ~~~~~~~~~~~~~~~~~~~~~
• ~~~~~~~~~~~~~~~~~~~~~
• ~~~~~~~~~~~~~~~~~~~~~
• Security
• Performance
• Accessibility ☹

3. So what do they care about?

4. Conversions?
https://www.flickr.com/photos/z0/19814021505

5. But it’s not just about getting visitors to buy things!
- sign ups / subscriptions
- completing tasks
- interaction with adverts
- sharing things
- finding a store / restaurant
- reducing costs - hosting / acquisition

6. Science explains visitor behaviour

7. Response Time in Man-computer Conversational Transactions
Robert B. Miller, 1968
Instant
Seamless
Yawn!
100ms 1s 10s
How we perceive response times

8. Response Time in Man-computer Conversational Transactions
Robert B. Miller, 1968
Instant
Seamless
Yawn!
100ms 1s 10s
Median start render time for 600+ UK retailers = 1.68s
How we perceive response times

9. So yes, our attention span is shorter than a goldfish
https://www.flickr.com/photos/watts_photos/23124797946

10. https://www.flickr.com/photos/ericparker/5416356195
“Badly performing web sites make us concentrate harder”
Foviance

11. https://www.flickr.com/photos/dhollister/2596483147
“Stress levels comparable to watching a horror movie”
Ericsson Mobility Report 2016

12. Challenge assumptions

13. Most people don’t have an iPhone
% sales by OS, three months to April ’17 - http://uk.kantar.com/tech/mobile/
0
10
20
30
40
50
60
70
Android iOS Windows Other

14. Everyone has fast broadband… right?

15. Average UK connection speed is 16.9Mbps!
Akamai State of the Internet Q1 2017
92% above 4Mbps
60% above 10Mbps
42% above 15Mbps

16. Which will load faster?
www.bbc.co.uk/news
10Mbps1Mbps

17. Which will load faster?
www.bbc.co.uk/news
10Mbps1Mbps

18. Which will load faster?
www.bbc.co.uk/news
10Mbps1Mbps

19. Which will load faster?
www.bbc.co.uk/news
10Mbps1Mbps

20. https://www.flickr.com/photos/joebenjamin/5009411920

21. Which will load faster?
www.bbc.co.uk/news
10Mbps1Mbps / 28ms / 280ms

22. Case studies demonstrate what’s possible

23. Wallmart demonstrated speed affects conversion in 2012
Walmart 2012

24. Cook reduced load times by 1 sec saw 7% increase in conversions
https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/march/how-we-predicted-an-increase-in-conversions-from-a-decrease-in-load-times

25. http://engineroom.ft.com/2016/04/04/a-faster-ft-com/
FT saw lower engagement when they made the site slower

26. Case studies are a great place to start
https://wpostats.com

27. “Case studies don’t represent our
demographic, our audience, our
products, our proposition…”
Has some truth

28. BounceRate
0%
25%
50%
75%
100%
Page Load Time (s)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Site 1
Site 2
Site 3
Bounce rates across three retailers

29. ConversionRate
0%
2.5%
5%
7.5%
10%
Page Load Time (s)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Site 1
Site 3
Site 4
Conversion rates vary between sites too
Noisy as few data
points at end of range

30. So what about real visitor experiences?

31. We’re used to the idea that conversion rate varies by device
iPhone 6/7
iPhone 5/5s
iPad
iPhone 6/7 Plus
Galaxy S6
Galaxy S7 Edge
Galaxy S7
Galaxy S6 Edge
Galaxy S5
Galaxy A3
Conversion / Bounce Rate
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Conversion Rate
Bounce Rate

32. But existing analytics tools are weak when it comes to speed data

33. But existing analytics tools are weak when it comes to speed data
Sample contains
< 4% of
impressions

34. They’re no substitute for measuring your own visitor’s experience
Performance Analytics aka Real User Monitoring

35. To understand how performance affects visitor behaviour

36. ConversionRate
0%
1%
2%
3%
4%
5%
6%
BounceRate
0%
20%
40%
60%
80%
100%
Page Load Time (s)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Bounce Rate
Conversion Rate
The impact on key business metrics

37. Conversion Rate
drops 1% over 3
seconds
ConversionRate
0%
1%
2%
3%
4%
5%
6%
BounceRate
0%
20%
40%
60%
80%
100%
Page Load Time (s)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Bounce Rate
Conversion Rate
The impact on key business metrics

38. And modelling the benefits improved performance delivers
Only NCC Group, Soasta and Blue Triangle’s Real User Monitoring products do this

39. Real User Monitoring is a strong driver of
performance improvement

40. Other complementary approaches too

41. Filmstrips (and videos) are hugely powerful
https://webpagetest.org

42. http://heatmap.webperf.tools
Can be sumarised into heat maps

43. Waterfalls require explanation
(I tend to avoid them with non-technical audiences)

44. http://requestmap.webperf.tools
But enable fascinating insights

45. Block third-parties understand how they affect performance
Produce comparison videos / filmstrips to show others

46. Can also compare against competitors

47. Can also compare against competitors

48. Videos prolong the painful moment

49. Or for fun…
…compare against an F1 pit crew

50. https://www.youtube.com/watch?v=aHSUp7msCIE
Vs
iPhone 6
on WiFi

51. https://www.youtube.com/watch?v=aHSUp7msCIE
Vs
iPhone 6
on WiFi

52. https://www.youtube.com/watch?v=aHSUp7msCIE
Vs
iPhone 6
on WiFi

53. https://www.youtube.com/watch?v=aHSUp7msCIE
Vs
iPhone 6
on WiFi

54. Wait… they changed all four wheels?
https://www.flickr.com/photos/cellardoorfilms/7620375702

55. So where next?

56. Establish which metrics people care about

57. Experiment
- Change the performance of a page
- Measure how metrics change
(Same principles as AB / MV testing)
Can build negative experiments
too e.g make pages bigger

58. Measure Visitors Experience
- Use the data to predict how improvements will
change the metrics
- Improve performance
- Verify the result

59. Even ‘small’ performance improvements
can have a significant impact

60. How much was 150ms improvement worth?
(was 300ms on Android)

61. Share your wins!
https://www.flickr.com/photos/borkazoid/221185817

62. http://www.flickr.com/photos/auntiep/5024494612
@AndyDavies
andy.davies@nccgroup.trust
http://slideshare.net/andydavies