My presentation at the Zycko breakfast session... About why your users don't like to wait and why you should care as a site owner. This presentation covers the importance of perception of speed, navigation and how to do proper performance monitoring...

1. w
@jeroentjepkema
Zycko Breakfast session
June 18 2014
The “Waiting” Experience

2. Waiting = Engagement

11. Mobile?

18. “Both offline and online consumers
associate long wait times with
poor customer service”

19. Click away slide
Performance tolerance

20. 0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10
Average Bouncerate per pagetype/sessionBouncerate(%)
Page load time (sec.)
Median Campaign Product search
Aggregated Real User Monitoring data collected by MeasureWorks for Dutch Commerce websites

21. 0
10
20
30
40
50
60
70
80
90
100
0 1 2 3 4 5 6 7 8 9 10
Bouncerate(%)
Page load time (sec.)
Median Campaign Product search
Average Bouncerate per pagetype/session
Aggregated Real User Monitoring data collected by MeasureWorks for Dutch Commerce websites

22. Purpose vs. Context

23. Online users often lack context for delays...

24. ...and see no other option than to click away

25. Good Design
+
Fast Delivery
=
Great User Experience

26. NS Hispeed: Site optimization
0
12,5
25
37,5
50
62,5
75
87,5
100
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Non-Bouncerate(%)
Categorienaam
Pageviews(#)
Bouncerate # Pageviews
Bol.com: Speed vs. Engagement per month
Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum

27. Load time
Availability
Homepage
Transaction time
Availability
Ticket ordering funnel (7 steps)
Before After
11,5s
95,61%
3,8s
99,52%
Before After
43,2s
88,72%
35,7s
97,65%
30% higher
site success
rate
NS Hispeed: Site optimization

28. Design vs. Speed?

29. 0
25
50
75
100
1 2 3 4 5
Slow Average Fast
%ofrespondentsthatratedtheactualwebsitespeed
Design score (1=bad - 5=beautiful)
Perception of good design is influenced by fast page load times

30. Good Design
+
Fast Delivery
=
Great User Experience

31. Navigating the web is not easy...

32. https://www.youtube.com/watch?v=3Sk7cOqB9Dk&feature=youtu.be

33. Let’s design for fast experience!

34. 1. Recognizable navigation

35. Source: Jakob Nielsen
99% of time online is spent on other sites
than yours...

36. Experience vs. surfing online....

37. Always compared to past experiences
Experience vs. surfing online....

38. Task completion has positive impact
Always compared to past experiences
Experience vs. surfing online....

39. Task completion has positive impact
Slow downs have more impact
Always compared to past experiences
Experience vs. surfing online....

40. Sensitive for specific tasks
Task completion has positive impact
Slow downs have more impact
Always compared to past experiences
Experience vs. surfing online....

41. Why is this important?

42. Consumers visit 18-23 travel websites
before they finally convert
Source: TNS/Nipo

43. Search/Orientation phase
Perception of experience
1
Performance influence cycle
http://benchmark.measureworks.nl

44. Delivered
experience
Your website
3
5
2
Stimulate content/
conversion
4
Performance influence cycle
Search/Orientation phase
Perception of experience
1
http://benchmark.measureworks.nl

45. The waiting experience

47. 2. Speed up your content!

48. Speed-up your perception

49. http://www.nytimes.com/2012/08/19/opinion/sunday/why-waiting-in-line-is-torture.html?pagewanted=all&_r=0

50. http://answers.yahoo.com/question/index?qid=1005081200005

51. nytimes.com/2004/02/27/nyregion/27BUTT.html

53. Perception = Fast Experience

54. http://www.fastcodesign.com/1669788/the-3-white-lies-behind-instagrams-lightning-speed

55. @jeroentjepkema, MeasureWorks
How fast should I be?

56. “Speed is at it’s best when it creates the feeling
that you don’t have to work to achieve your goal”

57. 0,3
0 Source: Jakob Nielsen
Instantaneous: I like it!

58. 1
0,3
0 Source: Jakob Nielsen
Instantaneous: I like it!
Interaction: Let’s conversate...

59. 1
0,3
0 Source: Jakob Nielsen
Instantaneous: I like it!
Interaction: Let’s conversate...

60. 1
3
0,3
0 Source: Jakob Nielsen
Instantaneous: I like it!
Interaction: Let’s conversate...
Mmm, shall I click away?

61. 10
1
3
0,3
Instantaneous: I like it!
Interaction: Let’s conversate...
Mmm, shall I click away?
Only if the task/content is relevant
0 Source: Jakob Nielsen

62. It’s (also) about the customer journey...

63. Provide relevant content to
support task completion...

64. ...deliver it fast, focus on perception!

65. “Flow state”

66. Speed vs. Content

67. What’s your time to first tweet?
https://blog.twitter.com/2012/improving-performance-on-twittercom

68. 3 (performance) design questions to ask...

69. Above the fold?

70. Which priority?

71. Improving which metrics?

72. Trouble is stickyness?
Gum/Shoe
3. Measure to make it stick...

73. www.donothing.com

74. Put performance in your SLA
Service Levels = Your baseline

75. A practical (and easy DIY) example:

76. Purchasing a book,
must be completed (speed),
where every page loads under 3 sec.,
from any location in the Netherlands,
for 90% of all users,
every day between 6am and 12pm,
using IE9 and higher,
Customer journey
Metric: Speed
Target: Sec
User scenario
User locations
Percentile
Service Window
measured with Real User Monitoring. Monitoring type
Read more: Metrics 101, Velocityconf 2010

77. Repeatable, End-User focused & Quantifiable

78. Setting up performance monitoring

79. INTERNET CUSTOMERS
Third-party/
Cloud Services
Content
Delivery
Networks
Local
ISP
Mobile
Carriers
Major
ISP
INTERNAL USERS(CLOUD) DATA CENTER
Storage DB Servers Web Servers
App
Servers
Middleware
ServersMainframe
Load
Balancers
Network
Application Delivery Chain

80. INTERNET CUSTOMERS
Third-party/
Cloud Services
Content
Delivery
Networks
Local
ISP
Mobile
Carriers
Major
ISP
INTERNAL USERS(CLOUD) DATA CENTER
Storage DB Servers Web Servers
App
Servers
Middleware
ServersMainframe
Load
Balancers
Network
Application Delivery Chain
This is what you control...

81. INTERNET CUSTOMERS
Third-party/
Cloud Services
Content
Delivery
Networks
Local
ISP
Mobile
Carriers
Major
ISP
INTERNAL USERS(CLOUD) DATA CENTER
Storage DB Servers Web Servers
App
Servers
Middleware
ServersMainframe
Load
Balancers
Network
Application Delivery Chain
This is what you control... What you’re blamed for..

82. Measuring performance is not easy...

83. Measuring performance is not easy...
You need diagnostic details for things you can
change and/or control

84. Measuring performance is not easy...
You need insights in the things you can’t control,
but do impact your bottom line
You need diagnostic details for things you can
change and/or control

85. Measuring performance?

86. Measuring performance? Always outside-
in, from the browser perspective...

87. Internet DB
Load
balancer
Web
server
App
server
User
Datacenter

88. Internet DB
Load
balancer
Web
server
App
server
User
Datacenter
Synthetic
monitoring

89. Internet DB
Load
balancer
Web
server
App
server
User
Datacenter
RUM
Synthetic
monitoring
RUM

90. Internet DB
Load
balancer
Web
server
App
server
User
Datacenter
RUM
APM APM
APM
Synthetic
monitoring
RUM

91. Internet DB
Load
balancer
Web
server
App
server
User
Datacenter
RUM
APM APM
APM
Synthetic
monitoring
DWH
Dashboard Reports Alerts
RUM

92. Internet DB
Load
balancer
Web
server
App
server
User
Datacenter
RUM
APM APM
APM
Synthetic
monitoring
DWH
Dashboard Reports Alerts
RUM
From pageview to
application code
Deep dive diagnostics
End to End User Experience

93. Synthetic monitoring = Your heartbeat

94. Synthetic monitoring = Your heartbeat
Ultimately, synthetic monitoring shows you if your
site’s working or not...
(best used for operations and service reports)

95. Simulate business transactions

96. Error detection & Root Cause analysis

97. However, the problem with synthetic is ...

98. Synthetic heartbeat
Real Users
..it’s ‘1’ user, not your end user

99. Real User Monitoring = Your Real Users

100. Ultimately, Real User Monitoring shows you how many
users are affected by bad performance...
(best used for capacity management and conversion optimization)
Real User Monitoring = Your Real Users

101. 1
Insert tag (.js file) into
(mobile) web pages
Pages are requested
from browser/device
As pages execute,
tag collects
performance metrics
After onload tag send
detailed report for
further analysis
tag.js
tag.js
tag.js
tag.js
2 3 4

102. 7
0%
0%6
5
4
3
2
1
0
LoadTime(Sec.)Pageviews(%)
100%
50%
75%
0%
25%
TCP connection SSL Handshake DNS Resolution Front-End time DOM Ready Page Load
Pageviews
2.5
0
1.5
2.0
1.0
LoadTime(Sec.)
Page views vs. Load time breakdown
Time (days)

103. Application Performance = Your mechanic

104. Application Performance = Your mechanic
Ultimately, Application Performance Monitoring shows
you how your application behaves under pressure...
(best used for root cause analysis, profiling and development)

108. A performance baseline...

109. Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum
0
12,5
25
37,5
50
62,5
75
87,5
100
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Non-Bouncerate(%)
Categorienaam
Pageviews(#)
Bouncerate # Pageviews
Bol.com: Speed vs. Engagement per month

110. 0
12,5
25
37,5
50
62,5
75
87,5
100
0-0,5 0,5-1 1-1,5 1,5-2 2-2,5 2,5-3 3-3,5 3,5-4 4-5 5-6 6-7 7-8 8-9 9-10 >10
Non-Bouncerate(%)
Categorienaam
Pageviews(#)
Bouncerate # Pageviews
Bol.com: Speed vs. Engagement per month
Non-Bounce Mobile
LD50: 4.85 sec.
Baseline: 2,7 sec.
Poverty Line: 10,6 sec.
Data presented by MeasureWorks at Velocity Conference London 2013: http://bit.ly/MW-VEUrum

111. Building reports...

112. Competition
0
1
2
3
4
5
33 34 35 36 37 38 39 40
Performance(avg)
Week
You Competitors Branche
40% of your
compeitors are
faster than you
40%
is the average
performance of your
competitors
3,4 sec
Is your ranking
against
competitors
# 6
Availability
Loadtime.
(Document.
complete)
Loadtime
(.fully.loaded.)
082 084
www.homepage.nl 97,0% 2,20 3,50 10% 85%
ge.nl/autoverzekeren 99,0% 2,50 3,30 20% 90%
overzekeren/afsluiten 95,0% 2,30 3,60 15% 70%
homepage.nl/contact 95,0% 2,43 3,57 20% 67%
homepage.nl/zoeken 94,0% 2,48 3,62 23% 59%
e.nl/productoverzicht 93,0% 2,53 3,67 25% 52%
ge.nl/ab8test/variant1 92,0% 2,58 3,72 28% 44%
ge.nl/ab8test/variant2 91,0% 2,63 3,77 30% 37%
ge.nl/facebookplugins 90,0% 2,68 3,82 33% 29%
page.nl/muziekpagina 89,0% 2,73 3,87 35% 22%
ww.homepage.nl/faq 88,0% 2,78 3,92 38% 14%
KPI’s
Service levels Service Level Report
Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
Page 7
Page 8
Page 9
Page 10
Page 11

113. 0
15
30
45
60
75
90
33 34 35 36 37 38 39 40
33 34 35 36 37 38 39 40
Speed (sec.) 2.9 2.4 2.2 2.8 3.9 2.9 3.1 2.9
URL’s (#) 67 68 65 69 80 72 69 73
3d Party (#) 7 8 6 9 8 9 7 7
Page Size (Kb) 2000 1800 1850 2100 2700 2200 2100 2150
6
Speed
URL’s
3d Party
Performance Trend report
5
4
3
2
1
0
Objects(#)
Loadtime(sec.)
Week (#)

114. Revenue risked (Performance/Time Period/Percentiles)
Optimal Flow (P_25) Speed (sec.) Availability (%) Bouncerate (%) Potential Conversion
Page 1 2.1 100 49% 51.000
Page 2 4.0 100 51% 24.990
Page 3 2.0 100 60% 9996
Page 4 3.5 85 25% 6372
Actual Flow (P_85) Speed (sec.) Availability (%) Bouncerate (%) Actual Conversion
Page 1 4.1 100 59% 41.000
Page 2 6.3 100 54% 18.860
Page 3 2.0 100 60% 7544
Page 4 4.7 85 53% 3013
100.000 visits
0-2,7 sec. 2,7-3,6 sec. >3,6 sec.

115. Questions so far?

116. Building your (monitoring) strategy...

117. Control
Quality vs.
Service Levels
User Impacted?
Application Chain vs.
Business impact
Resolution
Root cause vs.
Uptime
Task Data
Need for
details
- -
++
Manage
Deliverables?

118. Synthetic
monitoring
Real User
Monitoring
Application
Performance
Monitoring
Control
Quality vs.
Service Levels
User Impacted?
Application Chain vs.
Business impact
Resolution
Root cause vs.
Uptime
Task Data
Manage
Deliverables?
Need for
details
- -
++
Tooling

119. One dataset to rule them all...

120. The “waiting” experience
1. Understand your customer
2. Prioritize content
3. Deliver fast
4. Measure to optimize
5. Repeat this for every iteration

121. Thanks! More questions?
M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl
View slides: bit.ly/MW-breakFAST