This document provides performance metrics for over 100 websites tested on various browsers. Key findings include: - Internet Explorer 7 and 8 had significantly slower page load times than other browsers for many sites. - Page load time differences between browsers correlated with differences in conversion rates and revenue for sites. Faster sites on all browsers could lead to increased orders and revenue. - Building a business case for optimization, companies could estimate hours and costs for a DIY approach versus an automated solution, and correlate expected speed improvements to increased traffic, orders and revenue over time.

2. @jeroentjepkema
www. .nl

3. Performance Automation 101
Velocity Conference Europe,
8 November 2011

4. DIY vs. Automation
Test drive
Business benefits?
Performance Automation 101
Why optimize?
Velocity Conference Europe,
8 November 2011

5. Mainstream topic

6. Conversion Benchmark

7. Visitor Strangeloop Website
Decide whether
to optimize
Normal
Receive Accelerated
content
page
Insert
Process segment
scripts Optimize?
marker
Send
analytics Unaccelerated
Google Analytics

8. Here’s what they discovered....

9. Traffic levels
9.000
6.750
Total number of visits
4.500
8.505
2.250 4.740
0
Optimized Unoptimized

10. Average time on site
31
Time on site (minutes)
23
16 30,17
23,83
8
0
Optimized Unoptimized

11. Conversion rate / order value
20
Difference due to optimization
15
10
16,07
5
5,51
0
Conversion rate Order value

12. Performance by the numbers

13. Performance = Psychology

14. -
10 sec
4 sec
Engagement
2 sec
1 sec
+
Performance = Psychology

15. 10
4
2 Perceived load time
+ 15 % slower
1
Actual vs. Perceived performance
Performance = Psychology

16. Performance in numbers

17. 3 seconds

18. before60%
of online consumers
will click away
3 seconds

19. before60%
of online consumers
will click away
3 seconds
80% of them
will NOT return

20. before60%
of online consumers
will click away
3 seconds
Of these, 50% will
80% of them
tell others about their
will NOT return
negative experience

21. Too much variation is not good....

22. Page Abandonment 40
% Increase in
30
20 38%
33%
25%
10
7%
0
2 to 4 seconds 2 to 6 seconds 2 to 8 seconds 2 to 10 seconds
Change in page load time
Source: Forrester research 2010

23. Would you recommend me?

24. 1,5 100
1,25
96
1
92
0,75
88
0,5
84
0,25
0 80
17 18 19 20 21 22 23 24 25 26 27 28 29 30
Laadtijd Beschikbaarheid
17-30 Oct: Average Speed / Availability Homepage ING, Emerce eFinancials 2011

25. 1,5 100
1,25
96
1
92
0,75
88
0,5
84
0,25
Social
0 80
17 18 19 20 21 22 23 24 25 26 27 28
sentiment
29 30
Laadtijd Beschikbaarheid
17-30 Oct: Average Speed / Availability Homepage ING, Emerce eFinancials 2011

26. 6 100
5
80
4
60
3
40
2
20
1
0 0
17 18 19 20 21 22 23 24 25 26 27 28 29 30
Laadtijd Beschikbaarheid
17-30 Oct: Average Speed / Availability iPhone App, Emerce eFinancials 2011

27. 6 100
5
80
4
60
3
40
2
20
1
0 Social 0
17 18 19 20 21 22 23 24 25 26 27 28 29 30
sentiment
Laadtijd Beschikbaarheid
17-30 Oct: Average Speed / Availability iPhone App, Emerce eFinancials 2011

28. Faster is better!

29. DIY
Do it Yourself (DIY)

30. ormance Websites”

31. 1. Make fewer HTTP requests
ormance Websites” 2. Use a CDN
3. Add an Expires hea
4. Gzip components
5. Put CSS at the top
6. Move JS to the bottom
7. Avoid CSS expressions
8. Make JS and CSS external
9. Reduce DNS lookups
10. Minify JS
11. Avoid redirects
12. Remove duplicate scrips
13. Turn of Etags
14. Make AJAX cacheable and small

32. 1. Make fewer HTTP requests
ormance Websites” 2. Use a CDN
3. Add an Expires header
4. Gzip components
5. Put Stylesheets at the top
6. Move JS to the bottom
7. Avoid CSS expressions
8. Make JS and CSS external
9. Reduce DNS lookups
10. Minify JS
11. Avoid redirects
12. Remove duplicate scrips
13. Configure of Etags
14. Make AJAX cacheable and small

33. Do it yourself
Low hanging fruit
‣ Reduce page size
‣ Enable Gzip
‣ Reduce the number of roundtrips
‣ Structure the page to improve
rendering & download
‣ Enable Caching
‣ Clean up code & remove
duplicates

34. Do it yourself
Low hanging fruit More advanced
‣ Reduce page size ‣ Non Blocking scripts
‣ Enable Gzip ‣ Optimizing images
‣ Reduce the number of roundtrips ‣ Splitting payloads
‣ Structure the page to improve ‣ Asynchronous loading
rendering & download ‣ Pre-loading
‣ Enable Caching ‣ Etc.
‣ Clean up code & remove
duplicates

35. Example: Insurance Company

36. % visitors per browser
Other
9%
Chrome11
4%
Safari5
4%
MSIE8
Chrome12
7% 39%
Firefox4
8%
MSIE7
10%
MSIE9
19%
Source: Gomez Active & Browser RUM

37. Other
Chrome119%
4%
Safari5
4%
Chrome12
MSIE8 MSIE8 6,4
7% 39%
Firefox4
8%
MSIE9 5
MSIE7
10% MSIE9
19% MSIE7 14,7
Firefox4 5,6
Chrome12 5,8
Safari5 5,9
0 3,75 7,5 11,25 15
Page load time per browser (sec.)
Source: Gomez Active & Browser RUM

39. This is what the data tells us...
1. Focus on IE7/8
2. Focus on Front-End optimization first
3. Clean up the code
4. Reduce payload
5. Combine requests

40. IE7 IE8 IE9 Chrome
First view 14,7s 6,4s 5,0s 5,8s
Repeat view 11,7s 6,1s 4,1s 3,7s
Source: Web Analytics, Gomez Active & Browser RUM

41. IE7 IE8 IE9 Chrome
First view 14,7s 6,4s 5,0s 5,8s
Repeat view 11,7s 6,1s 4,1s 3,7s
Optimize!
‣Reduced pagesize from 760Kb to 390Kb
‣Reduced # request from 68 to 48
Source: Web Analytics, Gomez Active & Browser RUM

42. IE7 IE8 IE9 Chrome
First view 14,7s 6,4s 5,0s 5,8s
Repeat view 11,7s 6,1s 4,1s 3,7s
Optimize!
‣Reduced pagesize from 760Kb to 390Kb
‣Reduced # request from 68 to 48
IE7 IE8 IE9 Chrome
First view -11% -14% -7% -22%
Repeat view -20% -54% -39% -17%
Source: Web Analytics, Gomez Active & Browser RUM

43. Only 5
days after
5 days after
implementation
implementation

44. 30%
page 5 days after
load time
implementation
reduction!
7 300
270
6
240
5 210
4 180
150
3 120
2 90
60
1
30
0 0
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
Page Load Time (sec.) Actual Sales
Source: Web Analytics, Gomez Active & Browser RUM

45. 30%
page load time
20% average page
load time reduction!
reduction!
7 300
270
6
240
5 210
4 180
3 Better yet: 20% 150
120
2 increase in conversion! 90
60
1
30
0 0
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
Page Load Time (sec.) Actual Sales
Source: Web Analytics, Gomez Active & Browser RUM

46. Faster is better, but...

47. The downside of DIY
Manual optimization is time consuming, this is
only 1 sales funnel;
Getting priority in the release schedule can be
difficult;
Optimization is not a once off, but a continuous
cycle of improvement;
Difficult to keep up with new techniques, new
browsers, etc.

48. However...
the biggest challenge is:

49. Business IT
vs.

50. Making the
web faster
Marketing
initiatives

51. What if....?

52. Definition
Steve Souders “High-Performance Websites”
licationperformance.com Performance is our business!
tekst

53. Customer Web Site

54. Performance Automation

56. Web Application Web Application
Delivery Transformation
via: Joshua Bixby @ London Web Performance Group, Strangeloopnetworks

57. Web Application Transformation

58. Load Web App
Internet balancer server server DB
Customer
Your website

59. Option 1:
Cloud
Load Web App
Internet balancer server server DB
Customer
Your website

60. Option 2:
Option 1:
Hardware
Cloud
appliance
Load Web App
Internet balancer server server DB
Customer
Your website

61. Option 2:
Option 1:
Hardware
Cloud
appliance
Load Web App
Internet balancer server server DB
Customer
Your website
Option 3:
Software
modules

67. Building a Business Case

68. 2 Components

69. Budget for DIY
2 Components

70. Budget for DIY
# Hours to Build
2 Components

71. Budget for DIY
# Hours to Build
# Hours to Test
2 Components

72. Budget for DIY
# Hours to Build
# Hours to Test
# Hours to Deploy
2 Components

73. Budget for DIY
# Hours to Build
# Hours to Test
# Hours to Deploy
Time schedule?
2 Components

74. Budget for DIY Revenue
# Hours to Build
# Hours to Test
# Hours to Deploy
Time schedule?
2 Components

75. Budget for DIY Revenue
# Pageviews /
# Hours to Build
browser type
# Hours to Test
# Hours to Deploy
Time schedule?
2 Components

76. Budget for DIY Revenue
# Pageviews /
# Hours to Build
browser type
# Orders /
# Hours to Test
browser type
# Hours to Deploy
Time schedule?
2 Components

77. Budget for DIY Revenue
# Pageviews /
# Hours to Build
browser type
# Orders /
# Hours to Test
browser type
Page speed /
# Hours to Deploy
browser type
Time schedule?
2 Components

78. Budget for DIY Revenue
# Pageviews /
# Hours to Build
browser type
# Orders /
# Hours to Test
browser type
Page speed /
# Hours to Deploy
browser type
Time schedule? Correlate
2 Components

79. Example: Travel site

80. % Conversion/Visits
per browser type
47,7 48,5 13,7 0,3 13,4 10,3 6,8 12,7 5,3 5,8
IE8 IE6 IE7 IE9 FF3.6
Revenue
Source: Web Analytics, Gomez Active & Browser RUM

81. % Conversion/Visits
per browser type
47,7 48,5 13,7 0,3 13,4 10,3 6,8 12,7 5,3 5,8
IE8 IE6 IE7 IE9 FF3.6
Revenue
Page Load Time
per browser
Source: Web Analytics, Gomez Active & Browser RUM

82. Revenue Risked per Browser
47,7 48,5 13,7 0,3 13,4 10,3 6,8 12,7 5,3 5,8
IE8 IE6 IE7 IE9 FF3.6
Revenue
Source: Web Analytics, Gomez Active & Browser RUM

83. From accounting to optimization

85. autotrader.co.uk/
barclays.co.uk/
bouyguestelecom.fr/
bt.com/
100+ URLs de.bigpoint.com
global.metlife.com
meteo.it/
tested* meteofrance.com/
societegenerale.fr/
sports.yahoo.com
www.888.com/
www.aa.com
www.airfrance.fr/
www.allstate.com
www.amazon.com
www.autoscout24.de/
www.bahn.de/
www.bankofamerica.com
www.bbc.co.uk/
www.betfair.com/
www.bild.de/
www.bnpparibas.net/
www.bookit.com
www.chip.de/
www.cnet.com/
www.cnn.com
* Per site, homepage selected
www.commentcamarche.net/

86. ‣ Location Datacenter MeasureWorks: Almere/Amsterdam
‣ 20 private WPT instances on virtual machines running IE8
on Windows server 2003, with 512Mb memory each.
‣ Connection, DSL, 1.5 Mbs/384Kbs with 50ms RTT

88. Results

89. Overall:

90. 8,8
7,6
View
st
Fir 6,4
4,5
3,8
View
t
p ea 3,5
Re
0 1 2 3 4 5 6 7 8 9
Not-optimized Avg. optimizers Fastest optimizer

91. 8,8
View
7,6 -28%
st
Fir 6,4
4,5
3,8
t View -24%
p ea 3,5
Re
0 1 2 3 4 5 6 7 8 9
Not-optimized Avg. optimizers Fastest optimizer

92. From a business perspective:

93. Every site has a business model

94. E-commerce & Travel
Make people buy things

95. Media
Make you click on ads

96. Finance & Insurance
Perform financial transactions
& open savings account

97. Distribution of elements per type of sites (in Kb)
HTML
Javascript
CSS
Images
Flash
0 300 600 900
News Ecommerce Travel Gaming Finance

98. Every business model optimizes
differently

99. Images
Average usage of images (in Kb)
437
300
200 179
100
0
l e
ve nc
Tra Fina

100. Images
Average usage of images (in Kb)
437
300
Average reduction of Images (in %)
200 179
el 4%
Trav
100
ce 16%
n
F ina
0% 5% 10% 15% 20% 0
l e
ve nc
Tra Fina

101. Javascript
Average reduction of Javascript (in %)
ws
3%
N e
18%
e rce
o mm
Ec
0% 5% 10% 15% 20%

102. Javascript
Average reduction of Javascript (in %)
ws
3%
N e Average usage of javascript (in Kb)
300
260
18%
e rce 219
o mm 200
Ec
0% 5% 10% 15% 20%
100
0
ws rce
Ne e
o mm
Ec

103. Other...
avg. Keep Alive scores
100
98
93
85
75
67
50
25
0
Travel Ecommerce
Not-optimized
avg. Optimizers
+ 46% 9%

104. Other...
avg. Keep Alive scores avg. GZIP scores
100
98 100
93
85
76
75 75
67 65
59
50 50
30
25 25
0 0
Travel Ecommerce News Finance
Not-optimized
avg. Optimizers
+ 46% 9% 17% 97%

105. From a technical perspective:

106. Loadtime (sec.) Region
9,9
EU
8,3 12%
6,6
US
6,4 3%
0 2,5 5 7,5 10
Not-optimized
Optimizers

107. Loadtime (sec.) Region
9,9
EU
8,3 12%
6,6
Size (Kb)
VS
6,4 3%
1021
EU
0 2,5 5 7,5 10 862
Not-optimized
Optimizers
766
VS
695
0 275 550 825 1100
Not-optimized
avg. Optimizers

108. Improvement based on elements per site (%)
30
22,5
15
7,5
0
-7,5
ML ipt SS g es las
h
HT scr C
ma F
Java I
EU US

109. United States
Europe

110. Optimization will vary

111. Not-optimized Sitespect PageSpeedService
More requests Less requests

112. Not-optimized Sitespect PageSpeedService
More requests Less requests
Handle .swf object at the end

113. Not-optimized Mod_pagespeed Aptimize
Long First Byte Time Less requests

114. Some examples

115. .de
10#sec.#,#102#requests#
4,2#sec.#,#72#requests#

116. Not-Optimized: 12 sec. Optimized : 3.1 sec.
Accelera'on*of*small*elements*
m &&
S am e&file&fro &
ec.
0,6&to&0,1&s
*.swf&file&moved&last&in&line&

117. Taking care of the Javascript

118. Winners?

119. 89 89
88 88
86
85
84
83 83
81 81 81
79
77 77
76
75
74 74
68
67 67 67 67
64 64
63
57 57
55
54 54 54
53 53
52 52 52
51 51
48
44 44
43 43 43
42
41
37
30
Cache StaticCDN OneCDN Gzip Cookies KeepAlive Minify Combine Compression Etag
Non-optimized Aptimize Mod_Pagespeed PageSpeedService Sitespect

120. Who’s next....

121. 40%
30%
20%
32%
16%
10%
13%
9%
10% Finance
Gaming
0% Travel
Ecommerce
News
Average Loadtime Improvement (in %)

122. It works!

123. 15%
Total time
Savings
85%

124. 15%
27,9 hrs is the total time it took to complete the testrun with
non-optimized pages....
Total time
Savings
85%

125. 15%
27,9 hrs is the total time it took to complete the testrun with
non-optimized pages.... Optimization reduces it by 4,2 hrs
Total time
Savings
85%

126. 15%
27,9 hrs is the total time it took to complete the testrun with
non-optimized pages....Optimization reduces it by 4,2 hrs
Total time
Or....8,1Gb is the total non-optimized pagesize....Using
Savings
optimization this can be reduced with 1,2 Gb
85%

127. The average person in Europe spends
34,4 hours online per month
and visits 3515 pages
Source: Adformatie, March 2011

128. Just imagine.....

129. What you can do....

130. ...with all the extra time

131. Unfortunately, it’s not a push of the button

132. Your mileage may vary....

133. There’s the website itself
Place:
Place: View
View order my account
Task: history Task:
Create Log In
account
Task:
Place: Upgrade Task:
Read account Buy
review stuff
comments

134. Browsers Location
Browser versions
Last mile
Type of product
Used technology
Many variables to take into account

135. .....and so will your test results

136. Fully loaded, Non-Optimized Fully loaded, Optimized
Computerbild.de
NBC.com
Voyages-sncf.com

137. DIY vs. Performance Automation

138. Benefits of automation
Optimizes all pages automatically with a set of
standard treatments;
Quick and (can be) easy to install;
Provides continuous optimization;
Creates performance baseline, all pages are
made as fast as possible

139. But...

140. Downside of automation
You need to understand your website to get the
most out of automation;
Not all treatment settings are good for your
website;
Automation needs customization to reach full
potential;
3d Party content is not optimized ;-)

141. My advice: It’s both

142. Takeaways...

143. Start benchmarking multiple options

144. 1. Determine your business targets

145. 1. Determine your business targets
2. Create a test environment

146. 1. Determine your business targets
2. Create a test environment
3. Select measurement tool

147. 1. Determine your business targets
2. Create a test environment
3. Select measurement tool
4. Connect to automation providers

148. 1. Determine your business targets
2. Create a test environment
3. Select measurement tool
4. Connect to automation providers
5. Test, Correlate & Compare

149. It’s about the customer journey

150. Is this bad?....
Ordering Car Insurance
Source: Gomez

151. Is this bad?....
1: Landing page 4: Enter insurance details
2: Click order 5: Calculate offer
3: Enter personall details 6: Order insurance
Source: Gomez

152. Is this bad?....
1: Landing page 4: Enter insurance details
2: Click order 5: Calculate offer
3: Enter personall details 6: Order insurance
Source: Gomez

153. Business case: Focus on attributing KPI’s

154. Cart abandonments

155. http://www.flickr.com/photos/
mrmoorey/160654236
Bounce rate

156. Traffic / Page views

157. Time on site

158. And remember....

162. Publish data online
Start repeating tests with
specific scenarios
Stress-test the optimizers
Expand to different vendors
and/or solutions

163. Questions?

164. Thanks! More questions?
M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl