MeasureWorks - Velocity Conference Europe - Performance Automation 101

@jeroentjepkemawww. .nl

Performance Automation 101 Velocity Conference Europe, 8 November 2011

DIY vs. Automation Test drive Business beneﬁts?Performance Automation 101 Why optimize? Velocity Conference Europe, 8 November 2011

Mainstream topic

Conversion Benchmark

Visitor Strangeloop Website Decide whether to optimize Normal Receive Accelerated content page Insert Process segment scripts Optimize? marker Send analytics UnacceleratedGoogle Analytics

Here’s what they discovered....

Trafﬁc levels 9.000 6.750Total number of visits 4.500 8.505 2.250 4.740 0 Optimized Unoptimized

Average time on site 31Time on site (minutes) 23 16 30,17 23,83 8 0 Optimized Unoptimized

Conversion rate / order value 20Difference due to optimization 15 10 16,07 5 5,51 0 Conversion rate Order value

Performance by the numbers

Performance = Psychology

- 10 sec 4 sec Engagement 2 sec 1 sec +Performance = Psychology

10 4 2 Perceived load time + 15 % slower 1Actual vs. Perceived performance Performance = Psychology

Performance in numbers

3 seconds

before60%of online consumers will click away 3 seconds

before60%of online consumers will click away 3 seconds 80% of them will NOT return

before60% of online consumers will click away 3 secondsOf these, 50% will 80% of themtell others about their will NOT return negative experience

Too much variation is not good....

Page Abandonment 40 % Increase in 30 20 38% 33% 25% 10 7% 0 2 to 4 seconds 2 to 6 seconds 2 to 8 seconds 2 to 10 seconds Change in page load time Source: Forrester research 2010

Would you recommend me?

1,5 1001,25 96 1 920,75 88 0,5 840,25 0 80 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Laadtijd Beschikbaarheid 17-30 Oct: Average Speed / Availability Homepage ING, Emerce eFinancials 2011

1,5 1001,25 96 1 920,75 88 0,5 840,25 Social 0 80 17 18 19 20 21 22 23 24 25 26 27 28 sentiment 29 30 Laadtijd Beschikbaarheid 17-30 Oct: Average Speed / Availability Homepage ING, Emerce eFinancials 2011

6 1005 804 603 402 2010 0 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Laadtijd Beschikbaarheid 17-30 Oct: Average Speed / Availability iPhone App, Emerce eFinancials 2011

6 1005 804 603 402 2010 Social 0 17 18 19 20 21 22 23 24 25 26 27 28 29 30 sentiment Laadtijd Beschikbaarheid 17-30 Oct: Average Speed / Availability iPhone App, Emerce eFinancials 2011

Faster is better!

DIYDo it Yourself (DIY)

ormance Websites”

1. Make fewer HTTP requestsormance Websites” 2. Use a CDN 3. Add an Expires hea 4. Gzip components 5. Put CSS at the top 6. Move JS to the bottom 7. Avoid CSS expressions 8. Make JS and CSS external 9. Reduce DNS lookups 10. Minify JS 11. Avoid redirects 12. Remove duplicate scrips 13. Turn of Etags 14. Make AJAX cacheable and small

1. Make fewer HTTP requestsormance Websites” 2. Use a CDN 3. Add an Expires header 4. Gzip components 5. Put Stylesheets at the top 6. Move JS to the bottom 7. Avoid CSS expressions 8. Make JS and CSS external 9. Reduce DNS lookups 10. Minify JS 11. Avoid redirects 12. Remove duplicate scrips 13. Conﬁgure of Etags 14. Make AJAX cacheable and small

Do it yourselfLow hanging fruit ‣ Reduce page size ‣ Enable Gzip ‣ Reduce the number of roundtrips ‣ Structure the page to improve rendering & download ‣ Enable Caching ‣ Clean up code & remove duplicates

Do it yourselfLow hanging fruit More advanced ‣ Reduce page size ‣ Non Blocking scripts ‣ Enable Gzip ‣ Optimizing images ‣ Reduce the number of roundtrips ‣ Splitting payloads ‣ Structure the page to improve ‣ Asynchronous loading rendering & download ‣ Pre-loading ‣ Enable Caching ‣ Etc. ‣ Clean up code & remove duplicates

Example: Insurance Company

% visitors per browser Other 9% Chrome11 4% Safari5 4% MSIE8 Chrome12 7% 39% Firefox4 8% MSIE7 10% MSIE9 19%Source: Gomez Active & Browser RUM

Other Chrome119% 4% Safari5 4% Chrome12 MSIE8 MSIE8 6,4 7% 39% Firefox4 8% MSIE9 5 MSIE7 10% MSIE9 19% MSIE7 14,7 Firefox4 5,6 Chrome12 5,8 Safari5 5,9 0 3,75 7,5 11,25 15 Page load time per browser (sec.)Source: Gomez Active & Browser RUM

This is what the data tells us... 1. Focus on IE7/8 2. Focus on Front-End optimization ﬁrst 3. Clean up the code 4. Reduce payload 5. Combine requests

IE7 IE8 IE9 Chrome First view 14,7s 6,4s 5,0s 5,8s Repeat view 11,7s 6,1s 4,1s 3,7sSource: Web Analytics, Gomez Active & Browser RUM

IE7 IE8 IE9 Chrome First view 14,7s 6,4s 5,0s 5,8s Repeat view 11,7s 6,1s 4,1s 3,7s Optimize! ‣Reduced pagesize from 760Kb to 390Kb ‣Reduced # request from 68 to 48Source: Web Analytics, Gomez Active & Browser RUM

IE7 IE8 IE9 Chrome First view 14,7s 6,4s 5,0s 5,8s Repeat view 11,7s 6,1s 4,1s 3,7s Optimize! ‣Reduced pagesize from 760Kb to 390Kb ‣Reduced # request from 68 to 48 IE7 IE8 IE9 Chrome First view -11% -14% -7% -22% Repeat view -20% -54% -39% -17%Source: Web Analytics, Gomez Active & Browser RUM

Only 5 days after 5 days after implementationimplementation

30% page 5 days after load time implementation reduction! 7 300 270 6 240 5 210 4 180 150 3 120 2 90 60 1 30 0 0 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 Page Load Time (sec.) Actual SalesSource: Web Analytics, Gomez Active & Browser RUM

30% page load time 20% average page load time reduction! reduction! 7 300 270 6 240 5 210 4 180 3 Better yet: 20% 150 120 2 increase in conversion! 90 60 1 30 0 0 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 Page Load Time (sec.) Actual SalesSource: Web Analytics, Gomez Active & Browser RUM

Faster is better, but...

The downside of DIYManual optimization is time consuming, this isonly 1 sales funnel;Getting priority in the release schedule can bedifﬁcult;Optimization is not a once off, but a continuouscycle of improvement;Difﬁcult to keep up with new techniques, newbrowsers, etc.

However...the biggest challenge is:

Business IT vs.

Making theweb faster Marketing initiatives

What if....?

Definition Steve Souders “High-Performance Websites”licationperformance.com Performance is our business! tekst

Customer Web Site

Performance Automation

Web Application Web Application Delivery Transformation via: Joshua Bixby @ London Web Performance Group, Strangeloopnetworks

Web Application Transformation

Load Web App Internet balancer server server DBCustomer Your website

Option 1: Cloud Load Web App Internet balancer server server DBCustomer Your website

Option 2: Option 1: Hardware Cloud appliance Load Web App Internet balancer server server DBCustomer Your website

Option 2: Option 1: Hardware Cloud appliance Load Web App Internet balancer server server DBCustomer Your website Option 3: Software modules

Building a Business Case

2 Components

Budget for DIY 2 Components

Budget for DIY# Hours to Build 2 Components

Budget for DIY# Hours to Build# Hours to Test 2 Components

Budget for DIY# Hours to Build # Hours to Test# Hours to Deploy 2 Components

Budget for DIY# Hours to Build # Hours to Test# Hours to Deploy Time schedule? 2 Components

Budget for DIY Revenue# Hours to Build # Hours to Test# Hours to Deploy Time schedule? 2 Components

Budget for DIY Revenue # Pageviews /# Hours to Build browser type # Hours to Test# Hours to Deploy Time schedule? 2 Components

Budget for DIY Revenue # Pageviews /# Hours to Build browser type # Orders / # Hours to Test browser type# Hours to Deploy Time schedule? 2 Components

Budget for DIY Revenue # Pageviews /# Hours to Build browser type # Orders / # Hours to Test browser type Page speed /# Hours to Deploy browser type Time schedule? 2 Components

Budget for DIY Revenue # Pageviews /# Hours to Build browser type # Orders / # Hours to Test browser type Page speed /# Hours to Deploy browser type Time schedule? Correlate 2 Components

Example: Travel site

% Conversion/Visits per browser type 47,7 48,5 13,7 0,3 13,4 10,3 6,8 12,7 5,3 5,8 IE8 IE6 IE7 IE9 FF3.6 RevenueSource: Web Analytics, Gomez Active & Browser RUM

% Conversion/Visits per browser type 47,7 48,5 13,7 0,3 13,4 10,3 6,8 12,7 5,3 5,8 IE8 IE6 IE7 IE9 FF3.6 Revenue Page Load Time per browserSource: Web Analytics, Gomez Active & Browser RUM

Revenue Risked per Browser 47,7 48,5 13,7 0,3 13,4 10,3 6,8 12,7 5,3 5,8 IE8 IE6 IE7 IE9 FF3.6 RevenueSource: Web Analytics, Gomez Active & Browser RUM

From accounting to optimization

autotrader.co.uk/ barclays.co.uk/ bouyguestelecom.fr/ bt.com/100+ URLs de.bigpoint.com global.metlife.com meteo.it/ tested* meteofrance.com/ societegenerale.fr/ sports.yahoo.com www.888.com/ www.aa.com www.airfrance.fr/ www.allstate.com www.amazon.com www.autoscout24.de/ www.bahn.de/ www.bankofamerica.com www.bbc.co.uk/ www.betfair.com/ www.bild.de/ www.bnpparibas.net/ www.bookit.com www.chip.de/ www.cnet.com/ www.cnn.com* Per site, homepage selected www.commentcamarche.net/

‣ Location Datacenter MeasureWorks: Almere/Amsterdam‣ 20 private WPT instances on virtual machines running IE8on Windows server 2003, with 512Mb memory each.‣ Connection, DSL, 1.5 Mbs/384Kbs with 50ms RTT

Results

Overall:

8,8 7,6 View st Fir 6,4 4,5 3,8 View t p ea 3,5Re 0 1 2 3 4 5 6 7 8 9 Not-optimized Avg. optimizers Fastest optimizer

8,8 View 7,6 -28% st Fir 6,4 4,5 3,8 t View -24% p ea 3,5Re 0 1 2 3 4 5 6 7 8 9 Not-optimized Avg. optimizers Fastest optimizer

From a business perspective:

Every site has a business model

E-commerce & Travel Make people buy things

Media Make you click on ads

Finance & Insurance Perform ﬁnancial transactions & open savings account

Distribution of elements per type of sites (in Kb) HTMLJavascript CSS Images Flash 0 300 600 900 News Ecommerce Travel Gaming Finance

Every business model optimizesdifferently

Images Average usage of images (in Kb) 437 300 200 179 100 0 l e ve nc Tra Fina

Images Average usage of images (in Kb) 437 300 Average reduction of Images (in %) 200 179 el 4% Trav 100 ce 16% nF ina 0% 5% 10% 15% 20% 0 l e ve nc Tra Fina

Javascript Average reduction of Javascript (in %) ws 3% N e 18% e rce o mmEc 0% 5% 10% 15% 20%

Javascript Average reduction of Javascript (in %) ws 3% N e Average usage of javascript (in Kb) 300 260 18% e rce 219 o mm 200Ec 0% 5% 10% 15% 20% 100 0 ws rce Ne e o mm Ec

Other... avg. Keep Alive scores100 98 93 8575 675025 0 Travel Ecommerce Not-optimized avg. Optimizers+ 46% 9%

Other... avg. Keep Alive scores avg. GZIP scores100 98 100 93 85 7675 75 67 65 5950 50 3025 25 0 0 Travel Ecommerce News Finance Not-optimized avg. Optimizers+ 46% 9% 17% 97%

From a technical perspective:

Loadtime (sec.) Region 9,9EU 8,3 12% 6,6US 6,4 3% 0 2,5 5 7,5 10 Not-optimized Optimizers

Loadtime (sec.) Region 9,9EU 8,3 12% 6,6 Size (Kb)VS 6,4 3% 1021 EU 0 2,5 5 7,5 10 862 Not-optimized Optimizers 766 VS 695 0 275 550 825 1100 Not-optimized avg. Optimizers

Improvement based on elements per site (%) 3022,5 15 7,5 0-7,5 ML ipt SS g es las h HT scr C ma F Java I EU US

United StatesEurope

Optimization will vary

Not-optimized Sitespect PageSpeedService More requests Less requests

Not-optimized Sitespect PageSpeedService More requests Less requests Handle .swf object at the end

Not-optimized Mod_pagespeed AptimizeLong First Byte Time Less requests

Some examples

.de10#sec.#,#102#requests# 4,2#sec.#,#72#requests#

Not-Optimized: 12 sec. Optimized : 3.1 sec. Acceleraon*of*small*elements* m && S am e&ﬁle&fro & ec. 0,6&to&0,1&s *.swf&ﬁle&moved&last&in&line&

Taking care of the Javascript

Winners?

89 89 88 88 86 85 84 83 83 81 81 81 79 77 77 76 75 74 74 68 67 67 67 67 64 64 63 57 57 55 54 54 54 53 53 52 52 52 51 51 48 44 44 43 43 43 42 4137 30 Cache StaticCDN OneCDN Gzip Cookies KeepAlive Minify Combine Compression Etag Non-optimized Aptimize Mod_Pagespeed PageSpeedService Sitespect

Who’s next....

40%30%20% 32% 16% 10% 13% 9% 10% Finance Gaming 0% Travel Ecommerce News Average Loadtime Improvement (in %)

It works!

15% Total time Savings 85%

15%27,9 hrs is the total time it took to complete the testrun with non-optimized pages.... Total time Savings 85%

15%27,9 hrs is the total time it took to complete the testrun with non-optimized pages.... Optimization reduces it by 4,2 hrs Total time Savings 85%

15%27,9 hrs is the total time it took to complete the testrun with non-optimized pages....Optimization reduces it by 4,2 hrs Total time Or....8,1Gb is the total non-optimized pagesize....Using Savings optimization this can be reduced with 1,2 Gb 85%

The average person in Europe spends 34,4 hours online per month and visits 3515 pages Source: Adformatie, March 2011

Just imagine.....

What you can do....

...with all the extra time

Unfortunately, it’s not a push of the button

Your mileage may vary....

There’s the website itself Place: Place: View View order my account Task: history Task: Create Log Inaccount Task: Place: Upgrade Task: Read account Buy review stuff comments

Browsers Location Browser versions Last mile Type of productUsed technology Many variables to take into account

.....and so will your test results

Fully loaded, Non-Optimized Fully loaded, OptimizedComputerbild.de NBC.comVoyages-sncf.com

DIY vs. Performance Automation

Beneﬁts of automationOptimizes all pages automatically with a set ofstandard treatments;Quick and (can be) easy to install;Provides continuous optimization;Creates performance baseline, all pages aremade as fast as possible

But...

Downside of automation You need to understand your website to get the most out of automation; Not all treatment settings are good for your website; Automation needs customization to reach full potential; 3d Party content is not optimized ;-)

My advice: It’s both

Takeaways...

Start benchmarking multiple options

1. Determine your business targets

1. Determine your business targets 2. Create a test environment

1. Determine your business targets 2. Create a test environment 3. Select measurement tool

1. Determine your business targets 2. Create a test environment 3. Select measurement tool4. Connect to automation providers

1. Determine your business targets 2. Create a test environment 3. Select measurement tool4. Connect to automation providers 5. Test, Correlate & Compare

It’s about the customer journey

Is this bad?....Ordering Car Insurance Source: Gomez

Is this bad?....1: Landing page 4: Enter insurance details2: Click order 5: Calculate offer3: Enter personall details 6: Order insurance Source: Gomez

Business case: Focus on attributing KPI’s

Cart abandonments

http://www.ﬂickr.com/photos/ mrmoorey/160654236Bounce rate

Trafﬁc / Page views

Time on site

And remember....

Publish data online Start repeating tests with speciﬁc scenariosStress-test the optimizersExpand to different vendors and/or solutions

Questions?

Thanks! More questions?M: jtjepkema@measureworks.nlT: @jeroentjepkemaW: www.measureworks.nl

http://velocityconf.com	160
http://www.scoop.it	102
http://us-w1.rockmelt.com	4
http://paper.li	2
http://translate.googleusercontent.com	2
http://tweetedtimes.com	1
http://a0.twimg.com	1
http://lanyrd.com	1

MeasureWorks - Velocity Conference Europe - Performance Automation 101

by MeasureWorks on Nov 08, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

8 Embeds 273

Statistics

MeasureWorks - Velocity Conference Europe - Performance Automation 101 — Presentation Transcript