Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Making More Happen –
How to Measure Revenue in Milliseconds
Cliff Crocker
@cliffcrocker
SOASTA
VP Product, mPulse
Performance is a Business
Problem
Funnel Optimization is Everything
• Merchandizing
• SEO
• Multivariate testing
• Pricing
• Ad Campaigns & targeting
• User...
Yahoo! - 2008
Increase of 400ms
causes 5-9% increase in
user abandonment
http://www.slideshare.net/stubbornella/designing-...
Shopzilla - 2009
A reduction in Page Load
time of 5s increased site
conversion 7-12%!
http://assets.en.oreilly.com/1/event...
Walmart - 2012
http://minus.com/msM8y8nyh#1e
SF WebPerf –
2012
Up to 2%
conversion
increase for every
second
SIMULATION
Unfortunately, it’s not always
that straightforward
Not All Pages are Created Equal
For a typical
eCommerce site,
conversion rate
drops by up to 50%
when “browse”
pages incre...
Not All Pages are Created Equal
However, there is
much less impact
to conversion
when “checkout”
pages degrade
Getting Started
• Organizational challenges
• Process challenges
• Visibility
• Expertise
• Focus & prioritization
Steve Skroce
Staples
Sr. Manager, Site Operations
steven.skroce@staples.com
How Did I Get Here
Developer SEO Performance
SiteSpeed & Staples
• A top goal for Staples eCommerce
• It’s about customer experience
• Speed is essential to UX
• Becau...
First Things First
• You can’t manage what you can’t measure
• Earliest work involved selecting the proper tools
• Busines...
First Things First
• Assembling a cross-functional team
• Business
• Decides which features and functionality lives on the...
Why Is Speed Important to Staples?
Bounce Rate increases as load time increases
Why Is Speed Important to Staples?
Conversion rate decreases as load time increases
The First Pass
• Governance meetings to evaluate functionality
• Cataloged all site features
• Weekly meeting to discuss o...
The First Pass
• Onsite promotional banners were too “heavy”
• Inconsistent image optimization
• Depended on the individua...
The First Pass
• Testing practices were inefficient
• Two vehicles for A/B testing
• Eliminate one
• Process improvement o...
The First Pass
• Needed “sanity” around third party calls
• Categorization of calls
• Ads
• Marketing
• Analytics
• Recomm...
The First Pass
• Database calls were inefficient
• Some pages went from 200 calls to 10
Evolution
• Go bolder, sooner
• Sprint model allowed us to deliver something every release
• Something, not something subs...
How fast are we, really?
• Examples of pagespeed for Staples
• Median vs. percentiles
• Make it simple for a wide audience...
Make Faster Page Load Times Happen
• mPulse
• WebPagetest.org
• Dev Tools
• NetPanel
• “Motivational” emails from the top
Make ‘Faster Page Load Times’ Happen
• Confirmation of what we already knew:
• Front-end vs. back-end time
X
JS refactoring
Reduced k size by 48%
Reduced lines of code by 52%
CSS refactoring
Reduced k size by 83%
Reduced lines of c...
Early Results
• Shaved 1 full second off of Home page median
• Shaved 6 seconds off of the 98th%
• Conversion improved by ...
Upcoming SlideShare
Loading in …5
×

Velocity NY - How to Measure Revenue in Milliseconds

5,096 views

Published on

Cliff from SOASTA and Steve from Staples discuss the three questions: How fast are you? How fast should you be? How do you get there? An overview of real world performance optimization and RUM.

Published in: Internet

Velocity NY - How to Measure Revenue in Milliseconds

  1. 1. Making More Happen – How to Measure Revenue in Milliseconds
  2. 2. Cliff Crocker @cliffcrocker SOASTA VP Product, mPulse
  3. 3. Performance is a Business Problem
  4. 4. Funnel Optimization is Everything • Merchandizing • SEO • Multivariate testing • Pricing • Ad Campaigns & targeting • User Experience Performance IS User Experience!!
  5. 5. Yahoo! - 2008 Increase of 400ms causes 5-9% increase in user abandonment http://www.slideshare.net/stubbornella/designing-fast-websites-presentation
  6. 6. Shopzilla - 2009 A reduction in Page Load time of 5s increased site conversion 7-12%! http://assets.en.oreilly.com/1/event/29/Shopzilla%27s%20Site%20Redo%20- %20You%20Get%20What%20You%20Measure%20Presentation.ppt
  7. 7. Walmart - 2012 http://minus.com/msM8y8nyh#1e SF WebPerf – 2012 Up to 2% conversion increase for every second
  8. 8. SIMULATION
  9. 9. Unfortunately, it’s not always that straightforward
  10. 10. Not All Pages are Created Equal For a typical eCommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds
  11. 11. Not All Pages are Created Equal However, there is much less impact to conversion when “checkout” pages degrade
  12. 12. Getting Started • Organizational challenges • Process challenges • Visibility • Expertise • Focus & prioritization
  13. 13. Steve Skroce Staples Sr. Manager, Site Operations steven.skroce@staples.com
  14. 14. How Did I Get Here Developer SEO Performance
  15. 15. SiteSpeed & Staples • A top goal for Staples eCommerce • It’s about customer experience • Speed is essential to UX • Because Staples is highly matrixed • Visibility into the organization is key • Approach as a discipline and institutionalize the process
  16. 16. First Things First • You can’t manage what you can’t measure • Earliest work involved selecting the proper tools • Business and Engineering led team • Already had synthetic transaction monitoring tool • Backend tools were also available • RUM tool was missing from the set • Needed RUM info if we were going to affect our customers’ experiences
  17. 17. First Things First • Assembling a cross-functional team • Business • Decides which features and functionality lives on the site • Engineering • How something gets built can make or break page load time • Marketing • Asset optimization is important • Merchandising • On-site content • Analytics • Analytics team ties everything together • And others…
  18. 18. Why Is Speed Important to Staples? Bounce Rate increases as load time increases
  19. 19. Why Is Speed Important to Staples? Conversion rate decreases as load time increases
  20. 20. The First Pass • Governance meetings to evaluate functionality • Cataloged all site features • Weekly meeting to discuss one or two features in detail • Feature owner comes prepared to review all aspects • Full Page Load vs. Time to Interactive • How Staples defines them • FPL ensured wide exposure / awareness through the organization • Everyone was involved from the beginning • Set the expectation of accountability
  21. 21. The First Pass • Onsite promotional banners were too “heavy” • Inconsistent image optimization • Depended on the individual, not the process • 25k and 250k • Put a new process in place
  22. 22. The First Pass • Testing practices were inefficient • Two vehicles for A/B testing • Eliminate one • Process improvement opportunity for ending campaigns • Eliminate lingering artifacts
  23. 23. The First Pass • Needed “sanity” around third party calls • Categorization of calls • Ads • Marketing • Analytics • Recommendations • Social Buttons • Etc. • Removed some calls, optimized others • Further work needs to be done
  24. 24. The First Pass • Database calls were inefficient • Some pages went from 200 calls to 10
  25. 25. Evolution • Go bolder, sooner • Sprint model allowed us to deliver something every release • Something, not something substantial • Not enough time for big, bold changes • We needed to step-back and reevaluate
  26. 26. How fast are we, really? • Examples of pagespeed for Staples • Median vs. percentiles • Make it simple for a wide audience to understand the metrics • Key page performance – target specific pages to focus on • Home, Product, Search, Class, Login, Cart
  27. 27. Make Faster Page Load Times Happen • mPulse • WebPagetest.org • Dev Tools • NetPanel • “Motivational” emails from the top
  28. 28. Make ‘Faster Page Load Times’ Happen • Confirmation of what we already knew: • Front-end vs. back-end time X
  29. 29. JS refactoring Reduced k size by 48% Reduced lines of code by 52% CSS refactoring Reduced k size by 83% Reduced lines of code by 88%
  30. 30. Early Results • Shaved 1 full second off of Home page median • Shaved 6 seconds off of the 98th% • Conversion improved by roughly 10%

×