eCommerce Performance
What is it costing you, and what
can you do about it?
Peter Holditch
Technologist

pholditch@appdyna...
The Business Impact of One Second

“One second increase in
Amazon‟s page load
would annually cost $1.6
billion in sales”
B...
Because a 1 second delay equates to…

A 16% decrease in customer satisfaction

11% fewer page views

A 7% loss in conversi...
Google and Microsoft research

• Experiments to introduce delay into web
searches to measure the impact

http://velocityco...
50ms
200ms
-0.3% -0.4%
500ms
-0.6% -1.2% -1.0% -0.9%
1000ms -0.7% -0.9% -2.8% -1.9% -1.6%
2000ms -1.8% -2.1% -4.3% -4.4% -...
Impact measured by

• Slower performance  abandoned searches
• More active users more sensitive to this
• Effect got wors...
Conclusion

• Revenue is a function of user behaviour
• User behaviour is quite sensitive to
performance
• Effects of poor...
This is made very hard by the modern technology landscape

Distributed
Monolithic

Release 1.1
Release 1.2
Release 1.23
To...
Where and what is the problem?
Release 1.1
Release 1.2
Release 1.23
Tomcat Release 1.5

.NET

Amazon EC2
Windows Azure

CL...
Where and what is the problem?
Release 1.1
Release 1.2
Release 1.23
Tomcat Release 1.5

.NET

Amazon EC2
Windows Azure

CL...
What if the problem is outside the application?
Release 1.1
Release 1.2
Release 1.23
Tomcat Release 1.5

.NET

Amazon EC2
...
Real-User Monitoring gets Real Results*

>91% transaction
completion

End-users
„completely satisfied‟

12

*Source: Aberd...
And beyond performance monitoring…
Release 1.1
Release 1.2
Release 1.23
Tomcat Release 1.5

.NET

Amazon EC2
Windows Azure...
Case Study – One Year

Pre-Production

Dev

Production

QA

• Agile Releases 12 > 18
• Spent 3,060 hours less firefighting...
Want to learn more?

Visit the AppDynamics Booth
Near the eCommerce Theatre and Mobile &
Social Media World Theatre

Downl...
Thank You!
Peter Holditch
Technologist

pholditch@appdynamics.com
Upcoming SlideShare
Loading in …5
×

eCommerce Performance: What is it costing you, and what can you do about it? AppDynamics

469 views
354 views

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
469
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • A study by Borland identified an overwhelming correlation between sales-generated traffic rises and increases in website response times – a nightmare situation for any retailer hoping to capitalize on the seasonal online rush of bargain-hunting consumers. Research has shown that even minor delays to website response times can have a sizable impact on customer satisfaction, page views, conversion rates and site abandonment. A one second delay in website response time equals11% fewer page views,16% decrease in customer satisfaction and a 7% loss in conversions.The study thus concludes that a one second increase in Amazon’s page load would annually cost $1.6 billion in sales, and  38% of UK online shoppers abandon websites or apps that take more than 10 seconds to load.The average online shopper expects web pages to load in 2 seconds or less, after 3 seconds, up to 40% will abandon the site. Seventy four per cent of users will abandon a mobile site after waiting only five seconds for it to load.Once visitors leave, it’s very difficult to get them back.  88% of online consumers are less likely to return to a site after a bad experience.Play.com, the UK arm of the Rakuten Group, saw performance drop by 500% as its site slowed from a load time of 2 seconds to 12 when site traffic peaked on the 4th January. Other online retailers that also suffered significant increases in load times during the first few days of the January sales included John Lewis, Amazon.co.uk, Asos.com and Tesco.com. Increases ranged between 3 and 4.5 seconds for their landing page to load.“There is lots of data available showing that users are losing patience with poor performing websites,” said Archie Roboostoff, product director at Borland. “It looks like a number of the sites monitored over the seasonal period will have missed out on potential revenue as a result of their website’s inability to process high levels of traffic. The sites we monitored in the UK had normal load times averaging 2.9 seconds, but saw load times increase by an average of 4.5 seconds during peak traffic periods – a 55% deterioration.Developing a robust performance strategy takes time, and peak period preparation should begin early with testing starting about six months beforehand. Putting in this groundwork is crucial if retailers are to take full advantage of peak shopping times throughout the year.”http://www.retail-digital.com/retail_technology/one-second-delay-on-amazon-16-billion-loss-a-year[source data: http://www.aberdeen.com/aberdeen-library/5136/RA-performance-web-application.aspx]
  • http://velocityconf.com/velocity2009/public/schedule/detail/8523
  • The application landscape is complex, and so is the transaction landscapeSome transactions will be more important to track than others – with conventional monitoring it’s impossible to focus on the important things, and impossible to understand if monitoring anomalies have any business impactMoreover, it’s impossible to troubleshoot the important things – just
  • Find the point of a problem quicklyGather enough detail to troubleshoot it in situDo the same during development, to avoid issues getting to production
  • Find the point of a problem quicklyGather enough detail to troubleshoot it in situDo the same during development, to avoid issues getting to production
  • Find the point of a problem quicklyGather enough detail to troubleshoot it in situDo the same during development, to avoid issues getting to production
  • http://v1.aberdeen.com/launch/report/perspective/8371-AI-application-performance-management.asp?lan=US
  • Find the point of a problem quicklyGather enough detail to troubleshoot it in situDo the same during development, to avoid issues getting to production
  • Objective of SlideHighlight our value proposition across Development, QA, Operations and the business.ScriptFor example, here’s a customer case study from Edmunds.com which highlights the annual benefits of AppDynamics across their organization and lifecycle.Development was able to double their innovation as a result of spending less time firefighting, and implementing more business requirements.QA were able to detect performance defects twice as fast, therefore increasing testing productivity and accelerating time to market.Operations increased application availability by .04%, and cut MTTR in half which had a significant impact on the business.All these benefits translated an enhanced end user experience combined with significant lost revenue and productivity annual savings totaling almost $800,000.Bank of New Zealand, Expedia and Fox News also had similar savings to Edmunds.com.
  • Objective of SlideHighlight our value proposition across Development, QA, Operations and the business.ScriptFor example, here’s a customer case study from Edmunds.com which highlights the annual benefits of AppDynamics across their organization and lifecycle.Development was able to double their innovation as a result of spending less time firefighting, and implementing more business requirements.QA were able to detect performance defects twice as fast, therefore increasing testing productivity and accelerating time to market.Operations increased application availability by .04%, and cut MTTR in half which had a significant impact on the business.All these benefits translated an enhanced end user experience combined with significant lost revenue and productivity annual savings totaling almost $800,000.Bank of New Zealand, Expedia and Fox News also had similar savings to Edmunds.com.
  • eCommerce Performance: What is it costing you, and what can you do about it? AppDynamics

    1. 1. eCommerce Performance What is it costing you, and what can you do about it? Peter Holditch Technologist pholditch@appdynamics.com
    2. 2. The Business Impact of One Second “One second increase in Amazon‟s page load would annually cost $1.6 billion in sales” Borland Research - March 2013
    3. 3. Because a 1 second delay equates to… A 16% decrease in customer satisfaction 11% fewer page views A 7% loss in conversions 3
    4. 4. Google and Microsoft research • Experiments to introduce delay into web searches to measure the impact http://velocityconf.com/velocity2009/public/schedule/detail/8523 http://vimeo.com/5310021 4
    5. 5. 50ms 200ms -0.3% -0.4% 500ms -0.6% -1.2% -1.0% -0.9% 1000ms -0.7% -0.9% -2.8% -1.9% -1.6% 2000ms -1.8% -2.1% -4.3% -4.4% -3.8% - Means no statistically significant change li cre ase ck in m s) oC (in et Tim on isfa c ti Sat Dis t in Qu c t e ri es/ Us Qu er e ry Re fin em en Re t ven ue /U ser An yC lick s Server Delays Experiment: Results 500 1200 1900 3100 • Strong negative impacts • Roughly linear changes with increasing delay • Time to Click changed by roughly double the delay
    6. 6. Impact measured by • Slower performance  abandoned searches • More active users more sensitive to this • Effect got worse over time, and persisted once performance was restored wk1 wk2 delay removed 0.2% 0% 200 ms delay 400 ms delay -1% -0.8% -0.6% -0.4% -0.2% 0% -0.8% -0.6% -0.4% -0.2% actual trend daily searches per user relative to control 0.2% Persistent Impact of Post-header Delay 200 ms delay 400 ms delay -1% daily searches per user relative to control Impact of Post-header Delays Over Time wk3 wk4 wk5 wk6 actual trend wk3 wk4 wk5 wk6 wk7 wk8 wk9 wk10 wk11 6
    7. 7. Conclusion • Revenue is a function of user behaviour • User behaviour is quite sensitive to performance • Effects of poor performance outlast the problems • It is necessary to have a constant watch on performance of critical transactions, fix problems quickly and continuously improve over time 7
    8. 8. This is made very hard by the modern technology landscape Distributed Monolithic Release 1.1 Release 1.2 Release 1.23 Tomcat Release 1.5 .NET Amazon EC2 Windows Azure CLOUD Release 2.4 Release 2.5 Release 2.6 Release 3.0 Login Search Flight View Flight Status Make Reservation Tomcat Mule, Tibco, AG Tomcat ESB VMWare WEB 2.0 Memcached Oracle Weblogic Release 1.4 Release 1.5 Release 1.6 Release 2.0 Browser Logic AJAX Web Frameworks Coherence Hadoop Cassandra MongoDB SOA .NET MQ AGILE Release 3.4 Release 3.5 Release 3.6 Release 4.0 SQL Server Release 4.4 Release 4.5 Release 4.6 Release 5.0 JBoss Release 1.4 Release 1.5 Release 1.6 Release 2.0 BIG DATA ATG, Vignette, Sharepoint 8
    9. 9. Where and what is the problem? Release 1.1 Release 1.2 Release 1.23 Tomcat Release 1.5 .NET Amazon EC2 Windows Azure CLOUD Release 2.4 Release 2.5 Release 2.6 Release 3.0 Login Search Flight View Flight Status Make Reservation Tomcat Mule, Tibco, AG Tomcat ESB VMWare WEB 2.0 Memcached Oracle Weblogic Release 1.4 Release 1.5 Release 1.6 Release 2.0 Browser Logic AJAX Web Frameworks Coherence Hadoop Cassandra MongoDB SOA .NET MQ AGILE Release 3.4 Release 3.5 Release 3.6 Release 4.0 SQL Server Release 4.4 Release 4.5 Release 4.6 Release 5.0 JBoss Release 1.4 Release 1.5 Release 1.6 Release 2.0 BIG DATA ATG, Vignette, Sharepoint 9
    10. 10. Where and what is the problem? Release 1.1 Release 1.2 Release 1.23 Tomcat Release 1.5 .NET Amazon EC2 Windows Azure CLOUD Release 2.4 Release 2.5 Release 2.6 Release 3.0 Login Search Flight View Flight Status Make Reservation Tomcat Mule, Tibco, AG Tomcat ESB VMWare WEB 2.0 Memcached Oracle Weblogic Release 1.4 Release 1.5 Release 1.6 Release 2.0 Browser Logic AJAX Web Frameworks Coherence Hadoop Cassandra MongoDB SOA .NET MQ AGILE Release 3.4 Release 3.5 Release 3.6 Release 4.0 SQL Server Release 4.4 Release 4.5 Release 4.6 Release 5.0 JBoss Release 1.4 Release 1.5 Release 1.6 Release 2.0 BIG DATA ATG, Vignette, Sharepoint 10
    11. 11. What if the problem is outside the application? Release 1.1 Release 1.2 Release 1.23 Tomcat Release 1.5 .NET Amazon EC2 Windows Azure CLOUD Release 2.4 Release 2.5 Release 2.6 Release 3.0 Login Search Flight View Flight Status Make Reservation Tomcat Mule, Tibco, AG Tomcat ESB VMWare WEB 2.0 Memcached Oracle Weblogic Release 1.4 Release 1.5 Release 1.6 Release 2.0 Browser Logic AJAX Web Frameworks Coherence Hadoop Cassandra MongoDB SOA .NET MQ AGILE Release 3.4 Release 3.5 Release 3.6 Release 4.0 SQL Server Release 4.4 Release 4.5 Release 4.6 Release 5.0 JBoss Release 1.4 Release 1.5 Release 1.6 Release 2.0 BIG DATA ATG, Vignette, Sharepoint 11
    12. 12. Real-User Monitoring gets Real Results* >91% transaction completion End-users „completely satisfied‟ 12 *Source: Aberdeen Group, July 2012 >30% increase in App Availability Businesses NOT doing Real User Monitoring Businesses doing Real User Monitoring >10% decrease in end-user complaints
    13. 13. And beyond performance monitoring… Release 1.1 Release 1.2 Release 1.23 Tomcat Release 1.5 .NET Amazon EC2 Windows Azure CLOUD Release 2.4 Release 2.5 Release 2.6 Release 3.0 Login Search Flight View Flight Status Make Reservation Tomcat Mule, Tibco, AG Tomcat ESB VMWare WEB 2.0 Memcached Oracle Weblogic Release 1.4 Release 1.5 Release 1.6 Release 2.0 Browser Logic AJAX Web Frameworks Coherence Hadoop Cassandra MongoDB SOA .NET MQ AGILE Release 3.4 Release 3.5 Release 3.6 Release 4.0 SQL Server Release 4.4 Release 4.5 Release 4.6 Release 5.0 JBoss Release 1.4 Release 1.5 Release 1.6 Release 2.0 BIG DATA ATG, Vignette, Sharepoint 13
    14. 14. Case Study – One Year Pre-Production Dev Production QA • Agile Releases 12 > 18 • Spent 3,060 hours less firefighting • Delivered More Innovation Ops Business • Availability 99.91% > 99.95% • MTTR 40 hours > 22 hours • 1,528 hours less troubleshooting • Identify & Fix Defect 20 hours > 13 hours • Spent 4,024 hours less testing • Faster Time to Market • • • • End User Experience 500ms > 150ms $167,475 lost revenue savings $627,691 productivity savings $795,166 Total savings 14
    15. 15. Want to learn more? Visit the AppDynamics Booth Near the eCommerce Theatre and Mobile & Social Media World Theatre Download free monitoring solution: Bit.ly/IWLite Follow @AppDynamics 15
    16. 16. Thank You! Peter Holditch Technologist pholditch@appdynamics.com

    ×