Cutting-edge Performance Testing on eCommerce Websites
Upcoming SlideShare
Loading in...5
×
 

Cutting-edge Performance Testing on eCommerce Websites

on

  • 172 views

Having problems with your website’s performance? Does it take too much time and effort to determine the cause of a particular page’s poor performance? Would you like to find the root cause of ...

Having problems with your website’s performance? Does it take too much time and effort to determine the cause of a particular page’s poor performance? Would you like to find the root cause of client-side issues in an automated way? If you answered yes to any of these questions, then this session is for you. At GSI Commerce, an eBay company, Ron Woody manages a large team of performance engineers working on nearly nearly 100 eCommerce websites. Ron and his team have developed cutting-edge approaches for automating client- and server-side performance testing. Learn the specific approaches Ron’s team uses today for pre-release performance tests, production performance management, and website optimization. Find out the ways they’ve automated cross-browser performance testing—and analysis—to increase productivity and efficiency. Covering these and additional topics Ron shares a toolkit of performance testing ideas and approaches your team can use to ensure optimal application performance and a better user experience.

Statistics

Views

Total Views
172
Views on SlideShare
168
Embed Views
4

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 4

http://www.stickyminds.com 4

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cutting-edge Performance Testing on eCommerce Websites Cutting-edge Performance Testing on eCommerce Websites Document Transcript

  • W11 Performance Testing 5/1/2013 1:45:00 PM Cutting-edge Performance Testing on eCommerce Websites Presented by: Ron Woody GSI Commerce Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • Ron Woody Senior manager of QA performance at GSI Commerce (an eBay company), Ron Woody manages a team of twenty-one performance engineers dedicated to ensuring that web-based performance is within targeted requirements for all client sites and launches. Ron brings more than fifteen years of automation, performance, QA management, and process experience to his current role. He heads the effort to improve performance for all web stores released and supported by GSI while providing performance testing and engineering vision to the company. Ron has served as a consultant to companies such as Comcast, Bloomberg, Vanguard, and Bank of America.
  • 4/16/2013 Automating Client- and ServerClientServerSide Web Performance Testing and Optimization Ron Woody Director of Performance GSI Commerce, an eBay Inc. company |1 Who is GSI Commerce? |2 1
  • 4/16/2013 About GSI Commerce ■ Companies of all sizes have selected GSI Commerce® to help deliver an exceptional and engaging eCommerce shopping experience to their customers. |3 GSI Commerce: QA Performance Processes ■ Performance Testing – Platform/Product – Projects – Store Launches ■ Client-Side Testing ■ Server-Side Testing ■ Performance Analysis |4 2
  • 4/16/2013 Business Impact of Performance ■ Performance impacts sales! ■ Poor performance increases the risk of site abandonment ■ Slow performance may impact brand – People remember their experience when a site is slow; Users may be reluctant to return ■ Our clients’ competitors are only a click away |5 Recent Performance Testing Objectives ■ Phase 1: Improve the end-user experience via client-side and server-side testing ■ Phase 2: Increase productivity via Automation ■ Phase 3: Create more realistic client-side test scenarios and enhance our testing further |6 3
  • 4/16/2013 Phase 1 – Improve end-user experience via endclientclient- and server-side testing server■ Client-Side Testing – dynaTrace AJAX Edition – Improve the End-User Experience ■ Server-Side Testing – dynaTrace Development & Test Center Editions – Diagnosing server-side bottlenecks ■ End-to-End visibility |7 ClientClient-Side Testing ■ Started with dynaTrace’s free AJAX Edition – More accurate rendering times; greater visibility |8 4
  • 4/16/2013 ServerServer-Side Testing ■ Added dynaTrace Development Team & Test Center Editions |9 Phase 2 – Increase Productivity via Automation ■ Server-side Automation – LoadRunner with dynaTrace Test Edition – dynaTrace AJAX Edition + Test Center Editions provide in-depth views ■ Client-side Automation ■ Use of dynaTrace AJAX API’s to extend our reporting | 10 5
  • 4/16/2013 Benefits of using dynaTrace: Visibility ■ dynaTrace provided visibility we hadn’t had before – But testing browser use-cases was manual, and very time-consuming ■ Increasing Productivity via Automation – LoadRunner / Performance Center – QuickTest Professional / Selenium – Quality Center / ALM | 11 Solution: Automated Browser Lab ■ Virtualized browser lab to capture the end-user experience with the touch of a button ■ Cutting-edge automated home-grown solution – – – – – dynaTrace AJAX Edition Selenium Google Page Speed Webpagetest.org Shunra network emulation tools ■ Real-Time reporting system to show results | 12 6
  • 4/16/2013 GSI Commerce’s Browser Lab | 13 GSI Commerce’s Browser Lab | 14 7
  • 4/16/2013 Why dynaTrace? ■ Visibility into Rendering – As well as other Web 2.0 challenges – Was not available in HTTPWatch ■ IE 7,8,9 support – Firebug didn’t support IE ■ Now dynaTrace supports Firefox as well | 15 Basic Browser Lab Diagram | 16 8
  • 4/16/2013 GSI Commerce’s Browser Lab Front-End Front- | 17 A Sample Browser Lab Report | 18 9
  • 4/16/2013 A Sample Browser Lab Report | 19 Using Automation to Save Time and Money ■ With automation, GSI saved hours per test – Reducing test time from 20 hours to ~1 hour ■ Nearly a 20x time savings ■ Reproducible results ■ Shortens project time – Allows for more iterative testing | 20 10
  • 4/16/2013 Additional Benefits of our Browser Lab ■ Pinpoint problems quickly – Problems that were difficult / impossible to see before – Full Waterfall / timeline – Rendering – DOM Manipulations ■ Allows us to be proactive ■ Leads to a faster site ■ Enables clients to make informed decisions based on the results of our testing in QA | 21 Benefits of dynaTrace: Better Collaboration ■ Dev and QA collaboration – dynaTrace AJAX Edition sessions shared with Development – Now Development sees exactly what Test sees – Less back & forth – Faster resolution / optimization ■ Dev Teams can now use dynaTrace prior to QA ■ Developed Performance Requirements that include dynaTrace rankings – Dev teams now design to these requirements | 22 11
  • 4/16/2013 Phase 3 – Create more realistic client-side clienttest scenarios and enhance our testing further ■ Network emulation for more realistic traffic ■ Larger samplings with automation ■ Browser weighting to generate more accurate timings across browsers ■ Automatic removal of outliers | 23 Performance Objectives Revisited (ROI) ■ Phase 1: Improve the end-user experience – ROI: We now have visibility into performance issues and how to resolve them quickly and more efficiently thus saving us time and money ■ Phase 2: Increase productivity via Automation – ROI: ~20x savings with our Browser Lab solution – can now do testing and analysis simultaneously – faster time to market; more complete testing ■ Phase 3: Create more realistic test scenarios – ROI: More accurate timings through browser weighting and network emulation; increased consistency by reducing noise in our results | 24 12
  • 4/16/2013 Recent additions ■ Leveraging dynaTrace browser agent (AJAX Edition Premium Version) – Ability to see end-to-end performance in QA (PurePath) via Performance Center scripts ■ Use of dynaTrace (Production Edition) to become even MORE proactive – Ability to find issues that exist in Production (including issues that clients haven’t reported yet) – Determine where users are abandoning our sites – Ability to monitor Apache | 25 What’s Next? ■ Browser Lab aspirations – – – – Mobile testing Cached vs. Non-Cached pages Film strip Integration with webpagetest.org ■ Expand to Continuous Integration – Enable Dev to find/fix problems before they reach QA | 26 13
  • 4/16/2013 GSI Commerce’s Performance Best Practices ■ Performance matters! ■ Define Performance Targets ■ Test both client-side AND server-side performance – Cross-browser testing – Add server-side testing – Tie everything together | 27 GSI Commerce’s Performance Best Practices ■ Automate! ■ Get Test and Development on the same page ■ Get Proactive ■ Benchmark your site against your competition | 28 14
  • 4/16/2013 QUESTIONS? | 29 15