Your SlideShare is downloading. ×
Test Expo 2009   Site Confidence & Seriti Consulting   Load Test Case Study
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Test Expo 2009 Site Confidence & Seriti Consulting Load Test Case Study

1,398

Published on

Presentation given at the 2009 TestExpo in London www.testexpo.co.uk

Presentation given at the 2009 TestExpo in London www.testexpo.co.uk

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,398
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. SERITI CONSULTING Load Testing Case Study Some hints and tips on load-testing your website... 1 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 2. Agenda  Introductions  Before you start  Designing the test  On the Day…  Analysing the results and taking action  The vendor bit…  Questions? 2 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 3. SERITI CONSULTING Before you start Working out what to test and how much to test it 5 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 4. Why do you want to load test?  New website still under development?  Existing website about to head into a business peak period?  New marketing campaign about to launch?  The Met Office is predicting snow... 6 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 5. “Load test” vs. “Performance Test”  Performance vs Load Test?  This affects: How you will do it What you will do during the test Who needs to be involved What information you record! Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 7
  • 6. Load vs. Performance?  Load Test = quot;Black Boxquot; I only care if the whole system can handle the load If it can't I probably don't have a lot of clues as to why not...  Performance Test = quot;White Boxquot; Looking inside the App and Infrastructure architecture to identify the key elements Designing tests to stress each element Monitoring those key elements during the test Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 8
  • 7. Test In-house or use 3rd party?  Project triangle decision Time Money Scope  Resources required... Testing software e.g. LoadRunner, Selenium, Site Confidence People to script it Hardware & network to generate the load Accessibility of test environments (or use Prod) Reporting interface to get the right stats & graphs People to interpret the results... 9 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 8. SERITI CONSULTING Designing the test Working out what to test and how much to test it 1 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 0
  • 9. Decide on your critical user journeys  Home page  Search pages  Details pages  Upload/download  Checkout/e-commerce  SEO Crawl – Google can hammer sites! Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 12
  • 10. Set your targets  How many transactions/sec?  How many concurrent users?  What is the mix of the user journeys? Don't underestimate the impact of concurrence...  What's your quot;safety marginquot; We used 40% Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 13
  • 11. TIP - Web Analytics is your BFF  Leverage your site metrics! Google Analytics, WebTrends or Omniture etc Use the information to identify your peak loads across different user journeys It is very common for people to test their sites at very unrealistic rates... Or focus on the wrong parts… • Don’t forget deep linking! Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 14
  • 12. TIP – Management Speak..  The Management tends to ask “how many users will the site support”…  How many users do what, how often? Users on playing online poker are different to users reading “The Register”…  I find it best to talk about “page impressions” since that is a metrics they are probably familiar with by now… Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 15
  • 13. Concurrent users & “think time”  Take the number of page impression per hour  Divide by 3600  That gives you the page impressions (quot;transactionsquot;) per second target  Then take the number of quot;concurrent usersquot; and divide by the quot;think timequot; (dwell time) to give you transactions a second  1200 users with a 60s think time = 20 transactions /sec  600 users with 30s think time = 20 transactions /sec  300 users with 15s think time = 20 transactions /sec  100 users with a 5s think time = 20 transactions /sec Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 16
  • 14. Design the test… Scripts Profiles Scenarios  Scripts - the actual user journey (“clicks”)  Profiles - how many, how fast, how long etc  Scenarios - combining profiles to match the traffic patterns on your site… The “mix” of user journeys at any one time Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 17
  • 15. What does a Profile look like? Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 19
  • 16. 3 Quick Tips…  Baseline your network first  Do a “GET” for a 100K JPEG and see how many you get before you saturate your network  Do “single profile” scenarios first  Work from simple to complex  Test each element before you try combinations  Think about “keyhole” tests to isolate specific application functions  Rather than simulate the “click by click” of the user just “POST” (form submit) directly to the site  This enables you to focus on the app performance rather than the UI elements (you need to do both though!) Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 21
  • 17. One more tip…  DON’T FORGET YOUR BATCH PROCESSES  How does the system perform during… • Backups • Re-indexes • ETL activities • Any other batch process that stresses the system…  Many a site dies when these processes run  And if you are truly 24x7 online – when is your window? Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 22
  • 18. On the day – running the test! Who does what, where & when? 23 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 19. Plan out the timetable!  Make the most of your testing window…  Plan carefully but BE REALISTIC!  Allow a lot of time for review between each scenario run...  You WILL change your plans on the day... Be flexible!  Get the right people – Operations, QA, Network, Database etc Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 24
  • 20. On the day...  DON’T - Suddenly remember that you forgot to tell your hosting provider...  And watch their IDS system think it was a DDOS attack and drop all your packets...  DO – keep an eye on the time (and ensure that time is sync’s across the systems)  You will want to correlate events!  DO – use a meaningful name for each run and add comments  So you know how many users are added, over what ramp up period, and run for how long at a glance  DO – think about cleaning up after you!  If you have been running against Prod you might have created a lot of dummy transactions (and your web logs will be HUGE!) Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 25
  • 21. SERITI CONSULTING Analysing the results and taking action Data -> Information -> Knowledge -> Action 26 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved
  • 22. Lots of questions…  Did you hit your targets?  Where there any odd peaks/troughs?  Was “mixed scenario” performance different?  Was performance consistent across test runs?  Did your releases make things better (or worse)?  What was the Max/Mean/Min values  And what was the variance? 2 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 7
  • 23. Before and After a Network upgrade… Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 28
  • 24. Lower & longer is better Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 29
  • 25. Faster response, with more users… Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 30
  • 26. Taking action…  Use the performance stat (if you gathered any) to isolate the weak links Upgrade/re-factor & re-test  Use the (bad) stats to get more budget to fix the problem…  Use the (good) stats to get more a bonus because you did such a good job… 3 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 1
  • 27. SERITI CONSULTING The vendor bit… Over to James…  3 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 2
  • 28. Site Confidence Clients… Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 34
  • 29. Contact Details – Site Confidence  Talk to Site Confidence on their stand at the TestExpo after the presentation or if you would like a copy of the presentation  Email James – James.Bavington@SiteConfidence.com  Visit their website - www.siteconfidence.com 3 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 5
  • 30. Contact Details – Seriti Consulting  If you have any questions around  web hosting (IIS, Cloud)  Performance tools (Appmetrics, Avicode)  Operations & ITIL (service-now.com)  Configuration Management (Configuresoft ECM)  Near-shore development, functional test (Endava)  Email me – Stephen.Thair@SeritiConsulting.com  Connect with me on LinkedIn  http://www.linkedin.com/in/stephenthair 3 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 6
  • 31. Any Questions? 3 Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved 7

×