Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Webinar: Load Testing for Your Peak Season


Published on

With end of year just around the corner, many companies are entering their peak season, whether it's back-to-school, holiday shopping, end-of-year and next year planning.

If you don't know what to test, you might miss key paths through your site and customers might experience poor performance on your web and mobile site and apps. Save yourself multiple headaches and join us to learn not only what to test but how to test your web and mobile performance before your peak demand.

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Webinar: Load Testing for Your Peak Season

  1. 1. ©2016 SOASTA, All rights reserved. Tom Chavez @tomchavez Mike Ostenberg @mikeostenberg Load testing for peak season: What to test and how to test
  2. 2. ©2016 SOASTA, All rights reserved. • How fast is your site now? • Measure everything • Too much data? Science! • Prioritize what to fix • Optimize your site • What to test? • Test early and often – shift left and right • How to test?
  3. 3. ©2016 SOASTA, All rights reserved. 3 Measure, Optimize, Test – Repeat
  4. 4. How fast is your site now?
  5. 5. ©2016 SOASTA, All rights reserved. 5 How fast is your site now? • Google says 1000ms! • No single right answer • Not all pages are equal o Some pages should be faster than others o The slowest pages might not be the ones to fix first
  6. 6. ©2016 SOASTA, All rights reserved. 6 How is your site now? • mCommerce is global o US is not the benchmark • 39% of mobile users are unhappy with their online experiences o Top reasons: slow pages and site freezes/crashes • 57% have experienced problems when using mobile • 3 out of 4 consider mobile slower than desktop • 46% will not return to a site that performs poorly • 27% of smartphone users exceed data cap every month
  7. 7. ©2016 SOASTA, All rights reserved. 7 When is your Peak Season? • Peak is year-round o Super Bowl Sunday – 110 million viewers o Bi-weekly or monthly payroll processing o Valentine’s Day, President’s Day, Mother’s Day, Father’s Day o Quarterly processing and filing o US Tax filing o Spring, Summer, Fall & Winter season sales o July 4th, Christmas in July o Back to School  We are here o Olympics o Halloween o Election Day o Singles Day – world’s biggest shopping day, $9B in 12 hours o Black Friday, Cyber Monday and Holidays
  8. 8. Measure Everything
  9. 9. ©2016 SOASTA, All rights reserved. 9 Measure Real, Synthetic, and Virtual Users • Real Users (RUM) provide deep data about your visitors o No other way to know everything about your users on your site • Synthetic Users tell you how parts of your site are doing o When no real users are visiting, synthetic users are still coming • Virtual Users tell you how your site will do at scale o Can’t get a million friends to visit your site at the same time? • Collect all the data and save it in the cloud! o Storage is cheap in the cloud, plus easier to analyze there • Add App Performance Management (APM) for a complete picture The Three Musketeers of Performance
  10. 10. ©2016 SOASTA, All rights reserved. 10 Other Things to Measure • Session paths o How are your users going through your site • Mobile App Performance o How are your mobile apps performing? o What are users doing in your mobile apps? • Revenue o How much money are you making from each user and where? • How fast are your competitors’ pages? o You can’t insert code on their sites but you can measure Your company may be measuring these already
  11. 11. Too much data? Science!
  12. 12. ©2016 SOASTA, All rights reserved. 12 Analyze • Benchmark and baseline your site o How fast is your site now? How about the competition? • Correlate performance and outcome o Is poor performance affecting the business? • Find root causes for performance issues o Use RUM and APM data • Fix pages that impact most o Make a prioritized punch list of the pages to fix Use the data you’ve collected to gain insights
  13. 13. ©2016 SOASTA, All rights reserved. 13 Baseline Your Performance How is your site doing? How does it compare to the competition?
  14. 14. ©2016 SOASTA, All rights reserved. 14 Find Root Causes for Poor Performance Not all problems are from your app or back end systems
  15. 15. ©2016 SOASTA, All rights reserved. 15 Page Visits vs. Performance
  16. 16. ©2016 SOASTA, All rights reserved. 16 Page Visits vs. Performance by Device
  17. 17. ©2016 SOASTA, All rights reserved. 17 Prioritize What to Fix – Greatest Impact The poorest performing page might not be the most important page
  18. 18. Fix and Optimize
  19. 19. ©2016 SOASTA, All rights reserved. 19 Page Bloat The average web page is more than 2MB in size
  20. 20. ©2016 SOASTA, All rights reserved. 20 Page Bloat Where are all those kilobytes going
  21. 21. ©2016 SOASTA, All rights reserved. 21 Doesn’t Bandwidth Solve the Problem?
  22. 22. ©2016 SOASTA, All rights reserved. 22 Comparing Network Performance
  23. 23. ©2016 SOASTA, All rights reserved. 23 Comparing Network Performance
  24. 24. ©2016 SOASTA, All rights reserved. 24 Doesn’t Bandwidth Solve the Problem?
  25. 25. ©2016 SOASTA, All rights reserved. 25 Doesn’t Bandwidth Solve the Problem? 25x Faster Bandwidth != 25x Faster Load Time
  26. 26. ©2016 SOASTA, All rights reserved. 26 Prepare for Peak Events Optimize content for peak load events • Simplify website content o Keep only the most important content on the page • Reduce infrastructure requests o Block unnecessary requests, out-of-region visitors • Leverage historical comparisons o Review past traffic levels to predict future levels
  27. 27. What Should I Test?
  28. 28. ©2016 SOASTA, All rights reserved. 28 Creating a Test Plan – Session Analysis Which paths are the most important to users?
  29. 29. ©2016 SOASTA, All rights reserved. 29 Creating a Test Plan – Session Analysis Which paths are the most important to users?
  30. 30. ©2016 SOASTA, All rights reserved. 30 Creating a Test Plan – Peak Load What was the busiest Day, Hour, Minute, Second?
  31. 31. ©2016 SOASTA, All rights reserved. 31 Creating a Test Plan – Page Flows What were most popular page flows with think times?
  32. 32. ©2016 SOASTA, All rights reserved. 32 Creating a Test Plan - Which paths should I test, what think times, at what load, from where?
  33. 33. ©2016 SOASTA, All rights reserved. 33 Network Emulation • Testing to match realistic situations o Users are not all from one location on a fast network o Packets get delayed, lost, re-ordered • Apps perform differently as network conditions worsen o Reduced throughput o Timeouts occur, connections are lost o Transactions take longer, longer sessions Adding real network modeling to load and performance testing
  34. 34. Test Early and Often
  35. 35. ©2016 SOASTA, All rights reserved. 35 Start Testing Yesterday • How long will it take? o 300 test cases x 2 browsers = 600 cases o 600 cases x 30-minutes/case = 300 hours o 300 hours / 6 hours/day = 50 days o 50 days / 5 people = 10 days • And for mobile: o 300 text cases x 26 devices = 7800 cases o 7800 cases x 30 minutes/case = 3900 hours o 3900 hours / 6 hours/day = 650 days o 650- days/5 people = 130 days of test – 26 weeks! And even earlier for Mobile
  36. 36. ©2016 SOASTA, All rights reserved. 36 Don’t Wait Til the Last Minute to Test!
  37. 37. ©2016 SOASTA, All rights reserved. 37 Test Right and Test Left, even into Production • Performance Testing in development o Set performance SLAs for new code and test in development, same as unit and functional tests which code must pass • Performance Testing in staging o Higher loads in near-production systems • Performance Testing in Production o Full scale load tests to expected load and beyond o Find weak and stress points and prioritize fixes into project Performance Testing is Continuous
  38. 38. ©2016 SOASTA, All rights reserved. 38 Test Continuously Test from Development to Integration to Staging to Production
  39. 39. ©2016 SOASTA, All rights reserved. 39 Test in Production Some issues will only be found at scale load on production systems Don’t forget your third party providers: CDNs, external scripts
  40. 40. ©2016 SOASTA, All rights reserved. 40 Measure Mobile App Performance Under Load All mobile apps perform well when there is no load
  41. 41. ©2016 SOASTA, All rights reserved. 41 Automate Your Testing Use Continuous Integration tools to automate testing
  42. 42. Demo
  43. 43. ©2016 SOASTA, All rights reserved. 43 Now That the Test is Over • Analyze results for future areas to address o Weak or stress areas that aren’t likely this time but … • Identify third parties that were affected by load o Coordinate with suppliers to address issues o Align schedules or at least be aware of their changes Even if the test passed, you’re not done
  44. 44. ©2016 SOASTA, All rights reserved. 44 Issues Will Happen • “I am 100% confident that everything we tested will work just fine.” – Nordstrom Performance Leader o When loads are different than what you modeled, may have problems o E.g. At Nordstrom, during a recent anniversary sale, mobile sign- ins were 9x greater than anticipated o Used RUM to get to the root cause of the issue Realize that Issues Will Happen Where You Can’t Predict Them
  45. 45. Q & A Resources Next Steps
  46. 46. ©2016 SOASTA, All rights reserved. 46 Questions, Resources & Next Steps • Holiday Readiness o • The Performance Beacon o o • Upcoming Webinars o RUM-Based Testing o Recordings of past webinars • CloudTest on-Demand o SOASTA will build & run tests for you Resources
  47. 47. ©2016 SOASTA, All rights reserved. 47 “The leader in performance analytics.” SOASTA
  48. 48. ©2016 SOASTA, All rights reserved. 48 CloudTest • Load testing from development to production o Continuous testing at any scale, on-prem or in the cloud • Real-time analytics with customizable dashboards o Safely test in production with full visibility • Run tests live or automatically o Full hands-on control or hands-off automation • Full end-to-end view o Front-end performance to back-end systems Fast, scalable, affordable load testing for performance optimization