This presentation can serve as a template for presenting load testing results with team members. In particular this presentation differentiates performance into two discrete areas: throughput and response time.

1. Ecommerce v2 StackLoad Test Testing Performance through response time and throughput Stephen Rylander

2. Session Basis for Load Test What is Performance? Page Views Physical Tiers Response Times IIS Counters Preliminary Findings

3. Basis for load test

4. Purpose Generate baseline of site by…. increasing load on the infrastructure and application in a… Controlled and predictable manner

5. Specifics The ramp up… 4 steps 250 ::: 6:14 min 500 ::: 8:07 min 1000 ::: 8:53 min 2000 ::: 11:45 min

6. Location AlertSite’s cloud based (Amazon Web Services) load test infrastructure

8. TLP (team landing)

9. Dlp(department)

10. Dlp page 2

11. Post filter / get filtered dlp

12. T-shirt (product)

13. Dlp (t-shirt)

14. Dlp (bedding)

15. Pdp bedding set

16. Add to cart / get View cart

17. Continue to checkout/get sign in page

18. Post checkout start/ get address info (guest)

19. Post address / view payment info

21. 2 ways to measure…

22. 1 - Throughput Throughput Requests per second Transactions per minute Executions per click

23. 2 - Response Response Time Duration of a task Seconds per click Page load DNS lookup Length of time between <click> and seeing page

24. Together The system should handle 3M page views an hour with each page returning in less than 2.5 seconds.

25. Thinking about Measurements Averages Casual statistic Not bad for short periods Heavy variance masked Can hurt customer Percentiles Different way of viewing Like a standardized test “Home page should load in under 2 seconds for 95% of executions.”

26. Our customers feel the variance, not the mean. GE – Six Sigma

27. Shoppers vs. Page Views 11/14 6PM 166,397 page views = 30K visitors/hour (sitetracker) 7 PM 172,716 page views = 31K visitors/hour (sitetracker) Shoppers/Visitorsis only a business concept applied to historical averaged user behavior. It is a predictive tool not a capacity planning tool.

28. Page views The numbers via throughput

29. Reported

30. Real Feel This year high: NK /hour

31. Response Time

34. Wednesday Nov. 11 ::: 123K page views/hour

35. Queuing?

36. Queuing?

37. Trending over position in test 1

38. Trending over position in test 2

39. Trending over position in test 3

40. Web counters

41. Reqs Queued Req/sec Reqs/sec: avg. 49; max 117 Queued: avg. 273; max 1138

42. Reqs current Reqs queued Reqs executing

43. Servers

44. WebTrans02

45. WEB02

46. Take aways

47. What we saw ~20% load increase over our busiest hour this year At peak ~10K orders/hour Linear scaling; linear slow down Physical server capacity ok

48. Concerns IIS queuing because of… Down stream services Payment, inventory, etc Connection limits SQL connection limits per/pool Web server worker thread starvation

49. What we’ve been doing Adding more web tier capacity Splitting more app pools Bumping up worker processes in external web services

50. Next steps Verifying new web tier and services capacity Analyzing daily performance monitors Investigate downstream connections

51. end Thanks.