Top ten secret weapons for performance testing in an agile environment<br />patrick.kua@thoughtworks.com<br />(and alistai...
Make Performance Explicit<br />© ThoughtWorks 2011<br />  1<br />
So that I can make better investment decisions<br />As an investor<br />I want to see the value of my portfolio presented ...
© ThoughtWorks 2011<br />So that investors have a high-quality experience as the business grows<br />As the Operations Man...
One Team<br />© ThoughtWorks 2011<br />   2<br />
Team Dynamics<br />© ThoughtWorks 2011<br />
Performance Testers Part of Team<br />© ThoughtWorks 2011<br />
© ThoughtWorks 2011<br />
Performance Testers Part of Team<br />© ThoughtWorks 2011<br />
Pair on Performance Test Stories<br />© ThoughtWorks 2011<br />
Rotate Pairs<br />© ThoughtWorks 2011<br />
Customer Driven<br />© ThoughtWorks 2011<br />      3<br />
What was a good source of requirements?<br />© ThoughtWorks 2011<br />
© ThoughtWorks 2011<br />Existing Pain Points<br />
An example...<br />© ThoughtWorks 2011<br />
So that we can budget for future hardware needs as we grow<br />As the data centre manager<br />I want to know how much tr...
Another example<br />© ThoughtWorks 2011<br />
© ThoughtWorks 2011<br />So that we have confidence in meeting our SLA<br />As the Operations Manager<br />I want to ensur...
Personas<br />© ThoughtWorks 2011<br />
Who is the customer?<br />© ThoughtWorks 2011<br />Investors<br />Marketing<br />End Users<br />Power <br />Users<br />Ope...
Discipline<br />© ThoughtWorks 2011<br />  4<br />
© ThoughtWorks 2011<br />Observe test results<br />What do you see?<br />Formulate an hypothesis<br />Why is it doing that...
??????????<br />© ThoughtWorks 2011<br />
© ThoughtWorks 2011<br />Observe test results<br />Saw tooth pattern  (1 minute intervals)<br />Formulate an hypothesis<br...
One Directory<br />© ThoughtWorks 2011<br />
Play Performance Early<br />© ThoughtWorks 2011<br />   5<br />
© ThoughtWorks 2011<br />End<br />Start<br />Other projects start performance testing here<br />End<br />Start<br />Agile ...
Iterate Don’t (Just) Increment<br />© ThoughtWorks 2011<br />       6<br />
© ThoughtWorks 2011<br />
We               Sashimi<br />© ThoughtWorks 2011<br />
Sashimi Slice By... Presentation<br />© ThoughtWorks 2011<br />
© ThoughtWorks 2011<br />So that I can better see trends in performance<br />As the Operations Manager<br />I want a graph...
© ThoughtWorks 2011<br />So that I can better see trends in performance<br />As the Operations Manager<br />I want a graph...
© ThoughtWorks 2011<br />So that I can easily scan results at a single glance<br />As the Operations Manager<br />I want a...
Sashimi Slice By...  Scenario<br />© ThoughtWorks 2011<br />
© ThoughtWorks 2011<br />So that we never have a day like “October 10”<br />As the Operations Manager<br />I want to ensur...
© ThoughtWorks 2011<br />So that we never have a day like “November 12”<br />As the Operations Manager<br />I want to ensu...
Automate, Automate, Automate<br />© ThoughtWorks 2011<br />   7<br />
© ThoughtWorks 2011<br />Automated<br />Compilation<br />Automated<br />Tests<br />Automated<br />Packaging<br />Automated...
Automation => Reproducible and Consistent<br />Automation => Faster Feedback<br />Automation => Higher Productivity<br />W...
© ThoughtWorks 2011<br />Automated<br />Test Orchestration<br />Automated<br />Analysis<br />Automated Scheduling<br />Aut...
Continuous Performance Testing<br />© ThoughtWorks 2011<br />   8<br />
© ThoughtWorks 2011<br />
Application<br />Build Pipelines<br />© ThoughtWorks 2011<br />Performance<br />
© ThoughtWorks 2011<br />
Test Drive Your Performance Test Code<br />© ThoughtWorks 2011<br />   9<br />
V Model Testing<br />© ThoughtWorks 2011<br />Slower + Longer<br />Performance Testing<br />Speed<br />Fast<br />http://en...
We make mistakes<br />© ThoughtWorks 2011<br />
V Model Testing<br />© ThoughtWorks 2011<br />Slower + Longer<br />Performance Testing<br />Speed<br />Unit test performan...
Fail Fast!<br />© ThoughtWorks 2011<br />Fast feedback!<br />Faster learning<br />Faster results<br />
Classic Performance Areas to Test<br />© ThoughtWorks 2011<br />Analysis<br />Information Collection<br />Presentation<br ...
Get Feedback<br />© ThoughtWorks 2011<br />10<br />
Frequently (Weekly) Showcase<br />© ThoughtWorks 2011<br />Here is what we learned this week....<br />
Frequently (Weekly) Showcase<br />© ThoughtWorks 2011<br />And based on this... We changed our directory structure.<br />
Frequently (Weekly) Showcase<br />© ThoughtWorks 2011<br />Should we do something different knowing this new information?<...
List of All Secret Weapons<br />Make Performance Explicit<br />One Team<br />Customer Driven<br />Discipline<br />Play Per...
Photo Credits (Creative Commons licence)<br />Barbed wire picture: http://www.flickr.com/photos/lapideo/446201948/<br />Et...
Upcoming SlideShare
Loading in...5
×

Top ten secret weapons for performance testing in an agile environment

783

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
783
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Top ten secret weapons for performance testing in an agile environment

  1. 1. Top ten secret weapons for performance testing in an agile environment<br />patrick.kua@thoughtworks.com<br />(and alistair.jones@thoughtworks.com)<br />© ThoughtWorks 2011<br />
  2. 2. Make Performance Explicit<br />© ThoughtWorks 2011<br /> 1<br />
  3. 3. So that I can make better investment decisions<br />As an investor<br />I want to see the value of my portfolio presented on a single web page<br />must have “good” performance, less than 0.2s page load for about 10,000 concurrent users<br />© ThoughtWorks 2011<br />
  4. 4. © ThoughtWorks 2011<br />So that investors have a high-quality experience as the business grows<br />As the Operations Manager<br />I want the portfolio value page to render within 0.2s when 10,000 users are logged in<br />
  5. 5. One Team<br />© ThoughtWorks 2011<br /> 2<br />
  6. 6. Team Dynamics<br />© ThoughtWorks 2011<br />
  7. 7. Performance Testers Part of Team<br />© ThoughtWorks 2011<br />
  8. 8. © ThoughtWorks 2011<br />
  9. 9. Performance Testers Part of Team<br />© ThoughtWorks 2011<br />
  10. 10. Pair on Performance Test Stories<br />© ThoughtWorks 2011<br />
  11. 11. Rotate Pairs<br />© ThoughtWorks 2011<br />
  12. 12. Customer Driven<br />© ThoughtWorks 2011<br /> 3<br />
  13. 13. What was a good source of requirements?<br />© ThoughtWorks 2011<br />
  14. 14. © ThoughtWorks 2011<br />Existing Pain Points<br />
  15. 15. An example...<br />© ThoughtWorks 2011<br />
  16. 16. So that we can budget for future hardware needs as we grow<br />As the data centre manager<br />I want to know how much traffic we can handle now<br />© ThoughtWorks 2011<br />
  17. 17. Another example<br />© ThoughtWorks 2011<br />
  18. 18. © ThoughtWorks 2011<br />So that we have confidence in meeting our SLA<br />As the Operations Manager<br />I want to ensure that a sustained peak load does not take out our service<br />
  19. 19. Personas<br />© ThoughtWorks 2011<br />
  20. 20. Who is the customer?<br />© ThoughtWorks 2011<br />Investors<br />Marketing<br />End Users<br />Power <br />Users<br />Operations<br />
  21. 21. Discipline<br />© ThoughtWorks 2011<br /> 4<br />
  22. 22. © ThoughtWorks 2011<br />Observe test results<br />What do you see?<br />Formulate an hypothesis<br />Why is it doing that?<br />Design an experiment<br />How can I prove that’s what’s happening?<br />Run the experiment<br />Take the time to gather the evidence.<br />Is the hypothesis valid?<br />Change the application code<br />Safe in the knowledge that I’m making it faster<br />
  23. 23. ??????????<br />© ThoughtWorks 2011<br />
  24. 24. © ThoughtWorks 2011<br />Observe test results<br />Saw tooth pattern (1 minute intervals)<br />Formulate an hypothesis<br />Directory structure of (yyyy/mm/minuteofday)?. Slow down due to # of files in directory?<br />Design an experiment<br /> 1 directory should result in even worse performance...<br />Run the experiment<br />We ran the test…<br />Is the hypothesis valid?<br />Change the application code<br />
  25. 25. One Directory<br />© ThoughtWorks 2011<br />
  26. 26. Play Performance Early<br />© ThoughtWorks 2011<br /> 5<br />
  27. 27. © ThoughtWorks 2011<br />End<br />Start<br />Other projects start performance testing here<br />End<br />Start<br />Agile projects start performance testing as early as possible<br />
  28. 28. Iterate Don’t (Just) Increment<br />© ThoughtWorks 2011<br /> 6<br />
  29. 29. © ThoughtWorks 2011<br />
  30. 30. We Sashimi<br />© ThoughtWorks 2011<br />
  31. 31. Sashimi Slice By... Presentation<br />© ThoughtWorks 2011<br />
  32. 32. © ThoughtWorks 2011<br />So that I can better see trends in performance<br />As the Operations Manager<br />I want a graph of requests per second<br />
  33. 33. © ThoughtWorks 2011<br />So that I can better see trends in performance<br />As the Operations Manager<br />I want a graph of average latency per second<br />
  34. 34. © ThoughtWorks 2011<br />So that I can easily scan results at a single glance<br />As the Operations Manager<br />I want a one page showing all results <br />
  35. 35. Sashimi Slice By... Scenario<br />© ThoughtWorks 2011<br />
  36. 36. © ThoughtWorks 2011<br />So that we never have a day like “October 10”<br />As the Operations Manager<br />I want to ensure that a sustained peak load does not take out our service<br />
  37. 37. © ThoughtWorks 2011<br />So that we never have a day like “November 12”<br />As the Operations Manager<br />I want to ensure that an escalating load up to xxx requests/second does not take out our service<br />
  38. 38. Automate, Automate, Automate<br />© ThoughtWorks 2011<br /> 7<br />
  39. 39. © ThoughtWorks 2011<br />Automated<br />Compilation<br />Automated<br />Tests<br />Automated<br />Packaging<br />Automated<br />Deployment<br />
  40. 40. Automation => Reproducible and Consistent<br />Automation => Faster Feedback<br />Automation => Higher Productivity<br />Why Automation?<br />© ThoughtWorks 2011<br />
  41. 41. © ThoughtWorks 2011<br />Automated<br />Test Orchestration<br />Automated<br />Analysis<br />Automated Scheduling<br />Automated<br />Load Generation<br />Automated<br />Application Deployment<br />Automated Result Archiving<br />
  42. 42. Continuous Performance Testing<br />© ThoughtWorks 2011<br /> 8<br />
  43. 43. © ThoughtWorks 2011<br />
  44. 44. Application<br />Build Pipelines<br />© ThoughtWorks 2011<br />Performance<br />
  45. 45. © ThoughtWorks 2011<br />
  46. 46. Test Drive Your Performance Test Code<br />© ThoughtWorks 2011<br /> 9<br />
  47. 47. V Model Testing<br />© ThoughtWorks 2011<br />Slower + Longer<br />Performance Testing<br />Speed<br />Fast<br />http://en.wikipedia.org/wiki/V-Model_(software_development)<br />
  48. 48. We make mistakes<br />© ThoughtWorks 2011<br />
  49. 49. V Model Testing<br />© ThoughtWorks 2011<br />Slower + Longer<br />Performance Testing<br />Speed<br />Unit test performance code to fail faster<br />Fast<br />http://en.wikipedia.org/wiki/V-Model_(software_development)<br />
  50. 50. Fail Fast!<br />© ThoughtWorks 2011<br />Fast feedback!<br />Faster learning<br />Faster results<br />
  51. 51. Classic Performance Areas to Test<br />© ThoughtWorks 2011<br />Analysis<br />Information Collection<br />Presentation<br />Publishing<br />Visualisation<br />
  52. 52. Get Feedback<br />© ThoughtWorks 2011<br />10<br />
  53. 53. Frequently (Weekly) Showcase<br />© ThoughtWorks 2011<br />Here is what we learned this week....<br />
  54. 54. Frequently (Weekly) Showcase<br />© ThoughtWorks 2011<br />And based on this... We changed our directory structure.<br />
  55. 55. Frequently (Weekly) Showcase<br />© ThoughtWorks 2011<br />Should we do something different knowing this new information?<br />
  56. 56. List of All Secret Weapons<br />Make Performance Explicit<br />One Team<br />Customer Driven<br />Discipline<br />Play Performance Early<br />Iterate Don't (Just) Increment<br />Automate, Automate, Automate <br />Test Drive Your Performance Code<br />Continuous Performance Testing<br />Get Feedback<br />© ThoughtWorks 2011<br />
  57. 57. Photo Credits (Creative Commons licence)<br />Barbed wire picture: http://www.flickr.com/photos/lapideo/446201948/<br />Eternal clock: http://www.flickr.com/photos/robbie73/3387189144/<br />Sashimi from http://www.flickr.com/photos/mac-ash/3719114621/<br />Questions<br />© ThoughtWorks 2011<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×