Performance Testingatthe Edge<br />Alois Reitbauer, dynaTrace Software<br />
3,000,000,000<br />10,500,000,000<br />
The Classical Approach<br />
Waterfallsarepretty<br />
But mightgetscary<br />
The dynaTrace Approach<br />
Manyplatforms<br />Different usagescenarios<br />High numberofconfigurations<br />No easy waytopatchsoftware<br />
8<br />OurArchitecture<br />DYNATRACE CLIENT <br />DYNATRACE SERVER<br />DYNATRACE COLLECTOR<br />(OPTIONAL)<br />DYNATRAC...
Lessonslearned<br />
Profiling was not enough<br />Goodforfindingproblems<br />Resultcomparisonhard<br />Only valid untilnextcheck-in<br />Toom...
The Life of a Log Statement<br />Enterthecode<br />publicvoidfoo (){<br />   … // do something<br />  bar ();<br />}<br />...
The Life of a Log Statement<br />Somebodychangessomething<br />publicvoidfoo (){<br />   … // do something<br />if (condit...
The Life of a Log Statement<br />Yourcodegetsdeprecated<br />publicvoidfoo (){<br />   … // do something<br />newBar ();<b...
Methodology<br />
Definingourstrategy<br />Startearly<br />Break in pieces<br />Test Continuously<br />
Frequency vs.Granularity<br />JUnit-based<br />Tests (2x day)<br />Total System<br />Tests<br />Long-running<br />Stabilti...
Granularity<br />Comparability<br />Complexity<br />Quality<br />
Avoid Re-Runs<br />Whatcould happen?<br />Whichinformation do youwant?<br />Whatdescribesyoursystem?<br />Whatis different...
Aim high …<br />… test50%more<br />
Create Instability<br />.. addingsomevolatilityincreasesthelikelyness<br />todiscoverproblems …“<br />
„Last MileTesting“<br />
Measurements<br />
Stabilityof Tests<br />
UseDedicated Hardware<br />Comparability<br />Stability<br /> Efficiency<br />
Trends in Unstable Tests<br />
Testingscalability<br />Big DumpOperations<br />Small DumpOperations<br />
Understandyourmeasurements<br />Response Time and GC<br />Response Time only<br />
BeSpecific on whattotest<br />Throughput<br />Response Time<br />Memory Consumption<br />Other KPI …<br />
Beyond Response Time<br />KPI Chart: Server Throughput  Over Time<br />
Motivateyourteam<br />
Howtomakedeveloperswritetests<br />#1 Heroism<br />#2 Boomerang<br />#3 The otherguy<br />#4 Bug me not<br />#5 Feedback<b...
Test Case Complexity<br />First<br />Start dynaTrace infrastructure<br />Whenready<br />Start n WebSphereinstances on serv...
Making complexthings easy<br />    @DtdRemoteSud(<br />            host = "lab2",<br />            name = "WAS7.0",<br /> ...
Findingtheresponsiblecode<br />Version ControlHistory Lookup<br />
Alwaysavailable<br />Continuous Integration Reports<br />
E-Mail Notification<br />
alois.reitbauer@dynatrace.com        Mail<br />blog.dynatrace.com        Blog<br />AloisReitbauerTwitter<br />
Upcoming SlideShare
Loading in …5
×

Q Con Performance Testing At The Edge

837 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
837
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Q Con Performance Testing At The Edge

  1. 1. Performance Testingatthe Edge<br />Alois Reitbauer, dynaTrace Software<br />
  2. 2. 3,000,000,000<br />10,500,000,000<br />
  3. 3. The Classical Approach<br />
  4. 4. Waterfallsarepretty<br />
  5. 5. But mightgetscary<br />
  6. 6. The dynaTrace Approach<br />
  7. 7. Manyplatforms<br />Different usagescenarios<br />High numberofconfigurations<br />No easy waytopatchsoftware<br />
  8. 8. 8<br />OurArchitecture<br />DYNATRACE CLIENT <br />DYNATRACE SERVER<br />DYNATRACE COLLECTOR<br />(OPTIONAL)<br />DYNATRACE COLLECTOR<br />(OPTIONAL)<br />WAN<br />Java Server<br />.NET Server<br />Database <br />Web Server<br />APPLICATION <br />
  9. 9. Lessonslearned<br />
  10. 10. Profiling was not enough<br />Goodforfindingproblems<br />Resultcomparisonhard<br />Only valid untilnextcheck-in<br />Toomuchwork<br />
  11. 11. The Life of a Log Statement<br />Enterthecode<br />publicvoidfoo (){<br /> … // do something<br /> bar ();<br />}<br />publicvoid bar (){<br /> … // do somethingelse<br /> Logger.log (“executiontook“ + time);<br />}<br />
  12. 12. The Life of a Log Statement<br />Somebodychangessomething<br />publicvoidfoo (){<br /> … // do something<br />if (condition)<br /> bar ();<br />else<br />otherBar ();<br />}<br />publicvoid bar (){<br /> … // do somethingelse<br /> Logger.log (“executiontook“ + time);<br />}<br />
  13. 13. The Life of a Log Statement<br />Yourcodegetsdeprecated<br />publicvoidfoo (){<br /> … // do something<br />newBar ();<br />}<br />publicvoid bar (){<br /> … // do somethingelse<br /> Logger.log (“executiontook“ + time);<br />}<br />
  14. 14. Methodology<br />
  15. 15. Definingourstrategy<br />Startearly<br />Break in pieces<br />Test Continuously<br />
  16. 16. Frequency vs.Granularity<br />JUnit-based<br />Tests (2x day)<br />Total System<br />Tests<br />Long-running<br />Stabiltiy Tests<br />(2 w duration)<br />Granularity<br />Frequency<br />
  17. 17. Granularity<br />Comparability<br />Complexity<br />Quality<br />
  18. 18. Avoid Re-Runs<br />Whatcould happen?<br />Whichinformation do youwant?<br />Whatdescribesyoursystem?<br />Whatis different fromthe last run?<br />
  19. 19. Aim high …<br />… test50%more<br />
  20. 20. Create Instability<br />.. addingsomevolatilityincreasesthelikelyness<br />todiscoverproblems …“<br />
  21. 21. „Last MileTesting“<br />
  22. 22. Measurements<br />
  23. 23. Stabilityof Tests<br />
  24. 24. UseDedicated Hardware<br />Comparability<br />Stability<br /> Efficiency<br />
  25. 25. Trends in Unstable Tests<br />
  26. 26. Testingscalability<br />Big DumpOperations<br />Small DumpOperations<br />
  27. 27. Understandyourmeasurements<br />Response Time and GC<br />Response Time only<br />
  28. 28. BeSpecific on whattotest<br />Throughput<br />Response Time<br />Memory Consumption<br />Other KPI …<br />
  29. 29. Beyond Response Time<br />KPI Chart: Server Throughput Over Time<br />
  30. 30. Motivateyourteam<br />
  31. 31. Howtomakedeveloperswritetests<br />#1 Heroism<br />#2 Boomerang<br />#3 The otherguy<br />#4 Bug me not<br />#5 Feedback<br />#6 Code vs. Wine<br />#7 Newb vs. Noob<br />
  32. 32. Test Case Complexity<br />First<br />Start dynaTrace infrastructure<br />Whenready<br />Start n WebSphereinstances on servers … <br />Whenready<br />Start LoadtestagainstWebSphereservers<br />After loadteststart<br />Execute testcase<br />
  33. 33. Making complexthings easy<br />    @DtdRemoteSud(<br />            host = "lab2",<br />            name = "WAS7.0",<br />            startupPriority = 1,<br />            postStartClosure = WaitForWebSphereSudIsUp.class<br />    )<br />    private SudInterfacewebSphereSud;<br />
  34. 34. Findingtheresponsiblecode<br />Version ControlHistory Lookup<br />
  35. 35. Alwaysavailable<br />Continuous Integration Reports<br />
  36. 36. E-Mail Notification<br />
  37. 37. alois.reitbauer@dynatrace.com Mail<br />blog.dynatrace.com Blog<br />AloisReitbauerTwitter<br />

×