Techniques for Agile Performance Testing

226
-1

Published on

The performance of your application affects your business more than you might think. Top engineering organizations think of performance as not just a nice-to-have feature but as a crucial feature of their product. Those organizations understand that performance has a direct impact on user experience and, ultimately, their bottom line. Unfortunately, most engineering teams do not regularly test the performance and scalability of their infrastructure. Jim Hirschauer shares the latest performance testing tools and insights into why your team should add performance testing to an agile development process. Learn how to evaluate performance and scalability with MultiMechanize, Bees with Machine Guns, and Google PageSpeed. Take back an understanding of how to automate performance testing and evaluate the impact it has on performance and on your business.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
226
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Techniques for Agile Performance Testing

  1. 1.      Session    Presented by:  Jim Hirschauer      Brought to you by:      340 Corporate Way, Suite   Orange Park, FL 32073  888‐2 W14  Concurrent 4/9/2014    3:15 PM          “Techniques for Agile Performance Testing”      AppDynamics              300, 68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com 
  2. 2. Jim Hirschauer AppDynamics   Technology evangelist for AppDynamics Jim Hirschauer has an extensive background in highly available, business critical, large enterprise IT operations environments. Jim has been interested in application performance testing and monitoring since his days as a systems administrator in a retail bank. His passion for performance analysis led him down a path to design, implement, and manage the cloud computing monitoring architecture for a top-ten investment bank. During his tenure at the bank, Jim created new processes and procedures that reduced the time spent in the test and QA cycle while simultaneously increasing overall code release quality and dramatically improving end user experience.
  3. 3. 2/18/2014 1 PERFORMANCE TESTING CRASH COURSE Jim Hirschauer @HirschOnAPM Lockheed Martin Aeronautical Systems Duke Energy Wachovia Wells Fargo AppDynamics
  4. 4. 2/18/2014 2 Wh dWhy does performance matter?matter? Mi ft f d th t BiMicrosoft found that Bing searches that were 2 seconds slower resulted in a 4.3% drop in revenue per userdrop in revenue per user
  5. 5. 2/18/2014 3 Wh M ill h d 2 2When Mozilla shaved 2.2 seconds off their landing page, Firefox downloads increased 15.4%increased 15.4% Making Barack Obama’s website 60% faster increased donation conversions by 14%
  6. 6. 2/18/2014 4 Amazon and WalmartAmazon and Walmart increased revenue 1% for every 100ms of improvementimprovement
  7. 7. 2/18/2014 5 Performance directly impacts the bottom linee
  8. 8. 2/18/2014 6 T l f th t dTools of the trade for performance testingtesting
  9. 9. 2/18/2014 7 Understand your baseline performance Static vs Hello World vsvs App
  10. 10. 2/18/2014 8 Apache Bench ab -c 10 -t 10 -k http://dustinwhittle.com/
  11. 11. 2/18/2014 9 Server Software: nginx Server Hostname: dustinwhittle.com Server Port: 80 Document Path: / Document Length: 6642 bytes Concurrency Level: 10 Time taken for tests: 10.042 seconds C l 286Complete requests: 286 Failed requests: 0 Write errors: 0 Keep-Alive requests: 0 Total transferred: 2080364 bytes HTML transferred: 1899612 bytes Requests per second: 28.48 [#/sec] (mean) Time per request: 351.133 [ms] (mean) Time per request: 35.113 [ms] (mean, across all concurrent requests) Transfer rate: 202.30 [Kbytes/sec] received Connection Times (ms)Connection Times (ms) min mean[+/-sd] median max Connect: 9 20 14.2 17 148 Processing: 117 325 47.8 323 574 Waiting: 112 317 46.3 314 561 Total: 140 346 46.0 341 589 Percentage of the requests served within a certain time (ms) 50% 341 66% 356 75% 366 80% 372 Siege
  12. 12. 2/18/2014 10 siege -c 10 -b -t 10S http://dustinwhittle.com/ ** SIEGE 2.72 ** Preparing 10 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 263 hits Availability: 100.00 % Elapsed time: 9.36 secs Data transferred: 0.35 MB Response time: 0.35 secs Transaction rate: 28.10 trans/sec Throughput: 0.04 MB/sec Concurrency: 9.82y Successful transactions: 263 Failed transactions: 0 Longest transaction: 0.54 Shortest transaction: 0.19
  13. 13. 2/18/2014 11 Crawl the entire app to discover all urls sproxy -o ./urls.txt
  14. 14. 2/18/2014 12 SPROXY v1.02 listening on port 9001 ...appending HTTP requests to: ./urls.txt ...default connection timeout: 120 seconds wget -r -o verbose txt -l 0 -t 1 --spider -w 1 -ewget -r -o verbose.txt -l 0 -t 1 --spider -w 1 -e robots=on -e "http_proxy = http://127.0.0.1:9001" "http://dustinwhittle.com/" sort -u -o urls.txt urls.txt
  15. 15. 2/18/2014 13 Benchmark traffic across all unique urls with siegeu s t s ege siege -v -c 50 -i -t 3M -f urls.txt - d 10
  16. 16. 2/18/2014 14 Apache JMeter
  17. 17. 2/18/2014 15 M lti M h i iMulti-Mechanize is an open source framework for performance and load testingtesting
  18. 18. 2/18/2014 16 pip install multi- mechanize multimech-newproject demo
  19. 19. 2/18/2014 17 import requestsimport requests class Transaction(object): def run(self): r = requests.get('http://dustinwhittle.com/') d()r.raw.read() class Transaction(object): def run(self): br = mechanize.Browser() br.set_handle_robots(False) start_timer = time.time() resp = br.open('http://www.dustinwhittle.com/') resp.read() latency = time.time() - start_timer self custom timers['homepage'] = latencyself.custom_timers[ homepage ] latency start_timer = time.time() resp = br.open('http://www.dustinwhittle.com/blog') resp.read() latency = time time() start timer
  20. 20. 2/18/2014 18 [global] run_time = 10 5rampup = 5 results_ts_interval = 1 progress_bar = on console_logging = off xml_report = on [user_group-1] threads = 1 script = demo.py multimech-run demo
  21. 21. 2/18/2014 19
  22. 22. 2/18/2014 20 What about when you need more than one machine?o e ac e Who lives in the cloud?
  23. 23. 2/18/2014 21 Bees with Machine Guns A utility for arming (creating)A utility for arming (creating) many bees (micro EC2 instances) to attack (load test) t t ( b li ti )targets (web applications)
  24. 24. 2/18/2014 22 pip install beeswithmachineguns # ~/.boto [Credentials] aws_access_key_id=xxx aws_secret_access_key=xxx [Boto] ec2_region_name = us-west-2 ec2_region_endpoint = ec2.us-west- 2.amazonaws.com
  25. 25. 2/18/2014 23 bees up -s 2 -g default -z us-west- 2b -i ami-bc05898c -k appdynamics- dustinwhittle-aws-us-west-2 -l ec2- user Connecting to the hiveConnecting to the hive. Attempting to call up 2 bees. Waiting for bees to load their machine guns... . . . . Bee i-3828400c is ready for the attack. Bee i-3928400d is ready for the attack. The swarm has assembled 2 bees.
  26. 26. 2/18/2014 24 bees report Read 2 bees from the roster. Bee i-3828400c: running @ 54.212.22.176 Bee i-3928400d: running @ 50.112.6.191
  27. 27. 2/18/2014 25 bees attack -n 1000 -c 50 - u http://dustinwhittle.com/ Read 2 bees from the roster. Connecting to the hive. Assembling bees. Each of 2 bees will fire 50000 rounds, 125 at a time. Stinging URL so it will be cached for the attack. Organizing the swarm. Bee 0 is joining the swarm. Bee 1 is joining the swarm. Bee 0 is firing his machine gun. Bang bang! Bee 1 is firing his machine gun. Bang bang! Bee 1 is out of ammo. Bee 0 is out of ammo. Offensive complete. Complete requests: 100000 Requests per second: 1067.110000 [#/sec] (mean) Time per request: 278.348000 [ms] (mean) 50% response time: 47.500000 [ms] (mean) 90% response time: 114.000000 [ms] (mean) Mission Assessment: Target crushed bee offensive. The swarm is awaiting new orders.
  28. 28. 2/18/2014 26 bees down What about the client side?
  29. 29. 2/18/2014 27 Google PageSpeed Google PageSpeed Insights
  30. 30. 2/18/2014 28
  31. 31. 2/18/2014 29 Google PageSpeed API curl "https://www.googleapis.com/pagespe edonline/v1/runPagespeed?url=http:// dustinwhittle.com/&key=xxx"
  32. 32. 2/18/2014 30 WBench gem install wbench
  33. 33. 2/18/2014 31 wbench http://dustinwhittle.com/
  34. 34. 2/18/2014 32 Automate client-side performance testing with Gruntt G u t Use Bower (forUse Bower (for dependencies), Grunt (for automation), and Yeoman (for bootstrapping)
  35. 35. 2/18/2014 33
  36. 36. 2/18/2014 34 H lHow many people understand exactly how fast their site runs in production?production?
  37. 37. 2/18/2014 35
  38. 38. 2/18/2014 36
  39. 39. 2/18/2014 37
  40. 40. 2/18/2014 38
  41. 41. 2/18/2014 39 http://webpagetest.org/
  42. 42. 2/18/2014 40
  43. 43. 2/18/2014 41 L d t tiLoad testing services from the cloudcloud
  44. 44. 2/18/2014 42
  45. 45. 2/18/2014 43 I t t t t dIntegrate automated performance testing into continuous integration for server-side and client-sideserver side and client side
  46. 46. 2/18/2014 44 U d t d thUnderstand the performance implications of every deployment and package upgradepackage upgrade Monitor end user experience from end to end in productionto e d p oduct o
  47. 47. 2/18/2014 45 Questions?

×