Your SlideShare is downloading. ×
0
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Performance and Scalability Testing with Python and Multi-Mechanize

17,338

Published on

Performance and Scalability Testing …

Performance and Scalability Testing
with Python and Multi-Mechanize.

visit http://multimechanize.com

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
17,338
On Slideshare
0
From Embeds
0
Number of Embeds
39
Actions
Shares
0
Downloads
129
Comments
0
Likes
9
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Performance and Scalability Testing with Python and Multi-Mechanize Corey Goldberg   2011 web: goldb.org email: [email_address] twitter: @cgoldberg multimechanize.com
  • 2. Multi-Mechanize <ul><ul><li>Open Source (LGPLv3) </li></ul></ul><ul><ul><li>Started Development in 2010 </li></ul></ul><ul><ul><li>Python 2 (2.6+) </li></ul></ul><ul><ul><li>Project Hosted on Google Code (SVN) </li></ul></ul><ul><ul><li>Website: multimechanize.com </li></ul></ul><ul><ul><ul><li>Download </li></ul></ul></ul><ul><ul><ul><li>Code Samples / Script Dev Guide </li></ul></ul></ul><ul><ul><ul><li>Wiki </li></ul></ul></ul><ul><ul><ul><li>Discussion Group </li></ul></ul></ul>
  • 3. What is Multi-Mechanize? <ul><li>&quot;Python framework for performance and load testing remote API's&quot; </li></ul><ul><li>Enables you to: </li></ul><ul><ul><li>run simultaneous client scripts to generate workload against a remote service </li></ul></ul><ul><ul><li>run transactions at an increasing series of load levels against your API and measure performance </li></ul></ul><ul><ul><li>simulate realistic and projected API usage </li></ul></ul>
  • 4. Who is Multi-Mechanize For? <ul><ul><li>Developers </li></ul></ul><ul><ul><li>Testers </li></ul></ul><ul><ul><li>Ops </li></ul></ul><ul><li>... who need to test and tune the performance and scalability of a web site or API service </li></ul><ul><li>Assumptions: </li></ul><ul><ul><li>You know how to code in Python </li></ul></ul><ul><ul><li>You understand basic performance testing </li></ul></ul>
  • 5. Testing for Performance & Scalability <ul><li>You have a remote API or website and need to: </li></ul><ul><ul><li>stress test server and network infrastructure </li></ul></ul><ul><ul><li>find application bottlenecks </li></ul></ul><ul><ul><li>expose errors under load </li></ul></ul><ul><ul><li>performance benchmarks </li></ul></ul><ul><ul><li>capacity planning </li></ul></ul>
  • 6. Which API's Can You Test? <ul><li>You write the virtual user (client) scripts in Python, so any remote protocol/lib you want: </li></ul><ul><ul><li>Web/HTTP </li></ul></ul><ul><ul><ul><li>Raw HTTP (urllib2, httplib) </li></ul></ul></ul><ul><ul><ul><li>Web Forms (mechanize) </li></ul></ul></ul><ul><ul><ul><li>REST/SOAP/XMLRPC </li></ul></ul></ul><ul><ul><li>Sockets </li></ul></ul><ul><ul><li>Database, NoSQL, Cache </li></ul></ul><ul><ul><li>Any Python API </li></ul></ul>
  • 7. Things I Like <ul><ul><li>Free </li></ul></ul><ul><ul><li>Runs on multiple platforms (nix, win) </li></ul></ul><ul><ul><li>Python code base </li></ul></ul><ul><ul><li>Scripts are in Python (batteries included, python libraries) </li></ul></ul><ul><ul><li>Scalable load generator </li></ul></ul><ul><ul><li>Automatic report generation </li></ul></ul><ul><ul><li>Aggregate stats (response times, throughput, percentiles) </li></ul></ul><ul><ul><li>Graphs (scatter, time-series) </li></ul></ul>
  • 8. Inside Multi-Mechanize <ul><ul><li>Multi-process/Multi-threaded Test Harness </li></ul></ul><ul><ul><li>Virtual User Scripting in Python </li></ul></ul><ul><ul><li>Configurable Test Runner </li></ul></ul><ul><ul><li>Report Generator </li></ul></ul><ul><ul><li>Input: Virtual User Test Scripts, Config File </li></ul></ul><ul><ul><li>Output:  HTML Report with PNG images (graphs) </li></ul></ul>
  • 9. Virtual User Scripting <ul><li>You create test scripts in Python for Multi-Mechanize to run.   </li></ul><ul><li>Each test script must implement a Transaction() class . This class must implement a run() method . </li></ul><ul><li>So a basic test script consists of: </li></ul><ul><li>class Transaction: </li></ul><ul><li>    def run(self): </li></ul><ul><li>        # do something here </li></ul>
  • 10. Virtual User Scripting <ul><li>During a test run, your Transaction() class is instantiated once, and then its run() method is called repeatedly: </li></ul><ul><li>class Transaction: </li></ul><ul><li>    def __init__(self): </li></ul><ul><li>        # this gets called once </li></ul><ul><li>     </li></ul><ul><li>    def run(self): </li></ul><ul><li>        # this gets called repeatedly </li></ul>
  • 11. Virtual User Scripting <ul><li>A full script that will issue an HTTP GET using urllib2: </li></ul><ul><li>     </li></ul><ul><li>import urllib2 </li></ul><ul><li>class Transaction(object): </li></ul><ul><li>    def run(self): </li></ul><ul><li>        url = 'http://www.example.com/' </li></ul><ul><li>        urllib2.urlopen(url).read() </li></ul>
  • 12. Virtual User Scripting <ul><li>web request with timer and assertions: </li></ul><ul><li>import mechanize </li></ul><ul><li>import time </li></ul><ul><li>class Transaction: </li></ul><ul><li>    def run(self): </li></ul><ul><li>        self.custom_timers = {} </li></ul><ul><li>        br = mechanize.Browser() </li></ul><ul><li>        br.set_handle_robots(False) </li></ul><ul><li>         </li></ul><ul><li>        start_timer = time.time() </li></ul><ul><li>        resp = br.open('http://www.example.com/') </li></ul><ul><li>        resp.read() </li></ul><ul><li>        latency = time.time() - start_timer </li></ul><ul><li>         </li></ul><ul><li>        self.custom_timers['Example_Homepage'] = latency </li></ul><ul><li>         </li></ul><ul><li>        assert (resp.code == 200), 'Bad HTTP Response' </li></ul><ul><li>        assert ('Example Web Page' in resp.get_data()),  </li></ul><ul><li>            'Failed Content Verification' </li></ul>
  • 13. Test Configuration <ul><li>sample config file: </li></ul><ul><li>  [global] </li></ul><ul><li>  run_time: 300 </li></ul><ul><li>  rampup: 300 </li></ul><ul><li>  console_logging: off </li></ul><ul><li>  results_ts_interval: 60 </li></ul><ul><li>   </li></ul><ul><li>  [user_group-1] </li></ul><ul><li>  threads: 10 </li></ul><ul><li>  script: search_simple.py </li></ul><ul><li>   </li></ul><ul><li>  [user_group-2] </li></ul><ul><li>  threads: 5 </li></ul><ul><li>  script: search_complex.py </li></ul>
  • 14. Running a Test
  • 15. Results Reports
  • 16. Results Reports (Stats) <ul><ul><li>test summary </li></ul></ul><ul><ul><li>transaction timers </li></ul></ul><ul><ul><li>custom timers (from instrumented client code) </li></ul></ul><ul><ul><li>time-series/interval data </li></ul></ul><ul><ul><li>counts </li></ul></ul><ul><ul><li>rate/throughput </li></ul></ul><ul><ul><li>response times </li></ul></ul><ul><ul><ul><li>average, min, max, stdev </li></ul></ul></ul><ul><ul><ul><li>percentiles (80th, 90th, 95th) </li></ul></ul></ul>
  • 17. Results Reports (Graphs) <ul><li>response time scatter plot: </li></ul>
  • 18. Results Reports (Graphs) <ul><li>throughput time-series plot: </li></ul>
  • 19. Results Reports (Graphs) <ul><li>latency time-series plot: </li></ul>
  • 20. The End <ul><li>visit:   http://multimechanize.com </li></ul>

×