Performance and Scalability Testing with Python and Multi-Mechanize

Performance and Scalability Testing with Python and Multi-Mechanize Corey Goldberg 2011 web: goldb.org email: [email_address] twitter: @cgoldberg multimechanize.com

Multi-Mechanize
Open Source (LGPLv3)
Started Development in 2010
Python 2 (2.6+)
Project Hosted on Google Code (SVN)
Website: multimechanize.com
Download
Code Samples / Script Dev Guide
Wiki
Discussion Group

What is Multi-Mechanize?
"Python framework for performance and load testing remote API's"
Enables you to:
run simultaneous client scripts to generate workload against a remote service
run transactions at an increasing series of load levels against your API and measure performance
simulate realistic and projected API usage

Who is Multi-Mechanize For?
Developers
Testers
Ops
... who need to test and tune the performance and scalability of a web site or API service
Assumptions:
You know how to code in Python
You understand basic performance testing

Testing for Performance & Scalability
You have a remote API or website and need to:
stress test server and network infrastructure
find application bottlenecks
expose errors under load
performance benchmarks
capacity planning

Which API's Can You Test?
You write the virtual user (client) scripts in Python, so any remote protocol/lib you want:
Web/HTTP
Raw HTTP (urllib2, httplib)
Web Forms (mechanize)
REST/SOAP/XMLRPC
Sockets
Database, NoSQL, Cache
Any Python API

Things I Like
Free
Runs on multiple platforms (nix, win)
Python code base
Scripts are in Python (batteries included, python libraries)
Scalable load generator
Automatic report generation
Aggregate stats (response times, throughput, percentiles)
Graphs (scatter, time-series)

Inside Multi-Mechanize
Multi-process/Multi-threaded Test Harness
Virtual User Scripting in Python
Configurable Test Runner
Report Generator
Input: Virtual User Test Scripts, Config File
Output: HTML Report with PNG images (graphs)

Virtual User Scripting
You create test scripts in Python for Multi-Mechanize to run.
Each test script must implement a Transaction() class . This class must implement a run() method .
So a basic test script consists of:
class Transaction:
def run(self):
# do something here

During a test run, your Transaction() class is instantiated once, and then its run() method is called repeatedly:
class Transaction:
def __init__(self):
# this gets called once

def run(self):
# this gets called repeatedly

A full script that will issue an HTTP GET using urllib2:

import urllib2
class Transaction(object):
def run(self):
url = 'http://www.example.com/'
urllib2.urlopen(url).read()

web request with timer and assertions:
import mechanize
import time
class Transaction:
def run(self):
self.custom_timers = {}
br = mechanize.Browser()
br.set_handle_robots(False)

start_timer = time.time()
resp = br.open('http://www.example.com/')
resp.read()
latency = time.time() - start_timer

self.custom_timers['Example_Homepage'] = latency

assert (resp.code == 200), 'Bad HTTP Response'
assert ('Example Web Page' in resp.get_data()),
'Failed Content Verification'

Test Configuration
sample config file:
[global]
run_time: 300
rampup: 300
console_logging: off
results_ts_interval: 60

[user_group-1]
threads: 10
script: search_simple.py

[user_group-2]
threads: 5
script: search_complex.py

Running a Test

Results Reports

Results Reports (Stats)
test summary
transaction timers
custom timers (from instrumented client code)
time-series/interval data
counts
rate/throughput
response times
average, min, max, stdev
percentiles (80th, 90th, 95th)

Results Reports (Graphs)
response time scatter plot:

throughput time-series plot:

latency time-series plot:

The End
visit: http://multimechanize.com

http://coreygoldberg.blogspot.com	4100
http://feeds.feedburner.com	181
http://coreygoldberg.blogspot.in	72
http://www.coreygoldberg.blogspot.com	17
http://coreygoldberg.blogspot.com.au	9
http://translate.googleusercontent.com	5
http://coreygoldberg.blogspot.fr	4
http://feeds.nytimes.com	4
http://webcache.googleusercontent.com	3
http://coreygoldberg.blogspot.de	3
http://static.slidesharecdn.com	2
http://coreygoldberg.blogspot.co.uk	2
http://207.46.192.232	2
http://coreygoldberg.blogspot.com.es	2
http://coreygoldberg.blogspot.ca	2
http://coreygoldberg.blogspot.com.br	1
http://coreygoldberg.blogspot.jp	1
http://coreygoldberg.blogspot.mx	1
http://www.directrss.co.il	1
http://www.coreygoldberg.blogspot.in	1
http://tpbitalia.appspot.com	1
http://feeds2.feedburner.com	1
http://paper.li	1
http://www.goldb.org	1
http://localhost	1
http://www.netvibes.com	1
http://coreygoldberg.blogspot.se	1

Performance and Scalability Testing with Python and Multi-Mechanize

by coreygoldberg on Mar 30, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

27 Embeds 4,420

Statistics

Performance and Scalability Testing with Python and Multi-Mechanize — Presentation Transcript