vbench: lightweight
performance testing
Wes McKinney
@wesmckinn
PyCon 2012
Sunday, March 11,

Why do we test?
Sunday, March 11,

Freedom from fear
Sunday, March 11,

Testing for
performance?
Sunday, March 11,

Who made my code slower???
Sunday, March 11,

Sunday, March 11,

Who made my code slower???
Sunday, March 11,

Me
Sunday, March 11,

speed.pypy.org is a
one-off solution
Sunday, March 11,

Stop. Help is near
• GitHub: wesm/vbench
• Integrates with git: runs benchmarks for
each revision in your repo
• Persists results locally in SQLite
• Generates graphs using matplotlib
Sunday, March 11,

Catch performance
regressions sooner
Sunday, March 11,

Writing vbenchmarks
setup = common_setup + """
setup
values = np.concatenate([np.arange(100000),
np.random.randn(100000),
np.arange(100000)])
s = Series(values)
"""
stats_rank_average = Benchmark('s.rank()', setup)
Sunday, March 11,

Use them in your
workflow
In [2]: stats_rank_average.run()
Out[2]:
{'loops': 10,
'repeat': 3,
'succeeded': True,
'timing': 33.135390281677246,
'units': 'ms'}
Same code as %timeit in IPython
Sunday, March 11,

Get involved
• A useful weekend hack
• git bisect integration
• More version control systems
• Upload results to codespeed instance
• More setup/teardown control
Sunday, March 11,

GitHub: wesm/vbench
@wesmckinn
Sunday, March 11,