django-lean, Akoha's open-source a/b experimentation framework @ Montreal Python 9

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

more share options
Embed For WordPress.com

Without related presentations

Notes on slide 1

Erik Wright, Dir. Eng @ AkohaHere to present django-leanAkoha’s recently open-sourced split-testing framework for Django

A tool for measuring differences between alternative implementations* font sizes, texts, prices, images, and so on.Recently, use for measuring effectiveness of new features and enhancements is gainingSimultaneously run both implementations,randomly assign users to one or the other, compare performance

Rarely confident of the impact our changes will have on users / customersEven when confident, sometimes we are dead wrongSplit testing reduces the risk of wrong decisions / find, correct, and learn from mistakes quickly.

Trivially implement split-test experiments in Javascript, Django Templates, PythonMeasures user performance according to metrics you defineGenerates and publishes nightly reports of experiment resultsIncluding calculating statistical significance so that you know when you can stop testingThanks to django-admin, starting, stopping, and promoting tests is easy

Now that I’ve told you what django-lean can do, allow me to show you

2 Favorites

David Paccoud at Home, favorited this 1 month ago
Tags django ab testing
Austin Hill, CEO, Angel Investor, Executive Instigator at Akoha, Brudder Ventures, favorited this 1 month ago
Tags leanstartup akoha

django-lean, Akoha's open-source a/b experimentation framework @ Montreal Python 9 - Presentation Transcript

django-lean
An open-source framework for performing and analyzing a/b split-test experiments in Django
60 sec. intro to split testing
What is split testing?
A technique for comparing two alternative implementations
Done properly, allows you to eliminate other variables
For the gory details, seehttp://elem.com/~btilly/effective-ab-testing/
Why do people split test?
Improve conversion, ARPU, or other key metrics systematically
Eliminate guesswork and opinions
Determine, scientifically, which implementation is best
Test your hypotheses
Introducing django-lean
django-lean
Experiment
JavaScript
Django templates
Python
Measure
User-defined goals
User-defined metric
Analyze
Manage
- Nightly reports
- Statistical significance
- Start, stop, and promote experiments via django-admin
Examples
Python
Django template
JavaScript
In your HTML template:
In your JavaScript:
Reports
future
Wishlist
Experimentation improvements
Multivariate testing
Arbitrary numbers of test cases
Tools for automated (integration, unit, ...) testing
Support more test scenarios
Conversion funnels
Decoupling from Django
More “Lean Startup” tools
Cohort analysis
In-site surveys (net promoter score, etc.)
Find out more:
http://bitbucket.org/akoha/django-lean/
http://akoha.com
erik@erikwright.com / @erikwright