• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
django-lean, Akoha's open-source a/b experimentation framework @ Montreal Python 9
 

django-lean, Akoha's open-source a/b experimentation framework @ Montreal Python 9

on

  • 7,369 views

'django-lean' is an open source a/b split-test experimentation framework for Django. ...

'django-lean' is an open source a/b split-test experimentation framework for Django.

'django-lean' was created by Akoha, a Montreal-based startup building the world's first social reality game.

This presentation was delivered on September 30th, 2009 at Montreal Python 9, by Erik Wright, Director of Engineering for Akoha.

For further details visit:

http://bitbucket.org/akoha/django-lean/
http://akoha.com
http://twitter.com/erikwright

Statistics

Views

Total Views
7,369
Views on SlideShare
7,339
Embed Views
30

Actions

Likes
7
Downloads
0
Comments
0

3 Embeds 30

http://www.slideshare.net 24
http://www.linkedin.com 5
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Erik Wright, Dir. Eng @ AkohaHere to present django-leanAkoha’s recently open-sourced split-testing framework for Django
  • A tool for measuring differences between alternative implementations* font sizes, texts, prices, images, and so on.Recently, use for measuring effectiveness of new features and enhancements is gainingSimultaneously run both implementations,randomly assign users to one or the other, compare performance
  • Rarely confident of the impact our changes will have on users / customersEven when confident, sometimes we are dead wrongSplit testing reduces the risk of wrong decisions / find, correct, and learn from mistakes quickly.
  • Trivially implement split-test experiments in Javascript, Django Templates, PythonMeasures user performance according to metrics you defineGenerates and publishes nightly reports of experiment resultsIncluding calculating statistical significance so that you know when you can stop testingThanks to django-admin, starting, stopping, and promoting tests is easy
  • Now that I’ve told you what django-lean can do, allow me to show you

django-lean, Akoha's open-source a/b experimentation framework @ Montreal Python 9 django-lean, Akoha's open-source a/b experimentation framework @ Montreal Python 9 Presentation Transcript

  • django-lean
    An open-source framework for performing and analyzing a/b split-test experiments in Django
  • 60 sec. intro to split testing
  • What is split testing?
    A technique for comparing two alternative implementations
    Done properly, allows you to eliminate other variables
    For the gory details, seehttp://elem.com/~btilly/effective-ab-testing/
  • Why do people split test?
    Improve conversion, ARPU, or other key metrics systematically
    Eliminate guesswork and opinions
    Determine, scientifically, which implementation is best
    Test your hypotheses
  • Introducing django-lean
  • django-lean
    Experiment
    JavaScript
    Django templates
    Python
    Measure
    User-defined goals
    User-defined metric
    Analyze
    Manage
    • Nightly reports
    • Statistical significance
    • Start, stop, and promote experiments via django-admin
  • Examples
  • Python
  • Django template
  • JavaScript
    In your HTML template:
    In your JavaScript:
  • Reports
  • future
  • Wishlist
    Experimentation improvements
    Multivariate testing
    Arbitrary numbers of test cases
    Tools for automated (integration, unit, ...) testing
    Support more test scenarios
    Conversion funnels
    Decoupling from Django
    More “Lean Startup” tools
    Cohort analysis
    In-site surveys (net promoter score, etc.)
  • Find out more:
    http://bitbucket.org/akoha/django-lean/
    http://akoha.com
    erik@erikwright.com / @erikwright