Abingo

1,226 views

Published on

Talk at Philly.rb 7/13/2010

  • Be the first to comment

Abingo

  1. 1. A/B and A/Bingo Rob Di Marco
  2. 2. A/B Testing • Test (Version A and Version B) • Conversion Event
  3. 3. Who Cares?
  4. 4. A/B Helps If... • Seeking quantitative answer over taste • Looking to avoid HIPPO • HIghest Paid Persons Opinion • Have a clear test/conversion point
  5. 5. A/Bingo is.. • Methods for ActionController::Base and ActionView::Base • ActiveRecord objects • Experiment • Alternative • Dashboard
  6. 6. Install A/Bingo ‣ script/plugin install git:// git.bingocardcreator.com/abingo.git ‣ script/generate abingo_migration ‣ rake db:migrate ‣ script/generate controller abingo_dashboard
  7. 7. Before Using... • Add a before_filter that sets the ABingo identity • Normally, some value that can be tested across requests • User ID if logged in
  8. 8. A/Bingo a Signup Form • Define a test with ab_test • views/users/new.html.erb • Define a conversion with bingo! • controllers/users_controllers.rb • Go...
  9. 9. Forcing Options • test_name=<value> • signup_title=Registration • signup_intro=true • Useful in functional tests and real life
  10. 10. The Dashboard • Shows tests • Shows significance of tests • 95% certainty is normal standard • Statistical certainty != Impact
  11. 11. `
  12. 12. A/Bingo Coolness
  13. 13. Easy Install
  14. 14. Supports client or server side tests
  15. 15. Easy to judge progress
  16. 16. No URL changes required
  17. 17. Multi-Request/Session Tracking
  18. 18. Minuses....
  19. 19. No cohort selection
  20. 20. No over-weighting of alternatives
  21. 21. Unconfident Multi-variate
  22. 22. Before A/B Testing, ponder this...
  23. 23. Size == Confidence
  24. 24. Can You Convert?
  25. 25. Representative Samples
  26. 26. Multiple Tests / Correlation Effects
  27. 27. Tactics vs. Strategy
  28. 28. Today’s code • Railscast Episode 214 • http://github.com/ryanb/railscasts-episodes/ tree/master/episode-214/
  29. 29. Links • http://www.bingocardcreator.com/abingo • http://abtester.com/calculator/ • http://20bits.com/articles/statistical-analysis- and-ab-testing/
  30. 30. Alternatives • 7 Minute Abs - http://github.com/paulmars/ seven_minute_abs • Vanity - http://vanity.labnotes.org/ • Google Website Analyzer - https:// www.google.com/analytics/siteopt/exptlist • Omniture Test and Target - http:// www.omniture.com/en/products/ conversion/testandtarget

×