Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

8

Share

Download to read offline

A/B Testing for Lean Startups

Download to read offline

Primer on A/B testing for your startup.

Presented to Mucker Labs, February 27, 2012

Related Books

Free with a 30 day trial from Scribd

See all

A/B Testing for Lean Startups

  1. 1. A/B Testing for Lean StartupsEverything you ever Secrets from thewanted to know experts revealedabout split testing. Watch Presentation Learn More Pete Mauro @mauronic Mucker Labs, February 27, 2012
  2. 2. In 2009 Tropicana wanted a new juice box
  3. 3. So they redesigned the package New box launched in thousands of stores nationwide
  4. 4. How did it go? • “Ugly, stupid, cheap and generic looking” • Ad agency defends it • Sales plunge 20% • Agency CEO fired • Total cost: $33 Million
  5. 5. 8 Months +$33 Million Dollars = This Package Sucks
  6. 6. Very un-lean, dude
  7. 7. Lean Startups are Data Driven
  8. 8. Lean Startups Listen to their Customers
  9. 9. Lean Startups are Scientific
  10. 10. The Scientific Method Ask Question Do research Hypothesis Analyze Data Conclusion
  11. 11. Believe it or not, some people leave Facebook
  12. 12. Let’s pretend we work at FacebookAsk Question How can we stop more people from deactivating their Facebook account?Do research The reason why people use Facebook is because their friends are on Facebook.Hypothesis If you remind people about their friends who are being left behind, we will lower the deactivation rate.
  13. 13. Facebook’s Actual A/B Test Control Experiment Source: Julie Zhou, Facebook
  14. 14. The guilt trip worked Control Experiment ✔ Reduced Deactivation Rate by 7% Source: Julie Zhou, Facebook
  15. 15. Quiz: Guess the Winner A BSource: http://www.abtests.com/
  16. 16. 5 Quiz: Guess the Winner Points A B ✔ 80% improvement on sign upsSource: http://www.abtests.com/
  17. 17. Quiz: Guess the Winner A BSource: http://www.abtests.com/
  18. 18. 10 Quiz: Guess the Winner Points A BSource: http://www.abtests.com/ ✔ +60% signups
  19. 19. Designing a Test• Anyone at the company should contribute• Ideas can come from user testing• Use best practices• Be creative, try crazy ideas• Use common sense: don’t piss off your users for the sake of metrics
  20. 20. The 41 Shades of Blue Test at Google Link Color
  21. 21. The Local MaximaSource: Joshua Porter
  22. 22. A/B Testing Tools• Kissmetrics• Visual Website Optimizer• Optimizely• Unbounce• Mailchimp (email campaigns)• Google Website Optimizer• And more…
  23. 23. Mailchimp Split Test
  24. 24. Further Reading• Metrics Driven Design (See Joshua Porter on Slideshare)• The New Science of Product Development (See James Birchler on Slideshare)• Quora: “What are the most unexpected things people have learned from A/B tests?”• Abtests.com & whichtestwon.com
  25. 25. Thanks!Follow me I’m on Twitter@mauronic @mauronic
  • beluckyy

    Mar. 27, 2016
  • lofipunk

    Jan. 29, 2015
  • torrg

    Aug. 1, 2014
  • lcquadros

    Apr. 25, 2014
  • KostasZhukov

    Jan. 28, 2014
  • Jojas

    Nov. 25, 2013
  • a_noskov

    Jun. 10, 2013
  • ssuser05c1fe

    Apr. 3, 2012

Primer on A/B testing for your startup. Presented to Mucker Labs, February 27, 2012

Views

Total views

6,821

On Slideshare

0

From embeds

0

Number of embeds

381

Actions

Downloads

104

Shares

0

Comments

0

Likes

8

×