Your SlideShare is downloading. ×
0
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
KISSmetrics Summer 2012 State of AB Testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

KISSmetrics Summer 2012 State of AB Testing

1,668

Published on

3. We’ll cover...1 State of A/B Testing2 Q&A3 Demo …

3. We’ll cover...1 State of A/B Testing2 Q&A3 Demo
4. Let’s start with the data!
5. WHO AREWe?
6. Who are we? 176 in total 60 45 30 15 0 Ecommerce SaaS Agency/Cons Publishing Mobile Other
7. What’s the size of our teams? 90 67.5 45 22.5 0 1-5 6-10 11-25 26-60 51-200 200+
8. What testing tools does everyone use? 90 67.5 45 22.5 0 Visual Website GA Content Optimizely None Other Optimizer Experiments
9. How many A/B tests did we run last month? 150112.5 75 37.5 0 0 1-4 5-9 10-19 20+
10. It’s easy to get ahead of our competitors
11. How do we generate ideas for testing? 60 45 30 15 0 Test Experience Best Practices Research Needs Randomly
12. Step 1: Get Qualitative
13. Step 2: Make a prediction
14. Step 3: Confirm with an A/B test
15. How KISSmetrics generates testing ideas 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
16. OURResults
17. Do we regularly increase conversions? 80 60 40 20 0 Always Sometimes Rarely Never
18. The revenue we gain from A/B testing 70 52.5 35 17.5 0 $1001- $10,001- Over None $1-1,000 No Idea 10,000 100,000 $100,000
19. 2 ways to make money with A/B testing
20. Option 1: Understand your customers first 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
21. You’ll find the BIG wins much faster
22. Option 2: Get relentless
23. How pros like Facebook and Twitter do it 1 Build an entire growth team 2 Back-to-back testing 3 Look for 100s of small improvements
24. How do we A/B test with small amounts of data?
25. Do tests with customer interviews 1 Talk to people, give different options 2 Look for trends 3 Build and see if actions match
26. Do we track results over the long term? 130 97.5 65 32.5 0 Yes No
27. The pitfalls of not tracking long term results
28. Variation B performs the best in this funnel
29. Variation A gives the most revenue
30. LET’S TRACK THE RETURN ON A/B Testing
31. First, make sure you can track lifetime value You’ll need customer analytics that can track multiple purchases from the same person.
32. Where do we find customer analytics?
33. Second, connect A/B tools to customer analytics OR
34. What about Google Analytics? Connects tests to goals, not ecommerce transactions.

Published in: Technology, Education
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,668
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
71
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. SUMMER 2012 STATE OF A/B TESTING Lars Lofgren Marketing Analyst - August 2012 info@kissmetrics.com - Confidential - Do not distribute
  • 2. @larslofgren #KISSwebinar
  • 3. We’ll cover...1 State of A/B Testing2 Q&A3 Demo
  • 4. Let’s start with the data!
  • 5. WHO AREWe?
  • 6. Who are we? 176 in total 60 45 30 15 0 Ecommerce SaaS Agency/Cons Publishing Mobile Other
  • 7. What’s the size of our teams? 90 67.5 45 22.5 0 1-5 6-10 11-25 26-60 51-200 200+
  • 8. What testing tools does everyone use? 90 67.5 45 22.5 0 Visual Website GA Content Optimizely None Other Optimizer Experiments
  • 9. How many A/B tests did we run last month? 150112.5 75 37.5 0 0 1-4 5-9 10-19 20+
  • 10. It’s easy to get ahead of our competitors
  • 11. How do we generate ideas for testing? 60 45 30 15 0 Test Experience Best Practices Research Needs Randomly
  • 12. Step 1: Get Qualitative
  • 13. Step 2: Make a prediction
  • 14. Step 3: Confirm with an A/B test
  • 15. How KISSmetrics generates testing ideas 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
  • 16. OURResults
  • 17. Do we regularly increase conversions? 80 60 40 20 0 Always Sometimes Rarely Never
  • 18. The revenue we gain from A/B testing 70 52.5 35 17.5 0 $1001- $10,001- Over None $1-1,000 No Idea 10,000 100,000 $100,000
  • 19. 2 ways to make money with A/B testing
  • 20. Option 1: Understand your customers first 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
  • 21. You’ll find the BIG wins much faster
  • 22. Option 2: Get relentless
  • 23. How pros like Facebook and Twitter do it 1 Build an entire growth team 2 Back-to-back testing 3 Look for 100s of small improvements
  • 24. How do we A/B test with small amounts of data?
  • 25. Do tests with customer interviews 1 Talk to people, give different options 2 Look for trends 3 Build and see if actions match
  • 26. Do we track results over the long term? 130 97.5 65 32.5 0 Yes No
  • 27. The pitfalls of not tracking long term results
  • 28. Variation B performs the best in this funnel
  • 29. Variation A gives the most revenue
  • 30. LET’S TRACK THE RETURN ON A/B Testing
  • 31. First, make sure you can track lifetime value You’ll need customer analytics that can track multiple purchases from the same person.
  • 32. Where do we find customer analytics?
  • 33. Second, connect A/B tools to customer analytics OR
  • 34. What about Google Analytics? Connects tests to goals, not ecommerce transactions.
  • 35. Q&A Time! Lars Lofgren llofgren@kissmetrics.com @larslofgren

×