A/B testing #eMetrics London
•Download as PPTX, PDF•
0 likes•233 views
product development, a/b testing, case studies
Recommended
More Related Content
What's hot
What's hot (9)
Viewers also liked
Viewers also liked (20)
Similar to A/B testing #eMetrics London
Similar to A/B testing #eMetrics London (20)
Recently uploaded
Recently uploaded (20)
A/B testing #eMetrics London
- 6. Pre Test Significances A/A Testing
- 13. The Viral Gate Test
- 18. Daily Active Users 7 Day Moving Average 1,440,000 1,420,000 1,400,000 1,380,000 1,360,000 1,340,000 Control Test 1,320,000 1,300,000 1,280,000 1,260,000
- 19. 3 Day Retention 18.60 18.40 18.20 18.00 17.80 17.60 17.40 Control Test 17.20 17.00 16.80 16.60 16.40
- 21. Viral gate
- 22. Too good to be true?
- 23. The Opportunity Cost Test Could This Be Making More Money?
- 27. Daily Active Users 700,000 600,000 500,000 400,000 Cheap Plasma Expensive Plasma 300,000 200,000 100,000 0 2/28/13 3/7/13 3/14/13 3/21/13 3/28/13 4/4/13 4/11/13 4/18/13 4/25/13 5/2/13 5/9/13
- 28. Plasma bubbles sold 200,000 180,000 160,000 140,000 120,000 100,000 Cheap Plasma Expensive Plasma 80,000 60,000 40,000 20,000 0 3/6/13 3/13/13 3/20/13 3/27/13 4/3/13 4/10/13 4/17/13 4/24/13 5/1/13 5/8/13
- 29. Fire bubbles sold 100,000 90,000 80,000 70,000 60,000 50,000 Cheap Plasma Expensive Plasma 40,000 30,000 20,000 10,000 0 3/6/13 3/13/13 3/20/13 3/27/13 4/3/13 4/10/13 4/17/13 4/24/13 5/1/13 5/8/13
- 31. Long Run Testing
- 32. How long should I keep this test running for? Long enough
- 39. Could this be better?
- 40. Why did I not test this?
- 48. Make your life easy. Run an A/B test
- 49. • A/A Testing • Too good to be true (?) • Could this be better? • Long run tests • Test everything
Editor's Notes
- Insert images of our charaters
- $ 7.5 bn in 2013$ 5.5 bn USA by 2015
- If you have tests that show positive improvements, and when switched live they have little or a negative effect…. This might be the cause
- We’re going now into gamification territory
- The difference in the two groups is about 40-50,000 users.
- The difference is between – 0.4 and 0.8 percentage points at 95% confidence.Also looks like the gap is widening…. And the other retention measure showed a similar behaviour.
- Running the opportunity cost calculations I discovered that we could be making on average $1,500 / day more switching to the expensive group.