Ab Testing 101 Live Webinar Slides
•
0 likes•190 views
These are the slides of the A/B Testing 101 live webinar. --» Video: https://www.youtube.com/watch?v=Had65TUj3gg&t --» Check my website: https://data36.com --» For more similar content, subscribe to my newsletter: https://data36.com/newsletter-subscription/ --» Find me on Twitter: https://twitter.com/data36_com ************* Topic: A/B testing 101 Date: 16th May 2017 – 6.30pm, CEST Location: Online (Sign up for the link!) Brief: In this 30 minutes webinar I’ll give you an introduction into A/B testing. What is A/B testing? Why is it needed? The 5+1 rules of A/B testing. How to set it up? How to evaluate? Some A/B testing pitfalls to avoid. With case studies and with my personal best practices. It’s for you if you are: a business/marketing professional running an online business (startup, ecommerce, etc). Or an aspiring data analyst/scientist. Agenda: - Intro: A story, where analytics has failed. - What is A/B testing? Why is it needed? - Case studies, examples. - The 5+1 rules of A/B testing. - How to evaluate an A/B test? - Pitfalls to avoid. - Q&A
Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Ab Testing 101 Live Webinar Slides
- 1. A/B testing 101 Tomi Mester author of data36.com twitter: @data36_com —we start at 6.30pm— Q&A and comments: slido.com/data36
- 2. the pictures are illustrations Anna Eva twitter: @data36_com
- 3. the pictures are illustrations Anna Eva “I really like your sweater, so I think we could hang out in the next lunch break. XOXO” “I really like your backpack, so I think we could hang out in the next lunch break. XOXO” twitter: @data36_com
- 4. 0% conversion 0% conversion the pictures are illustrations Anna Eva “I really like your sweater, so I think we could hang out in the next lunch break. XOXO” “I really like your backpack, so I think we could hang out in the next lunch break. XOXO” twitter: @data36_com
- 6. WHY? twitter: @data36_comQ&A and comments: slido.com/data36
- 7. twitter: @data36_com ONLINE SAAS COMPANY
- 8. twitter: @data36_com ONLINE SAAS COMPANY Situation: There is a 5-mins video on the support site.
- 9. twitter: @data36_com ONLINE SAAS COMPANY Data analysis shows: Situation: There is a 5-mins video on the support site.
- 10. “Those visitors, who watched it, registered to the service with a +30% probability!” twitter: @data36_com ONLINE SAAS COMPANY Data analysis shows: Situation: There is a 5-mins video on the support site.
- 11. “Those visitors, who watched it, registered to the service with a +30% probability!” twitter: @data36_com ONLINE SAAS COMPANY Data analysis shows: Conclusion: “The video plays a significant role in the registration flow, move it to the front page!” Situation: There is a 5-mins video on the support site.
- 12. “Those visitors, who watched it, registered to the service with a +30% probability!” twitter: @data36_com ONLINE SAAS COMPANY Data analysis shows: Conclusion: “The video plays a significant role in the registration flow, move it to the front page!”WRONG Situation: There is a 5-mins video on the support site.
- 15. twitter: @data36_com POSSIBLE REASONS video commitment
- 16. twitter: @data36_com POSSIBLE REASONS video commitment registration
- 17. twitter: @data36_com POSSIBLE REASONS video commitment registration commitment
- 18. twitter: @data36_com POSSIBLE REASONS video commitment registration commitment video
- 19. twitter: @data36_com POSSIBLE REASONS video commitment registration commitment registration video
- 20. twitter: @data36_com POSSIBLE REASONS video commitment registration commitment commitment registration video
- 21. twitter: @data36_com POSSIBLE REASONS video commitment registration videocommitment commitment registration video
- 22. twitter: @data36_com POSSIBLE REASONS video commitment registration videocommitment registration commitment registration video
- 23. correlation vs. causation twitter: @data36_com aka. chicken or egg?
- 24. twitter: @data36_com POSSIBLE REASONS video commitment registration videocommitment registration video commitment registration
- 25. The only way to find it out: twitter: @data36_com A/B testing
- 26. 1. Do a 50-50% split. Test Group: remove the video Control Group: keep the video twitter: @data36_com
- 27. 1. Do a 50-50% split. Test Group: remove the video Control Group: keep the video twitter: @data36_com 2. Do a 50-50% split. Test Group: move the video to the front page Control Group: don’t move the video
- 32. source: VWO.com
- 33. twitter: @data36_com DEMO (Optimizely) Q&A and comments: slido.com/data36
- 34. twitter: @data36_com The 5+1 rules of A/B testing
- 36. 1. Define measurable goals before the test twitter: @data36_com
- 37. 1. Define measurable goals before the test 2. The different versions should run parallel twitter: @data36_com
- 38. 1. Define measurable goals before the test 2. The different versions should run parallel 3. The test and the control group should be selected randomly twitter: @data36_com
- 39. 1. Define measurable goals before the test 2. The different versions should run parallel 3. The test and the control group should be selected randomly 4. Don’t let the Users to know they are taking part in the test twitter: @data36_com
- 40. 1. Define measurable goals before the test 2. The different versions should run parallel 3. The test and the control group should be selected randomly 4. Don’t let the Users to know they are taking part in the test 5. Before evaluation - look at significance level twitter: @data36_com
- 41. 1. Define measurable goals before the test 2. The different versions should run parallel 3. The test and the control group should be selected randomly 4. Don’t let the Users to know they are taking part in the test 5. Before evaluation - look at significance level twitter: @data36_com
- 42. 1. Define measurable goals before the test 2. The different versions should run parallel 3. The test and the control group should be selected randomly 4. Don’t let the Users to know they are taking part in the test 5. Before evaluation - look at significance level +1 Change one thing at a time twitter: @data36_com
- 43. twitter: @data36_com How to evaluate an A/B test?
- 45. 1. Look on the significance level! twitter: @data36_com
- 46. 1. Look on the significance level! twitter: @data36_com
- 47. 1. Look on the significance level! 2. Look on the main KPI! twitter: @data36_com
- 49. Don’t evaluate before significance is 95% or above. twitter: @data36_com
- 50. Version A CONVERSION RATE Version B CONVERSION RATE 10% 20% twitter: @data36_com
- 51. Version A CONVERSION RATE Version B CONVERSION RATE 10% 20% twitter: @data36_com
- 52. Version A CONVERSION RATE Version B CONVERSION RATE 10% 20% “omg, 100% uplift!!!!” twitter: @data36_com
- 53. Version A CONVERSION RATE Version B CONVERSION RATE sample size: 10 converted: 1 sample size: 10 converted: 2 twitter: @data36_com
- 54. Version A CONVERSION RATE Version B CONVERSION RATE sample size: 10000 converted: 1000 sample size: 10000 converted: 2000 twitter: @data36_com
- 55. Version A CONVERSION RATE Version B CONVERSION RATE sample size: 10000 converted: 1000 sample size: 10000 converted: 2000 “omg, 100% uplift!!!!” twitter: @data36_com
- 56. Don’t change anything, but bugs on a running A/B test. twitter: @data36_com
- 57. Don’t use an A/B testing software (Optimizely) for 100% changes. twitter: @data36_com
- 58. twitter: @data36_com And one more thing…
- 61. data36.com/newsletter Tomi Mester data analyst @iZettle twitter: @data36_comQ&A and comments: slido.com/data36