Session slides from Future Insights Live, Vegas 2015:
Hiten Shah, co-founder of KISSmetrics and Crazy Egg, talks you through his 4-step data-driven process for successful A/B testing.
30. Increase Conversions
By A/B Testing
Do a Happy Dance
But Watch it Closely
Get More Traﬃc
So You Can Start A/B Testing
Get More Traﬃc
So You Can Do a Happy Dance
High ConversionsLow Conversions
How To Figure Out What To Improve
31. What percentage of your
customers or users have
adopted each feature?
Intercom on Product Management
When you’ve already got usage, analyze it.
32. Where do you have the most
visitors, users, customers,
emails, or notiﬁcations?
More volume helps you test faster.
33. ๏ Website Traﬃc / App Installs
๏ Welcome Emails
๏ Web / Mobile Onboarding
๏ Email Digests
๏ Triggered Notiﬁcations
๏ Cart / Funnel Abandonment
๏ Dormant / Churned users
Common areas with large volume and high impact potential
47. Use simple equations to assess potential impact of each idea
DAN MCKINLEY HTTP://KISS.LY/DANETSY
48. Assess the potential impact,
difficulty and priority of
each test idea you have.
Prioritize your tests
60. 1. Your starting hypothesis.
2. Dates for when you ran the test.
3. Screenshots of control and all variations used during the test.
4. Expected change in conversion.
5. Probability of change.
6. Screenshot of the raw data used in your analysis.
7. A screenshot to the A/B Test Report in KISSmetrics.
8. The decision you've made from the test.
9. What you learned from the test.
Document every A/B test and create a playbook
61. What learnings do we want to
carry over from the past?
Helps you avoid unknowingly
decreasing your conversions.
62. ✓ Keep the footer links
✓ Autofocus on the ﬁrst form ﬁeld
✓ Do not include social proof
✓ Link to the features page
✓ Use “Log in” copy on the bu on
Create experiment checklists based on learnings