17. Simple 4-step experimentation process
Data
Figure out what to improve
Hypothesis
Make an educated guess
Experiment
Test your guess
Act
Decide what’s next
29. Increase Conversions
By A/B Testing
Do a Happy Dance
But Watch it Closely
Get More Traffic
So You Can Start A/B Testing
Get More Traffic
So You Can Do a Happy Dance
High Traffic
Low Traffic
High ConversionsLow Conversions
How To Figure Out What To Improve
30. Where do you have the most
visitors, users, customers,
emails, or notifications?
More volume helps you test faster.
31. ๏ Website Traffic / App Installs
๏ Welcome Emails
๏ Web / Mobile Onboarding
๏ Email Digests
๏ Triggered Notifications
๏ Cart / Funnel Abandonment
๏ Dormant / Churned users
Common areas with large volume and high impact potential
32. What percentage of your
customers or users have
adopted each feature?
Intercom on Product Management
When you’ve already got usage, analyze it.
46. Use simple equations to assess potential impact of each idea
DAN MCKINLEY HTTP://KISS.LY/DANETSY
visits x conversion rate x order value x li4 = ?
users x order value x resurrec7on rate = ?
ESTIMATED
ESTIMATED
MODELED
MODELED
47. Assess the potential impact,
difficulty and priority of
each test idea you have.
Prioritize your tests
59. 1. Your starting hypothesis.
2. Dates for when you ran the test.
3. Screenshots of control and all variations used during the test.
4. Expected change in conversion.
5. Probability of change.
6. Screenshot of the raw data used in your analysis.
7. A screenshot to the A/B Test Report in KISSmetrics.
8. The decision you've made from the test.
9. What you learned from the test.
Document every A/B test and create a playbook
60. What learnings do we want to
carry over from the past?
Helps you avoid unknowingly
decreasing your conversions.
61. ✓ Keep the footer links
✓ Autofocus on the first form field
✓ Do not include social proof
✓ Link to the features page
✓ Use “Log in” copy on the button
Create experiment checklists based on learnings