Transcript of "Qualaroo & Optimizely Webinar - 5 Steps to Your First A/B Testing Win"
5 Steps to Your First
A/B Testing Win
Sean Ellis, CEO, Qualaroo (@seanellis)
Sean Ellis is the CEO of Qualaroo and founder of GrowthHackers.com. Prior to Qualaroo, Sean
held marketing leadership roles with breakout companies including Dropbox, LogMeIn (IPO),
Uproar (IPO), Eventbrite and Lookout.
Ryan Lillis, Strategic Optimization Consultant, Optimizely
Ryan Lillis is a web conversion optimization specialist who helps Optimizely clients get the
most out of their A/B, multivariate testing and targeting campaigns. He has worked in the
website optimization space for eight years and has worked with companies as small as 5
people all the way up to Fortune 100 companies.
Why Spend Time on A/B Testing?
$92 dollars spent on acquisition for every $1 on optimization
87% of companies (currently testing) run 1-5 tests per month
Best companies in the world run hundreds per month
Source: Econsultancy & Redeye Conversion Rate Optimization Report 2014
Optimization is a Process
Top 5 highest bounce rate pages
Top 5 abandonment points in your funnel
Top 5 most valuable pages to your business
#1: Find Your Best Opportunities
1. Look at top landing pages
2. Identify those with high
3. Conduct research on
4. Develop tests to improve
Image via: SmartInsights.com
Example: High Bounce Rates
#2: Understand Visitor Needs
Ask and Answer Four Key Questions:!
Why did they come to your website?
What stopped them from converting?
Did they ﬁnd what they were looking for?
If they did convert, what almost stopped them?
Example: Post Conversion Surveys
Testing Ideas to Start:!
4 Message Tests
4 "Aha Moment" Tests
2 Large Scale Design Tests
#4: Where to Start?
Measure impact through the entire funnel, not just at the
Bigger changes yield faster results.
Different levels of statistical signiﬁcance will speed up or
slow down decision making.
#5: Measuring Success