0
5 Steps to Your First
A/B Testing Win
Sean Ellis, CEO, Qualaroo (@seanellis)
Sean Ellis is the CEO of Qualaroo and founder of GrowthHackers.com. Prior to Qualar...
#winningtests
Why Spend Time on A/B Testing?
$92 dollars spent on acquisition for every $1 on optimization

87% of companies (currently ...
Optimization is a Process
Quantitative

Insights
Qualitative

Insights
Testing
#winningtests
Understanding Conversion
Optimization Goals
image via: http://www.shayhowe.com/web-design/defining-multivariate-testing/
Brainstorm
Test Ideas
Make a Small Change
See ...
Ideal Testing Process
User Research Lots of Testing Ideas
Quantitative

Insights
Qualitative

Insights
Testing
Continuous ...
Guessing is Expensive
Top 5 highest bounce rate pages
Top 5 abandonment points in your funnel
Top 5 most valuable pages to your business
#1: Fin...
1. Look at top landing pages
2. Identify those with high
bounce rates
3. Conduct research on
those pages
4. Develop tests ...
#2: Understand Visitor Needs
Ask and Answer Four Key Questions:!
Why did they come to your website?
What stopped them from...
Example: Post Conversion Surveys
#winningtests
Use Data & Insights to
Drive Testing
Start small to get a win and validation for process
Test for impact, avoid ‘Meek Tweaking’
Use test result data to inform ...
Image via Conversion Rate Experts
Meek Tweaking
#winningtests
Running Your 

First Tests
Testing Ideas to Start:!
4 Message Tests
4 "Aha Moment" Tests
2 Large Scale Design Tests
#4: Where to Start?
#winningtests
Measure impact through the entire funnel, not just at the
page level.
Bigger changes yield faster results.
Different level...
Putting It All
Together
Quantitative

Insights
Qualitative

Insights
Testing
Optimization is a Process
#winningtests
1. Plan and run your first five to ten tests.
2. Get your first win.
3. Start the cycle of continuous improvement.
Next Steps...
qualaroo.com/beginners-guide-to-cro/
Resources
optimize.ly/datadna
#winningtests
Q & A
#winningtests
Upcoming SlideShare
Loading in...5
×

Qualaroo & Optimizely Webinar - 5 Steps to Your First A/B Testing Win

10,231

Published on

Published in: Data & Analytics, Technology
1 Comment
13 Likes
Statistics
Notes
No Downloads
Views
Total Views
10,231
On Slideshare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
63
Comments
1
Likes
13
Embeds 0
No embeds

No notes for slide

Transcript of "Qualaroo & Optimizely Webinar - 5 Steps to Your First A/B Testing Win"

  1. 1. 5 Steps to Your First A/B Testing Win
  2. 2. Sean Ellis, CEO, Qualaroo (@seanellis) Sean Ellis is the CEO of Qualaroo and founder of GrowthHackers.com. Prior to Qualaroo, Sean held marketing leadership roles with breakout companies including Dropbox, LogMeIn (IPO), Uproar (IPO), Eventbrite and Lookout. Ryan Lillis, Strategic Optimization Consultant, Optimizely (@Optimizely) Ryan Lillis is a web conversion optimization specialist who helps Optimizely clients get the most out of their A/B, multivariate testing and targeting campaigns. He has worked in the website optimization space for eight years and has worked with companies as small as 5 people all the way up to Fortune 100 companies.
  3. 3. #winningtests
  4. 4. Why Spend Time on A/B Testing? $92 dollars spent on acquisition for every $1 on optimization
 87% of companies (currently testing) run 1-5 tests per month
 Best companies in the world run hundreds per month Source: Econsultancy & Redeye Conversion Rate Optimization Report 2014 #winningtests
  5. 5. Optimization is a Process Quantitative
 Insights Qualitative
 Insights Testing #winningtests
  6. 6. Understanding Conversion Optimization Goals
  7. 7. image via: http://www.shayhowe.com/web-design/defining-multivariate-testing/ Brainstorm Test Ideas Make a Small Change See Mixed or No Results Typical Testing Process #winningtests
  8. 8. Ideal Testing Process User Research Lots of Testing Ideas Quantitative
 Insights Qualitative
 Insights Testing Continuous Process 
 of Learning #winningtests
  9. 9. Guessing is Expensive
  10. 10. Top 5 highest bounce rate pages Top 5 abandonment points in your funnel Top 5 most valuable pages to your business #1: Find Your Best Opportunities #winningtests
  11. 11. 1. Look at top landing pages 2. Identify those with high bounce rates 3. Conduct research on those pages 4. Develop tests to improve page performance Image via: SmartInsights.com Example: High Bounce Rates #winningtests
  12. 12. #2: Understand Visitor Needs Ask and Answer Four Key Questions:! Why did they come to your website? What stopped them from converting? Did they find what they were looking for? If they did convert, what almost stopped them? #winningtests
  13. 13. Example: Post Conversion Surveys #winningtests
  14. 14. Use Data & Insights to Drive Testing
  15. 15. Start small to get a win and validation for process Test for impact, avoid ‘Meek Tweaking’ Use test result data to inform the next test #3: Use Data & Insights to Inform Testing #winningtests
  16. 16. Image via Conversion Rate Experts Meek Tweaking #winningtests
  17. 17. Running Your 
 First Tests
  18. 18. Testing Ideas to Start:! 4 Message Tests 4 "Aha Moment" Tests 2 Large Scale Design Tests #4: Where to Start? #winningtests
  19. 19. Measure impact through the entire funnel, not just at the page level. Bigger changes yield faster results. Different levels of statistical significance will speed up or slow down decision making. #5: Measuring Success #winningtests
  20. 20. Putting It All Together
  21. 21. Quantitative
 Insights Qualitative
 Insights Testing Optimization is a Process #winningtests
  22. 22. 1. Plan and run your first five to ten tests. 2. Get your first win. 3. Start the cycle of continuous improvement. Next Steps #winningtests
  23. 23. qualaroo.com/beginners-guide-to-cro/ Resources optimize.ly/datadna #winningtests
  24. 24. Q & A #winningtests
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×