SlideShare a Scribd company logo
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
@shanelle_mullin
We’ll cover…
1. How to conduct conversion research to find smarter A/B test ideas.
2. How to prioritize your A/B test ideas in a meaningful way.
3. What a high velocity testing program is and why it works.
4. Common high velocity testing program mistakes to avoid.
5. How to increase your testing velocity responsibly.
How to Run Smarter A/B Tests, Faster
“You don’t have generic problems, you
have specific problems.”
~Peep Laja
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
1. Relevancy
Does the page’s content meet user expectation?
Does the page’s design meet user expectation?
How can it match what they expect / want even more?
How to Run Smarter A/B Tests, Faster
2. Clarity
Is the content on this page as clear as possible?
Is the value prop on this page as clear as possible?
How can we make it clearer?
How to Run Smarter A/B Tests, Faster
3. Motivation
What’s motivating the visitor to take action?
Is it clear, believable, timely and persuasive?
How can we increase motivation even more?
How to Run Smarter A/B Tests, Faster
4. Friction
What is causing doubts, hesitations and uncertainties?
What makes the process difficult?
How can we simplify the process?
How to Run Smarter A/B Tests, Faster
5. Distraction
What’s not helping the user take action?
What’s drawing attention?
Is everything that draws attention really necessary?
How to Run Smarter A/B Tests, Faster
Cross-Browser Testing
Audience > Technology > Browser & OS
Switch from the default “Data” view to the “Comparison” view.
How to Run Smarter A/B Tests, Faster
Cross-Browser Testing
Always compare within the same browser family.
How to Run Smarter A/B Tests, Faster
Cross-Device Testing
Audience > Mobile > Devices
Switch from the default “Data” view to the “Comparison” view.
How to Run Smarter A/B Tests, Faster
Speed Analysis
Page Load Time: Seconds until every element on the
page is done loading.
Page Interactive Time: Seconds until the site is usable.
How to Run Smarter A/B Tests, Faster
Speed Analysis
Your site loads in 3 seconds or less.
Your site loads in 4 to 7 seconds.
Your site loads in 7 to 10 seconds (or more).
How to Run Smarter A/B Tests, Faster
Speed Analysis
Behavior > Site Speed > Page Timings
Switch from the default “Data” view to the “Comparison” view.
How to Run Smarter A/B Tests, Faster
Google Analytics Health Check
1. Are you collecting all of the data you need?
2. Can you trust the data you are collecting?
3. Is anything broken or tracking / reporting
incorrectly? Why?
How to Run Smarter A/B Tests, Faster
Common Google Analytics Mistakes
1. Pages are missing.
2. Data is different in the shopping cart tool.
3. Cross-domain tracking isn’t working.
How to Run Smarter A/B Tests, Faster
Full Health Check Guide
bit.ly/gahealthcheck
How to Run Smarter A/B Tests, Faster
Start with the Highest Value
1. High volume, low value pages.
2. Low volume, high value pages.
How to Run Smarter A/B Tests, Faster
Finding Broken Links
How to Run Smarter A/B Tests, Faster
Finding Broken Links
How to Run Smarter A/B Tests, Faster
Auditing Your Funnel
The further down the funnel, the smaller the
necessary impact.
How to Run Smarter A/B Tests, Faster
Auditing Your Funnel
How to Run Smarter A/B Tests, Faster
Auditing Your Funnel
How to Run Smarter A/B Tests, Faster
Exploring Internal Search
How to Run Smarter A/B Tests, Faster
Segmentation Is the Secret
bit.ly/gasegmentation
How to Run Smarter A/B Tests, Faster
Click Maps
1. This info is largely available in Google Analytics via
“Navigation Summary”.
2. See when people are clicking on non-links.
How to Run Smarter A/B Tests, Faster
Scroll Maps
1. Better prioritize your content and messaging.
2. Drastic colour changes mean a need for visual
cues… likely indicate false bottom.
How to Run Smarter A/B Tests, Faster
Session Replays
1. Real users with real money.
2. Where do they pause? Where do they give up?
3. Use this to gather more insight on poor performers.
How to Run Smarter A/B Tests, Faster
Form Analytics
1. Which form fields cause the most error messages?
2. Which do people hesitate to fill?
3. Which do people leave empty, even if required?
How to Run Smarter A/B Tests, Faster
On-Site Surveys
Exit surveys.
On-page surveys.
How to Run Smarter A/B Tests, Faster
On-Site Survey Questions
1. What was the purpose of your visit to our site?
2. Were you able to complete your task?
3. If not, why?
4. Is there anything holding you back?
5. Do you have unanswered questions?
Focus on actionable data, on the friction.
How to Run Smarter A/B Tests, Faster
On-Site Survey Options
1 open-ended question.
Y/N and then an explanation post-answer.
How to Run Smarter A/B Tests, Faster
On-Site Survey Guide
bit.ly/onsitesurveys
How to Run Smarter A/B Tests, Faster
Customer Surveys
1. First time buyers.
2. No previous relationship with you.
3. Try for ~200 responses.
How to Run Smarter A/B Tests, Faster
Customer Survey Questions
1. Who are you?
2. What are you using this product for?
3. What problem does it solve for you?
4. What do you like about the product most?
5. Did you consider any alternatives?
6. Why did you choose us over the competition?
7. What’s the one thing that nearly stopped you from buying?
8. What was your biggest concern or fear about us?
How to Run Smarter A/B Tests, Faster
User Testing Tasks
1. A specific task.
2. A broad task.
3. Funnel completion.
How to Run Smarter A/B Tests, Faster
Constant Ideation
1. Continue conducting conversion research.
2. Involve the entire team.
3. Focus on the entire funnel (acquisition to referral).
How to Run Smarter A/B Tests, Faster
“If you come to me with an idea and it’s not
live in 2 weeks, it’s not because it’s a bad
idea. It’s because I have better things to test.”
~Pauline Marol
How to Run Smarter A/B Tests, Faster
The Prioritization Problem
1. Not objective enough.
2. Not data-driven enough.
3. Not easily customized.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Benefits of PXL
1. Makes “potential” more objective.
2. Makes “ease” more objective.
3. Helps create a data-driven culture.
4. Easy to customize.
How to Run Smarter A/B Tests, Faster
Prioritization Spreadsheet
bit.ly/pxlmodel
How to Run Smarter A/B Tests, Faster
Better Management
1. Review KPIs.
2. Review tests launched (and not launched).
3. Review insights gained.
4. Choose tests for the upcoming week.
5. Discuss resources (or lack thereof).
How to Run Smarter A/B Tests, Faster
Archive (Or Live to Regret It)
1. Avoid repeating tests by mistake.
2. Easily communicate wins and insights.
3. Improve the quality of future tests.
4. Create an insights > wins culture.
How to Run Smarter A/B Tests, Faster
High Velocity Mistakes
1. Without buy-in and a testing culture, you’re
doomed.
2. Calling it too soon and non-representative samples.
3. Letting quality and effectiveness slip.
How to Run Smarter A/B Tests, Faster
Testing Capacity
(52 / Avg. Req. Test Duration) x Number of Testable Pages
(52 / 2) x 10 = 260
How to Run Smarter A/B Tests, Faster
Measuring Velocity
1. How well are you trending towards achieving your
capacity?
2. Trends are what matters… are you getting closer?
3. Chart it.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
@shanelle_mullin
1. Conduct proper conversion research.
2. Come up with ideas constantly.
3. Prioritize with data.
4. Manage the process.
5. Archive your learnings.
6. Achieve your testing capacity.
7. Stop listening to CRO BS you read online.
How to Run Smarter A/B Tests, Faster
Resources
Optimization Training Program
bit.ly/cxlinstitute
Original UX Research
bit.ly/cxlresearch
How to Run Smarter A/B Tests, Faster
Questions?
Ask any time via email: shanelle@conversionxl.com

More Related Content

What's hot

Ab testing 101
Ab testing 101Ab testing 101
Ab testing 101
Ashish Dua
 
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision MakingData-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
indeedeng
 
Weapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
Weapons of Math Instruction: Evolving from Data0-Driven to Science-DrivenWeapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
Weapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
indeedeng
 
Anyone Can Write User Stories. It's the (Shared) Understanding That's Important
Anyone Can Write User Stories. It's the (Shared) Understanding That's ImportantAnyone Can Write User Stories. It's the (Shared) Understanding That's Important
Anyone Can Write User Stories. It's the (Shared) Understanding That's Important
Kent McDonald
 
Optimizely Workshop: Take Action on Results with Statistics
Optimizely Workshop: Take Action on Results with StatisticsOptimizely Workshop: Take Action on Results with Statistics
Optimizely Workshop: Take Action on Results with Statistics
Optimizely
 
Content Testing & Optimization (CTO) Webinar, April 29 2010
Content Testing & Optimization (CTO) Webinar, April 29 2010Content Testing & Optimization (CTO) Webinar, April 29 2010
Content Testing & Optimization (CTO) Webinar, April 29 2010One to One
 
eMetrics London - The AB Testing Hype Cycle
eMetrics London - The AB Testing Hype CycleeMetrics London - The AB Testing Hype Cycle
eMetrics London - The AB Testing Hype Cycle
Craig Sullivan
 
Talks@Coursera - A/B Testing @ Internet Scale
Talks@Coursera - A/B Testing @ Internet ScaleTalks@Coursera - A/B Testing @ Internet Scale
Talks@Coursera - A/B Testing @ Internet Scale
courseratalks
 
4 Steps Toward Scientific A/B Testing
4 Steps Toward Scientific A/B Testing4 Steps Toward Scientific A/B Testing
4 Steps Toward Scientific A/B Testing
Janessa Lantz
 
How to stay relevant - An evolutionary talk for testers
How to stay relevant - An evolutionary talk for testersHow to stay relevant - An evolutionary talk for testers
How to stay relevant - An evolutionary talk for testers
Augusto Evangelisti
 
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
indeedeng
 
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to meDigital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Craig Sullivan
 
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
MeetMagentoNY2014
 
Patterns in Testing and a claim - iCheckWebsite
Patterns in Testing and a claim - iCheckWebsitePatterns in Testing and a claim - iCheckWebsite
Patterns in Testing and a claim - iCheckWebsite
Anand Ramdeo
 
Agile Testing
Agile TestingAgile Testing
Agile Testing
Anand Ramdeo
 
Why do my AB tests suck? measurecamp
Why do my AB tests suck?   measurecampWhy do my AB tests suck?   measurecamp
Why do my AB tests suck? measurecamp
Craig Sullivan
 
Meet Magento Belarus - Viacheslav Kravchuk
Meet Magento Belarus - Viacheslav KravchukMeet Magento Belarus - Viacheslav Kravchuk
Meet Magento Belarus - Viacheslav Kravchuk
Amasty
 
How to Test Anything
How to Test AnythingHow to Test Anything
How to Test Anything
James Thomas
 
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Mobile presentation - Sydney Online Retailer - 26 Sep 2011Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Craig Sullivan
 
Workshop 6: Build Your Organization's Optimization Culture
Workshop 6: Build Your Organization's Optimization CultureWorkshop 6: Build Your Organization's Optimization Culture
Workshop 6: Build Your Organization's Optimization Culture
Optimizely
 

What's hot (20)

Ab testing 101
Ab testing 101Ab testing 101
Ab testing 101
 
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision MakingData-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
 
Weapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
Weapons of Math Instruction: Evolving from Data0-Driven to Science-DrivenWeapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
Weapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
 
Anyone Can Write User Stories. It's the (Shared) Understanding That's Important
Anyone Can Write User Stories. It's the (Shared) Understanding That's ImportantAnyone Can Write User Stories. It's the (Shared) Understanding That's Important
Anyone Can Write User Stories. It's the (Shared) Understanding That's Important
 
Optimizely Workshop: Take Action on Results with Statistics
Optimizely Workshop: Take Action on Results with StatisticsOptimizely Workshop: Take Action on Results with Statistics
Optimizely Workshop: Take Action on Results with Statistics
 
Content Testing & Optimization (CTO) Webinar, April 29 2010
Content Testing & Optimization (CTO) Webinar, April 29 2010Content Testing & Optimization (CTO) Webinar, April 29 2010
Content Testing & Optimization (CTO) Webinar, April 29 2010
 
eMetrics London - The AB Testing Hype Cycle
eMetrics London - The AB Testing Hype CycleeMetrics London - The AB Testing Hype Cycle
eMetrics London - The AB Testing Hype Cycle
 
Talks@Coursera - A/B Testing @ Internet Scale
Talks@Coursera - A/B Testing @ Internet ScaleTalks@Coursera - A/B Testing @ Internet Scale
Talks@Coursera - A/B Testing @ Internet Scale
 
4 Steps Toward Scientific A/B Testing
4 Steps Toward Scientific A/B Testing4 Steps Toward Scientific A/B Testing
4 Steps Toward Scientific A/B Testing
 
How to stay relevant - An evolutionary talk for testers
How to stay relevant - An evolutionary talk for testersHow to stay relevant - An evolutionary talk for testers
How to stay relevant - An evolutionary talk for testers
 
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
 
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to meDigital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
 
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
 
Patterns in Testing and a claim - iCheckWebsite
Patterns in Testing and a claim - iCheckWebsitePatterns in Testing and a claim - iCheckWebsite
Patterns in Testing and a claim - iCheckWebsite
 
Agile Testing
Agile TestingAgile Testing
Agile Testing
 
Why do my AB tests suck? measurecamp
Why do my AB tests suck?   measurecampWhy do my AB tests suck?   measurecamp
Why do my AB tests suck? measurecamp
 
Meet Magento Belarus - Viacheslav Kravchuk
Meet Magento Belarus - Viacheslav KravchukMeet Magento Belarus - Viacheslav Kravchuk
Meet Magento Belarus - Viacheslav Kravchuk
 
How to Test Anything
How to Test AnythingHow to Test Anything
How to Test Anything
 
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Mobile presentation - Sydney Online Retailer - 26 Sep 2011Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
 
Workshop 6: Build Your Organization's Optimization Culture
Workshop 6: Build Your Organization's Optimization CultureWorkshop 6: Build Your Organization's Optimization Culture
Workshop 6: Build Your Organization's Optimization Culture
 

Viewers also liked

Scoopit's Content Marketing Grader for ROI
Scoopit's Content Marketing Grader for ROIScoopit's Content Marketing Grader for ROI
Scoopit's Content Marketing Grader for ROI
Scoop.it
 
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Orbit Media Studios
 
Applied Analytics: Insights and Actions from 13 Google Analytics Reports
Applied Analytics: Insights and Actions from 13 Google Analytics ReportsApplied Analytics: Insights and Actions from 13 Google Analytics Reports
Applied Analytics: Insights and Actions from 13 Google Analytics Reports
Orbit Media Studios
 
Advanced Content Marketing & SEO: How to Publish More and Rank Higher
Advanced Content Marketing & SEO: How to Publish More and Rank HigherAdvanced Content Marketing & SEO: How to Publish More and Rank Higher
Advanced Content Marketing & SEO: How to Publish More and Rank Higher
Andy Crestodina
 
Content Strategy at the Big Ooga!
Content Strategy at the Big Ooga!Content Strategy at the Big Ooga!
Content Strategy at the Big Ooga!
Andy Crestodina
 
Clickbait: A Guide To Writing Un-Ignorable Headlines
Clickbait: A Guide To Writing Un-Ignorable HeadlinesClickbait: A Guide To Writing Un-Ignorable Headlines
Clickbait: A Guide To Writing Un-Ignorable Headlines
Venngage
 
Secrets of the Most Successful Bloggers
Secrets of the Most Successful BloggersSecrets of the Most Successful Bloggers
Secrets of the Most Successful Bloggers
Barry Feldman
 
Intro to Mozcon 2016
Intro to Mozcon 2016Intro to Mozcon 2016
Intro to Mozcon 2016
Rand Fishkin
 
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders 2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
Bryan Kramer
 
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North AmericaB2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
Content Marketing Institute
 

Viewers also liked (10)

Scoopit's Content Marketing Grader for ROI
Scoopit's Content Marketing Grader for ROIScoopit's Content Marketing Grader for ROI
Scoopit's Content Marketing Grader for ROI
 
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
 
Applied Analytics: Insights and Actions from 13 Google Analytics Reports
Applied Analytics: Insights and Actions from 13 Google Analytics ReportsApplied Analytics: Insights and Actions from 13 Google Analytics Reports
Applied Analytics: Insights and Actions from 13 Google Analytics Reports
 
Advanced Content Marketing & SEO: How to Publish More and Rank Higher
Advanced Content Marketing & SEO: How to Publish More and Rank HigherAdvanced Content Marketing & SEO: How to Publish More and Rank Higher
Advanced Content Marketing & SEO: How to Publish More and Rank Higher
 
Content Strategy at the Big Ooga!
Content Strategy at the Big Ooga!Content Strategy at the Big Ooga!
Content Strategy at the Big Ooga!
 
Clickbait: A Guide To Writing Un-Ignorable Headlines
Clickbait: A Guide To Writing Un-Ignorable HeadlinesClickbait: A Guide To Writing Un-Ignorable Headlines
Clickbait: A Guide To Writing Un-Ignorable Headlines
 
Secrets of the Most Successful Bloggers
Secrets of the Most Successful BloggersSecrets of the Most Successful Bloggers
Secrets of the Most Successful Bloggers
 
Intro to Mozcon 2016
Intro to Mozcon 2016Intro to Mozcon 2016
Intro to Mozcon 2016
 
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders 2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
 
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North AmericaB2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
 

Similar to How to Run Smarter A/B Tests, Faster

Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Future Insights
 
Growing Your Business Through Experimentation
Growing Your Business Through ExperimentationGrowing Your Business Through Experimentation
Growing Your Business Through Experimentation
Hiten Shah
 
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
Mariia Bocheva
 
How to Create Winning A/B Tests through Stronger Research
How to Create Winning A/B Tests through Stronger ResearchHow to Create Winning A/B Tests through Stronger Research
How to Create Winning A/B Tests through Stronger Research
VWO
 
Web Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test Ideas
Web Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test IdeasWeb Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test Ideas
Web Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test Ideas
Shanelle Mullin
 
Master the Essentials of Conversion Optimization
Master the Essentials of Conversion OptimizationMaster the Essentials of Conversion Optimization
Master the Essentials of Conversion Optimization
joshuapaulharper
 
How to do usability testing and eye tracking
How to do usability testing and eye trackingHow to do usability testing and eye tracking
How to do usability testing and eye tracking
Objective Experience
 
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
VWO
 
Mindmaps and heuristics tester's best friends - lalit bhamare
Mindmaps and heuristics  tester's best friends - lalit bhamareMindmaps and heuristics  tester's best friends - lalit bhamare
Mindmaps and heuristics tester's best friends - lalit bhamareLalit Bhamare
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
VWO
 
Dev's Guide to Feedback Driven Development
Dev's Guide to Feedback Driven DevelopmentDev's Guide to Feedback Driven Development
Dev's Guide to Feedback Driven Development
Marty Haught
 
Open Source Software Development Practices that Works
Open Source Software Development Practices that WorksOpen Source Software Development Practices that Works
Open Source Software Development Practices that Works
Choong Ping Teo
 
Is Video the Future of Qualitative Research?
Is Video the Future of Qualitative Research? Is Video the Future of Qualitative Research?
Is Video the Future of Qualitative Research?
Kantar
 
Conversion Optimization Willa Fogarty
Conversion Optimization Willa FogartyConversion Optimization Willa Fogarty
Conversion Optimization Willa Fogarty
Willa Fogarty
 
Master the essentials of conversion optimization
Master the essentials of conversion optimization Master the essentials of conversion optimization
Master the essentials of conversion optimization
Steve Clough
 
Creating a culture that provokes failure and boosts improvement
Creating a culture that provokes failure and boosts improvementCreating a culture that provokes failure and boosts improvement
Creating a culture that provokes failure and boosts improvement
Ben Dressler
 
A Happy Marriage between Context-Driven and Agile
A Happy Marriage between Context-Driven and AgileA Happy Marriage between Context-Driven and Agile
A Happy Marriage between Context-Driven and Agile
Ilari Henrik Aegerter
 
Taming The HiPPO
Taming The HiPPOTaming The HiPPO
Taming The HiPPO
Google A/NZ
 
Digital Olympus Presentation - 3 Tips to Optimize for Voice Search
Digital Olympus Presentation - 3 Tips to Optimize for Voice SearchDigital Olympus Presentation - 3 Tips to Optimize for Voice Search
Digital Olympus Presentation - 3 Tips to Optimize for Voice Search
Phoebe Fasulo
 
Conversion Rate Optimization
Conversion Rate OptimizationConversion Rate Optimization
Conversion Rate Optimization
2Checkout
 

Similar to How to Run Smarter A/B Tests, Faster (20)

Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
 
Growing Your Business Through Experimentation
Growing Your Business Through ExperimentationGrowing Your Business Through Experimentation
Growing Your Business Through Experimentation
 
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
A/B testing, optimization and results analysis by Mariia Bocheva, ATD'18
 
How to Create Winning A/B Tests through Stronger Research
How to Create Winning A/B Tests through Stronger ResearchHow to Create Winning A/B Tests through Stronger Research
How to Create Winning A/B Tests through Stronger Research
 
Web Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test Ideas
Web Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test IdeasWeb Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test Ideas
Web Analytics Analysis: How to Find Low-Hanging Fruit and A/B Test Ideas
 
Master the Essentials of Conversion Optimization
Master the Essentials of Conversion OptimizationMaster the Essentials of Conversion Optimization
Master the Essentials of Conversion Optimization
 
How to do usability testing and eye tracking
How to do usability testing and eye trackingHow to do usability testing and eye tracking
How to do usability testing and eye tracking
 
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
 
Mindmaps and heuristics tester's best friends - lalit bhamare
Mindmaps and heuristics  tester's best friends - lalit bhamareMindmaps and heuristics  tester's best friends - lalit bhamare
Mindmaps and heuristics tester's best friends - lalit bhamare
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
 
Dev's Guide to Feedback Driven Development
Dev's Guide to Feedback Driven DevelopmentDev's Guide to Feedback Driven Development
Dev's Guide to Feedback Driven Development
 
Open Source Software Development Practices that Works
Open Source Software Development Practices that WorksOpen Source Software Development Practices that Works
Open Source Software Development Practices that Works
 
Is Video the Future of Qualitative Research?
Is Video the Future of Qualitative Research? Is Video the Future of Qualitative Research?
Is Video the Future of Qualitative Research?
 
Conversion Optimization Willa Fogarty
Conversion Optimization Willa FogartyConversion Optimization Willa Fogarty
Conversion Optimization Willa Fogarty
 
Master the essentials of conversion optimization
Master the essentials of conversion optimization Master the essentials of conversion optimization
Master the essentials of conversion optimization
 
Creating a culture that provokes failure and boosts improvement
Creating a culture that provokes failure and boosts improvementCreating a culture that provokes failure and boosts improvement
Creating a culture that provokes failure and boosts improvement
 
A Happy Marriage between Context-Driven and Agile
A Happy Marriage between Context-Driven and AgileA Happy Marriage between Context-Driven and Agile
A Happy Marriage between Context-Driven and Agile
 
Taming The HiPPO
Taming The HiPPOTaming The HiPPO
Taming The HiPPO
 
Digital Olympus Presentation - 3 Tips to Optimize for Voice Search
Digital Olympus Presentation - 3 Tips to Optimize for Voice SearchDigital Olympus Presentation - 3 Tips to Optimize for Voice Search
Digital Olympus Presentation - 3 Tips to Optimize for Voice Search
 
Conversion Rate Optimization
Conversion Rate OptimizationConversion Rate Optimization
Conversion Rate Optimization
 

Recently uploaded

Adapt or Die - Jon Lakefish, Lakefish Group LLC
Adapt or Die - Jon Lakefish, Lakefish Group LLCAdapt or Die - Jon Lakefish, Lakefish Group LLC
Top 3 Ways to Align Sales and Marketing Teams for Rapid Growth
Top 3 Ways to Align Sales and Marketing Teams for Rapid GrowthTop 3 Ways to Align Sales and Marketing Teams for Rapid Growth
Top 3 Ways to Align Sales and Marketing Teams for Rapid Growth
Demandbase
 
Is AI-Generated Content the Future of Content Creation?
Is AI-Generated Content the Future of Content Creation?Is AI-Generated Content the Future of Content Creation?
Is AI-Generated Content the Future of Content Creation?
Cut-the-SaaS
 
Generative AI - Unleash Creative Opportunity - Peter Weltman
Generative AI - Unleash Creative Opportunity - Peter WeltmanGenerative AI - Unleash Creative Opportunity - Peter Weltman
Generative AI - Unleash Creative Opportunity - Peter Weltman
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly BulletinBLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BalmerLawrie
 
Your Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius
Your Path to Profits - The Game-Changing Power of a Marketing - Daniel BussiusYour Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius
Your Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
May 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group SessionMay 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group Session
Vbout.com
 
SMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the worldSMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the world
smmpanel567
 
DMF Portfolio Piece Smart Goals - Artist Management.docx
DMF Portfolio Piece Smart Goals - Artist Management.docxDMF Portfolio Piece Smart Goals - Artist Management.docx
DMF Portfolio Piece Smart Goals - Artist Management.docx
TravisMalana
 
Offissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdf
Offissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdfOffissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdf
Offissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdf
offisadizayn
 
Email Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris FerrisEmail Marketing Master Class - Chris Ferris
Marketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee LevittMarketing as a Primary Revenue Driver - Lee Levitt
Winning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis YuWinning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis YuWinning local SEO in the Age of AI - Dennis Yu
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly BulletinBLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BalmerLawrie
 
Playlist and Paint Event with Sony Music U
Playlist and Paint Event with Sony Music UPlaylist and Paint Event with Sony Music U
Playlist and Paint Event with Sony Music U
SemajahParker
 
My Personal Brand Exploration by Mariano
My Personal Brand Exploration by MarianoMy Personal Brand Exploration by Mariano
My Personal Brand Exploration by Mariano
marianooscos
 
How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
How to use Short Form Video To Grow Your Brand and Business - Keenya KellyHow to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
ThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness ReportThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness Report
ThinkNow
 
The What, Why & How of 3D and AR in Digital Commerce
The What, Why & How of 3D and AR in Digital CommerceThe What, Why & How of 3D and AR in Digital Commerce
The What, Why & How of 3D and AR in Digital Commerce
PushON Ltd
 

Recently uploaded (20)

Adapt or Die - Jon Lakefish, Lakefish Group LLC
Adapt or Die - Jon Lakefish, Lakefish Group LLCAdapt or Die - Jon Lakefish, Lakefish Group LLC
Adapt or Die - Jon Lakefish, Lakefish Group LLC
 
Top 3 Ways to Align Sales and Marketing Teams for Rapid Growth
Top 3 Ways to Align Sales and Marketing Teams for Rapid GrowthTop 3 Ways to Align Sales and Marketing Teams for Rapid Growth
Top 3 Ways to Align Sales and Marketing Teams for Rapid Growth
 
Is AI-Generated Content the Future of Content Creation?
Is AI-Generated Content the Future of Content Creation?Is AI-Generated Content the Future of Content Creation?
Is AI-Generated Content the Future of Content Creation?
 
Generative AI - Unleash Creative Opportunity - Peter Weltman
Generative AI - Unleash Creative Opportunity - Peter WeltmanGenerative AI - Unleash Creative Opportunity - Peter Weltman
Generative AI - Unleash Creative Opportunity - Peter Weltman
 
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly BulletinBLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
 
Your Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius
Your Path to Profits - The Game-Changing Power of a Marketing - Daniel BussiusYour Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius
Your Path to Profits - The Game-Changing Power of a Marketing - Daniel Bussius
 
May 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group SessionMay 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group Session
 
SMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the worldSMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the world
 
DMF Portfolio Piece Smart Goals - Artist Management.docx
DMF Portfolio Piece Smart Goals - Artist Management.docxDMF Portfolio Piece Smart Goals - Artist Management.docx
DMF Portfolio Piece Smart Goals - Artist Management.docx
 
Offissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdf
Offissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdfOffissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdf
Offissa Dizayn - Otel, Kafe, Restoran Kataloqu_240603_011042.pdf
 
Email Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris FerrisEmail Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris Ferris
 
Marketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee LevittMarketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee Levitt
 
Winning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis YuWinning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis Yu
 
Winning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis YuWinning local SEO in the Age of AI - Dennis Yu
Winning local SEO in the Age of AI - Dennis Yu
 
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly BulletinBLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
 
Playlist and Paint Event with Sony Music U
Playlist and Paint Event with Sony Music UPlaylist and Paint Event with Sony Music U
Playlist and Paint Event with Sony Music U
 
My Personal Brand Exploration by Mariano
My Personal Brand Exploration by MarianoMy Personal Brand Exploration by Mariano
My Personal Brand Exploration by Mariano
 
How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
How to use Short Form Video To Grow Your Brand and Business - Keenya KellyHow to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
 
ThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness ReportThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness Report
 
The What, Why & How of 3D and AR in Digital Commerce
The What, Why & How of 3D and AR in Digital CommerceThe What, Why & How of 3D and AR in Digital Commerce
The What, Why & How of 3D and AR in Digital Commerce
 

How to Run Smarter A/B Tests, Faster

  • 1. How to Run Smarter A/B Tests, Faster How to Run Smarter A/B Tests, Faster @shanelle_mullin We’ll cover… 1. How to conduct conversion research to find smarter A/B test ideas. 2. How to prioritize your A/B test ideas in a meaningful way. 3. What a high velocity testing program is and why it works. 4. Common high velocity testing program mistakes to avoid. 5. How to increase your testing velocity responsibly.
  • 2. How to Run Smarter A/B Tests, Faster “You don’t have generic problems, you have specific problems.” ~Peep Laja
  • 3. How to Run Smarter A/B Tests, Faster
  • 4.
  • 5.
  • 6. How to Run Smarter A/B Tests, Faster 1. Relevancy Does the page’s content meet user expectation? Does the page’s design meet user expectation? How can it match what they expect / want even more?
  • 7. How to Run Smarter A/B Tests, Faster 2. Clarity Is the content on this page as clear as possible? Is the value prop on this page as clear as possible? How can we make it clearer?
  • 8. How to Run Smarter A/B Tests, Faster 3. Motivation What’s motivating the visitor to take action? Is it clear, believable, timely and persuasive? How can we increase motivation even more?
  • 9. How to Run Smarter A/B Tests, Faster 4. Friction What is causing doubts, hesitations and uncertainties? What makes the process difficult? How can we simplify the process?
  • 10. How to Run Smarter A/B Tests, Faster 5. Distraction What’s not helping the user take action? What’s drawing attention? Is everything that draws attention really necessary?
  • 11.
  • 12.
  • 13. How to Run Smarter A/B Tests, Faster Cross-Browser Testing Audience > Technology > Browser & OS Switch from the default “Data” view to the “Comparison” view.
  • 14. How to Run Smarter A/B Tests, Faster Cross-Browser Testing Always compare within the same browser family.
  • 15. How to Run Smarter A/B Tests, Faster Cross-Device Testing Audience > Mobile > Devices Switch from the default “Data” view to the “Comparison” view.
  • 16.
  • 17. How to Run Smarter A/B Tests, Faster Speed Analysis Page Load Time: Seconds until every element on the page is done loading. Page Interactive Time: Seconds until the site is usable.
  • 18. How to Run Smarter A/B Tests, Faster Speed Analysis Your site loads in 3 seconds or less. Your site loads in 4 to 7 seconds. Your site loads in 7 to 10 seconds (or more).
  • 19. How to Run Smarter A/B Tests, Faster Speed Analysis Behavior > Site Speed > Page Timings Switch from the default “Data” view to the “Comparison” view.
  • 20.
  • 21.
  • 22.
  • 23. How to Run Smarter A/B Tests, Faster Google Analytics Health Check 1. Are you collecting all of the data you need? 2. Can you trust the data you are collecting? 3. Is anything broken or tracking / reporting incorrectly? Why?
  • 24. How to Run Smarter A/B Tests, Faster Common Google Analytics Mistakes 1. Pages are missing. 2. Data is different in the shopping cart tool. 3. Cross-domain tracking isn’t working.
  • 25. How to Run Smarter A/B Tests, Faster Full Health Check Guide bit.ly/gahealthcheck
  • 26. How to Run Smarter A/B Tests, Faster Start with the Highest Value 1. High volume, low value pages. 2. Low volume, high value pages.
  • 27.
  • 28. How to Run Smarter A/B Tests, Faster Finding Broken Links
  • 29. How to Run Smarter A/B Tests, Faster Finding Broken Links
  • 30. How to Run Smarter A/B Tests, Faster Auditing Your Funnel The further down the funnel, the smaller the necessary impact.
  • 31. How to Run Smarter A/B Tests, Faster Auditing Your Funnel
  • 32. How to Run Smarter A/B Tests, Faster Auditing Your Funnel
  • 33. How to Run Smarter A/B Tests, Faster Exploring Internal Search
  • 34.
  • 35. How to Run Smarter A/B Tests, Faster Segmentation Is the Secret bit.ly/gasegmentation
  • 36. How to Run Smarter A/B Tests, Faster Click Maps 1. This info is largely available in Google Analytics via “Navigation Summary”. 2. See when people are clicking on non-links.
  • 37. How to Run Smarter A/B Tests, Faster Scroll Maps 1. Better prioritize your content and messaging. 2. Drastic colour changes mean a need for visual cues… likely indicate false bottom.
  • 38.
  • 39. How to Run Smarter A/B Tests, Faster Session Replays 1. Real users with real money. 2. Where do they pause? Where do they give up? 3. Use this to gather more insight on poor performers.
  • 40. How to Run Smarter A/B Tests, Faster Form Analytics 1. Which form fields cause the most error messages? 2. Which do people hesitate to fill? 3. Which do people leave empty, even if required?
  • 41. How to Run Smarter A/B Tests, Faster On-Site Surveys Exit surveys. On-page surveys.
  • 42. How to Run Smarter A/B Tests, Faster On-Site Survey Questions 1. What was the purpose of your visit to our site? 2. Were you able to complete your task? 3. If not, why? 4. Is there anything holding you back? 5. Do you have unanswered questions? Focus on actionable data, on the friction.
  • 43. How to Run Smarter A/B Tests, Faster On-Site Survey Options 1 open-ended question. Y/N and then an explanation post-answer.
  • 44. How to Run Smarter A/B Tests, Faster On-Site Survey Guide bit.ly/onsitesurveys
  • 45. How to Run Smarter A/B Tests, Faster Customer Surveys 1. First time buyers. 2. No previous relationship with you. 3. Try for ~200 responses.
  • 46. How to Run Smarter A/B Tests, Faster Customer Survey Questions 1. Who are you? 2. What are you using this product for? 3. What problem does it solve for you? 4. What do you like about the product most? 5. Did you consider any alternatives? 6. Why did you choose us over the competition? 7. What’s the one thing that nearly stopped you from buying? 8. What was your biggest concern or fear about us?
  • 47.
  • 48. How to Run Smarter A/B Tests, Faster User Testing Tasks 1. A specific task. 2. A broad task. 3. Funnel completion.
  • 49.
  • 50.
  • 51. How to Run Smarter A/B Tests, Faster Constant Ideation 1. Continue conducting conversion research. 2. Involve the entire team. 3. Focus on the entire funnel (acquisition to referral).
  • 52. How to Run Smarter A/B Tests, Faster “If you come to me with an idea and it’s not live in 2 weeks, it’s not because it’s a bad idea. It’s because I have better things to test.” ~Pauline Marol
  • 53. How to Run Smarter A/B Tests, Faster The Prioritization Problem 1. Not objective enough. 2. Not data-driven enough. 3. Not easily customized.
  • 54. How to Run Smarter A/B Tests, Faster
  • 55. How to Run Smarter A/B Tests, Faster Benefits of PXL 1. Makes “potential” more objective. 2. Makes “ease” more objective. 3. Helps create a data-driven culture. 4. Easy to customize.
  • 56. How to Run Smarter A/B Tests, Faster Prioritization Spreadsheet bit.ly/pxlmodel
  • 57. How to Run Smarter A/B Tests, Faster Better Management 1. Review KPIs. 2. Review tests launched (and not launched). 3. Review insights gained. 4. Choose tests for the upcoming week. 5. Discuss resources (or lack thereof).
  • 58. How to Run Smarter A/B Tests, Faster Archive (Or Live to Regret It) 1. Avoid repeating tests by mistake. 2. Easily communicate wins and insights. 3. Improve the quality of future tests. 4. Create an insights > wins culture.
  • 59.
  • 60. How to Run Smarter A/B Tests, Faster High Velocity Mistakes 1. Without buy-in and a testing culture, you’re doomed. 2. Calling it too soon and non-representative samples. 3. Letting quality and effectiveness slip.
  • 61. How to Run Smarter A/B Tests, Faster Testing Capacity (52 / Avg. Req. Test Duration) x Number of Testable Pages (52 / 2) x 10 = 260
  • 62. How to Run Smarter A/B Tests, Faster Measuring Velocity 1. How well are you trending towards achieving your capacity? 2. Trends are what matters… are you getting closer? 3. Chart it.
  • 63.
  • 64. How to Run Smarter A/B Tests, Faster How to Run Smarter A/B Tests, Faster @shanelle_mullin 1. Conduct proper conversion research. 2. Come up with ideas constantly. 3. Prioritize with data. 4. Manage the process. 5. Archive your learnings. 6. Achieve your testing capacity. 7. Stop listening to CRO BS you read online.
  • 65. How to Run Smarter A/B Tests, Faster Resources Optimization Training Program bit.ly/cxlinstitute Original UX Research bit.ly/cxlresearch
  • 66. How to Run Smarter A/B Tests, Faster Questions? Ask any time via email: shanelle@conversionxl.com