Download to read offline
Uploaded byPirate Skills
A/B Testing Blueprint | Pirate Skills
The document outlines various resources and tools for A/B testing, including a checklist for setting up tests, statistical significance calculators, and optimization frameworks like the lift model. It emphasizes the importance of data-driven decision-making in marketing and provides links to external guides and calculators for further insights. Additionally, it discusses the necessity of understanding statistical principles to effectively evaluate A/B test results.
More Related Content
![[Elite Camp 2016] Annemarie Klaassen & Tom van de Berg - Moving Beyond Testin...](https://cdn.slidesharecdn.com/ss_thumbnails/06elitecamp-annemarieklaassentomvandeberg-howweoptimizeoptimizeoptimizeoptimizeatthethebiggesthotelc-160708114439-thumbnail.jpg?width=640&height=640&fit=bounds)
![Conquering the perfect storm share[1]](https://cdn.slidesharecdn.com/ss_thumbnails/conqueringtheperfectstormshare1-170713124720-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to A/B Testing Blueprint | Pirate Skills
![AB Testing [INFOGRAPHIC]](https://cdn.slidesharecdn.com/ss_thumbnails/4imprintabtestingfnl-140630144834-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
A/B Testing Blueprint | Pirate Skills
- 1.
- 2. A/B TESTING BLUEPRINT pirateskills.com/abtesting Newtable Metric Not started Started Completed -20% 4% 5% 5% Registration / Visitors 17 8 4 5 Registration Page 8% 10.8% 10% 11% Workshop LP 2000 25% 2500 2000 CR% Revenue 12 5 4 3 20 5 8 7 4 3 2 1 Homepage Fail Unclear Success 9% 15% 2300 What's your best test idea? What's your best test idea? What's your best test idea? What's your best test idea? 5 What's your best test idea? 6 What's your best test idea? 7 What's your best test idea? 8 What's your best test idea? 9 What's your best test idea? 10 What's your best test idea? A/B TESTING BLUEPRINT Make better data driven choice blog.hubspot.com How to Do A/B Testing: A Checklist You'll Want to Bookmark When marketers like us create landing pages, write email copy, or design call- to-action buttons, it can be tempting to use our intuition to predict what will make people click and convert. But basing marketing decisions off of a "feeling" can be pretty … abtestguide.com A/B-Test Calculator - Power & Significance - ABTestGuide.com Calculate statistical significance and the Power of your A/B-test vwo.com VWO | #1 A/B Testing Tool in the World Thousands of brands across the globe use VWO as their experimentation platform to run A/B tests on their websites, apps and products. Import and aggregate all metrics you care about and measure how they get impacted by your experiments. cxl.com 40+ Conversion Optimization Tools (Reviewed by Experts) Men have become the tools of their tools. Henry David Thoreau AB Tasty is another testing tool designed for the masses, this one hailing from France. They offer client-side, bayesian-based A/B testing along with full factorial multivariate testing -the … cxl.com AB Test Sample Size Calculators by CXL One test calculator to answer all your pre- and post-test analysis questions. vwo.com A/B testing statistical significance calculator - VWO Get comprehensive insights about testing, optimization, UX, design, and more. Are your results significant? Learn more about how to research A/B testing ideas and create winning tests through our detailed guide. Deliver great experiences. Grow faster, s… www.evanmiller.org Sample Size Calculator Need A/B sample sizes on your iPhone or iPad? Download A/B Buddy today. Question: How many subjects are needed for an A/B test? cxl.com A/B Testing Statistics: An Easy-to-Understand Guide | CXL Testing tools are getting more sophisticated. Blogs are brimming with "inspiring" case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind. This post is filled with clear explanations of A/… www.optimizesmart.com Understanding A/B testing statistics to get REAL Lift in Conversions If you want to increase your chances of getting a real lift through A/B tests then you need to understand the statistics behind it. If you don’t like learning statistics then I am afraid A/B testing is not for you. fivethirtyeight.com Science Isn't Broken If you follow the headlines, your confidence in science may have taken a hit lately. Peer review? More like self- review. An investigation in November uncovered a scam in which researchers were rubber-stamping their own work, circumventing peer review at… conversionsciences.com The Ultimate A/B Testing Guide: Everything You Need, All In One Place Welcome to the ultimate A/B testing guide! In this post, I'm going to cover everything you need to know about A/B testing (also referred to as "split" testing), from start to finish. Here's what we'll cover: By the end of this guide, you'll have a thoro… How to setup A/B testing Generate strong testing ideas Learn more about A/B testing Baseline Result Change Variation Control ICE Score Ease Confi- dence Impact Test Idea URL / Screen # Status Hypothesis 1. Install Google Analytics 2. Setup Conversion Goals 3. Install Google Optimize 4. Configure Google Optimze 5. Create first A/B Test Google Analytics Insights Hotjar Screen Recordings marketingplatform.google.com Analytics Tools & Solutions for Your Business - Google Analytics Easily access data from other Google solutions while working in Analytics, for a seamless workflow that saves you time and increases efficiency. Google Ads Gain deeper insights into how users from your Google Ads campaigns engage with your site. Data St… marketingplatform.google.com Website, A/B Testing & Optimization Tools - Google Optimize Optimize natively integrate with other Google solutions, saving you valuable time while increasing efficiencies across your team. Analytics Use your Analytics data to identify where to improve your site, measure experiments against business goals, and s… Why your website reminds me of "50 First Dates" - Part 1/5 YouTube Want to learn how deep the rabit hole goes? (in order) How to calculate sample size and significance? How to get enough traffic to make A/B testing viable? pirateskills.com Traffic Channel Compass | Pirate Skills The Traffic Channel Compass will give you an overview of the most effective growth marketing channels and help you focus on the right ones. Should focus scarce resources on Facebook, SEO, Newsletter,... or maybe even TikTok? Deciding on your most import… Ads are the most fun In E-Commerce YouTube Targeting Your B2B Audience with LinkedIn Ads YouTube Get more Instagram followers with this ad strategy (mobile only) - Part 1/6 YouTube How to get more Traffic with Native Ads YouTube Google Analytics Demystified | Live Stream YouTube Qualaroo Customer Feedback User- Testing .com GTmetrix Speed Test Typeform User Surveys Customer Journey Map Tools Frameworks www.widerfunn… LIFT Model: Most Popular Conversion Optimization Framework In 2009, Chris Goward introduced the LIFT Model®, a framework for analyzing web and mobile experiences and developing A/B test hypotheses. Today, it is the go-to conversion optimization framework for leading companies around the world. Motivation = Benefits - Costs vwo.com What is A/B Testing? A Practical Guide With Examples A/B testing (also known as split testing) is a process of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions. Start with research, end with it www.shopify.com The Complete Guide to A/B Testing: Expert Tips from Google, HubSpot and More This probably isn't the first time you've read about A/B testing. You might even already A/B test your email subject lines or your social media posts. Despite the fact that there's been plenty said about A/B testing in the field of marketing, a lot of p… Fact & Act model
- 3. pirateskills.com/abtesting 10 What's yourbest test idea? How to setup A/B testing 1. Install Google Analytics 2. Setup Conversion Goals 3. Install Google Optimize 4. Configure Google Optimze 5. Create first A/B Test marketingplatform.google.com Analytics Tools & Solutions for Your Business - Google Analytics Easily access data from other Google solutions while working in Analytics, for a seamless workflow that saves you time and increases efficiency. Google Ads Gain deeper insights into how users from your Google Ads campaigns engage with your site. Data St… marketingplatform.google.com Website, A/B Testing & Optimization Tools - Google Optimize Optimize natively integrate with other Google solutions, saving you valuable time while increasing efficiencies across your team. Analytics Use your Analytics data to identify where to improve your site, measure experiments against business goals, and s… Why your website reminds me of "50 First Dates" - Part 1/5 YouTube Google Analytics Demystified | Live Stream YouTube
- 4. pirateskills.com/abtesting Generate strong testingideas Google Analytics Insights Hotjar Screen Recordings Qualaroo Customer Feedback User- Testing .com GTmetrix Speed Test Typeform User Surveys Customer Journey Map Tools Frameworks www.widerfunn… LIFT Model: Most Popular Conversion Optimization Framework In 2009, Chris Goward introduced the LIFT Model®, a framework for analyzing web and mobile experiences and developing A/B test hypotheses. Today, it is the go-to conversion optimization framework for leading companies around the world. Motivation = Benefits - Costs vwo.com What is A/B Testing? A Practical Guide With Examples A/B testing (also known as split testing) is a process of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions. Start with research, end with it www.shopify.com The Complete Guide to A/B Testing: Expert Tips from Google, HubSpot and More This probably isn't the first time you've read about A/B testing. You might even already A/B test your email subject lines or your social media posts. Despite the fact that there's been plenty said about A/B testing in the field of marketing, a lot of p… Fact & Act model
- 5. pirateskills.com/abtesting blog.hubspot.com How to DoA/B Testing: A Checklist You'll Want to Bookmark When marketers like us create landing pages, write email copy, or design call- to-action buttons, it can be tempting to use our intuition to predict what will make people click and convert. But basing marketing decisions off of a "feeling" can be pretty … abtestguide.com A/B-Test Calculator - Power & Significance - ABTestGuide.com Calculate statistical significance and the Power of your A/B-test vwo.com VWO | #1 A/B Testing Tool in the World Thousands of brands across the globe use VWO as their experimentation platform to run A/B tests on their websites, apps and products. Import and aggregate all metrics you care about and measure how they get impacted by your experiments. cxl.com 40+ Conversion Optimization Tools (Reviewed by Experts) Men have become the tools of their tools. Henry David Thoreau AB Tasty is another testing tool designed for the masses, this one hailing from France. They offer client-side, bayesian-based A/B testing along with full factorial multivariate testing -the … cxl.com AB Test Sample Size Calculators by CXL One test calculator to answer all your pre- and post-test analysis questions. vwo.com A/B testing statistical significance calculator - VWO Get comprehensive insights about testing, optimization, UX, design, and more. Are your results significant? Learn more about how to research A/B testing ideas and create winning tests through our detailed guide. Deliver great experiences. Grow faster, s… www.evanmiller.org Sample Size Calculator Need A/B sample sizes on your iPhone or iPad? Download A/B Buddy today. Question: How many subjects are needed for an A/B test? cxl.com A/B Testing Statistics: An Easy-to-Understand Guide | CXL Testing tools are getting more sophisticated. Blogs are brimming with "inspiring" case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind. This post is filled with clear explanations of A/… www.optimizesmart.com Understanding A/B testing statistics to get REAL Lift in Conversions If you want to increase your chances of getting a real lift through A/B tests then you need to understand the statistics behind it. If you don’t like learning statistics then I am afraid A/B testing is not for you. fivethirtyeight.com Science Isn't Broken If you follow the headlines, your confidence in science may have taken a hit lately. Peer review? More like self- review. An investigation in November uncovered a scam in which researchers were rubber-stamping their own work, circumventing peer review at… conversionsciences.com The Ultimate A/B Testing Guide: Everything You Need, All In One Place Welcome to the ultimate A/B testing guide! In this post, I'm going to cover everything you need to know about A/B testing (also referred to as "split" testing), from start to finish. Here's what we'll cover: By the end of this guide, you'll have a thoro… Learn more about A/B testing Want to learn how deep the rabit hole goes? (in order) How to calculate sample size and significance? How to get enough traffic to make A/B testing viable? pirateskills.com Traffic Channel Compass | Pirate Skills The Traffic Channel Compass will give you an overview of the most effective growth marketing channels and help you focus on the right ones. Should focus scarce resources on Facebook, SEO, Newsletter,... or maybe even TikTok? Deciding on your most import… Ads are the most fun In E-Commerce YouTube Targeting Your B2B Audience with LinkedIn Ads YouTube Get more Instagram followers with this ad strategy (mobile only) - Part 1/6 YouTube How to get more Traffic with Native Ads YouTube
- 6. Get the blueprint pirateskills.com/abtesting Newtable Metric Not started Started Completed -20% 4% 5% 5% Registration / Visitors 17 8 4 5 Registration Page 8% 10.8% 10% 11% Workshop LP 2000 25% 2500 2000 CR% Revenue 12 5 4 3 20 5 8 7 4 3 2 1 Homepage Fail Unclear Success 9% 15% 2300 What's your best test idea? What's your best test idea? What's your best test idea? What's your best test idea? 5 What's your best test idea? 6 What's your best test idea? 7 What's your best test idea? 8 What's your best test idea? 9 What's your best test idea? 10 What's your best test idea? A/B TESTING BLUEPRINT Make better data driven choice blog.hubspot.com How to Do A/B Testing: A Checklist You'll Want to Bookmark When marketers like us create landing pages, write email copy, or design call- to-action buttons, it can be tempting to use our intuition to predict what will make people click and convert. But basing marketing decisions off of a "feeling" can be pretty … abtestguide.com A/B-Test Calculator - Power & Significance - ABTestGuide.com Calculate statistical significance and the Power of your A/B-test vwo.com VWO | #1 A/B Testing Tool in the World Thousands of brands across the globe use VWO as their experimentation platform to run A/B tests on their websites, apps and products. Import and aggregate all metrics you care about and measure how they get impacted by your experiments. cxl.com 40+ Conversion Optimization Tools (Reviewed by Experts) Men have become the tools of their tools. Henry David Thoreau AB Tasty is another testing tool designed for the masses, this one hailing from France. They offer client-side, bayesian-based A/B testing along with full factorial multivariate testing -the … cxl.com AB Test Sample Size Calculators by CXL One test calculator to answer all your pre- and post-test analysis questions. vwo.com A/B testing statistical significance calculator - VWO Get comprehensive insights about testing, optimization, UX, design, and more. Are your results significant? Learn more about how to research A/B testing ideas and create winning tests through our detailed guide. Deliver great experiences. Grow faster, s… www.evanmiller.org Sample Size Calculator Need A/B sample sizes on your iPhone or iPad? Download A/B Buddy today. Question: How many subjects are needed for an A/B test? cxl.com A/B Testing Statistics: An Easy-to-Understand Guide | CXL Testing tools are getting more sophisticated. Blogs are brimming with "inspiring" case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind. This post is filled with clear explanations of A/… www.optimizesmart.com Understanding A/B testing statistics to get REAL Lift in Conversions If you want to increase your chances of getting a real lift through A/B tests then you need to understand the statistics behind it. If you don’t like learning statistics then I am afraid A/B testing is not for you. fivethirtyeight.com Science Isn't Broken If you follow the headlines, your confidence in science may have taken a hit lately. Peer review? More like self- review. An investigation in November uncovered a scam in which researchers were rubber-stamping their own work, circumventing peer review at… conversionsciences.com The Ultimate A/B Testing Guide: Everything You Need, All In One Place Welcome to the ultimate A/B testing guide! In this post, I'm going to cover everything you need to know about A/B testing (also referred to as "split" testing), from start to finish. Here's what we'll cover: By the end of this guide, you'll have a thoro… How to setup A/B testing Generate strong testing ideas Learn more about A/B testing Baseline Result Change Variation Control ICE Score Ease Confi- dence Impact Test Idea URL / Screen # Status Hypothesis 1. Install Google Analytics 2. Setup Conversion Goals 3. Install Google Optimize 4. Configure Google Optimze 5. Create first A/B Test Google Analytics Insights Hotjar Screen Recordings marketingplatform.google.com Analytics Tools & Solutions for Your Business - Google Analytics Easily access data from other Google solutions while working in Analytics, for a seamless workflow that saves you time and increases efficiency. Google Ads Gain deeper insights into how users from your Google Ads campaigns engage with your site. Data St… marketingplatform.google.com Website, A/B Testing & Optimization Tools - Google Optimize Optimize natively integrate with other Google solutions, saving you valuable time while increasing efficiencies across your team. Analytics Use your Analytics data to identify where to improve your site, measure experiments against business goals, and s… Why your website reminds me of "50 First Dates" - Part 1/5 YouTube Want to learn how deep the rabit hole goes? (in order) How to calculate sample size and significance? How to get enough traffic to make A/B testing viable? pirateskills.com Traffic Channel Compass | Pirate Skills The Traffic Channel Compass will give you an overview of the most effective growth marketing channels and help you focus on the right ones. Should focus scarce resources on Facebook, SEO, Newsletter,... or maybe even TikTok? Deciding on your most import… Ads are the most fun In E-Commerce YouTube Targeting Your B2B Audience with LinkedIn Ads YouTube Get more Instagram followers with this ad strategy (mobile only) - Part 1/6 YouTube How to get more Traffic with Native Ads YouTube Google Analytics Demystified | Live Stream YouTube Qualaroo Customer Feedback User- Testing .com GTmetrix Speed Test Typeform User Surveys Customer Journey Map Tools Frameworks www.widerfunn… LIFT Model: Most Popular Conversion Optimization Framework In 2009, Chris Goward introduced the LIFT Model®, a framework for analyzing web and mobile experiences and developing A/B test hypotheses. Today, it is the go-to conversion optimization framework for leading companies around the world. Motivation = Benefits - Costs vwo.com What is A/B Testing? A Practical Guide With Examples A/B testing (also known as split testing) is a process of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions. Start with research, end with it www.shopify.com The Complete Guide to A/B Testing: Expert Tips from Google, HubSpot and More This probably isn't the first time you've read about A/B testing. You might even already A/B test your email subject lines or your social media posts. Despite the fact that there's been plenty said about A/B testing in the field of marketing, a lot of p… Fact & Act model
- 7. Ben Su fi ani • Foundersince 2008 • Growth Marketer • Host of Pirate Skills • Irrationally passioned about the nerdy parts of marketing
- 8. Conversion Optimization • Improve theuser fl ow • Lower cost per acquisition • Increase customer lifetime value • CPA < CLV 💥🚀
- 9. How to reachyour conversion goals Ideas Priorities Experiments Results
- 18. Let’s dive intoit pirateskills.com/abtesting New table Metric Not started Started Completed -20% 4% 5% 5% Registration / Visitors 17 8 4 5 Registration Page 8% 10.8% 10% 11% Workshop LP 2000 25% 2500 2000 CR% Revenue 12 5 4 3 20 5 8 7 4 3 2 1 Homepage Fail Unclear Success 9% 15% 2300 What's your best test idea? What's your best test idea? What's your best test idea? What's your best test idea? 5 What's your best test idea? 6 What's your best test idea? 7 What's your best test idea? 8 What's your best test idea? 9 What's your best test idea? 10 What's your best test idea? A/B TESTING BLUEPRINT Make better data driven choice blog.hubspot.com How to Do A/B Testing: A Checklist You'll Want to Bookmark When marketers like us create landing pages, write email copy, or design call- to-action buttons, it can be tempting to use our intuition to predict what will make people click and convert. But basing marketing decisions off of a "feeling" can be pretty … abtestguide.com A/B-Test Calculator - Power & Significance - ABTestGuide.com Calculate statistical significance and the Power of your A/B-test vwo.com VWO | #1 A/B Testing Tool in the World Thousands of brands across the globe use VWO as their experimentation platform to run A/B tests on their websites, apps and products. Import and aggregate all metrics you care about and measure how they get impacted by your experiments. cxl.com 40+ Conversion Optimization Tools (Reviewed by Experts) Men have become the tools of their tools. Henry David Thoreau AB Tasty is another testing tool designed for the masses, this one hailing from France. They offer client-side, bayesian-based A/B testing along with full factorial multivariate testing -the … cxl.com AB Test Sample Size Calculators by CXL One test calculator to answer all your pre- and post-test analysis questions. vwo.com A/B testing statistical significance calculator - VWO Get comprehensive insights about testing, optimization, UX, design, and more. Are your results significant? Learn more about how to research A/B testing ideas and create winning tests through our detailed guide. Deliver great experiences. Grow faster, s… www.evanmiller.org Sample Size Calculator Need A/B sample sizes on your iPhone or iPad? Download A/B Buddy today. Question: How many subjects are needed for an A/B test? cxl.com A/B Testing Statistics: An Easy-to-Understand Guide | CXL Testing tools are getting more sophisticated. Blogs are brimming with "inspiring" case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind. This post is filled with clear explanations of A/… www.optimizesmart.com Understanding A/B testing statistics to get REAL Lift in Conversions If you want to increase your chances of getting a real lift through A/B tests then you need to understand the statistics behind it. If you don’t like learning statistics then I am afraid A/B testing is not for you. fivethirtyeight.com Science Isn't Broken If you follow the headlines, your confidence in science may have taken a hit lately. Peer review? More like self- review. An investigation in November uncovered a scam in which researchers were rubber-stamping their own work, circumventing peer review at… conversionsciences.com The Ultimate A/B Testing Guide: Everything You Need, All In One Place Welcome to the ultimate A/B testing guide! In this post, I'm going to cover everything you need to know about A/B testing (also referred to as "split" testing), from start to finish. Here's what we'll cover: By the end of this guide, you'll have a thoro… How to setup A/B testing Generate strong testing ideas Learn more about A/B testing Baseline Result Change Variation Control ICE Score Ease Confi- dence Impact Test Idea URL / Screen # Status Hypothesis 1. Install Google Analytics 2. Setup Conversion Goals 3. Install Google Optimize 4. Configure Google Optimze 5. Create first A/B Test Google Analytics Insights Hotjar Screen Recordings marketingplatform.google.com Analytics Tools & Solutions for Your Business - Google Analytics Easily access data from other Google solutions while working in Analytics, for a seamless workflow that saves you time and increases efficiency. Google Ads Gain deeper insights into how users from your Google Ads campaigns engage with your site. Data St… marketingplatform.google.com Website, A/B Testing & Optimization Tools - Google Optimize Optimize natively integrate with other Google solutions, saving you valuable time while increasing efficiencies across your team. Analytics Use your Analytics data to identify where to improve your site, measure experiments against business goals, and s… Why your website reminds me of "50 First Dates" - Part 1/5 YouTube Want to learn how deep the rabit hole goes? (in order) How to calculate sample size and significance? How to get enough traffic to make A/B testing viable? pirateskills.com Traffic Channel Compass | Pirate Skills The Traffic Channel Compass will give you an overview of the most effective growth marketing channels and help you focus on the right ones. Should focus scarce resources on Facebook, SEO, Newsletter,... or maybe even TikTok? Deciding on your most import… Ads are the most fun In E-Commerce YouTube Targeting Your B2B Audience with LinkedIn Ads YouTube Get more Instagram followers with this ad strategy (mobile only) - Part 1/6 YouTube How to get more Traffic with Native Ads YouTube Google Analytics Demystified | Live Stream YouTube Qualaroo Customer Feedback User- Testing .com GTmetrix Speed Test Typeform User Surveys Customer Journey Map Tools Frameworks www.widerfunn… LIFT Model: Most Popular Conversion Optimization Framework In 2009, Chris Goward introduced the LIFT Model®, a framework for analyzing web and mobile experiences and developing A/B test hypotheses. Today, it is the go-to conversion optimization framework for leading companies around the world. Motivation = Benefits - Costs vwo.com What is A/B Testing? A Practical Guide With Examples A/B testing (also known as split testing) is a process of showing two variants of the same web page to different segments of website visitors at the same time and comparing which variant drives more conversions. Start with research, end with it www.shopify.com The Complete Guide to A/B Testing: Expert Tips from Google, HubSpot and More This probably isn't the first time you've read about A/B testing. You might even already A/B test your email subject lines or your social media posts. Despite the fact that there's been plenty said about A/B testing in the field of marketing, a lot of p… Fact & Act model
- 19.
- 20.
- 21.
- 22. GROWTH MARKETING WORKSHOPS Master the Artand Science pirateskills.com/workshops of Growth Marketing
- 23.
- 24.
- 25.