A Crash Course on A/B Testing
•
127 likes•8,051 views
A brief, high-level overview and introduction on A/B testing - What can you test? - Famous A/B Testing Case Studies from the 2008 Presidential Election - The technology behind A/B testing including Javascript, jQuery - Tools you can use including: Optimizely, Visual Website Optimizer, Unbounce, Google Analytics - Pluses and minues to A/B testing
Report
Share
Report
Share
Download to read offline
Recommended
A/B Testing: You Might be Driving in the Wrong Direction
A/B Testing
You Might Be Driving in the Wrong Direction TOMASZ BORYS
@Kissmetrics #KissWebinar @thuelmadsen
Thue is the Kissmetrics Webinar Wizard and Marketing Ops Manager. Before joining forces with Kissmetrics, he was a Lyft driver in SF, which is also how he ended up as a Kissmetrics marketer. Whenever Thue is not trying to automate everything around him, you can find him hiking in the Sierras. THUE MADSEN Marketing Operations Manager, Kissmetrics @ThueLMadsen Tomasz loves dipping his feet in the river while fishing, injuring his thumb while gaming, hacking away at a golf club, and driving demand at Kissmetrics. He’s also the biggest fan of gummy bears. TOMASZ BORYS Director of Marketing, Kissmetrics @tbcali
@Kissmetrics #KissWebinar @tbcali
1 The optimum strategies for A/B testing 2 How we A/B Test at KISSmetrics 3 A/B Testing beyond click conversions - having a pulse on the entire funnel and why it’s important TABLE OF CONTENTS 4 The influence social traffic has in the funnel
• Set a Goal • Baby Steps • Aim for Statistical Significance • Never Lose Sight THE OPTIMUM STRATEGIES FOR A/B TESTING
Baby Steps People want to move the needle and see results fast, by implementing multiple changes at once. But how can you be sure what element had an impact?
Aim for Statistical Significance SET A GOAL Aim for Statistical Significance getdatadriven.com
Never Lose Sight We can caught up on what we think is best to optimize conversions or traffic…but data doesn’t lie
• 4,000 and above sample size • 99% Statistical Confidence • Blind Eye for 1 Week HOW WE A/B TEST AT KISSMETRICS
4,000 and Above Sample Size Significant or decent amount of data is needed
when you’re testing beyond click conversions
4,000 and Above Sample Size significant or decent amount of data is needed when you’re testing beyond click conversions 99% Statistical Confidence Police
Blind Eye for 1 Week Data can be very erratic the first several days, so it’s easy to hit the panic button
A/B Testing Beyond Click Conversions
ORIGINAL
VARIANT
SIGNUPS
Signups Install JS
Custom Data
Custom Data Opportunities Created
SOCIAL AD FUNNEL
TOMASZ BORYS Director of Marketing, Kissmetrics @tbcali tborys@kissmetrics.com THUE MADSEN Marketing Operations Manager, Kissmetrics @ThueLMadsen tmadsen@kissmetrics.com Questions?
Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing ...
Keynote by @tonw / Ton Wesseling - CEO of http://testing.agency - he shares all the failures he saw and also made himself on analytics and A/B-testing.
Are you also questioning why your total website revenue is not exploding after you've put all those A/B-test winners live? Find the answers in these slides.
eMetrics London - The AB Testing Hype Cycle
My talk at eMetrics about how to escape from stupid testing that doesn't work.
Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...
Chief Persuasion Officer Bart Schutz will be keynoting about his Wheel of Persuasion on marketing & optimization conferences throughout Europe in october & november 2014:
Conversion Jam - Stockhom, Sweden
Conversions@Google - Dublin, Ireland
iLive - Riga, Latvia
Conversion Conference - London, England
Conversion Conference - Berlin, Germany
Conversion Summit - Istanbul, Turkey
Conversion Hotel - Texel, the Netherlands
At these conferences he will present his Maxima (System 1) and Willem (System 2) story and inspire you how to persuade these two parts of your customers brains. This slideset is the main slideset for each conference and is extended with extra A/B-testcases.
More info:
http://wheel-of-persuasion.com (the knowledge source)
http://online-dialogue.com (hire our team)
http://testing.agency (fast and quality A/B-test development)
http://abtestguide.com (you A/B-test workflow tool)
and http://Persuasion.tips for an overview of Bart his recent work.
A/B Testing With Real Examples And Results
Here is a bunch of A/B test examples and results that I gathered from all over web. Hope this will help people understand A/B test more :)
Google Partner Agency Digital Breakfast with Geary LSF
Geary LSF and Google partnered up to present the latest innovation at Google and in digital marketing at an exclusive Google Partner Agency event in San Francisco.
Recommended
A/B Testing: You Might be Driving in the Wrong Direction
A/B Testing
You Might Be Driving in the Wrong Direction TOMASZ BORYS
@Kissmetrics #KissWebinar @thuelmadsen
Thue is the Kissmetrics Webinar Wizard and Marketing Ops Manager. Before joining forces with Kissmetrics, he was a Lyft driver in SF, which is also how he ended up as a Kissmetrics marketer. Whenever Thue is not trying to automate everything around him, you can find him hiking in the Sierras. THUE MADSEN Marketing Operations Manager, Kissmetrics @ThueLMadsen Tomasz loves dipping his feet in the river while fishing, injuring his thumb while gaming, hacking away at a golf club, and driving demand at Kissmetrics. He’s also the biggest fan of gummy bears. TOMASZ BORYS Director of Marketing, Kissmetrics @tbcali
@Kissmetrics #KissWebinar @tbcali
1 The optimum strategies for A/B testing 2 How we A/B Test at KISSmetrics 3 A/B Testing beyond click conversions - having a pulse on the entire funnel and why it’s important TABLE OF CONTENTS 4 The influence social traffic has in the funnel
• Set a Goal • Baby Steps • Aim for Statistical Significance • Never Lose Sight THE OPTIMUM STRATEGIES FOR A/B TESTING
Baby Steps People want to move the needle and see results fast, by implementing multiple changes at once. But how can you be sure what element had an impact?
Aim for Statistical Significance SET A GOAL Aim for Statistical Significance getdatadriven.com
Never Lose Sight We can caught up on what we think is best to optimize conversions or traffic…but data doesn’t lie
• 4,000 and above sample size • 99% Statistical Confidence • Blind Eye for 1 Week HOW WE A/B TEST AT KISSMETRICS
4,000 and Above Sample Size Significant or decent amount of data is needed
when you’re testing beyond click conversions
4,000 and Above Sample Size significant or decent amount of data is needed when you’re testing beyond click conversions 99% Statistical Confidence Police
Blind Eye for 1 Week Data can be very erratic the first several days, so it’s easy to hit the panic button
A/B Testing Beyond Click Conversions
ORIGINAL
VARIANT
SIGNUPS
Signups Install JS
Custom Data
Custom Data Opportunities Created
SOCIAL AD FUNNEL
TOMASZ BORYS Director of Marketing, Kissmetrics @tbcali tborys@kissmetrics.com THUE MADSEN Marketing Operations Manager, Kissmetrics @ThueLMadsen tmadsen@kissmetrics.com Questions?
Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing ...
Keynote by @tonw / Ton Wesseling - CEO of http://testing.agency - he shares all the failures he saw and also made himself on analytics and A/B-testing.
Are you also questioning why your total website revenue is not exploding after you've put all those A/B-test winners live? Find the answers in these slides.
eMetrics London - The AB Testing Hype Cycle
My talk at eMetrics about how to escape from stupid testing that doesn't work.
Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...
Chief Persuasion Officer Bart Schutz will be keynoting about his Wheel of Persuasion on marketing & optimization conferences throughout Europe in october & november 2014:
Conversion Jam - Stockhom, Sweden
Conversions@Google - Dublin, Ireland
iLive - Riga, Latvia
Conversion Conference - London, England
Conversion Conference - Berlin, Germany
Conversion Summit - Istanbul, Turkey
Conversion Hotel - Texel, the Netherlands
At these conferences he will present his Maxima (System 1) and Willem (System 2) story and inspire you how to persuade these two parts of your customers brains. This slideset is the main slideset for each conference and is extended with extra A/B-testcases.
More info:
http://wheel-of-persuasion.com (the knowledge source)
http://online-dialogue.com (hire our team)
http://testing.agency (fast and quality A/B-test development)
http://abtestguide.com (you A/B-test workflow tool)
and http://Persuasion.tips for an overview of Bart his recent work.
A/B Testing With Real Examples And Results
Here is a bunch of A/B test examples and results that I gathered from all over web. Hope this will help people understand A/B test more :)
Google Partner Agency Digital Breakfast with Geary LSF
Geary LSF and Google partnered up to present the latest innovation at Google and in digital marketing at an exclusive Google Partner Agency event in San Francisco.
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...
A presentation covering the what, why, and how for using Google Analytics as the central hub for gaining marketing insights and acting on them to drive business growth.
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
Joe Nguyen presents trends for online in Vietnam for Google's first ever Think Digital in VN
How Hospital Administrators Make Purchase Decision - Think with Google Report
What and how digital resources influence hospital administrators during their purchase journey.
Evolution of TV Measuring Across All Screens | Think With Google
No matter how measurement is structured in the future, or what changes are on the horizon in terms of technology and viewer behavior, digital will continue to lead the way to greater clarity and precision. Advertisers will have better efficiency, programmers can maximize the value of their inventory, and consumers will have a better viewing experience. And that’s a win-win for everyone.
The Best A/B Test Idea You Haven’t Thought Of
The problem
Not so much Winner! Not so much Not so much Not so much Not so much Not so much Not so much Not so much Not so much
The real problem
One month Winner! Two months Three months Four months Five months Six months Seven months Eight months Nine months
1 winner per year
QUOTE FROM ELON MUSK BIOGRAPHY “ He would place this urgency that he expected the revenue in ten years to be ten million dollars a day and that every day we were slower to achieve our goals was a day of missing out on that money.
We’re missing out on this
But…why?
TOO MANY IDEAS!
170 ideas in 3 minutes
Try new CTA Try new CTA location Test multiple CTAs Test different button colors Add an “About” section Try a different headline Add an image to the homepage Shorten my form Auto play the video Try modals for forms Test a free trial Try annual billing Try a high anchor price
Er.
We need a process
Find out what’s stopping folks from converting…
…fix it.
Thanks
I’ll be here all night
Step 1: Take these ideas…
Try new CTA Try new CTA location Test multiple CTAs Test different button colors Add an “About” section Try a different headline Add an image to the homepage Shorten my form Auto play the video Try modals for forms Test a free trial Try annual billing Try a high anchor price
…and throw them away.
Step 2: Listen to your customers and prospects
Try new CTA Try new CTA location Test multiple CTAs Test different button colors Add an “About” section Try a different headline Add an image to the homepage Shorten my form Auto play the video Try modals for forms Test a free trial Try annual billing Try a high anchor price These ideas are good. They just might not be for you.
Gut Your gut is good. It just knows too much.
Curse of knowledge
USER TESTING
Send 3 users through your funnel
Send 3 users to your competition
Send 3 users to a HUGE company in your space
PEEK
QUALAROO
CHAT TRANSCRIPTS
SALES PEOPLE AND SUPPORT
Step 3: Use a framework to make decisions
Test summary Test objective Hypothesis Opportunity size Time to test Likely scenarios Next steps What worked? What didn’t work? HTTP://BIT.LY/ABTEST-FRAMEWORK
The problem
Not so much Winner! Not so much Not so much Not so much Not so much Not so much Not so much Not so much Not so much
The problem opportunity
Not so much Winner! Not so much Not so much Not so much Not so much Not so much Not so much Not so much Not so much
Not so much Winner! Not so much Not so much Not so much Winner! Not so much Not so much Not so much Not so much
Not so much Not so much Not so much Winner! Not so much Not so much Winner! Not so much Not so much Winner!
Auto aftermarket digital driver research studies
The needs of aftermarket shoppers haven’t changed. There are two types of behaviors when it comes to vehicle parts and accessories purchases: Proactive drivers buy for the season, enhancement and regular maintenance, while reactive ones purchase for repairs. But what has changed?
The anatomy of an A/B Test - JSConf Colombia Workshop
“In God we trust, all others must bring data”. Intuition, experience and well known patterns may give us good indications of successful ideas and features, but nothing gets closer to the truth than data analysis and A/B testing. In this workshop, we’ll show how we do experimentation at Booking: what we test, how to get data through templates and JavaScript, and how we analyse the resulting metrics. We’ll live-code examples, see all potential caveats of dealing with the user tracking on the client-side, and show existent tools you can use to test your own ideas.
How to Successfully Run Your First Website A/B Test
What is A/B testing? A SCIENTIFIC METHOD OF PROOF
WHAT IS A/B TESTING FOR MARKETING
Where do you start testing? THE DON’TS
WHERE DO YOU START TESTING?
DON’T JUST GUESS
SUB OPTIMIZE • Button colors • Small copy changes • Small layout changes • Fonts, font size
WATCH OUT FOR BUZZ • Don’t just test what the gurus say • What works for others may not work for you
PREMATURE TESTING • Testing without traffic • Before you know what and why you are testing
Where do you start testing? THE DOS
DO YOUR HOMEWORK You have to put in the work to get great results • Learn as much as you can about A/B testing • Study the great people out there • What’s their process and insights?
A framework for success The basis of a successful testing program
Research Know what to test and see the greatest improvements
DIVE INTO YOUR DATA
FIND YOUR BIGGEST OPPORTUNITIES
INVESTIGATE THE OPPORTUNITY TO UNDERSTAND IT
FORM A HYPOTHESIS By doing A, B will happen, Because of C
BUILD THE TEST
LAUNCH THE TEST VS. Control Variant
THE TEST VS. +24.01% increase in registrations!! 100% Significance
DOCUMENT AND SHARE YOUR FINDINGS Documenting all your tests and results • Increases company transparency • Helps others know where tests are going on • Makes it easier to call tests • Makes learnings available for everyone in company
REPEAT THE PROCESS AS OFTEN AS POSSIBLE
Control vs. Variant
“Only 1 out of 8 A/B tests have driven significant change.” Noah Kagan AppSumo*
“The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift,” Dr. Flint McGlaughlin MECLABS
RULES OF THUMB FOR TESTING Things to remember when we are testing • Min. 2000 visitors • Min. a few hundred conversions • At least one full cycle > 2 week • >99% statistical significance • Keep a backlog of test hypothesis • Build next tests as the current one is running *More info at http://kiss.ly/1vF4wUc and http://kiss.ly/1Q4vTm0
Tooltips AND BURGERS
How I like my testing burger Tooltips for your testing framework
Tool Company Why do I like it? Analytics Kissmetrics Shows me my actual customers and their behavior Surveys Qualaroo and/or SurveyMonkey Easy to use/Deep qualitative insights Project Management Trello Manage my time + communicate with collaborators
Tool Company Why do I like it? Company Wiki Confluence Document initial thoughts for everyone Testing platform* Optimizely, Pardot, Adwords Easily build tests and get data Analytics Kissmetrics Let’s me confirm and verify down my funnel Company Wiki Confluence Documentation and learnings for everyone *Includes actual testing platform and tools that have testing capabilities
Not everyone likes their burger the same way How do you take yours?
Trust the process Share your leanings Stack your results A/B TESTING FOR SUCCESS
A/B Testing at Pinterest: Building a Culture of Experimentation
Presenter: Andrea Burbank, Pinterest
A successful experimentation program consists of much more than mere randomization and measurement. How do you help stakeholders understand the right things to measure, avoid common pitfalls, and learn to rely on A/B tests as the best way to measure a new system or feature? In this talk, Andrea will explain how building a culture of experimentation and the right tools to support it is just as important as the statistics behind the comparisons themselves - and potentially much trickier to get right.
Millward brown vermeer organizational design think brand 2014 snf
MB Vermeer partnered with Google for Organizing for Marketing in a Digital Age – a thought-leadership initiative that codified best practices and practical insights into how winning teams are organizing for the evolving digital landscape. CMO Marc de Swaan Arons presented the findings to +500 senior level marketers at the Google ThinkBrand conference this October.
A/B Testing Best Practices - Do's and Don'ts
A/B Testing best practices from strategic vision to operational considerations to communication and finally expectations management. We need to adhere to fundamental project management, technology, statistical, experimental design, UX Design, Customer Relationship, business and data principles to ensure that the insights and hence the decision is as trustworthy as possible.
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
Videos are more engaging, more memorable, and more popular than any other type of content out there. That’s why it’s estimated that 82% of consumer traffic will come from videos by 2025.
And with videos evolving from landscape to portrait and experts promoting shorter clips, one thing remains constant – our brains LOVE videos.
So is there science behind what makes people absolutely irresistible on camera?
The answer: definitely yes.
In this jam-packed session with Stephanie Garcia, you’ll get your hands on a steal-worthy guide that uncovers the art and science to being irresistible on camera. From body language to words that convert, she’ll show you how to captivate on command so that viewers are excited and ready to take action.Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
Most small businesses struggle to see marketing results. In this session, we will eliminate any confusion about what to do next, solving your marketing problems so your business can thrive. You’ll learn how to create a foundational marketing OS (operating system) based on neuroscience and backed by real-world results. You’ll be taught how to develop deep customer connections, and how to have your CRM dynamically segment and sell at any stage in the customer’s journey. By the end of the session, you’ll remove confusion and chaos and replace it with clarity and confidence for long-term marketing success.
Key Takeaways:
• Uncover the power of a foundational marketing system that dynamically communicates with prospects and customers on autopilot.
• Harness neuroscience and Tribal Alignment to transform your communication strategies, turning potential clients into fans and those fans into loyal customers.
• Discover the art of automated segmentation, pinpointing your most lucrative customers and identifying the optimal moments for successful conversions.
• Streamline your business with a content production plan that eliminates guesswork, wasted time, and money.BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
Digital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on HowDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
The digital marketing industry is changing faster than ever and those who don’t adapt with the times are losing market share. Where should marketers be focusing their efforts? What strategies are the experts seeing get the best results? Get up-to-speed with the latest industry insights, trends and predictions for the future in this panel discussion with some leading digital marketing experts.The What, Why & How of 3D and AR in Digital Commerce
Vladimir Mulhem has over 20 years of experience in commercialising cutting edge creative technology across construction, marketing and retail.
Previously the founder and Tech and Innovation Director of Creative Content Works working with the likes of Next, John Lewis and JD Sport, he now helps retailers, brands and agencies solve challenges of applying the emerging technologies 3D, AR, VR and Gen AI to real-world problems.
In this webinar, Vladimir will be covering the following topics:
Applications of 3D and AR in Digital Commerce,
Benefits of 3D and AR,
Tools to create, manage and publish 3D and AR in Digital Commerce.
10 Videos Any Business Can Make Right Now! - Shelly Nathan
10 Videos Any Business Can Make Right Now! - Shelly NathanDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
10 Video Ideas Any Business Can Make RIGHT NOW!
You'll never draw a blank again on what kind of video to make for your business. Go beyond the basic categories and truly reimagine a brand new advanced way to brainstorm video content creation. During this masterclass you'll be challenged to think creatively and outside of the box and view your videos through lenses you may have never thought of previously. It's guaranteed that you'll leave with more than 10 video ideas, but I like to under-promise and over-deliver. Don't miss this session.
Key Takeaways:
How to use the Video Matrix
How to use additional "Lenses"
Where to source original video ideasHow to Run Landing Page Tests On and Off Paid Social Platforms
Join us for an exclusive webinar featuring Mariate, Alexandra and Nima where we will unveil a comprehensive blueprint for crafting a successful paid media strategy focused on landing page testing.With escalating costs in paid advertising, understanding how to maximize each visitor’s experience is crucial for retention and conversion.
This session will dive into the methodologies for executing and analyzing landing page tests within paid social channels, offering a blend of theoretical knowledge and practical insights.
The Pearmill team will guide you through the nuances of setting up and managing landing page experiments on paid social platforms. You will learn about the critical rules to follow, the structure of effective tests, optimal conversion duration and budget allocation.
The session will also cover data analysis techniques and criteria for graduating landing pages.
In the second part of the webinar, Pearmill will explore the use of A/B testing platforms. Discover common pitfalls to avoid in A/B testing and gain insights into analyzing A/B tests results effectively.
Coca Cola Branding Strategy and strategic marketing plan
A group presentation of BBA students, University of Okara, Punjab, Pakistan.
De-risk Your Digital Evolution - Hannah Grap
De-risk Your Digital Evolution - Hannah GrapDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
It's another new era of digital and marketers are faced with making big bets on their digital strategy. If you are looking at modernizing your tech stack to support your digital evolution, there are a few can't miss (often overlooked) areas that should be part of every conversation. We'll cover setting your vision, avoiding siloes, adding a democratized approach to data strategy, localization, creating critical governance requirements and more. Attendees will walk away with actions they can take into initiatives they are running today and consider for the future.Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
When most people in the industry talk about online or digital reputation management, what they're really saying is Google search and PPC. And it's usually reactive, left dealing with the aftermath of negative information published somewhere online. That's outdated. It leaves executives, organizations and other high-profile individuals at a high risk of a digital reputation attack that spans channels and tactics. But the tools needed to safeguard against an attack are more cybersecurity-oriented than most marketing and communications professionals can manage. Business leaders Leaders grasp the importance; 83% of executives place reputation in their top five areas of risk, yet only 23% are confident in their ability to address it. To succeed in 2024 and beyond, you need to turn online reputation on its axis and think like an attacker.
Key Takeaways:
- New framework for examining and safeguarding an online reputation
- Tools and techniques to keep you a step ahead
- Practical examples that demonstrate when to act, how to act and how to recoverMore Related Content
Viewers also liked
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...
A presentation covering the what, why, and how for using Google Analytics as the central hub for gaining marketing insights and acting on them to drive business growth.
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
Joe Nguyen presents trends for online in Vietnam for Google's first ever Think Digital in VN
How Hospital Administrators Make Purchase Decision - Think with Google Report
What and how digital resources influence hospital administrators during their purchase journey.
Evolution of TV Measuring Across All Screens | Think With Google
No matter how measurement is structured in the future, or what changes are on the horizon in terms of technology and viewer behavior, digital will continue to lead the way to greater clarity and precision. Advertisers will have better efficiency, programmers can maximize the value of their inventory, and consumers will have a better viewing experience. And that’s a win-win for everyone.
The Best A/B Test Idea You Haven’t Thought Of
The problem
Not so much Winner! Not so much Not so much Not so much Not so much Not so much Not so much Not so much Not so much
The real problem
One month Winner! Two months Three months Four months Five months Six months Seven months Eight months Nine months
1 winner per year
QUOTE FROM ELON MUSK BIOGRAPHY “ He would place this urgency that he expected the revenue in ten years to be ten million dollars a day and that every day we were slower to achieve our goals was a day of missing out on that money.
We’re missing out on this
But…why?
TOO MANY IDEAS!
170 ideas in 3 minutes
Try new CTA Try new CTA location Test multiple CTAs Test different button colors Add an “About” section Try a different headline Add an image to the homepage Shorten my form Auto play the video Try modals for forms Test a free trial Try annual billing Try a high anchor price
Er.
We need a process
Find out what’s stopping folks from converting…
…fix it.
Thanks
I’ll be here all night
Step 1: Take these ideas…
Try new CTA Try new CTA location Test multiple CTAs Test different button colors Add an “About” section Try a different headline Add an image to the homepage Shorten my form Auto play the video Try modals for forms Test a free trial Try annual billing Try a high anchor price
…and throw them away.
Step 2: Listen to your customers and prospects
Try new CTA Try new CTA location Test multiple CTAs Test different button colors Add an “About” section Try a different headline Add an image to the homepage Shorten my form Auto play the video Try modals for forms Test a free trial Try annual billing Try a high anchor price These ideas are good. They just might not be for you.
Gut Your gut is good. It just knows too much.
Curse of knowledge
USER TESTING
Send 3 users through your funnel
Send 3 users to your competition
Send 3 users to a HUGE company in your space
PEEK
QUALAROO
CHAT TRANSCRIPTS
SALES PEOPLE AND SUPPORT
Step 3: Use a framework to make decisions
Test summary Test objective Hypothesis Opportunity size Time to test Likely scenarios Next steps What worked? What didn’t work? HTTP://BIT.LY/ABTEST-FRAMEWORK
The problem
Not so much Winner! Not so much Not so much Not so much Not so much Not so much Not so much Not so much Not so much
The problem opportunity
Not so much Winner! Not so much Not so much Not so much Not so much Not so much Not so much Not so much Not so much
Not so much Winner! Not so much Not so much Not so much Winner! Not so much Not so much Not so much Not so much
Not so much Not so much Not so much Winner! Not so much Not so much Winner! Not so much Not so much Winner!
Auto aftermarket digital driver research studies
The needs of aftermarket shoppers haven’t changed. There are two types of behaviors when it comes to vehicle parts and accessories purchases: Proactive drivers buy for the season, enhancement and regular maintenance, while reactive ones purchase for repairs. But what has changed?
The anatomy of an A/B Test - JSConf Colombia Workshop
“In God we trust, all others must bring data”. Intuition, experience and well known patterns may give us good indications of successful ideas and features, but nothing gets closer to the truth than data analysis and A/B testing. In this workshop, we’ll show how we do experimentation at Booking: what we test, how to get data through templates and JavaScript, and how we analyse the resulting metrics. We’ll live-code examples, see all potential caveats of dealing with the user tracking on the client-side, and show existent tools you can use to test your own ideas.
How to Successfully Run Your First Website A/B Test
What is A/B testing? A SCIENTIFIC METHOD OF PROOF
WHAT IS A/B TESTING FOR MARKETING
Where do you start testing? THE DON’TS
WHERE DO YOU START TESTING?
DON’T JUST GUESS
SUB OPTIMIZE • Button colors • Small copy changes • Small layout changes • Fonts, font size
WATCH OUT FOR BUZZ • Don’t just test what the gurus say • What works for others may not work for you
PREMATURE TESTING • Testing without traffic • Before you know what and why you are testing
Where do you start testing? THE DOS
DO YOUR HOMEWORK You have to put in the work to get great results • Learn as much as you can about A/B testing • Study the great people out there • What’s their process and insights?
A framework for success The basis of a successful testing program
Research Know what to test and see the greatest improvements
DIVE INTO YOUR DATA
FIND YOUR BIGGEST OPPORTUNITIES
INVESTIGATE THE OPPORTUNITY TO UNDERSTAND IT
FORM A HYPOTHESIS By doing A, B will happen, Because of C
BUILD THE TEST
LAUNCH THE TEST VS. Control Variant
THE TEST VS. +24.01% increase in registrations!! 100% Significance
DOCUMENT AND SHARE YOUR FINDINGS Documenting all your tests and results • Increases company transparency • Helps others know where tests are going on • Makes it easier to call tests • Makes learnings available for everyone in company
REPEAT THE PROCESS AS OFTEN AS POSSIBLE
Control vs. Variant
“Only 1 out of 8 A/B tests have driven significant change.” Noah Kagan AppSumo*
“The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift,” Dr. Flint McGlaughlin MECLABS
RULES OF THUMB FOR TESTING Things to remember when we are testing • Min. 2000 visitors • Min. a few hundred conversions • At least one full cycle > 2 week • >99% statistical significance • Keep a backlog of test hypothesis • Build next tests as the current one is running *More info at http://kiss.ly/1vF4wUc and http://kiss.ly/1Q4vTm0
Tooltips AND BURGERS
How I like my testing burger Tooltips for your testing framework
Tool Company Why do I like it? Analytics Kissmetrics Shows me my actual customers and their behavior Surveys Qualaroo and/or SurveyMonkey Easy to use/Deep qualitative insights Project Management Trello Manage my time + communicate with collaborators
Tool Company Why do I like it? Company Wiki Confluence Document initial thoughts for everyone Testing platform* Optimizely, Pardot, Adwords Easily build tests and get data Analytics Kissmetrics Let’s me confirm and verify down my funnel Company Wiki Confluence Documentation and learnings for everyone *Includes actual testing platform and tools that have testing capabilities
Not everyone likes their burger the same way How do you take yours?
Trust the process Share your leanings Stack your results A/B TESTING FOR SUCCESS
A/B Testing at Pinterest: Building a Culture of Experimentation
Presenter: Andrea Burbank, Pinterest
A successful experimentation program consists of much more than mere randomization and measurement. How do you help stakeholders understand the right things to measure, avoid common pitfalls, and learn to rely on A/B tests as the best way to measure a new system or feature? In this talk, Andrea will explain how building a culture of experimentation and the right tools to support it is just as important as the statistics behind the comparisons themselves - and potentially much trickier to get right.
Millward brown vermeer organizational design think brand 2014 snf
MB Vermeer partnered with Google for Organizing for Marketing in a Digital Age – a thought-leadership initiative that codified best practices and practical insights into how winning teams are organizing for the evolving digital landscape. CMO Marc de Swaan Arons presented the findings to +500 senior level marketers at the Google ThinkBrand conference this October.
A/B Testing Best Practices - Do's and Don'ts
A/B Testing best practices from strategic vision to operational considerations to communication and finally expectations management. We need to adhere to fundamental project management, technology, statistical, experimental design, UX Design, Customer Relationship, business and data principles to ensure that the insights and hence the decision is as trustworthy as possible.
Viewers also liked (12)
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
How Hospital Administrators Make Purchase Decision - Think with Google Report
How Hospital Administrators Make Purchase Decision - Think with Google Report
Evolution of TV Measuring Across All Screens | Think With Google
Evolution of TV Measuring Across All Screens | Think With Google
The anatomy of an A/B Test - JSConf Colombia Workshop
The anatomy of an A/B Test - JSConf Colombia Workshop
How to Successfully Run Your First Website A/B Test
How to Successfully Run Your First Website A/B Test
A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation
Millward brown vermeer organizational design think brand 2014 snf
Millward brown vermeer organizational design think brand 2014 snf
Recently uploaded
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
Videos are more engaging, more memorable, and more popular than any other type of content out there. That’s why it’s estimated that 82% of consumer traffic will come from videos by 2025.
And with videos evolving from landscape to portrait and experts promoting shorter clips, one thing remains constant – our brains LOVE videos.
So is there science behind what makes people absolutely irresistible on camera?
The answer: definitely yes.
In this jam-packed session with Stephanie Garcia, you’ll get your hands on a steal-worthy guide that uncovers the art and science to being irresistible on camera. From body language to words that convert, she’ll show you how to captivate on command so that viewers are excited and ready to take action.Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
Most small businesses struggle to see marketing results. In this session, we will eliminate any confusion about what to do next, solving your marketing problems so your business can thrive. You’ll learn how to create a foundational marketing OS (operating system) based on neuroscience and backed by real-world results. You’ll be taught how to develop deep customer connections, and how to have your CRM dynamically segment and sell at any stage in the customer’s journey. By the end of the session, you’ll remove confusion and chaos and replace it with clarity and confidence for long-term marketing success.
Key Takeaways:
• Uncover the power of a foundational marketing system that dynamically communicates with prospects and customers on autopilot.
• Harness neuroscience and Tribal Alignment to transform your communication strategies, turning potential clients into fans and those fans into loyal customers.
• Discover the art of automated segmentation, pinpointing your most lucrative customers and identifying the optimal moments for successful conversions.
• Streamline your business with a content production plan that eliminates guesswork, wasted time, and money.BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
Digital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on HowDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
The digital marketing industry is changing faster than ever and those who don’t adapt with the times are losing market share. Where should marketers be focusing their efforts? What strategies are the experts seeing get the best results? Get up-to-speed with the latest industry insights, trends and predictions for the future in this panel discussion with some leading digital marketing experts.The What, Why & How of 3D and AR in Digital Commerce
Vladimir Mulhem has over 20 years of experience in commercialising cutting edge creative technology across construction, marketing and retail.
Previously the founder and Tech and Innovation Director of Creative Content Works working with the likes of Next, John Lewis and JD Sport, he now helps retailers, brands and agencies solve challenges of applying the emerging technologies 3D, AR, VR and Gen AI to real-world problems.
In this webinar, Vladimir will be covering the following topics:
Applications of 3D and AR in Digital Commerce,
Benefits of 3D and AR,
Tools to create, manage and publish 3D and AR in Digital Commerce.
10 Videos Any Business Can Make Right Now! - Shelly Nathan
10 Videos Any Business Can Make Right Now! - Shelly NathanDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
10 Video Ideas Any Business Can Make RIGHT NOW!
You'll never draw a blank again on what kind of video to make for your business. Go beyond the basic categories and truly reimagine a brand new advanced way to brainstorm video content creation. During this masterclass you'll be challenged to think creatively and outside of the box and view your videos through lenses you may have never thought of previously. It's guaranteed that you'll leave with more than 10 video ideas, but I like to under-promise and over-deliver. Don't miss this session.
Key Takeaways:
How to use the Video Matrix
How to use additional "Lenses"
Where to source original video ideasHow to Run Landing Page Tests On and Off Paid Social Platforms
Join us for an exclusive webinar featuring Mariate, Alexandra and Nima where we will unveil a comprehensive blueprint for crafting a successful paid media strategy focused on landing page testing.With escalating costs in paid advertising, understanding how to maximize each visitor’s experience is crucial for retention and conversion.
This session will dive into the methodologies for executing and analyzing landing page tests within paid social channels, offering a blend of theoretical knowledge and practical insights.
The Pearmill team will guide you through the nuances of setting up and managing landing page experiments on paid social platforms. You will learn about the critical rules to follow, the structure of effective tests, optimal conversion duration and budget allocation.
The session will also cover data analysis techniques and criteria for graduating landing pages.
In the second part of the webinar, Pearmill will explore the use of A/B testing platforms. Discover common pitfalls to avoid in A/B testing and gain insights into analyzing A/B tests results effectively.
Coca Cola Branding Strategy and strategic marketing plan
A group presentation of BBA students, University of Okara, Punjab, Pakistan.
De-risk Your Digital Evolution - Hannah Grap
De-risk Your Digital Evolution - Hannah GrapDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
It's another new era of digital and marketers are faced with making big bets on their digital strategy. If you are looking at modernizing your tech stack to support your digital evolution, there are a few can't miss (often overlooked) areas that should be part of every conversation. We'll cover setting your vision, avoiding siloes, adding a democratized approach to data strategy, localization, creating critical governance requirements and more. Attendees will walk away with actions they can take into initiatives they are running today and consider for the future.Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
When most people in the industry talk about online or digital reputation management, what they're really saying is Google search and PPC. And it's usually reactive, left dealing with the aftermath of negative information published somewhere online. That's outdated. It leaves executives, organizations and other high-profile individuals at a high risk of a digital reputation attack that spans channels and tactics. But the tools needed to safeguard against an attack are more cybersecurity-oriented than most marketing and communications professionals can manage. Business leaders Leaders grasp the importance; 83% of executives place reputation in their top five areas of risk, yet only 23% are confident in their ability to address it. To succeed in 2024 and beyond, you need to turn online reputation on its axis and think like an attacker.
Key Takeaways:
- New framework for examining and safeguarding an online reputation
- Tools and techniques to keep you a step ahead
- Practical examples that demonstrate when to act, how to act and how to recoverThinkNow 2024 Consumer Financial Wellness Report
Financial curveballs sent many American families reeling in 2023. Household budgets were squeezed by rising interest rates, surging prices on everyday goods, and a stagnating housing market. Consumers were feeling strapped. That sentiment, however, appears to be waning. The question is, to what extent?
To take the pulse of consumers’ feelings about their financial well-being ahead of a highly anticipated election, ThinkNow conducted a nationally representative quantitative survey. The survey highlights consumers’ hopes and anxieties as we move into 2024. Let's unpack the key findings to gain insights about where we stand.
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The New Era Of SEO - How AI Has Changed SEO Forever - Danny LeibrandtDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
In this presentation, Danny Leibrandt explains the impact of AI on SEO and what Google has been doing about it. Learn how to take your SEO game to the next level and win over Google with his new strategy anyone can use. Get actionable steps to rank your name, your business, and your clients on Google - the right way.
Key Takeaways:
1. Real content is king
2. Find ways to show EEAT
3. Repurpose across all platformsHow to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
How to use Short Form Video To Grow Your Brand and Business - Keenya KellyDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
Short video marketing has sweeped the nation and is the fastest way to build an online brand on social media in 2024. In this session you will learn:- What is short video marketing- Which platforms work best for your business- Content strategies that are on brand for your business- How to sell organically without paying for ads.AI-Powered Personalization: Principles, Use Cases, and Its Impact on CRO
In today’s era of AI, personalization is more than just a trend—it’s a fundamental strategy that unlocks numerous opportunities.
When done effectively, personalization builds trust, loyalty, and satisfaction among your users—key factors for business success. However, relying solely on AI capabilities isn’t enough. You need to anchor your approach in solid principles, understand your users’ context, and master the art of persuasion.
Join us as Sarjak Patel and Naitry Saggu from 3rd Eye Consulting unveil a transformative framework. This approach seamlessly integrates your unique context, consumer insights, and conversion goals, paving the way for unparalleled success in personalization.
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Digital Marketing Trends - Experts Insights on How to Gain a Competitive EdgeDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
The digital marketing industry is changing faster than ever and those who don’t adapt with the times are losing market share. Where should marketers be focusing their efforts? What strategies are the experts seeing get the best results? Get up-to-speed with the latest industry insights, trends and predictions for the future in this panel discussion with some leading digital marketing experts.FullSail: HOF - Presentation Phil Pallen
From FullSail University's Creative Presentation Hall Of fame Assignment
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminDigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
When most people in the industry talk about online or digital reputation management, what they're really saying is Google search and PPC. And it's usually reactive, left dealing with the aftermath of negative information published somewhere online. That's outdated. It leaves executives, organizations and other high-profile individuals at a high risk of a digital reputation attack that spans channels and tactics. But the tools needed to safeguard against an attack are more cybersecurity-oriented than most marketing and communications professionals can manage. Business leaders Leaders grasp the importance; 83% of executives place reputation in their top five areas of risk, yet only 23% are confident in their ability to address it. To succeed in 2024 and beyond, you need to turn online reputation on its axis and think like an attacker.\
Key Takeaways:
- New framework for examining and safeguarding an online reputation
- Tools and techniques to keep you a step ahead
- Practical examples that demonstrate when to act, how to act and how to recoverRecently uploaded (20)
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024. Balmer Lawrie Online Monthly Bulletin
Digital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on How
The What, Why & How of 3D and AR in Digital Commerce
The What, Why & How of 3D and AR in Digital Commerce
10 Videos Any Business Can Make Right Now! - Shelly Nathan
10 Videos Any Business Can Make Right Now! - Shelly Nathan
How to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social Platforms
Coca Cola Branding Strategy and strategic marketing plan
Coca Cola Branding Strategy and strategic marketing plan
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The Old Oak - Press Kit - Cannes Film Festival 2023
The Old Oak - Press Kit - Cannes Film Festival 2023
How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
How to use Short Form Video To Grow Your Brand and Business - Keenya Kelly
20221005110010_633d63baa84f6_learn___week_3_ch._5.pdf
20221005110010_633d63baa84f6_learn___week_3_ch._5.pdf
AI-Powered Personalization: Principles, Use Cases, and Its Impact on CRO
AI-Powered Personalization: Principles, Use Cases, and Its Impact on CRO
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
A Crash Course on A/B Testing
- 7. Awesome Cat Co Awesome Cat Co
- 8. ♥…
- 27. …
- 32. May 2014