SlideShare a Scribd company logo

Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing analytics errors

Online Dialogue
Online Dialogue
Online DialogueOnline Dialogue

Keynote by @tonw / Ton Wesseling - CEO of http://testing.agency - he shares all the failures he saw and also made himself on analytics and A/B-testing. Are you also questioning why your total website revenue is not exploding after you've put all those A/B-test winners live? Find the answers in these slides.

Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing analytics errors

1 of 52
Download to read offline
< H a v e m o r e f u n A / B - t e s t i n g > 
A/B-t e s t i n g Errors 
To n We s s e l i n g – C EO Te s t i n g . A g e n c y 
#DDTT A m s t e r d a m 2 0 1 4 / 0 6 / 2 4
Why me? 
ü 10 years+ A/B-test experience 
ü I always loved the numbers side 
ü I’ve made all these mistakes 
Email: ton@testing.agency Twitter: @TonW #DDTT
Email: ton@testing.agency Twitter: @TonW #DDTT
We optimize 
Email: ton@testing.agency Twitter: @TonW #DDTT
Nature 
Email: ton@testing.agency Twitter: @TonW #DDTT
A better example 
Email: ton@testing.agency Twitter: @TonW #DDTT

Recommended

Visualized andy cotgreave
Visualized andy cotgreaveVisualized andy cotgreave
Visualized andy cotgreaveAndy Cotgreave
 
Using A/B-testing to grow your company / visitor knowledge
Using A/B-testing to grow your company / visitor knowledgeUsing A/B-testing to grow your company / visitor knowledge
Using A/B-testing to grow your company / visitor knowledgeOnline Dialogue
 
Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...
Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...
Keynote Wheel of Persuasion - Bart Schutz of Online Dialogue on how to persua...Online Dialogue
 
Corrall & Goldstein - Using the Researcher Development Framework to Develop P...
Corrall & Goldstein - Using the Researcher Development Framework to Develop P...Corrall & Goldstein - Using the Researcher Development Framework to Develop P...
Corrall & Goldstein - Using the Researcher Development Framework to Develop P...IL Group (CILIP Information Literacy Group)
 
'Demystifying Knowledge Transfer- an introduction to Implementation Science M...
'Demystifying Knowledge Transfer- an introduction to Implementation Science M...'Demystifying Knowledge Transfer- an introduction to Implementation Science M...
'Demystifying Knowledge Transfer- an introduction to Implementation Science M...NEQOS
 
A/B Testing With Real Examples And Results
A/B Testing With Real Examples And ResultsA/B Testing With Real Examples And Results
A/B Testing With Real Examples And ResultsWilson Pang
 
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...University of Nebraska Think Tank: Google Analytics as your Digital Marketing...
University of Nebraska Think Tank: Google Analytics as your Digital Marketing...Caleb Whitmore
 

More Related Content

Viewers also liked

Google Partner Agency Digital Breakfast with Geary LSF
Google Partner Agency Digital Breakfast with Geary LSFGoogle Partner Agency Digital Breakfast with Geary LSF
Google Partner Agency Digital Breakfast with Geary LSFKatie Fellenz
 
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012action.vn
 
How Hospital Administrators Make Purchase Decision - Think with Google Report
How Hospital Administrators Make Purchase Decision - Think with Google Report How Hospital Administrators Make Purchase Decision - Think with Google Report
How Hospital Administrators Make Purchase Decision - Think with Google Report Medico Mart
 
Evolution of TV Measuring Across All Screens | Think With Google
Evolution of TV Measuring Across All Screens | Think With GoogleEvolution of TV Measuring Across All Screens | Think With Google
Evolution of TV Measuring Across All Screens | Think With GoogleArgent media agency
 
The Best A/B Test Idea You Haven’t Thought Of
The Best A/B Test Idea You Haven’t Thought OfThe Best A/B Test Idea You Haven’t Thought Of
The Best A/B Test Idea You Haven’t Thought OfKissmetrics on SlideShare
 
Wheel of Persuasion door Bart Schutz - Marviq -20121114
Wheel of Persuasion door Bart Schutz - Marviq -20121114Wheel of Persuasion door Bart Schutz - Marviq -20121114
Wheel of Persuasion door Bart Schutz - Marviq -20121114Online Dialogue
 
Auto aftermarket digital driver research studies
Auto aftermarket digital driver research studiesAuto aftermarket digital driver research studies
Auto aftermarket digital driver research studiesHouseLead Italia
 
The anatomy of an A/B Test - JSConf Colombia Workshop
The anatomy of an A/B Test - JSConf Colombia WorkshopThe anatomy of an A/B Test - JSConf Colombia Workshop
The anatomy of an A/B Test - JSConf Colombia WorkshopEduardo Shiota Yasuda
 
Why everything is an A/B Test at Pinterest
Why everything is an A/B Test at PinterestWhy everything is an A/B Test at Pinterest
Why everything is an A/B Test at PinterestKrishna Gade
 
How to Successfully Run Your First Website A/B Test
How to Successfully Run Your First Website A/B TestHow to Successfully Run Your First Website A/B Test
How to Successfully Run Your First Website A/B TestKissmetrics on SlideShare
 
A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation WrangleConf
 
Millward brown vermeer organizational design think brand 2014 snf
Millward brown vermeer organizational design think brand 2014 snfMillward brown vermeer organizational design think brand 2014 snf
Millward brown vermeer organizational design think brand 2014 snfVermeer
 
The Impact of Emotional Intelligence: Understanding Consumer Behavior
The Impact of Emotional Intelligence: Understanding Consumer BehaviorThe Impact of Emotional Intelligence: Understanding Consumer Behavior
The Impact of Emotional Intelligence: Understanding Consumer BehaviorAshford University
 
Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...
Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...
Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...SL Ecommerce and ReviewsReputation.com
 
A/B Testing: You Might be Driving in the Wrong Direction
A/B Testing: You Might be Driving in the Wrong DirectionA/B Testing: You Might be Driving in the Wrong Direction
A/B Testing: You Might be Driving in the Wrong DirectionKissmetrics on SlideShare
 
The Operating Model That is Eating the World
The Operating Model That is Eating the WorldThe Operating Model That is Eating the World
The Operating Model That is Eating the Worldundercurrent
 
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Future Insights
 

Viewers also liked (18)

Mobile A/B Test
Mobile A/B TestMobile A/B Test
Mobile A/B Test
 
Google Partner Agency Digital Breakfast with Geary LSF
Google Partner Agency Digital Breakfast with Geary LSFGoogle Partner Agency Digital Breakfast with Geary LSF
Google Partner Agency Digital Breakfast with Geary LSF
 
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
comScore's Presentation Google's Think Digital Vietnam 11-Dec-2012
 
How Hospital Administrators Make Purchase Decision - Think with Google Report
How Hospital Administrators Make Purchase Decision - Think with Google Report How Hospital Administrators Make Purchase Decision - Think with Google Report
How Hospital Administrators Make Purchase Decision - Think with Google Report
 
Evolution of TV Measuring Across All Screens | Think With Google
Evolution of TV Measuring Across All Screens | Think With GoogleEvolution of TV Measuring Across All Screens | Think With Google
Evolution of TV Measuring Across All Screens | Think With Google
 
The Best A/B Test Idea You Haven’t Thought Of
The Best A/B Test Idea You Haven’t Thought OfThe Best A/B Test Idea You Haven’t Thought Of
The Best A/B Test Idea You Haven’t Thought Of
 
Wheel of Persuasion door Bart Schutz - Marviq -20121114
Wheel of Persuasion door Bart Schutz - Marviq -20121114Wheel of Persuasion door Bart Schutz - Marviq -20121114
Wheel of Persuasion door Bart Schutz - Marviq -20121114
 
Auto aftermarket digital driver research studies
Auto aftermarket digital driver research studiesAuto aftermarket digital driver research studies
Auto aftermarket digital driver research studies
 
The anatomy of an A/B Test - JSConf Colombia Workshop
The anatomy of an A/B Test - JSConf Colombia WorkshopThe anatomy of an A/B Test - JSConf Colombia Workshop
The anatomy of an A/B Test - JSConf Colombia Workshop
 
Why everything is an A/B Test at Pinterest
Why everything is an A/B Test at PinterestWhy everything is an A/B Test at Pinterest
Why everything is an A/B Test at Pinterest
 
How to Successfully Run Your First Website A/B Test
How to Successfully Run Your First Website A/B TestHow to Successfully Run Your First Website A/B Test
How to Successfully Run Your First Website A/B Test
 
A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation
 
Millward brown vermeer organizational design think brand 2014 snf
Millward brown vermeer organizational design think brand 2014 snfMillward brown vermeer organizational design think brand 2014 snf
Millward brown vermeer organizational design think brand 2014 snf
 
The Impact of Emotional Intelligence: Understanding Consumer Behavior
The Impact of Emotional Intelligence: Understanding Consumer BehaviorThe Impact of Emotional Intelligence: Understanding Consumer Behavior
The Impact of Emotional Intelligence: Understanding Consumer Behavior
 
Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...
Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...
Google Automotive Think Insights - Digital Drives Car Researches and Auto Pur...
 
A/B Testing: You Might be Driving in the Wrong Direction
A/B Testing: You Might be Driving in the Wrong DirectionA/B Testing: You Might be Driving in the Wrong Direction
A/B Testing: You Might be Driving in the Wrong Direction
 
The Operating Model That is Eating the World
The Operating Model That is Eating the WorldThe Operating Model That is Eating the World
The Operating Model That is Eating the World
 
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
 

Similar to Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing analytics errors

Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...
Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...
Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...Online Dialogue
 
#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...
#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...
#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...Webanalisten .nl
 
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experimentsKeynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experimentsTon Wesseling
 
Conversion Conference Boston keynote Ton Wesseling
Conversion Conference Boston keynote Ton WesselingConversion Conference Boston keynote Ton Wesseling
Conversion Conference Boston keynote Ton WesselingOnline Dialogue
 
Conversion Hotel 2018 Keynote: Lizzie Eardley
Conversion Hotel 2018 Keynote: Lizzie EardleyConversion Hotel 2018 Keynote: Lizzie Eardley
Conversion Hotel 2018 Keynote: Lizzie EardleyWebanalisten .nl
 
Winning at optimization - Ton Wesseling
Winning at optimization - Ton WesselingWinning at optimization - Ton Wesseling
Winning at optimization - Ton WesselingOnline Dialogue
 
DDTT11: Ton Wesseling - 21-01-20
DDTT11: Ton Wesseling - 21-01-20DDTT11: Ton Wesseling - 21-01-20
DDTT11: Ton Wesseling - 21-01-20Webanalisten .nl
 
Building Powerful Landing Pages
Building Powerful Landing PagesBuilding Powerful Landing Pages
Building Powerful Landing PagesScott Bales
 
Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...
Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...
Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...Online Dialogue
 
How a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO workedHow a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO workedDominic Woodman
 
Test for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing PagesTest for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing PagesOptimizely
 
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!Ton Wesseling
 
Conversion World 2015 Keynote by Ton Wesseling
Conversion World 2015 Keynote by Ton WesselingConversion World 2015 Keynote by Ton Wesseling
Conversion World 2015 Keynote by Ton WesselingOnline Dialogue
 
Progression by Regression: How to increase your A/B Test Velocity
Progression by Regression: How to increase your A/B Test VelocityProgression by Regression: How to increase your A/B Test Velocity
Progression by Regression: How to increase your A/B Test VelocityStitch Fix Algorithms
 
How to build an effective conversion program on a budget
 How to build an effective conversion program on a budget  How to build an effective conversion program on a budget
How to build an effective conversion program on a budget Joni Lindgren
 
Growth Hackers of Vienna - Meetup #2 Part1&2
Growth Hackers of Vienna - Meetup #2  Part1&2Growth Hackers of Vienna - Meetup #2  Part1&2
Growth Hackers of Vienna - Meetup #2 Part1&2Growth Hackers of Vienna
 
No estimates - a controversial way to improve estimation with results-handouts
No estimates - a controversial way to improve estimation with results-handoutsNo estimates - a controversial way to improve estimation with results-handouts
No estimates - a controversial way to improve estimation with results-handoutsVasco Duarte
 

Similar to Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing analytics errors (20)

Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...
Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...
Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion ...
 
#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...
#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...
#DDTT meetup #7 Amsterdam - Results on the state of the Dutch CRO Industry su...
 
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experimentsKeynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
Keynote Digital Elite Camp Estonia 2015 - ROAR - when to run more experiments
 
Conversion Conference Boston keynote Ton Wesseling
Conversion Conference Boston keynote Ton WesselingConversion Conference Boston keynote Ton Wesseling
Conversion Conference Boston keynote Ton Wesseling
 
Conversion Hotel 2018 Keynote: Lizzie Eardley
Conversion Hotel 2018 Keynote: Lizzie EardleyConversion Hotel 2018 Keynote: Lizzie Eardley
Conversion Hotel 2018 Keynote: Lizzie Eardley
 
Winning at optimization - Ton Wesseling
Winning at optimization - Ton WesselingWinning at optimization - Ton Wesseling
Winning at optimization - Ton Wesseling
 
DDTT11: Ton Wesseling - 21-01-20
DDTT11: Ton Wesseling - 21-01-20DDTT11: Ton Wesseling - 21-01-20
DDTT11: Ton Wesseling - 21-01-20
 
Building Powerful Landing Pages
Building Powerful Landing PagesBuilding Powerful Landing Pages
Building Powerful Landing Pages
 
Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...
Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...
Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize...
 
Conversion Jam
Conversion JamConversion Jam
Conversion Jam
 
How a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO workedHow a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO worked
 
Top 10 Tips for Your Split Test Program by Justin Rondeau
Top 10 Tips for Your Split Test Program by Justin RondeauTop 10 Tips for Your Split Test Program by Justin Rondeau
Top 10 Tips for Your Split Test Program by Justin Rondeau
 
Test for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing PagesTest for Success: A Guide to A/B Testing on Emails & Landing Pages
Test for Success: A Guide to A/B Testing on Emails & Landing Pages
 
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
Keynote Ton Wesseling at Superweek 2020: How an analyst can add value!
 
Conversion World 2015 Keynote by Ton Wesseling
Conversion World 2015 Keynote by Ton WesselingConversion World 2015 Keynote by Ton Wesseling
Conversion World 2015 Keynote by Ton Wesseling
 
Conversionboost 2015 tlb
Conversionboost 2015 tlbConversionboost 2015 tlb
Conversionboost 2015 tlb
 
Progression by Regression: How to increase your A/B Test Velocity
Progression by Regression: How to increase your A/B Test VelocityProgression by Regression: How to increase your A/B Test Velocity
Progression by Regression: How to increase your A/B Test Velocity
 
How to build an effective conversion program on a budget
 How to build an effective conversion program on a budget  How to build an effective conversion program on a budget
How to build an effective conversion program on a budget
 
Growth Hackers of Vienna - Meetup #2 Part1&2
Growth Hackers of Vienna - Meetup #2  Part1&2Growth Hackers of Vienna - Meetup #2  Part1&2
Growth Hackers of Vienna - Meetup #2 Part1&2
 
No estimates - a controversial way to improve estimation with results-handouts
No estimates - a controversial way to improve estimation with results-handoutsNo estimates - a controversial way to improve estimation with results-handouts
No estimates - a controversial way to improve estimation with results-handouts
 

More from Online Dialogue

Introduction to BOOM! - Behavioral online optimization method
Introduction to BOOM! - Behavioral online optimization methodIntroduction to BOOM! - Behavioral online optimization method
Introduction to BOOM! - Behavioral online optimization methodOnline Dialogue
 
GAUC 2020 - presentatie Hans en Reinier
GAUC 2020 - presentatie Hans en ReinierGAUC 2020 - presentatie Hans en Reinier
GAUC 2020 - presentatie Hans en ReinierOnline Dialogue
 
Online Dialogue Digital Analytics Congres 2019
Online Dialogue Digital Analytics Congres 2019Online Dialogue Digital Analytics Congres 2019
Online Dialogue Digital Analytics Congres 2019Online Dialogue
 
Presentation Emerce Conversion_Bart Schutz_April 11th 2019
Presentation Emerce Conversion_Bart Schutz_April 11th 2019Presentation Emerce Conversion_Bart Schutz_April 11th 2019
Presentation Emerce Conversion_Bart Schutz_April 11th 2019Online Dialogue
 
Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019
Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019
Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019Online Dialogue
 
Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...
Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...
Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...Online Dialogue
 
Dialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van Dam
Dialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van DamDialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van Dam
Dialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van DamOnline Dialogue
 
Dialogue Donderdag #28 Gerben Langendijk
Dialogue Donderdag #28 Gerben LangendijkDialogue Donderdag #28 Gerben Langendijk
Dialogue Donderdag #28 Gerben LangendijkOnline Dialogue
 
Joost Fromberg at the Global Digital Marketing Summit '18
Joost Fromberg at the Global Digital Marketing Summit '18Joost Fromberg at the Global Digital Marketing Summit '18
Joost Fromberg at the Global Digital Marketing Summit '18Online Dialogue
 
Amy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynote
Amy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynoteAmy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynote
Amy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynoteOnline Dialogue
 
Ronny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynote
Ronny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynoteRonny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynote
Ronny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynoteOnline Dialogue
 
Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...
Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...
Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...Online Dialogue
 
Matthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynote
Matthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynoteMatthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynote
Matthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynoteOnline Dialogue
 
Els Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynote
Els Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynoteEls Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynote
Els Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynoteOnline Dialogue
 
Joost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynote
Joost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynoteJoost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynote
Joost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynoteOnline Dialogue
 
Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...
Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...
Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...Online Dialogue
 
Workshop congres verborgen psycholoog
Workshop congres  verborgen psycholoogWorkshop congres  verborgen psycholoog
Workshop congres verborgen psycholoogOnline Dialogue
 
Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...
Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...
Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...Online Dialogue
 
Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...
Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...
Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...Online Dialogue
 

More from Online Dialogue (20)

Introduction to BOOM! - Behavioral online optimization method
Introduction to BOOM! - Behavioral online optimization methodIntroduction to BOOM! - Behavioral online optimization method
Introduction to BOOM! - Behavioral online optimization method
 
CRO & scrum combineren
CRO & scrum combinerenCRO & scrum combineren
CRO & scrum combineren
 
GAUC 2020 - presentatie Hans en Reinier
GAUC 2020 - presentatie Hans en ReinierGAUC 2020 - presentatie Hans en Reinier
GAUC 2020 - presentatie Hans en Reinier
 
Online Dialogue Digital Analytics Congres 2019
Online Dialogue Digital Analytics Congres 2019Online Dialogue Digital Analytics Congres 2019
Online Dialogue Digital Analytics Congres 2019
 
Presentation Emerce Conversion_Bart Schutz_April 11th 2019
Presentation Emerce Conversion_Bart Schutz_April 11th 2019Presentation Emerce Conversion_Bart Schutz_April 11th 2019
Presentation Emerce Conversion_Bart Schutz_April 11th 2019
 
Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019
Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019
Presentation Emerce Conversion_Tom vanden Berg_April 11th 2019
 
Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...
Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...
Validation in every Organisation - Ton Weseling - CRO Elite keynote London - ...
 
Dialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van Dam
Dialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van DamDialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van Dam
Dialogue Donderdag #28 Joost Baalbergen, Eline van Baal en Roos van Dam
 
Dialogue Donderdag #28 Gerben Langendijk
Dialogue Donderdag #28 Gerben LangendijkDialogue Donderdag #28 Gerben Langendijk
Dialogue Donderdag #28 Gerben Langendijk
 
Joost Fromberg at the Global Digital Marketing Summit '18
Joost Fromberg at the Global Digital Marketing Summit '18Joost Fromberg at the Global Digital Marketing Summit '18
Joost Fromberg at the Global Digital Marketing Summit '18
 
Amy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynote
Amy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynoteAmy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynote
Amy Cueva, Mad*Pow (USA) - Conversion Hotel 2017 - keynote
 
Ronny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynote
Ronny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynoteRonny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynote
Ronny Kohavi, Microsoft (USA) - Conversion Hotel 2017 - keynote
 
Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...
Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...
Michelle Kiss, Analytics Demystified (AUS/NL/USA) - Conversion Hotel 2017 - k...
 
Matthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynote
Matthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynoteMatthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynote
Matthew Roach, Sanoma (AUS/UK/NL) - Conversion Hotel 2017 - keynote
 
Els Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynote
Els Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynoteEls Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynote
Els Aerts, AGConsult (BE) - Conversion Hotel 2017 - keynote
 
Joost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynote
Joost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynoteJoost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynote
Joost de Schepper, Spotify (SWE) - Conversion Hotel 2017 - keynote
 
Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...
Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...
Keynote #gconvert17 - Bart Schutz of Online Dialogue - Online Persuasion - Sy...
 
Workshop congres verborgen psycholoog
Workshop congres  verborgen psycholoogWorkshop congres  verborgen psycholoog
Workshop congres verborgen psycholoog
 
Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...
Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...
Keynote #DAC17 - Bart Schutz - Marshmallows and the psychology behind evidenc...
 
Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...
Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...
Keynote on Online Experiments by Ton Wesseling - Online Dialogue at the Web A...
 

Recently uploaded

Hootsuite Platform Certification Course - Certificate
Hootsuite Platform Certification Course - CertificateHootsuite Platform Certification Course - Certificate
Hootsuite Platform Certification Course - CertificateAdriannaBednarz
 
5 Digital Marketing Tips . By Digital Dyynamo
5 Digital Marketing Tips . By Digital Dyynamo5 Digital Marketing Tips . By Digital Dyynamo
5 Digital Marketing Tips . By Digital Dyynamoshahanndmitteamf
 
20240214 Security-JAWS activity results for 2023 and activity goals for 2024
20240214 Security-JAWS activity results for 2023 and activity goals for 202420240214 Security-JAWS activity results for 2023 and activity goals for 2024
20240214 Security-JAWS activity results for 2023 and activity goals for 2024Typhon 666
 
Branding doesn’t mean logo - Easy guide to create a powerful branding for you...
Branding doesn’t mean logo - Easy guide to create a powerful branding for you...Branding doesn’t mean logo - Easy guide to create a powerful branding for you...
Branding doesn’t mean logo - Easy guide to create a powerful branding for you...Ahmad El-Saeed
 
How We Built A Strong $10 Million Agency: A Proven Framework
How We Built A Strong $10 Million Agency: A Proven FrameworkHow We Built A Strong $10 Million Agency: A Proven Framework
How We Built A Strong $10 Million Agency: A Proven FrameworkSearch Engine Journal
 
The Most Admired Women Leaders In Mortgage Industry, 2024..pdf
The Most Admired Women Leaders In Mortgage Industry, 2024..pdfThe Most Admired Women Leaders In Mortgage Industry, 2024..pdf
The Most Admired Women Leaders In Mortgage Industry, 2024..pdfwomenworldmagazine
 
I'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdf
I'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdfI'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdf
I'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdfPeter Mesarec
 
Research - Women's Jeans in Colombia (Feb-24).pdf
Research - Women's Jeans in Colombia (Feb-24).pdfResearch - Women's Jeans in Colombia (Feb-24).pdf
Research - Women's Jeans in Colombia (Feb-24).pdfJuan Renteria
 
5 Benefits of Digital Marketing-Theseo-expertz
5 Benefits of Digital Marketing-Theseo-expertz5 Benefits of Digital Marketing-Theseo-expertz
5 Benefits of Digital Marketing-Theseo-expertzTheseo Expertz
 
PPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies World
PPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies WorldPPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies World
PPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies WorldAnu Adegbola
 
4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf
4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf
4.1. PROFESSIONAL SALES REPRESENTATIVE.pdfBALASUNDARESAN M
 
Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...
Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...
Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...janithpandey88
 
Raport Algorithm Insights 2024-NowyMarketing.pdf
Raport Algorithm Insights 2024-NowyMarketing.pdfRaport Algorithm Insights 2024-NowyMarketing.pdf
Raport Algorithm Insights 2024-NowyMarketing.pdfagatadrynko
 
Assignment 5 - Task 1.pptx
Assignment 5 - Task 1.pptxAssignment 5 - Task 1.pptx
Assignment 5 - Task 1.pptxDeanClark42
 
Mastering Social Media, Mobile Marketing and Digital Strategy - Certificate
Mastering Social Media, Mobile Marketing and Digital Strategy - CertificateMastering Social Media, Mobile Marketing and Digital Strategy - Certificate
Mastering Social Media, Mobile Marketing and Digital Strategy - CertificateAdriannaBednarz
 
Social Media Marketing
Social Media MarketingSocial Media Marketing
Social Media MarketingNestor Pablo
 
DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?
DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?
DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?Andreas Kennardi Julianto
 
GroupM_TYNY_Trends_Final.pdf
GroupM_TYNY_Trends_Final.pdfGroupM_TYNY_Trends_Final.pdf
GroupM_TYNY_Trends_Final.pdfSocial Samosa
 

Recently uploaded (20)

Hootsuite Platform Certification Course - Certificate
Hootsuite Platform Certification Course - CertificateHootsuite Platform Certification Course - Certificate
Hootsuite Platform Certification Course - Certificate
 
5 Digital Marketing Tips . By Digital Dyynamo
5 Digital Marketing Tips . By Digital Dyynamo5 Digital Marketing Tips . By Digital Dyynamo
5 Digital Marketing Tips . By Digital Dyynamo
 
20240214 Security-JAWS activity results for 2023 and activity goals for 2024
20240214 Security-JAWS activity results for 2023 and activity goals for 202420240214 Security-JAWS activity results for 2023 and activity goals for 2024
20240214 Security-JAWS activity results for 2023 and activity goals for 2024
 
Branding doesn’t mean logo - Easy guide to create a powerful branding for you...
Branding doesn’t mean logo - Easy guide to create a powerful branding for you...Branding doesn’t mean logo - Easy guide to create a powerful branding for you...
Branding doesn’t mean logo - Easy guide to create a powerful branding for you...
 
How We Built A Strong $10 Million Agency: A Proven Framework
How We Built A Strong $10 Million Agency: A Proven FrameworkHow We Built A Strong $10 Million Agency: A Proven Framework
How We Built A Strong $10 Million Agency: A Proven Framework
 
The Most Admired Women Leaders In Mortgage Industry, 2024..pdf
The Most Admired Women Leaders In Mortgage Industry, 2024..pdfThe Most Admired Women Leaders In Mortgage Industry, 2024..pdf
The Most Admired Women Leaders In Mortgage Industry, 2024..pdf
 
SEO BRIEF.ppt
SEO BRIEF.pptSEO BRIEF.ppt
SEO BRIEF.ppt
 
I'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdf
I'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdfI'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdf
I'm not Sarah Connor but Focussing on Human Nature Content that Converts.pdf
 
Research - Women's Jeans in Colombia (Feb-24).pdf
Research - Women's Jeans in Colombia (Feb-24).pdfResearch - Women's Jeans in Colombia (Feb-24).pdf
Research - Women's Jeans in Colombia (Feb-24).pdf
 
5 Benefits of Digital Marketing-Theseo-expertz
5 Benefits of Digital Marketing-Theseo-expertz5 Benefits of Digital Marketing-Theseo-expertz
5 Benefits of Digital Marketing-Theseo-expertz
 
PPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies World
PPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies WorldPPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies World
PPC LIVE #9 - Amy Stamper - How to measure Paid Media in a Post-Cookies World
 
4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf
4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf
4.1. PROFESSIONAL SALES REPRESENTATIVE.pdf
 
Slideshare
SlideshareSlideshare
Slideshare
 
Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...
Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...
Elevate Your Marketing Game with Vidya Sarthi Best Digital Marketing Institut...
 
Raport Algorithm Insights 2024-NowyMarketing.pdf
Raport Algorithm Insights 2024-NowyMarketing.pdfRaport Algorithm Insights 2024-NowyMarketing.pdf
Raport Algorithm Insights 2024-NowyMarketing.pdf
 
Assignment 5 - Task 1.pptx
Assignment 5 - Task 1.pptxAssignment 5 - Task 1.pptx
Assignment 5 - Task 1.pptx
 
Mastering Social Media, Mobile Marketing and Digital Strategy - Certificate
Mastering Social Media, Mobile Marketing and Digital Strategy - CertificateMastering Social Media, Mobile Marketing and Digital Strategy - Certificate
Mastering Social Media, Mobile Marketing and Digital Strategy - Certificate
 
Social Media Marketing
Social Media MarketingSocial Media Marketing
Social Media Marketing
 
DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?
DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?
DOES SOCIAL ISSUE ALWAYS REQUIRE CREATIVE SOLUTION?
 
GroupM_TYNY_Trends_Final.pdf
GroupM_TYNY_Trends_Final.pdfGroupM_TYNY_Trends_Final.pdf
GroupM_TYNY_Trends_Final.pdf
 

Keynote Digital Data Tips Tuesday - Amsterdam - June 24th 2014 - A/B-testing analytics errors

  • 1. < H a v e m o r e f u n A / B - t e s t i n g > A/B-t e s t i n g Errors To n We s s e l i n g – C EO Te s t i n g . A g e n c y #DDTT A m s t e r d a m 2 0 1 4 / 0 6 / 2 4
  • 2. Why me? ü 10 years+ A/B-test experience ü I always loved the numbers side ü I’ve made all these mistakes Email: ton@testing.agency Twitter: @TonW #DDTT
  • 4. We optimize Email: ton@testing.agency Twitter: @TonW #DDTT
  • 5. Nature Email: ton@testing.agency Twitter: @TonW #DDTT
  • 6. A better example Email: ton@testing.agency Twitter: @TonW #DDTT
  • 7. Groundhog Day – The A/B-test movie Email: ton@testing.agency Twitter: @TonW #DDTT
  • 8. Trying to get the girl Email: ton@testing.agency Twitter: @TonW #DDTT
  • 9. Constant testing environment A / B Testing Email: ton@testing.agency Twitter: @TonW #DDTT
  • 10. Better Dialogue Optimization Default Variation B Variation Winner!" A Email: ton@testing.agency Twitter: @TonW #DDTT
  • 11. This is why you’re A/B-testing Small steps to learn how to make the next big step Email: ton@testing.agency Twitter: @TonW #DDTT
  • 12. Testing is not free Use it to LEARN Email: ton@testing.agency Twitter: @TonW #DDTT
  • 13. 50% of the A/B-tests I get to see is wrong Email: ton@testing.agency Twitter: @TonW #DDTT
  • 14. Look at your A/B-testing results! +24% +52% +126% +83% +12% Email: ton@testing.agency Twitter: @TonW #DDTT +16%
  • 15. Did you push the winners live to your site? You must be rich now! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 16. Or is this reality? Email: ton@testing.agency Twitter: @TonW #DDTT
  • 17. Or even worse: A/B-test winners are letting you down! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 18. Are these real winners? +24% +52% +83% +12% +126% Email: ton@testing.agency Twitter: @TonW #DDTT +16%
  • 19. 20 reasons why Your A/B-test analyzing sucks Email: ton@testing.agency Twitter: @TonW #DDTT
  • 20. Get your statistics right! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 21. 1: low significance levels ü Big chance the winner is not a real winner Email: ton@testing.agency Twitter: @TonW #DDTT
  • 22. 2: low power levels ü Big chance a real winner is not recognized Email: ton@testing.agency Twitter: @TonW #DDTT
  • 23. Example test set-up ü 90% significance level ü 100 conversions per variation ü Average conversion rate: 2% ü 20 out of 100 ideas is a winner ü With an average uplift of 10% Email: ton@testing.agency Twitter: @TonW #DDTT
  • 24. Average: 75% significance & 40% power Email: ton@testing.agency Twitter: @TonW #DDTT
  • 25. So: ü 100 tests will give 10 false positives (90% significance) ü 40% of the 20 winners are recognized (40% power) ü 8 real winners & 10 false positives ü NOT 18 improvements of 15%+ Email: ton@testing.agency Twitter: @TonW #DDTT
  • 26. 3: Not calculating traffic up front ü You thought the page had thousands of visitors Email: ton@testing.agency Twitter: @TonW #DDTT
  • 27. 4: Not calculating what you can improve ü You thought the page had lots and lots conversions Email: ton@testing.agency Twitter: @TonW #DDTT
  • 29. 5: Not choosing your segments up front ü You just go and dig unitil you find something (you will always find something that seems true BUT is not!) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 30. 6: You stop the test once it is significant Email: ton@testing.agency Twitter: @TonW #DDTT
  • 31. 6: You stop the test once it is significant Email: ton@testing.agency Twitter: @TonW #DDTT
  • 32. 7: You just let it run until it’s significant Email: ton@testing.agency Twitter: @TonW #DDTT
  • 33. 7: You just let it run until it’s significant Your test should have a fixed lenght! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 34. 8: Your test takes too long ü Cookie deletion (you will loose 10% in two weeks) They will re-enter the the and pollute your samples! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 35. 9: You just choose a test length ü At least 1 full purchase cycle! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 36. 10: You don’t test full weeks ü At least 1 full week (if your not testing for time of day / week effects) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 37. 11: You think they always use one decive ü Only test 1 device at a time! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 38. 12: You’re testing broken stuff ü Always check browser compatibility! ü Dont’ break dynamic stuff! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 39. 13: You have multiple tests that overlap ü Your variation did not won (it was a combination of variations) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 40. 14: You thought this page matters ü But it did not (so you decide to use some other metric to declare the winner) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 41. 15: You forgot to leave out some visitors ü Only test on those who can be changed (if you test a sales promotion – leave out your current clients who just logon) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 42. 16: You’re not slowing the control down ü Add the same code to the control Email: ton@testing.agency Twitter: @TonW #DDTT
  • 43. 17: You’re not testing on fresh people ü Start with control for a full purchase cycle (and then start sending traffic to your variations – setting the control to 0%) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 44. 18: You forgot to give them time to buy ü Stop getting fresh visitors after your test time (send new traffic to the control, but give you tested people time to finish converting) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 45. 19: You’re not diving into your analytics ü Always measure with analytics software (You will find conversions that are not supposed to be there!) Email: ton@testing.agency Twitter: @TonW #DDTT
  • 46. 20: You’re not measuring the right stuff ü Use unique visitors ü Against lifetime value predictors! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 47. If you keep on making mistakes: Stuff like this happens Email: ton@testing.agency Twitter: @TonW #DDTT
  • 48. Testing is not free Use it to LEARN Email: ton@testing.agency Twitter: @TonW #DDTT
  • 49. Bonus: You did not re-test your winners ü Always re-test to be sure it was not a glitch Email: ton@testing.agency Twitter: @TonW #DDTT
  • 50. So when you implement winners This will happen! Email: ton@testing.agency Twitter: @TonW #DDTT
  • 51. < H a v e m o r e f u n A / B - t e s t i n g > A/B-t e s t i n g Errors To n We s s e l i n g – Te s t Wi n n e r #DDTT A m s t e r d a m 2 0 1 4 / 0 6 / 2 4