SlideShare a Scribd company logo

[WMD 2015] ConversionXL >> Peep Laja, "WARNING: 13 Ways You're Screwing Up Your A/B Tests"

•Download as PPTX, PDF•
•
500 Startups
500 Startups

Peep Laja, Founder, ConversionXL

Technology
1 of 28
13 Ways You're Screwing Up Your A/B Tests by Peep Laja
Peep Laja, ConversionXL @peeplaja Typical scenario: A company runs 100 tests / year. After 12 months their conversion rate is still the same.
Testing mistake #1: Precious time wasted on stupid tests Peep Laja, ConversionXL @peeplaja
Testing mistake #2: You don’t have a data- driven, learning- oriented hypothesis Peep Laja, ConversionXL @peeplaja
Learn more: http://conversionxl.com/how-to-come-up-with-more- winning-tests-using-data/ Peep Laja, ConversionXL @peeplaja
Testing mistake #3: You think you know what will work Peep Laja, ConversionXL @peeplaja
Testing mistake #4: Your absolute conversions are way too low Peep Laja, ConversionXL @peeplaja
Peep Laja, ConversionXL @peeplaja
Calculate needed sample size in advance: Peep Laja, ConversionXL @peeplaja
I won’t even look at results until there’s at least 250-350 conversions PER variation. Way more if you want to segment the data Peep Laja, ConversionXL @peeplaja
Testing mistake #5: You run tests when you have no traffic Peep Laja, ConversionXL @peeplaja
Learn more: http://conversionxl.com/how-to-do-conversion- optimization-with-very-little-traffic/ Peep Laja, ConversionXL @peeplaja
Testing mistake #6: Your tests don’t run long enough Peep Laja, ConversionXL @peeplaja
Peep Laja, ConversionXL @peeplaja
Testing mistake #7: You don’t test full weeks at a time Peep Laja, ConversionXL @peeplaja
Peep Laja, ConversionXL @peeplaja
Testing mistake #8: Test data is not sent to third-party analytics Peep Laja, ConversionXL @peeplaja
Peep Laja, ConversionXL @peeplaja
Testing mistake #9: You give up after your first test for a hypothesis fails. Peep Laja, ConversionXL @peeplaja
Peep Laja, ConversionXL @peeplaja
Full case study: http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/ 79.3% uplift in conversions Peep Laja, ConversionXL @peeplaja
Testing mistake #10: You’re not aware of validity threats Peep Laja, ConversionXL @peeplaja
History effect Instrumentation effect Selection effect Peep Laja, ConversionXL @peeplaja
Testing mistake #11: You’re ignoring small gains Peep Laja, ConversionXL @peeplaja
5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how math works. Peep Laja, ConversionXL @peeplaja
Testing mistake #12: You copy competitors / leaders / other people’s tests Peep Laja, ConversionXL @peeplaja
Testing mistake #13: You’re not running tests at all times Peep Laja, ConversionXL @peeplaja
Follow me @peeplaja Thank you! Join 150,000+ monthly readers to learn from the best conversion optimization blog on the planet: Peep Laja

Recommended

Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.Martijn Scheijbeler
 
Spreadsheet Errors
Spreadsheet ErrorsSpreadsheet Errors
Spreadsheet Errorsguestcdc19eb
 
SEO Tips to Improve Site’s Google Ranking & Avoid Penalty
SEO Tips to Improve Site’s Google Ranking & Avoid PenaltySEO Tips to Improve Site’s Google Ranking & Avoid Penalty
SEO Tips to Improve Site’s Google Ranking & Avoid PenaltyEminenture
 
[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...
[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...
[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...500 Startups
 
How to Execute and A/B Test Cross-Channel Personalization
How to Execute and A/B Test Cross-Channel PersonalizationHow to Execute and A/B Test Cross-Channel Personalization
How to Execute and A/B Test Cross-Channel PersonalizationCXL
 
Startup Metrics for Pirates (Nov 2012)
Startup Metrics for Pirates (Nov 2012)Startup Metrics for Pirates (Nov 2012)
Startup Metrics for Pirates (Nov 2012)Dave McClure
 
Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...
Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...
Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...Sean Ellis
 
500’s Demo Day Batch 15 >> Metadata
500’s Demo Day Batch 15 >> Metadata500’s Demo Day Batch 15 >> Metadata
500’s Demo Day Batch 15 >> Metadata500 Startups
 

Recommended

Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.
Bad Experiments: The #18 Ways You’re A/B Tests are Going Wrong.Martijn Scheijbeler
 
Spreadsheet Errors
Spreadsheet ErrorsSpreadsheet Errors
Spreadsheet Errorsguestcdc19eb
 
SEO Tips to Improve Site’s Google Ranking & Avoid Penalty
SEO Tips to Improve Site’s Google Ranking & Avoid PenaltySEO Tips to Improve Site’s Google Ranking & Avoid Penalty
SEO Tips to Improve Site’s Google Ranking & Avoid PenaltyEminenture
 
[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...
[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...
[WMD 2015] Maven Ventures >> Jim Scheinman, "Science Meets Art: How To Build ...500 Startups
 
How to Execute and A/B Test Cross-Channel Personalization
How to Execute and A/B Test Cross-Channel PersonalizationHow to Execute and A/B Test Cross-Channel Personalization
How to Execute and A/B Test Cross-Channel PersonalizationCXL
 
Startup Metrics for Pirates (Nov 2012)
Startup Metrics for Pirates (Nov 2012)Startup Metrics for Pirates (Nov 2012)
Startup Metrics for Pirates (Nov 2012)Dave McClure
 
Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...
Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...
Scaling Authentic Growth - How Product and Marketing Work Together to Drive G...Sean Ellis
 
500’s Demo Day Batch 15 >> Metadata
500’s Demo Day Batch 15 >> Metadata500’s Demo Day Batch 15 >> Metadata
500’s Demo Day Batch 15 >> Metadata500 Startups
 
500’s Demo Day Batch 16 >> Qualio
500’s Demo Day Batch 16 >> Qualio500’s Demo Day Batch 16 >> Qualio
500’s Demo Day Batch 16 >> Qualio500 Startups
 
500’s Demo Day Batch 13 >> Storygami
500’s Demo Day Batch 13 >> Storygami 500’s Demo Day Batch 13 >> Storygami
500’s Demo Day Batch 13 >> Storygami 500 Startups
 
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013500 Startups
 
Welcu deckpublic
Welcu deckpublicWelcu deckpublic
Welcu deckpublic500 Startups
 
500’s Demo Day: Post-Seed >> Totspot
500’s Demo Day: Post-Seed >> Totspot500’s Demo Day: Post-Seed >> Totspot
500’s Demo Day: Post-Seed >> Totspot500 Startups
 
500’s Demo Day Batch 17 >> Wivity
500’s Demo Day Batch 17 >> Wivity500’s Demo Day Batch 17 >> Wivity
500’s Demo Day Batch 17 >> Wivity500 Startups
 
500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500 Startups
 
unSEXY Conf 2013: Jamie Sutherland, Xero
unSEXY Conf 2013: Jamie Sutherland, Xero unSEXY Conf 2013: Jamie Sutherland, Xero
unSEXY Conf 2013: Jamie Sutherland, Xero 500 Startups
 
Janice Fraser, LUXr presentation at Lean Startup SXSW
Janice Fraser, LUXr presentation at Lean Startup SXSWJanice Fraser, LUXr presentation at Lean Startup SXSW
Janice Fraser, LUXr presentation at Lean Startup SXSW500 Startups
 
500’s Demo Day: Post-Seed >> ActivityHero
500’s Demo Day: Post-Seed >> ActivityHero500’s Demo Day: Post-Seed >> ActivityHero
500’s Demo Day: Post-Seed >> ActivityHero500 Startups
 
500’s Demo Day Batch 17 >> Avocode
500’s Demo Day Batch 17 >> Avocode500’s Demo Day Batch 17 >> Avocode
500’s Demo Day Batch 17 >> Avocode500 Startups
 
Scott Chacon, GitHub, SXSW Lean Startup 2013
Scott Chacon, GitHub, SXSW Lean Startup 2013Scott Chacon, GitHub, SXSW Lean Startup 2013
Scott Chacon, GitHub, SXSW Lean Startup 2013500 Startups
 
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013500 Startups
 
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20500 Startups
 
500’s Demo Day Batch 11 >> Connected2me
500’s Demo Day Batch 11 >> Connected2me 500’s Demo Day Batch 11 >> Connected2me
500’s Demo Day Batch 11 >> Connected2me 500 Startups
 
TinfoilSecurity
TinfoilSecurityTinfoilSecurity
TinfoilSecurity500 Startups
 
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013500 Startups
 
500’s Demo Day Batch 13 >> Edools
500’s Demo Day Batch 13 >> Edools500’s Demo Day Batch 13 >> Edools
500’s Demo Day Batch 13 >> Edools500 Startups
 
Adwords Secrets and Strategies for Startups
Adwords Secrets and Strategies for StartupsAdwords Secrets and Strategies for Startups
Adwords Secrets and Strategies for Startups500 Startups
 
East Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
East Asia Geeks On A Plane >> Shenzhen Ecosystem OverviewEast Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
East Asia Geeks On A Plane >> Shenzhen Ecosystem Overview500 Startups
 
Validation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-KhalidiValidation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-KhalidiAbdallah Al-Khalidi
 
mm13 keynote presentation | Peep Laja
mm13 keynote presentation | Peep Lajamm13 keynote presentation | Peep Laja
mm13 keynote presentation | Peep Lajamesh group
 

More Related Content

Viewers also liked

500’s Demo Day Batch 16 >> Qualio
500’s Demo Day Batch 16 >> Qualio500’s Demo Day Batch 16 >> Qualio
500’s Demo Day Batch 16 >> Qualio500 Startups
 
500’s Demo Day Batch 13 >> Storygami
500’s Demo Day Batch 13 >> Storygami 500’s Demo Day Batch 13 >> Storygami
500’s Demo Day Batch 13 >> Storygami 500 Startups
 
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013500 Startups
 
500’s Demo Day: Post-Seed >> Totspot
500’s Demo Day: Post-Seed >> Totspot500’s Demo Day: Post-Seed >> Totspot
500’s Demo Day: Post-Seed >> Totspot500 Startups
 
500’s Demo Day Batch 17 >> Wivity
500’s Demo Day Batch 17 >> Wivity500’s Demo Day Batch 17 >> Wivity
500’s Demo Day Batch 17 >> Wivity500 Startups
 
500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500 Startups
 
unSEXY Conf 2013: Jamie Sutherland, Xero
unSEXY Conf 2013: Jamie Sutherland, Xero unSEXY Conf 2013: Jamie Sutherland, Xero
unSEXY Conf 2013: Jamie Sutherland, Xero 500 Startups
 
Janice Fraser, LUXr presentation at Lean Startup SXSW
Janice Fraser, LUXr presentation at Lean Startup SXSWJanice Fraser, LUXr presentation at Lean Startup SXSW
Janice Fraser, LUXr presentation at Lean Startup SXSW500 Startups
 
500’s Demo Day: Post-Seed >> ActivityHero
500’s Demo Day: Post-Seed >> ActivityHero500’s Demo Day: Post-Seed >> ActivityHero
500’s Demo Day: Post-Seed >> ActivityHero500 Startups
 
500’s Demo Day Batch 17 >> Avocode
500’s Demo Day Batch 17 >> Avocode500’s Demo Day Batch 17 >> Avocode
500’s Demo Day Batch 17 >> Avocode500 Startups
 
Scott Chacon, GitHub, SXSW Lean Startup 2013
Scott Chacon, GitHub, SXSW Lean Startup 2013Scott Chacon, GitHub, SXSW Lean Startup 2013
Scott Chacon, GitHub, SXSW Lean Startup 2013500 Startups
 
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013500 Startups
 
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20500 Startups
 
500’s Demo Day Batch 11 >> Connected2me
500’s Demo Day Batch 11 >> Connected2me 500’s Demo Day Batch 11 >> Connected2me
500’s Demo Day Batch 11 >> Connected2me 500 Startups
 
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013500 Startups
 
500’s Demo Day Batch 13 >> Edools
500’s Demo Day Batch 13 >> Edools500’s Demo Day Batch 13 >> Edools
500’s Demo Day Batch 13 >> Edools500 Startups
 
Adwords Secrets and Strategies for Startups
Adwords Secrets and Strategies for StartupsAdwords Secrets and Strategies for Startups
Adwords Secrets and Strategies for Startups500 Startups
 
East Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
East Asia Geeks On A Plane >> Shenzhen Ecosystem OverviewEast Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
East Asia Geeks On A Plane >> Shenzhen Ecosystem Overview500 Startups
 

Viewers also liked (20)

500’s Demo Day Batch 16 >> Qualio
500’s Demo Day Batch 16 >> Qualio500’s Demo Day Batch 16 >> Qualio
500’s Demo Day Batch 16 >> Qualio
 
500’s Demo Day Batch 13 >> Storygami
500’s Demo Day Batch 13 >> Storygami 500’s Demo Day Batch 13 >> Storygami
500’s Demo Day Batch 13 >> Storygami
 
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
Jake Johnson, Divshot – Startup Pitch, WarmGun 2013
 
Welcu deckpublic
Welcu deckpublicWelcu deckpublic
Welcu deckpublic
 
500’s Demo Day: Post-Seed >> Totspot
500’s Demo Day: Post-Seed >> Totspot500’s Demo Day: Post-Seed >> Totspot
500’s Demo Day: Post-Seed >> Totspot
 
500’s Demo Day Batch 17 >> Wivity
500’s Demo Day Batch 17 >> Wivity500’s Demo Day Batch 17 >> Wivity
500’s Demo Day Batch 17 >> Wivity
 
500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy500’s Demo Day Batch 11 >> CloudAcademy
500’s Demo Day Batch 11 >> CloudAcademy
 
unSEXY Conf 2013: Jamie Sutherland, Xero
unSEXY Conf 2013: Jamie Sutherland, Xero unSEXY Conf 2013: Jamie Sutherland, Xero
unSEXY Conf 2013: Jamie Sutherland, Xero
 
Janice Fraser, LUXr presentation at Lean Startup SXSW
Janice Fraser, LUXr presentation at Lean Startup SXSWJanice Fraser, LUXr presentation at Lean Startup SXSW
Janice Fraser, LUXr presentation at Lean Startup SXSW
 
500’s Demo Day: Post-Seed >> ActivityHero
500’s Demo Day: Post-Seed >> ActivityHero500’s Demo Day: Post-Seed >> ActivityHero
500’s Demo Day: Post-Seed >> ActivityHero
 
500’s Demo Day Batch 17 >> Avocode
500’s Demo Day Batch 17 >> Avocode500’s Demo Day Batch 17 >> Avocode
500’s Demo Day Batch 17 >> Avocode
 
Scott Chacon, GitHub, SXSW Lean Startup 2013
Scott Chacon, GitHub, SXSW Lean Startup 2013Scott Chacon, GitHub, SXSW Lean Startup 2013
Scott Chacon, GitHub, SXSW Lean Startup 2013
 
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
Neven Zeremski, Dubb – Startup Pitch, WarmGun 2013
 
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
Mike Greenfield w/ Circle of Moms @ MamaBear Conference, Mt. View 4/20
 
500’s Demo Day Batch 11 >> Connected2me
500’s Demo Day Batch 11 >> Connected2me 500’s Demo Day Batch 11 >> Connected2me
500’s Demo Day Batch 11 >> Connected2me
 
TinfoilSecurity
TinfoilSecurityTinfoilSecurity
TinfoilSecurity
 
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
Rohey Livne, Notelr – Startup Pitch, WarmGun 2013
 
500’s Demo Day Batch 13 >> Edools
500’s Demo Day Batch 13 >> Edools500’s Demo Day Batch 13 >> Edools
500’s Demo Day Batch 13 >> Edools
 
Adwords Secrets and Strategies for Startups
Adwords Secrets and Strategies for StartupsAdwords Secrets and Strategies for Startups
Adwords Secrets and Strategies for Startups
 
East Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
East Asia Geeks On A Plane >> Shenzhen Ecosystem OverviewEast Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
East Asia Geeks On A Plane >> Shenzhen Ecosystem Overview
 

Similar to [WMD 2015] ConversionXL >> Peep Laja, "WARNING: 13 Ways You're Screwing Up Your A/B Tests"

Validation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-KhalidiValidation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-KhalidiAbdallah Al-Khalidi
 
mm13 keynote presentation | Peep Laja
mm13 keynote presentation | Peep Lajamm13 keynote presentation | Peep Laja
mm13 keynote presentation | Peep Lajamesh group
 
Andreas Reiffen - SMX London Slidedeck
Andreas Reiffen - SMX London SlidedeckAndreas Reiffen - SMX London Slidedeck
Andreas Reiffen - SMX London SlidedeckCrealytics
 
Landing Page Testing To Increase Conversions
Landing Page Testing To Increase ConversionsLanding Page Testing To Increase Conversions
Landing Page Testing To Increase ConversionsAffiliate Summit
 
The 11 Biggest A/B Testing Mistakes and How To Avoid Them
The 11 Biggest A/B Testing Mistakes and How To Avoid ThemThe 11 Biggest A/B Testing Mistakes and How To Avoid Them
The 11 Biggest A/B Testing Mistakes and How To Avoid ThemHanapin Marketing
 
Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...
Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...
Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...PMX Agency
 
10 things to do and check in Google Analytics for Construction Companies
10 things to do and check in Google Analytics for Construction Companies10 things to do and check in Google Analytics for Construction Companies
10 things to do and check in Google Analytics for Construction Companiespauleycreative
 
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...CXL
 
How to build an effective conversion program on a budget
How to build an effective conversion program on a budget How to build an effective conversion program on a budget
How to build an effective conversion program on a budget Joni Lindgren
 
Milion Dollar Impact Through Metrics, Analytics & A/B Testing
Milion Dollar Impact Through Metrics, Analytics & A/B TestingMilion Dollar Impact Through Metrics, Analytics & A/B Testing
Milion Dollar Impact Through Metrics, Analytics & A/B TestingAzhar Bandeali
 
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingSAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingZack Notes
 
How to ab test yourself to a new website webinar version
How to ab test yourself to a new website webinar versionHow to ab test yourself to a new website webinar version
How to ab test yourself to a new website webinar versionEffin Amazing
 
Landing Page Testing to Attract Super Affiliates
Landing Page Testing to Attract Super AffiliatesLanding Page Testing to Attract Super Affiliates
Landing Page Testing to Attract Super AffiliatesAffiliate Summit
 
Test Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with ExperimentationTest Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with ExperimentationOptimizely
 
Webinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B TestingWebinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B TestingOptimizely
 
10NTC - Data Superheroes - DiJulio
10NTC - Data Superheroes - DiJulio10NTC - Data Superheroes - DiJulio
10NTC - Data Superheroes - DiJuliosarahdijulio
 
A/B Testing Blueprint | Pirate Skills
A/B Testing Blueprint | Pirate SkillsA/B Testing Blueprint | Pirate Skills
A/B Testing Blueprint | Pirate SkillsPirate Skills
 
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Future Insights
 
Growing Your Business Through Experimentation
Growing Your Business Through ExperimentationGrowing Your Business Through Experimentation
Growing Your Business Through ExperimentationHiten Shah
 

Similar to [WMD 2015] ConversionXL >> Peep Laja, "WARNING: 13 Ways You're Screwing Up Your A/B Tests" (20)

Validation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-KhalidiValidation and hypothesis based product management by Abdallah Al-Khalidi
Validation and hypothesis based product management by Abdallah Al-Khalidi
 
mm13 keynote presentation | Peep Laja
mm13 keynote presentation | Peep Lajamm13 keynote presentation | Peep Laja
mm13 keynote presentation | Peep Laja
 
Andreas Reiffen - SMX London Slidedeck
Andreas Reiffen - SMX London SlidedeckAndreas Reiffen - SMX London Slidedeck
Andreas Reiffen - SMX London Slidedeck
 
SPC Presentation Master
SPC Presentation MasterSPC Presentation Master
SPC Presentation Master
 
Landing Page Testing To Increase Conversions
Landing Page Testing To Increase ConversionsLanding Page Testing To Increase Conversions
Landing Page Testing To Increase Conversions
 
The 11 Biggest A/B Testing Mistakes and How To Avoid Them
The 11 Biggest A/B Testing Mistakes and How To Avoid ThemThe 11 Biggest A/B Testing Mistakes and How To Avoid Them
The 11 Biggest A/B Testing Mistakes and How To Avoid Them
 
Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...
Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...
Testing for Success: How to Infuse Consistent Testing Into Your Fundraising P...
 
10 things to do and check in Google Analytics for Construction Companies
10 things to do and check in Google Analytics for Construction Companies10 things to do and check in Google Analytics for Construction Companies
10 things to do and check in Google Analytics for Construction Companies
 
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...
[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...
 
How to build an effective conversion program on a budget
How to build an effective conversion program on a budget How to build an effective conversion program on a budget
How to build an effective conversion program on a budget
 
Milion Dollar Impact Through Metrics, Analytics & A/B Testing
Milion Dollar Impact Through Metrics, Analytics & A/B TestingMilion Dollar Impact Through Metrics, Analytics & A/B Testing
Milion Dollar Impact Through Metrics, Analytics & A/B Testing
 
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not makingSAMPLE SIZE – The indispensable A/B test calculation that you’re not making
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
 
How to ab test yourself to a new website webinar version
How to ab test yourself to a new website webinar versionHow to ab test yourself to a new website webinar version
How to ab test yourself to a new website webinar version
 
Landing Page Testing to Attract Super Affiliates
Landing Page Testing to Attract Super AffiliatesLanding Page Testing to Attract Super Affiliates
Landing Page Testing to Attract Super Affiliates
 
Test Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with ExperimentationTest Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with Experimentation
 
Webinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B TestingWebinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B Testing
 
10NTC - Data Superheroes - DiJulio
10NTC - Data Superheroes - DiJulio10NTC - Data Superheroes - DiJulio
10NTC - Data Superheroes - DiJulio
 
A/B Testing Blueprint | Pirate Skills
A/B Testing Blueprint | Pirate SkillsA/B Testing Blueprint | Pirate Skills
A/B Testing Blueprint | Pirate Skills
 
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
 
Growing Your Business Through Experimentation
Growing Your Business Through ExperimentationGrowing Your Business Through Experimentation
Growing Your Business Through Experimentation
 

More from 500 Startups

Connected Analytics
Connected AnalyticsConnected Analytics
Connected Analytics500 Startups
 

More from 500 Startups (20)

Get on Board
Get on BoardGet on Board
Get on Board
 
Connected Analytics
Connected AnalyticsConnected Analytics
Connected Analytics
 
Sira Medical
Sira MedicalSira Medical
Sira Medical
 
The Atlas
The AtlasThe Atlas
The Atlas
 
Trash Warrior
Trash WarriorTrash Warrior
Trash Warrior
 
Thematic
ThematicThematic
Thematic
 
Shiplyst
ShiplystShiplyst
Shiplyst
 
Renetec
RenetecRenetec
Renetec
 
Predina
PredinaPredina
Predina
 
Pluto
PlutoPluto
Pluto
 
Plant an App
Plant an AppPlant an App
Plant an App
 
Pilota
PilotaPilota
Pilota
 
Mero Technologies
Mero TechnologiesMero Technologies
Mero Technologies
 
Omnitron Sensors
Omnitron SensorsOmnitron Sensors
Omnitron Sensors
 
Juked
JukedJuked
Juked
 
GamerzClass
GamerzClassGamerzClass
GamerzClass
 
eino
einoeino
eino
 
Cenos
CenosCenos
Cenos
 
Bliinx
BliinxBliinx
Bliinx
 
Butlr
ButlrButlr
Butlr
 

Recently uploaded

DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...caitlingebhard1
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingWSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMECloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization - The evolution of API governanceAPI Governance and Monetization - The evolution of API governance
API Governance and Monetization - The evolution of API governanceWSO2
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceIES VE
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....rightmanforbloodline
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdfSandro Moreira
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Bhuvaneswari Subramani
 

Recently uploaded (20)

DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Quantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation ComputingQuantum Leap in Next-Generation Computing
Quantum Leap in Next-Generation Computing
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMECloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization - The evolution of API governanceAPI Governance and Monetization - The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational Performance
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 

[WMD 2015] ConversionXL >> Peep Laja, "WARNING: 13 Ways You're Screwing Up Your A/B Tests"

Editor's Notes

  1. TO BE PRESENTED IN GOOGLE DOCS- Press PRESENT in left top corner.
  2. Use the bottom area as breadcrumbs - so you don’t need to use headings on every page
  3. Use a line border for the images.
  4. Use a line border for the images.
  5. Use a line border for the images.
  6. Use a line border for the images.
  7. Use the bottom area as breadcrumbs - so you don’t need to use headings on every page
  8. Use the bottom area as breadcrumbs - so you don’t need to use headings on every page
  9. Use a line border for the images.
  10. Use a line border for the images.
  11. Use a line border for the images.
  12. Use a line border for the images.
  13. Use a line border for the images.
  14. Use a line border for the images.
  15. Use the bottom area as breadcrumbs - so you don’t need to use headings on every page
  16. Use a line border for the images.
  17. Use the bottom area as breadcrumbs - so you don’t need to use headings on every page
  18. Use a line border for the images.
  19. Use a line border for the images.
  20. Use the bottom area as breadcrumbs - so you don’t need to use headings on every page