SlideShare a Scribd company logo

Lessons from SEO split-testing

Computers might already be better at your job than you are. Are you ready to partner with them to keep your job? This presentation shows you how hard our jobs have become, gives you the results of some of our testing, and outlines a plan to keep our jobs.

1 of 98
Download to read offline
Lessons from SEO split-testing
Including evidence that we don’t know what we’re talking about
@willcritchlow
We used to have a pretty good
understanding of ranking factors
My mental model for ~2009 ranking factors had
three different modes:
One in the
hyper-competitive
head
My mental model for ~2009 ranking factors had
three different modes:
One in the
competitive
mid-tail
...and
one
in
the
long-tail
One in the
hyper-competitive
head
Tons of perfectly on-topic
pages to choose from
One in the
hyper-competitive
head

Recommended

SearchLeeds 2017 - Paul Madden - Co-Founder, Kerboo - Links as a metric in 20...
SearchLeeds 2017 - Paul Madden - Co-Founder, Kerboo - Links as a metric in 20...SearchLeeds 2017 - Paul Madden - Co-Founder, Kerboo - Links as a metric in 20...
SearchLeeds 2017 - Paul Madden - Co-Founder, Kerboo - Links as a metric in 20...Branded3
 
SearchLeeds 2017 - Head of Global Marketing - Olga Andrienko - SEMrush - SEMr...
SearchLeeds 2017 - Head of Global Marketing - Olga Andrienko - SEMrush - SEMr...SearchLeeds 2017 - Head of Global Marketing - Olga Andrienko - SEMrush - SEMr...
SearchLeeds 2017 - Head of Global Marketing - Olga Andrienko - SEMrush - SEMr...Branded3
 
Sample SEO presentation for clients
Sample SEO presentation for clientsSample SEO presentation for clients
Sample SEO presentation for clientsSiddu Hosageri
 
Affordable SEO - On Page SEO, The First Step Of GOOGLE Or BING Ranking
Affordable SEO - On Page SEO, The First Step Of GOOGLE Or BING RankingAffordable SEO - On Page SEO, The First Step Of GOOGLE Or BING Ranking
Affordable SEO - On Page SEO, The First Step Of GOOGLE Or BING RankingMS HOQUE
 
The Future of Visual Search by Gianluca Fiorelli for Brighton SEO
The Future of Visual Search by Gianluca Fiorelli for Brighton SEOThe Future of Visual Search by Gianluca Fiorelli for Brighton SEO
The Future of Visual Search by Gianluca Fiorelli for Brighton SEOGianluca Fiorelli
 
SEO for Ecommerce: A Comprehensive Guide
SEO for Ecommerce: A Comprehensive GuideSEO for Ecommerce: A Comprehensive Guide
SEO for Ecommerce: A Comprehensive GuideAdam Audette
 
Let Me Tell You of The Days of HIgh Adventure
Let Me Tell You of The Days of HIgh AdventureLet Me Tell You of The Days of HIgh Adventure
Let Me Tell You of The Days of HIgh AdventureDFWSEM
 
SEO 2020: On-SERP SEO & Beyond
SEO 2020: On-SERP SEO & BeyondSEO 2020: On-SERP SEO & Beyond
SEO 2020: On-SERP SEO & BeyondDiane Kulseth
 

More Related Content

What's hot

Unpacking the Local Search Ranking Factors for 2021 & Beyond
Unpacking the Local Search Ranking Factors for 2021 & BeyondUnpacking the Local Search Ranking Factors for 2021 & Beyond
Unpacking the Local Search Ranking Factors for 2021 & BeyondSeven Figure Agency
 
The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...
The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...
The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...Paul Shapiro
 
SEO 101 - Tips and Tactics for 2014
SEO 101 - Tips and Tactics for 2014SEO 101 - Tips and Tactics for 2014
SEO 101 - Tips and Tactics for 2014Prosperity Media
 
Talking Local SERP Market Share: Mixing Paid, Local, and Organic
Talking Local SERP Market Share: Mixing Paid, Local, and OrganicTalking Local SERP Market Share: Mixing Paid, Local, and Organic
Talking Local SERP Market Share: Mixing Paid, Local, and OrganicDFWSEM
 
Content Marketing Success by Design
Content Marketing Success by DesignContent Marketing Success by Design
Content Marketing Success by DesignEric Enge
 
Introduction to SEO
Introduction to SEOIntroduction to SEO
Introduction to SEORand Fishkin
 
Teaching seo in a classroom
Teaching seo in a classroomTeaching seo in a classroom
Teaching seo in a classroomstephencroome
 
The State of Search
The State of SearchThe State of Search
The State of SearchDFWSEM
 
Reverse Engineering Google's Local Search Algorithm
Reverse Engineering Google's Local Search AlgorithmReverse Engineering Google's Local Search Algorithm
Reverse Engineering Google's Local Search AlgorithmDFWSEM
 
SEO Checklist 2018 - Ranking in the first page of SERP organically.
SEO Checklist 2018 - Ranking in the first page of SERP organically.SEO Checklist 2018 - Ranking in the first page of SERP organically.
SEO Checklist 2018 - Ranking in the first page of SERP organically.AVIK BAL
 
Schema in 2016 - A New Frontier - Carrie Hill - Pubcon
Schema in 2016 - A New Frontier - Carrie Hill - PubconSchema in 2016 - A New Frontier - Carrie Hill - Pubcon
Schema in 2016 - A New Frontier - Carrie Hill - PubconCarrie Hill
 
In-Depth with Local SEO
In-Depth with Local SEOIn-Depth with Local SEO
In-Depth with Local SEORand Fishkin
 
14 Steps to Successful SEO
14 Steps to Successful SEO14 Steps to Successful SEO
14 Steps to Successful SEORyan Spoon
 
Killing Giants - How to compete with Big Brands in the SERPS
Killing Giants - How to compete with Big Brands in the SERPSKilling Giants - How to compete with Big Brands in the SERPS
Killing Giants - How to compete with Big Brands in the SERPSRoss Tavendale
 

What's hot (20)

Unpacking the Local Search Ranking Factors for 2021 & Beyond
Unpacking the Local Search Ranking Factors for 2021 & BeyondUnpacking the Local Search Ranking Factors for 2021 & Beyond
Unpacking the Local Search Ranking Factors for 2021 & Beyond
 
The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...
The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...
The Actionable Guide to Doing Better Semantic Keyword Research #BrightonSEO (...
 
SEO Training Slides October 2016
SEO Training Slides October 2016SEO Training Slides October 2016
SEO Training Slides October 2016
 
SEO 101 - Tips and Tactics for 2014
SEO 101 - Tips and Tactics for 2014SEO 101 - Tips and Tactics for 2014
SEO 101 - Tips and Tactics for 2014
 
Talking Local SERP Market Share: Mixing Paid, Local, and Organic
Talking Local SERP Market Share: Mixing Paid, Local, and OrganicTalking Local SERP Market Share: Mixing Paid, Local, and Organic
Talking Local SERP Market Share: Mixing Paid, Local, and Organic
 
Content Marketing Success by Design
Content Marketing Success by DesignContent Marketing Success by Design
Content Marketing Success by Design
 
Introduction to SEO
Introduction to SEOIntroduction to SEO
Introduction to SEO
 
Teaching seo in a classroom
Teaching seo in a classroomTeaching seo in a classroom
Teaching seo in a classroom
 
The State of Search
The State of SearchThe State of Search
The State of Search
 
Reverse Engineering Google's Local Search Algorithm
Reverse Engineering Google's Local Search AlgorithmReverse Engineering Google's Local Search Algorithm
Reverse Engineering Google's Local Search Algorithm
 
SEO Checklist 2018 - Ranking in the first page of SERP organically.
SEO Checklist 2018 - Ranking in the first page of SERP organically.SEO Checklist 2018 - Ranking in the first page of SERP organically.
SEO Checklist 2018 - Ranking in the first page of SERP organically.
 
Schema in 2016 - A New Frontier - Carrie Hill - Pubcon
Schema in 2016 - A New Frontier - Carrie Hill - PubconSchema in 2016 - A New Frontier - Carrie Hill - Pubcon
Schema in 2016 - A New Frontier - Carrie Hill - Pubcon
 
In-Depth with Local SEO
In-Depth with Local SEOIn-Depth with Local SEO
In-Depth with Local SEO
 
basic Seo ppt
basic Seo pptbasic Seo ppt
basic Seo ppt
 
SEO Basics and More 2013
SEO Basics and More 2013SEO Basics and More 2013
SEO Basics and More 2013
 
SEO Presentation
SEO PresentationSEO Presentation
SEO Presentation
 
Introduction SEO
Introduction SEOIntroduction SEO
Introduction SEO
 
Seo Success in 2020
Seo Success in 2020Seo Success in 2020
Seo Success in 2020
 
14 Steps to Successful SEO
14 Steps to Successful SEO14 Steps to Successful SEO
14 Steps to Successful SEO
 
Killing Giants - How to compete with Big Brands in the SERPS
Killing Giants - How to compete with Big Brands in the SERPSKilling Giants - How to compete with Big Brands in the SERPS
Killing Giants - How to compete with Big Brands in the SERPS
 

Similar to Lessons from SEO split-testing

Amazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOAmazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOWill Critchlow
 
Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....
Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....
Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....Matthew Woodward
 
The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...
The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...
The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...James Brockbank
 
Essay On Search Engine Optimization Strategies
Essay On Search Engine Optimization StrategiesEssay On Search Engine Optimization Strategies
Essay On Search Engine Optimization StrategiesPay For A Paper Singapore
 
Stephan Spencer - SEMPO Atlanta. October 1, 2010. Topic: Advanced SEO
Stephan Spencer - SEMPO Atlanta.  October 1, 2010.  Topic: Advanced SEOStephan Spencer - SEMPO Atlanta.  October 1, 2010.  Topic: Advanced SEO
Stephan Spencer - SEMPO Atlanta. October 1, 2010. Topic: Advanced SEOAllison Fabella
 
SearchCon 2016 | High Velocity Presentations
SearchCon 2016 | High Velocity PresentationsSearchCon 2016 | High Velocity Presentations
SearchCon 2016 | High Velocity PresentationsSearchCon
 
Search Engine Optimization
Search Engine OptimizationSearch Engine Optimization
Search Engine OptimizationTheju Paul
 
Split Testing for SEO - 9 Months of Learning
Split Testing for SEO - 9 Months of LearningSplit Testing for SEO - 9 Months of Learning
Split Testing for SEO - 9 Months of LearningDominic Woodman
 
C-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO StrategiesC-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO StrategiesIzzi Smith
 
SEO & CRO - Acquiring & Engaging Customers Through Digital Marketing
SEO & CRO - Acquiring & Engaging Customers Through Digital MarketingSEO & CRO - Acquiring & Engaging Customers Through Digital Marketing
SEO & CRO - Acquiring & Engaging Customers Through Digital MarketingRenee Girard
 
Google Ranking by Digital Fry
Google Ranking by Digital FryGoogle Ranking by Digital Fry
Google Ranking by Digital FryShaaz Malik
 
How to perform a technical SEO audit and ramp up your content strategy in 10 ...
How to perform a technical SEO audit and ramp up your content strategy in 10 ...How to perform a technical SEO audit and ramp up your content strategy in 10 ...
How to perform a technical SEO audit and ramp up your content strategy in 10 ...Waqar Ahmad
 
Content seo
Content seoContent seo
Content seotg xd
 
Content seo
Content seoContent seo
Content seolambcap
 
Sourav SEO Case Study - 1k to 100k traffic
Sourav SEO Case Study - 1k to 100k trafficSourav SEO Case Study - 1k to 100k traffic
Sourav SEO Case Study - 1k to 100k trafficsourav kumar
 
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' Distilled
 
Internet Marketing Services
Internet Marketing ServicesInternet Marketing Services
Internet Marketing ServicesAndrew Davidson
 
Introduction to SEONext Systems & Processes
Introduction to SEONext Systems & ProcessesIntroduction to SEONext Systems & Processes
Introduction to SEONext Systems & ProcessesJustin_Blackhurst
 

Similar to Lessons from SEO split-testing (20)

Amazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOAmazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEO
 
Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....
Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....
Take A Sneaky Peak At How Someone Built & Ranked A Site That Beats Penguin 3....
 
The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...
The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...
The ‘traditional approach’ to SEO is broken - how to prioritise your efforts ...
 
3 Key Areas areas for SEO Growth in 2019/2020 - James Norquay, Prosperity Media
3 Key Areas areas for SEO Growth in 2019/2020  - James Norquay, Prosperity Media3 Key Areas areas for SEO Growth in 2019/2020  - James Norquay, Prosperity Media
3 Key Areas areas for SEO Growth in 2019/2020 - James Norquay, Prosperity Media
 
Essay On Search Engine Optimization Strategies
Essay On Search Engine Optimization StrategiesEssay On Search Engine Optimization Strategies
Essay On Search Engine Optimization Strategies
 
Stephan Spencer - SEMPO Atlanta. October 1, 2010. Topic: Advanced SEO
Stephan Spencer - SEMPO Atlanta.  October 1, 2010.  Topic: Advanced SEOStephan Spencer - SEMPO Atlanta.  October 1, 2010.  Topic: Advanced SEO
Stephan Spencer - SEMPO Atlanta. October 1, 2010. Topic: Advanced SEO
 
SearchCon 2016 | High Velocity Presentations
SearchCon 2016 | High Velocity PresentationsSearchCon 2016 | High Velocity Presentations
SearchCon 2016 | High Velocity Presentations
 
SEO Strategy.pdf
SEO Strategy.pdfSEO Strategy.pdf
SEO Strategy.pdf
 
Search Engine Optimization
Search Engine OptimizationSearch Engine Optimization
Search Engine Optimization
 
Split Testing for SEO - 9 Months of Learning
Split Testing for SEO - 9 Months of LearningSplit Testing for SEO - 9 Months of Learning
Split Testing for SEO - 9 Months of Learning
 
C-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO StrategiesC-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO Strategies
 
SEO & CRO - Acquiring & Engaging Customers Through Digital Marketing
SEO & CRO - Acquiring & Engaging Customers Through Digital MarketingSEO & CRO - Acquiring & Engaging Customers Through Digital Marketing
SEO & CRO - Acquiring & Engaging Customers Through Digital Marketing
 
Google Ranking by Digital Fry
Google Ranking by Digital FryGoogle Ranking by Digital Fry
Google Ranking by Digital Fry
 
How to perform a technical SEO audit and ramp up your content strategy in 10 ...
How to perform a technical SEO audit and ramp up your content strategy in 10 ...How to perform a technical SEO audit and ramp up your content strategy in 10 ...
How to perform a technical SEO audit and ramp up your content strategy in 10 ...
 
Content seo
Content seoContent seo
Content seo
 
Content seo
Content seoContent seo
Content seo
 
Sourav SEO Case Study - 1k to 100k traffic
Sourav SEO Case Study - 1k to 100k trafficSourav SEO Case Study - 1k to 100k traffic
Sourav SEO Case Study - 1k to 100k traffic
 
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
 
Internet Marketing Services
Internet Marketing ServicesInternet Marketing Services
Internet Marketing Services
 
Introduction to SEONext Systems & Processes
Introduction to SEONext Systems & ProcessesIntroduction to SEONext Systems & Processes
Introduction to SEONext Systems & Processes
 

More from Will Critchlow

SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...
SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...
SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...Will Critchlow
 
BrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will CritchlowBrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will CritchlowWill Critchlow
 
BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...
BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...
BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...Will Critchlow
 
BrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO tests
BrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO testsBrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO tests
BrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO testsWill Critchlow
 
CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...Will Critchlow
 
SEO split tests you should run - Will Critchlow
SEO split tests you should run - Will CritchlowSEO split tests you should run - Will Critchlow
SEO split tests you should run - Will CritchlowWill Critchlow
 
What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015Will Critchlow
 
Will critchlow, meaningful metrics
Will critchlow, meaningful metricsWill critchlow, meaningful metrics
Will critchlow, meaningful metricsWill Critchlow
 
Internet trends for marketers
Internet trends for marketersInternet trends for marketers
Internet trends for marketersWill Critchlow
 
Inbound on a shoestring - Searchlove Boston
Inbound on a shoestring - Searchlove BostonInbound on a shoestring - Searchlove Boston
Inbound on a shoestring - Searchlove BostonWill Critchlow
 
Link building mediocre to great
Link building mediocre to greatLink building mediocre to great
Link building mediocre to greatWill Critchlow
 
Common Technical Mistakes SMX Munich 2012
Common Technical Mistakes SMX Munich 2012Common Technical Mistakes SMX Munich 2012
Common Technical Mistakes SMX Munich 2012Will Critchlow
 
Things I wish I'd known
Things I wish I'd knownThings I wish I'd known
Things I wish I'd knownWill Critchlow
 
Why you should love seo
Why you should love seoWhy you should love seo
Why you should love seoWill Critchlow
 

More from Will Critchlow (17)

SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...
SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...
SEO in Turbulent Times - BrightonSEO San Diego 2023 - Will Critchlow, SearchP...
 
BrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will CritchlowBrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
 
BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...
BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...
BrightonSEO - SearchPilot - Will Critchlow - When what’s good for users isn’t...
 
BrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO tests
BrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO testsBrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO tests
BrightonSEO - SearchPilot - Will Critchlow - Lessons from 100s of SEO tests
 
CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...
 
Gaps in the algorithm
Gaps in the algorithmGaps in the algorithm
Gaps in the algorithm
 
SEO split tests you should run - Will Critchlow
SEO split tests you should run - Will CritchlowSEO split tests you should run - Will Critchlow
SEO split tests you should run - Will Critchlow
 
What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015
 
Will critchlow, meaningful metrics
Will critchlow, meaningful metricsWill critchlow, meaningful metrics
Will critchlow, meaningful metrics
 
Internet trends for marketers
Internet trends for marketersInternet trends for marketers
Internet trends for marketers
 
Inbound on a shoestring - Searchlove Boston
Inbound on a shoestring - Searchlove BostonInbound on a shoestring - Searchlove Boston
Inbound on a shoestring - Searchlove Boston
 
Black hat politics
Black hat politicsBlack hat politics
Black hat politics
 
Link building mediocre to great
Link building mediocre to greatLink building mediocre to great
Link building mediocre to great
 
Common Technical Mistakes SMX Munich 2012
Common Technical Mistakes SMX Munich 2012Common Technical Mistakes SMX Munich 2012
Common Technical Mistakes SMX Munich 2012
 
Metrics that Matter
Metrics that MatterMetrics that Matter
Metrics that Matter
 
Things I wish I'd known
Things I wish I'd knownThings I wish I'd known
Things I wish I'd known
 
Why you should love seo
Why you should love seoWhy you should love seo
Why you should love seo
 

Recently uploaded

Report on the future of Digital Advertising in India
Report on the future of Digital Advertising in IndiaReport on the future of Digital Advertising in India
Report on the future of Digital Advertising in IndiaSocial Samosa
 
ChatGPT : How to use ChatGPT and Get updated Prompts
ChatGPT : How to use ChatGPT and Get updated PromptsChatGPT : How to use ChatGPT and Get updated Prompts
ChatGPT : How to use ChatGPT and Get updated PromptsSaurav Srivastava
 
Media Training 101: Discover the Basics of Being a Spokesperson
Media Training 101: Discover the Basics of Being a SpokespersonMedia Training 101: Discover the Basics of Being a Spokesperson
Media Training 101: Discover the Basics of Being a SpokespersonNapierPR
 
Advertising concepts , Importance and its Types
Advertising concepts , Importance and its TypesAdvertising concepts , Importance and its Types
Advertising concepts , Importance and its Typesparamanjegatheeswari
 
Advanced Marketing Management Promotional Marketing for Bourbon Bowl
Advanced Marketing Management Promotional Marketing for Bourbon BowlAdvanced Marketing Management Promotional Marketing for Bourbon Bowl
Advanced Marketing Management Promotional Marketing for Bourbon Bowlanahimariche
 
Porter's Five Forces Analysis: Starbucks vs. Costa Coffee
Porter's Five Forces Analysis: Starbucks vs. Costa CoffeePorter's Five Forces Analysis: Starbucks vs. Costa Coffee
Porter's Five Forces Analysis: Starbucks vs. Costa Coffeeakshayytanwar123
 
Advertising and Sales Promotion concepts
Advertising and Sales Promotion  conceptsAdvertising and Sales Promotion  concepts
Advertising and Sales Promotion conceptsparamanjegatheeswari
 
Simplifying Direct Booking With Integrated Websites
Simplifying Direct Booking With Integrated WebsitesSimplifying Direct Booking With Integrated Websites
Simplifying Direct Booking With Integrated WebsitesAnton Shulke
 
Advanced Email Marketing
Advanced Email MarketingAdvanced Email Marketing
Advanced Email MarketingAdriannaBednarz
 
Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14
Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14
Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14SearchNorwich
 
Advertising in Music Genre 2023 - TAM AdEx
Advertising in Music Genre 2023 - TAM AdExAdvertising in Music Genre 2023 - TAM AdEx
Advertising in Music Genre 2023 - TAM AdExSocial Samosa
 
Silvers_Alexander_SCBS_PB.1_2024-02.pptx
Silvers_Alexander_SCBS_PB.1_2024-02.pptxSilvers_Alexander_SCBS_PB.1_2024-02.pptx
Silvers_Alexander_SCBS_PB.1_2024-02.pptxhqyfxsvkhb
 
VBOUT Partners Meeting Group - February 2024
VBOUT Partners Meeting Group - February 2024VBOUT Partners Meeting Group - February 2024
VBOUT Partners Meeting Group - February 2024Vbout.com
 
How An Enterprise Digital PR Firm Earns 100’s Of Links In 30 Days
How An Enterprise Digital PR Firm Earns 100’s Of Links In 30 DaysHow An Enterprise Digital PR Firm Earns 100’s Of Links In 30 Days
How An Enterprise Digital PR Firm Earns 100’s Of Links In 30 DaysSearch Engine Journal
 
Market segmentation: Segments of a market refer to distinct groups of consumers
Market segmentation:  Segments of a market refer to distinct groups of consumersMarket segmentation:  Segments of a market refer to distinct groups of consumers
Market segmentation: Segments of a market refer to distinct groups of consumersNiveditha Achandira
 
Mattress Topper King Size | Kushimattress
Mattress Topper King Size | KushimattressMattress Topper King Size | Kushimattress
Mattress Topper King Size | KushimattressKushimattress
 
Presentation on Starbucks Marketing Strategy
Presentation on Starbucks Marketing StrategyPresentation on Starbucks Marketing Strategy
Presentation on Starbucks Marketing StrategySuman Kumari
 
Snapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdfSnapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdfEastern Online-iSURVEY
 
Advanced Pay Per Click (PPC)
Advanced Pay Per Click (PPC)Advanced Pay Per Click (PPC)
Advanced Pay Per Click (PPC)AdriannaBednarz
 
Redseer_Ear-It-All_Report Design 2_16022024_final.pdf
Redseer_Ear-It-All_Report Design 2_16022024_final.pdfRedseer_Ear-It-All_Report Design 2_16022024_final.pdf
Redseer_Ear-It-All_Report Design 2_16022024_final.pdfSocial Samosa
 

Recently uploaded (20)

Report on the future of Digital Advertising in India
Report on the future of Digital Advertising in IndiaReport on the future of Digital Advertising in India
Report on the future of Digital Advertising in India
 
ChatGPT : How to use ChatGPT and Get updated Prompts
ChatGPT : How to use ChatGPT and Get updated PromptsChatGPT : How to use ChatGPT and Get updated Prompts
ChatGPT : How to use ChatGPT and Get updated Prompts
 
Media Training 101: Discover the Basics of Being a Spokesperson
Media Training 101: Discover the Basics of Being a SpokespersonMedia Training 101: Discover the Basics of Being a Spokesperson
Media Training 101: Discover the Basics of Being a Spokesperson
 
Advertising concepts , Importance and its Types
Advertising concepts , Importance and its TypesAdvertising concepts , Importance and its Types
Advertising concepts , Importance and its Types
 
Advanced Marketing Management Promotional Marketing for Bourbon Bowl
Advanced Marketing Management Promotional Marketing for Bourbon BowlAdvanced Marketing Management Promotional Marketing for Bourbon Bowl
Advanced Marketing Management Promotional Marketing for Bourbon Bowl
 
Porter's Five Forces Analysis: Starbucks vs. Costa Coffee
Porter's Five Forces Analysis: Starbucks vs. Costa CoffeePorter's Five Forces Analysis: Starbucks vs. Costa Coffee
Porter's Five Forces Analysis: Starbucks vs. Costa Coffee
 
Advertising and Sales Promotion concepts
Advertising and Sales Promotion  conceptsAdvertising and Sales Promotion  concepts
Advertising and Sales Promotion concepts
 
Simplifying Direct Booking With Integrated Websites
Simplifying Direct Booking With Integrated WebsitesSimplifying Direct Booking With Integrated Websites
Simplifying Direct Booking With Integrated Websites
 
Advanced Email Marketing
Advanced Email MarketingAdvanced Email Marketing
Advanced Email Marketing
 
Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14
Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14
Empowering In-house Content Projects with AI - Yagmur Simsek - Search Norwich 14
 
Advertising in Music Genre 2023 - TAM AdEx
Advertising in Music Genre 2023 - TAM AdExAdvertising in Music Genre 2023 - TAM AdEx
Advertising in Music Genre 2023 - TAM AdEx
 
Silvers_Alexander_SCBS_PB.1_2024-02.pptx
Silvers_Alexander_SCBS_PB.1_2024-02.pptxSilvers_Alexander_SCBS_PB.1_2024-02.pptx
Silvers_Alexander_SCBS_PB.1_2024-02.pptx
 
VBOUT Partners Meeting Group - February 2024
VBOUT Partners Meeting Group - February 2024VBOUT Partners Meeting Group - February 2024
VBOUT Partners Meeting Group - February 2024
 
How An Enterprise Digital PR Firm Earns 100’s Of Links In 30 Days
How An Enterprise Digital PR Firm Earns 100’s Of Links In 30 DaysHow An Enterprise Digital PR Firm Earns 100’s Of Links In 30 Days
How An Enterprise Digital PR Firm Earns 100’s Of Links In 30 Days
 
Market segmentation: Segments of a market refer to distinct groups of consumers
Market segmentation:  Segments of a market refer to distinct groups of consumersMarket segmentation:  Segments of a market refer to distinct groups of consumers
Market segmentation: Segments of a market refer to distinct groups of consumers
 
Mattress Topper King Size | Kushimattress
Mattress Topper King Size | KushimattressMattress Topper King Size | Kushimattress
Mattress Topper King Size | Kushimattress
 
Presentation on Starbucks Marketing Strategy
Presentation on Starbucks Marketing StrategyPresentation on Starbucks Marketing Strategy
Presentation on Starbucks Marketing Strategy
 
Snapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdfSnapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdf
Snapshot of Consumer Behaviors of January 2024-EOLiSurvey (EN).pdf
 
Advanced Pay Per Click (PPC)
Advanced Pay Per Click (PPC)Advanced Pay Per Click (PPC)
Advanced Pay Per Click (PPC)
 
Redseer_Ear-It-All_Report Design 2_16022024_final.pdf
Redseer_Ear-It-All_Report Design 2_16022024_final.pdfRedseer_Ear-It-All_Report Design 2_16022024_final.pdf
Redseer_Ear-It-All_Report Design 2_16022024_final.pdf
 

Lessons from SEO split-testing

  • 1. Lessons from SEO split-testing Including evidence that we don’t know what we’re talking about @willcritchlow
  • 2. We used to have a pretty good understanding of ranking factors
  • 3. My mental model for ~2009 ranking factors had three different modes:
  • 4. One in the hyper-competitive head My mental model for ~2009 ranking factors had three different modes: One in the competitive mid-tail ...and one in the long-tail
  • 6. Tons of perfectly on-topic pages to choose from One in the hyper-competitive head
  • 7. So pick only perfectly-on-topic pages One in the hyper-competitive head
  • 8. (*) Page authority, but the domain inevitably factors into that calculation. This is why so many homepages ranked One in the hyper-competitive head ...and rank by authority (*)
  • 9. This resulted in a mix of homepages of mid-size sites, and inner pages on huge sites One in the hyper-competitive head
  • 10. But the general way to move up was through increased authority One in the hyper-competitive head
  • 11. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Long-tail
  • 12. One in the hyper-competitive head One in the competitive mid-tail
  • 13. Wealth of ROUGHLY on-topic pages to choose from One in the competitive mid-tail
  • 14. PERFECTLY on-topic could do well even on a relatively weak site One in the competitive mid-tail
  • 15. Rank the roughly on-topic pages by authority x “on-topicness” One in the competitive mid-tail
  • 16. Move up with better targeting or more authority One in the competitive mid-tail
  • 17. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail
  • 18. One in the hyper-competitive head One in the competitive mid-tail ...and one in the long-tail
  • 19. In the long-tail, a site of arbitrary weakness could rank if it was the most relevant ...and one in the long-tail
  • 20. Otherwise, massive sites rank with off-topic pages that mention something similar ...and one in the long-tail
  • 21. Generally, move up with better targeting ...and one in the long-tail
  • 22. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting.
  • 23. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting. So that was ~2009
  • 24. It’s not so simple any more. Google is harder to understand these days.
  • 25. PageRank (the first algorithm to use the link structure of the web) We know how we got to ~2009...
  • 28. Information retrieval PageRank Original research TWEAKS ...with growing complexity in subsequent years
  • 29. Particularly this comment from a user called Kevin Lacker (@lacker):
  • 30. I was thinking about it like it was a math puzzle and if I just thought really hard it would all make sense. -- Kevin Lacker (@lacker)
  • 31. Hey why don't you take the square root? -- Amit Singhal according to Kevin Lacker (@lacker)
  • 32. oh... am I allowed to write code that doesn't make any sense? -- Kevin Lacker (@lacker)
  • 33. -- Amit Singhal according to Kevin Lacker (@lacker) Multiply by 2 if it helps, add 5, whatever, just make things work and we can make it make sense later.
  • 34. No, but I’m still pretty good at this You’re thinking this to yourself right now.
  • 51. I can beat you with a simple machine Well. It turns out all we needed was a coin to flip. You’re all fired.
  • 55. It’s only going to get worse under Sundar Pichai
  • 56. So we need to test. Here’s how SEO split tests work
  • 60. You might have seen @TomAnthonySEO tweeting about the platform we’ve built to make this easy
  • 62. Excuse a brief diversion into geeky details
  • 63. Instead of comparing the performance of the control pages directly with the variant pages, we build a forecast of what’s called the counterfactual which is an estimate of what would have happened if we hadn’t made the change. We use the control group to make a counterfactual forecast that takes into account seasonality and site-wide changes. The black line on the chart above is the actual organic traffic to the variant pages. The blue line is the counterfactual. More: Distilled blog post and free forecasting tool
  • 64. It’s easiest to analyse the results by looking at the cumulative difference over time between the actual organic traffic and the counterfactual. The pale blue area is the 95% confidence interval. We can see a (statistically) zero effect for an initial time while Google crawls and indexes the test, followed by steady growth. A couple of weeks in, the confidence interval goes above zero and we have a winning test. More: Distilled blog
  • 65. It’s easiest to analyse the results by looking at the cumulative difference over time between the actual organic traffic and the counterfactual. The pale blue area is the 95% confidence interval. We can see a (statistically) zero effect for an initial time while Google crawls and indexes the test, followed by steady growth. A couple of weeks in, the confidence interval goes above zero and we have a winning test. More: Distilled blog Hashtag winning
  • 66. Further reading for those interested: ● Predicting the present with Bayesian structural time series [PDF] ● Inferring causal impact using Bayesian structural time series [PDF] ● CausalImpact R package ● Finding the ROI of title tag changes More: Distilled blog
  • 67. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  • 69. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  • 70. Category pages have lots of images and not much text
  • 71. Adding structured data to category pages
  • 73. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  • 74. Established wisdom would say ALT tags on images are good for SEO
  • 75. We found adding ALT tags to images had no effect
  • 76. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  • 77. Title tag before: Which TV should I buy? - Argos Title tag after: Which TV to buy? - Argos What happens when you match title tags to the greatest search volume?
  • 78. Organic sessions decreased by an average of 8%
  • 79. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  • 80. We can render Javascript!
  • 81. What happens if your content is only visible with Javascript? Javascript EnabledJavascript Disabled
  • 82. Making it visible increased organic sessions by ~ 6.2%
  • 83. Read more on our blog: early results from split-testing JS for SEO
  • 84. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  • 85. How does SEO text on category pages perform?
  • 86. E-commerce site number 1 ~ 3.1% increase in organic sessions
  • 87. E-commerce site number 2 - No effect/negative effect
  • 88. This is why we have been investing so much in split-testing Check out odn.distilled.net if you haven’t already. The team will be happy to demo for you. We served ~5 billion requests last quarter and recently published everything from response times to our +£100k / month split test.
  • 89. But I’m also seeing more subtle impacts on my recommendations: ● You can recommend small tweaks and see the benefits compound ● You can test wild hypotheses with unknown upsides ● You can try things that might have a downside (more focused targeting, less copy, etc.) And that’s even before you get the benefits of testing clickthrough rate, and the benefits of pretty charts to show the boss highlighting the impact of your work! More: blog post
  • 90. Oh, and one more thing
  • 91. Can we build a better machine than a coin flip? Seems like a low bar
  • 96. The goal is a winning combination of human and machine Human + computer beats computer (for now)
  • 98. ● Mobius strip ● Confusion ● Signal box ● Cigar ● Discontinuity ● Confidence ● Burt Totaro ● Sundar Pichai ● John Giannandrea ● Chuck Norris ● Jeff Dean ● Fencing ● Keyboard Image credits ● Go ● Robot ● Leopard print sofa ● Leopard ● Bug ● Lego robots ● Iron Man ● Leeds