BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget

Botify
BotifyBotify
Crawl Budget is dead, please
welcome Rendering Budget
Robin Eisenberg // Botify //
Javascript is complex and here
to stay.
Rendering problems ? Looks a
lot like Crawl Budget to me
What is already available to
optimise your JS?
Agenda
Crawl Budget
Google has finite power
Google doesn’t have the time to crawl
everything
Google needs to allocate
What if Google doesn’t have enough time
to crawl your whole website in 24 hours?
Crawl Budgets affect your bottom line
Search engines are visibly allocating resources
Direct correlation: page speed vs crawl volume
You might not be showing your whole website!
The modern web relies on JavaScript
Today is different
Render Budgets
JavaScript is Complex
JavaScript is Asynchronous
JavaScript is Slow
What if Google doesn’t have enough time
to render your whole website in 24 hours?
We’ve seen disasters like these:
JavaScript
Release
Drop
Rollback to
HTML
1
2
3
1 3
2
JavaScript complexities
Content changes over the page’s lifecycle
Rendering time is relative
JavaScript is not easily readable/debuggable
These difficulties apply to Google, not just you.
BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget
Javascript rendering resources
JavaScript Crawl Cost Management
Rendering is actually deferred until Google has the resources ready to render the client side content.
Once resources are available, Google index the content further.
Googlebot waits
for available
resources to
render Javascript
by itself.
A new Step in the Search Process
Please welcome rendering budget
Crawl Index
(HTML)
Render Index
(JS+HTML)
SAME SAME NEW! NEW!
Despite this, we should not duck out but rather
embrace JavaScript
JS is not going away
Help us help you
JavaScript is an
opportunity
-Google
Rendering Budget
Google still has finite power
Google doesn’t have the time to render
everything, everyday
Google still needs to allocate
Allocation has changed though, because
the cost and complexity has changed
Helping Google
Google has trouble with rendering
Anything you can do to help will be
rewarded
Or risk not being competitive anymore
And of what you already know about Crawl
Budget applies to Rendering Budget.
Analyzing your JavaScript
There’s nothing new under the sun.
You already know how to do that!
Page Delay
We counted the time for the
server to send the page
Time to Interactive
BEFORE:
We count the time for the
crawler to render the page
NOW:
Crawl Ratio
We parsed log files and
counted unique urls
Render Ratio
BEFORE:
We parse log files and count
unique referrers
...from googlebot, on API calls
NOW:
Test a URL & content that should be Indexed
Search in Google:
site:<your_url> <unique phrase>
We know what important LINKS and CONTENT are rendered client-side
Can we find this content in Google’s index?
Is it relevant to assess JavaScript rendering for this website ?
Is Google indexing my important content?
TEST:
site:https://reverb.com/marketplace/
acoustic-guitars "Martin D-18
Dreadnought Acoustic Guitar"
RESULTS:
Googlebot fetched the JS
The rendering was DONE
The 2-pass indexing is OK
Find an expression that is always there in the
HTML-only version: "used from"
What proportion of my pages are rendered?
Find an expression that is always there in the
rendered version (but not in HTML-only):
"Make an Offer"
What proportion of my pages are rendered?
Use site: in Google with the first expression, then the second, to get the number of indexed results and
calculate the rendering ratio.
+
What proportion of my pages are rendered?
79% 2-pass indexed (rendered)13,700 / 17,400 =
SEO.php
We added server-side
SEO tracking
SEO.js
BEFORE:
We add crawler-side
JS SEO tracking
NOW:
SEO.php Simple Example
...Before logfile analysis, we all used this trick to monitor SEs crawls.
SEO.js Simple Example
Use Google’s Index to understand its passage, and fetch vs. render.
Staying on top of your JavaScript SEO
Preparing for the future
STEP 1
Have a coffee with your developers
SHARING IS CARING
Set up regular discussions
Stay informed of their roadmap
Have dedicated points of contact
Help them understand your issues - most are unaware
Your devs are your best allies in this endeavour
STEP 2
Make your own targets
MONITOR THE RIGHT KPIS
Identify the right resources to allow/exclude:
Timing - FCP, TTI
Quality - Size, N* of requests
Metrics - Lighthouse scores
Start with regular manual monitoring - automate it if possible
STEP 3
Invest in a platform for JS
optimisation at scale
Crawl Budget is (almost) dead, long live Rendering Budget
Budgets are here to stay
You need to optimise for them
The method has not changed
Key Takeaways
Visit our Booth to discuss!
1 of 31

Recommended

BrightonSEO Oct 2022 Similar_ai.pdf by
BrightonSEO Oct 2022 Similar_ai.pdfBrightonSEO Oct 2022 Similar_ai.pdf
BrightonSEO Oct 2022 Similar_ai.pdfDylan Fuler
246 views49 slides
Keyword Research and Topic Modeling in a Semantic Web by
Keyword Research and Topic Modeling in a Semantic WebKeyword Research and Topic Modeling in a Semantic Web
Keyword Research and Topic Modeling in a Semantic WebBill Slawski
53.8K views37 slides
Entity Seo Mastery by
Entity Seo MasteryEntity Seo Mastery
Entity Seo MasteryDixon Jones
299 views48 slides
SEO Content Automation by
SEO Content AutomationSEO Content Automation
SEO Content AutomationDylan Fuler
58 views21 slides
Coronavirus and Future of SEO: Digital Marketing and Remote Culture by
Coronavirus and Future of SEO: Digital Marketing and Remote CultureCoronavirus and Future of SEO: Digital Marketing and Remote Culture
Coronavirus and Future of SEO: Digital Marketing and Remote CultureKoray Tugberk GUBUR
3.9K views61 slides
SGE, New Features in Google Search & How to Respond.pdf by
SGE, New Features in Google Search & How to Respond.pdfSGE, New Features in Google Search & How to Respond.pdf
SGE, New Features in Google Search & How to Respond.pdfLily Ray
808 views94 slides

More Related Content

What's hot

BrightonSEO - Amanda Jordan.pptx by
BrightonSEO - Amanda Jordan.pptxBrightonSEO - Amanda Jordan.pptx
BrightonSEO - Amanda Jordan.pptxAmanda Jordan
557 views46 slides
SMX_DevTools_Monaco_2.pdf by
SMX_DevTools_Monaco_2.pdfSMX_DevTools_Monaco_2.pdf
SMX_DevTools_Monaco_2.pdfSara Moccand-Sayegh
173 views90 slides
The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank? by
The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank?The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank?
The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank?Koray Tugberk GUBUR
1.9K views124 slides
BrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce Websites by
BrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce WebsitesBrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce Websites
BrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce WebsitesDan Taylor
557 views32 slides
LondonSEO Meetup - Cutting through the noise in SEO with data - Reina Hanada by
LondonSEO Meetup - Cutting through the noise in SEO with data - Reina HanadaLondonSEO Meetup - Cutting through the noise in SEO with data - Reina Hanada
LondonSEO Meetup - Cutting through the noise in SEO with data - Reina HanadaReinaHanada1
73 views88 slides
Slawski New Approaches for Structured Data:Evolution of Question Answering by
Slawski   New Approaches for Structured Data:Evolution of Question Answering Slawski   New Approaches for Structured Data:Evolution of Question Answering
Slawski New Approaches for Structured Data:Evolution of Question Answering Bill Slawski
10.3K views36 slides

What's hot(20)

BrightonSEO - Amanda Jordan.pptx by Amanda Jordan
BrightonSEO - Amanda Jordan.pptxBrightonSEO - Amanda Jordan.pptx
BrightonSEO - Amanda Jordan.pptx
Amanda Jordan557 views
The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank? by Koray Tugberk GUBUR
The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank?The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank?
The Reason Behind Semantic SEO: Why does Google Avoid the Word PageRank?
Koray Tugberk GUBUR1.9K views
BrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce Websites by Dan Taylor
BrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce WebsitesBrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce Websites
BrightonSEO October 2022 - Dan Taylor SEO - Indexing Ecommerce Websites
Dan Taylor557 views
LondonSEO Meetup - Cutting through the noise in SEO with data - Reina Hanada by ReinaHanada1
LondonSEO Meetup - Cutting through the noise in SEO with data - Reina HanadaLondonSEO Meetup - Cutting through the noise in SEO with data - Reina Hanada
LondonSEO Meetup - Cutting through the noise in SEO with data - Reina Hanada
ReinaHanada173 views
Slawski New Approaches for Structured Data:Evolution of Question Answering by Bill Slawski
Slawski   New Approaches for Structured Data:Evolution of Question Answering Slawski   New Approaches for Structured Data:Evolution of Question Answering
Slawski New Approaches for Structured Data:Evolution of Question Answering
Bill Slawski10.3K views
Data-driven SEO & content strategy to reduce your customer acquisition costs by adlift
Data-driven SEO & content strategy to reduce your customer acquisition costsData-driven SEO & content strategy to reduce your customer acquisition costs
Data-driven SEO & content strategy to reduce your customer acquisition costs
adlift582 views
Semantic Publishing and Entity SEO - Conteference 20-11-2022 by Massimiliano Geraci
Semantic Publishing and Entity SEO - Conteference 20-11-2022Semantic Publishing and Entity SEO - Conteference 20-11-2022
Semantic Publishing and Entity SEO - Conteference 20-11-2022
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And... by Dawn Anderson MSc DigM
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
How to Incorporate ML in your SERP Analysis, Lazarina Stoy -BrightonSEO Oct, ... by LazarinaStoyanova
How to Incorporate ML in your SERP Analysis, Lazarina Stoy -BrightonSEO Oct, ...How to Incorporate ML in your SERP Analysis, Lazarina Stoy -BrightonSEO Oct, ...
How to Incorporate ML in your SERP Analysis, Lazarina Stoy -BrightonSEO Oct, ...
SEO Case Study - Hangikredi.com From 12 March to 24 September Core Update by Koray Tugberk GUBUR
SEO Case Study - Hangikredi.com From 12 March to 24 September Core UpdateSEO Case Study - Hangikredi.com From 12 March to 24 September Core Update
SEO Case Study - Hangikredi.com From 12 March to 24 September Core Update
Agile SEO: Prioritise SEO Activities with Cadence and Risk Radius by Parth Suba
Agile SEO: Prioritise SEO Activities with Cadence and Risk RadiusAgile SEO: Prioritise SEO Activities with Cadence and Risk Radius
Agile SEO: Prioritise SEO Activities with Cadence and Risk Radius
Parth Suba1.1K views
Opinion-based Article Ranking for Information Retrieval Systems: Factoids and... by Koray Tugberk GUBUR
Opinion-based Article Ranking for Information Retrieval Systems: Factoids and...Opinion-based Article Ranking for Information Retrieval Systems: Factoids and...
Opinion-based Article Ranking for Information Retrieval Systems: Factoids and...
Koray Tugberk GUBUR1.4K views
Semantic Search Engine: Semantic Search and Query Parsing with Phrases and En... by Koray Tugberk GUBUR
Semantic Search Engine: Semantic Search and Query Parsing with Phrases and En...Semantic Search Engine: Semantic Search and Query Parsing with Phrases and En...
Semantic Search Engine: Semantic Search and Query Parsing with Phrases and En...
Koray Tugberk GUBUR8.3K views
Building a Search Intent-Driven Website Architecture (SEO Mastery Summit 2022... by LazarinaStoyanova
Building a Search Intent-Driven Website Architecture (SEO Mastery Summit 2022...Building a Search Intent-Driven Website Architecture (SEO Mastery Summit 2022...
Building a Search Intent-Driven Website Architecture (SEO Mastery Summit 2022...
LazarinaStoyanova3.8K views
Turning A Neglected YouTube Channel into a Traffic Generation Machine by Phil Nottingham
Turning A Neglected YouTube Channel into a Traffic Generation MachineTurning A Neglected YouTube Channel into a Traffic Generation Machine
Turning A Neglected YouTube Channel into a Traffic Generation Machine
Phil Nottingham958 views
Why the f*ck Doesn't This HREFLANG Work - BrightonSEO Autumn 2021 by Sophie Gibson
Why the f*ck Doesn't This HREFLANG Work - BrightonSEO Autumn 2021Why the f*ck Doesn't This HREFLANG Work - BrightonSEO Autumn 2021
Why the f*ck Doesn't This HREFLANG Work - BrightonSEO Autumn 2021
Sophie Gibson13.8K views
Technical SEO Myths Facts And Theories On Crawl Budget And The Importance Of ... by Dawn Anderson MSc DigM
Technical SEO Myths Facts And Theories On Crawl Budget And The Importance Of ...Technical SEO Myths Facts And Theories On Crawl Budget And The Importance Of ...
Technical SEO Myths Facts And Theories On Crawl Budget And The Importance Of ...

Similar to BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget

SEARCH Y : Benjamin Bussière - Javascript and seo misconceptions, misunders... by
SEARCH Y :  Benjamin Bussière - Javascript and seo  misconceptions, misunders...SEARCH Y :  Benjamin Bussière - Javascript and seo  misconceptions, misunders...
SEARCH Y : Benjamin Bussière - Javascript and seo misconceptions, misunders...SEARCH Y - Philippe Yonnet Evénements
612 views52 slides
How to make JavaScript websites successful in Google | iJS 2019 by
 How to make JavaScript websites successful in Google | iJS 2019 How to make JavaScript websites successful in Google | iJS 2019
How to make JavaScript websites successful in Google | iJS 2019Onely
128 views97 slides
L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic... by
L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic...L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic...
L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic...SEO Camp Association
4.9K views36 slides
Javascript SEO Devs and SEOs playing nicely by
Javascript SEO Devs and SEOs playing nicelyJavascript SEO Devs and SEOs playing nicely
Javascript SEO Devs and SEOs playing nicelyPeter Mead
89 views59 slides
Demystifying JavaScript & SEO by
Demystifying JavaScript & SEODemystifying JavaScript & SEO
Demystifying JavaScript & SEOBotify
877 views39 slides
Indexing Your Content is Optional (2020 Edition) by
Indexing Your Content is Optional (2020 Edition)Indexing Your Content is Optional (2020 Edition)
Indexing Your Content is Optional (2020 Edition)Onely
148 views87 slides

Similar to BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget(20)

How to make JavaScript websites successful in Google | iJS 2019 by Onely
 How to make JavaScript websites successful in Google | iJS 2019 How to make JavaScript websites successful in Google | iJS 2019
How to make JavaScript websites successful in Google | iJS 2019
Onely128 views
L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic... by SEO Camp Association
L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic...L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic...
L’importance du crawl du JavaScript : pourquoi, comment et pour quels bénéfic...
Javascript SEO Devs and SEOs playing nicely by Peter Mead
Javascript SEO Devs and SEOs playing nicelyJavascript SEO Devs and SEOs playing nicely
Javascript SEO Devs and SEOs playing nicely
Peter Mead89 views
Demystifying JavaScript & SEO by Botify
Demystifying JavaScript & SEODemystifying JavaScript & SEO
Demystifying JavaScript & SEO
Botify877 views
Indexing Your Content is Optional (2020 Edition) by Onely
Indexing Your Content is Optional (2020 Edition)Indexing Your Content is Optional (2020 Edition)
Indexing Your Content is Optional (2020 Edition)
Onely148 views
React seo tips to build seo friendly web applications by Katy Slemon
React seo tips to build seo friendly web applicationsReact seo tips to build seo friendly web applications
React seo tips to build seo friendly web applications
Katy Slemon65 views
Migration to a JS Framework without Losing Your Rankings and Mind by Maria Cieślak
Migration to a JS Framework without Losing Your Rankings and MindMigration to a JS Framework without Losing Your Rankings and Mind
Migration to a JS Framework without Losing Your Rankings and Mind
Maria Cieślak1.7K views
JavaScript Doesn't Have to be Evil | Digital Olympus by Onely
JavaScript Doesn't Have to be Evil | Digital OlympusJavaScript Doesn't Have to be Evil | Digital Olympus
JavaScript Doesn't Have to be Evil | Digital Olympus
Onely91 views
Website SEO Audits: How to Approach by Geoff Kennedy
Website SEO Audits: How to ApproachWebsite SEO Audits: How to Approach
Website SEO Audits: How to Approach
Geoff Kennedy2.2K views
Programmatic SEO: How to Dominate SEO Like TripAdvisor, Yelp and Zillow by Bernard Huang
Programmatic SEO: How to Dominate SEO Like TripAdvisor, Yelp and ZillowProgrammatic SEO: How to Dominate SEO Like TripAdvisor, Yelp and Zillow
Programmatic SEO: How to Dominate SEO Like TripAdvisor, Yelp and Zillow
Bernard Huang14K views
Javascript & SEO 2019 by Edd Wilson
Javascript & SEO 2019Javascript & SEO 2019
Javascript & SEO 2019
Edd Wilson136 views
SearchLove Boston 2018 - Bartosz Goralewicz - JavaScript: Looking Past the ... by Distilled
SearchLove Boston 2018 -  Bartosz Goralewicz -  JavaScript: Looking Past the ...SearchLove Boston 2018 -  Bartosz Goralewicz -  JavaScript: Looking Past the ...
SearchLove Boston 2018 - Bartosz Goralewicz - JavaScript: Looking Past the ...
Distilled2.3K views
How Much Content is Not Indexed in Google in 2019? by Onely
How Much Content is Not Indexed in Google in 2019?How Much Content is Not Indexed in Google in 2019?
How Much Content is Not Indexed in Google in 2019?
Onely2.8K views
Keeping Things Lean & Mean: Crawl Optimisation - Search Marketing Summit AU by Jason Mun
Keeping Things Lean & Mean: Crawl Optimisation - Search Marketing Summit AUKeeping Things Lean & Mean: Crawl Optimisation - Search Marketing Summit AU
Keeping Things Lean & Mean: Crawl Optimisation - Search Marketing Summit AU
Jason Mun3.1K views
Chewy Trewella - Google Searchtips by sounddelivery
Chewy Trewella - Google SearchtipsChewy Trewella - Google Searchtips
Chewy Trewella - Google Searchtips
sounddelivery2.9K views
How to scale SEO work NOBODY wants to do (including your competitors) to rapi... by Hamlet Batista
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
Hamlet Batista403 views

More from Botify

Faceted Navigation: (Almost) Everyone is Doing it Wrong by
Faceted Navigation: (Almost) Everyone is Doing it WrongFaceted Navigation: (Almost) Everyone is Doing it Wrong
Faceted Navigation: (Almost) Everyone is Doing it WrongBotify
738 views63 slides
From Search to Transaction: How to Master the Customer Experience by
From Search to Transaction: How to Master the Customer ExperienceFrom Search to Transaction: How to Master the Customer Experience
From Search to Transaction: How to Master the Customer ExperienceBotify
293 views49 slides
The Evolution of Customer Journeys & SEO by
The Evolution of Customer Journeys & SEOThe Evolution of Customer Journeys & SEO
The Evolution of Customer Journeys & SEOBotify
318 views28 slides
How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It? by
How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It?How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It?
How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It?Botify
179 views26 slides
How to Find Your Site's True Ranking Factors by
How to Find Your Site's True Ranking FactorsHow to Find Your Site's True Ranking Factors
How to Find Your Site's True Ranking FactorsBotify
229 views55 slides
Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D... by
Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D...Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D...
Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D...Botify
176 views23 slides

More from Botify(20)

Faceted Navigation: (Almost) Everyone is Doing it Wrong by Botify
Faceted Navigation: (Almost) Everyone is Doing it WrongFaceted Navigation: (Almost) Everyone is Doing it Wrong
Faceted Navigation: (Almost) Everyone is Doing it Wrong
Botify738 views
From Search to Transaction: How to Master the Customer Experience by Botify
From Search to Transaction: How to Master the Customer ExperienceFrom Search to Transaction: How to Master the Customer Experience
From Search to Transaction: How to Master the Customer Experience
Botify293 views
The Evolution of Customer Journeys & SEO by Botify
The Evolution of Customer Journeys & SEOThe Evolution of Customer Journeys & SEO
The Evolution of Customer Journeys & SEO
Botify318 views
How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It? by Botify
How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It?How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It?
How Is COVID-19 Impacting Organic Search by Industry & What Can We Do About It?
Botify179 views
How to Find Your Site's True Ranking Factors by Botify
How to Find Your Site's True Ranking FactorsHow to Find Your Site's True Ranking Factors
How to Find Your Site's True Ranking Factors
Botify229 views
Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D... by Botify
Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D...Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D...
Webinar: How to Make Data-Driven Marketing Decisions Without a Data Science D...
Botify176 views
The Total Economic Impact of Botify by Botify
The Total Economic Impact of BotifyThe Total Economic Impact of Botify
The Total Economic Impact of Botify
Botify245 views
Algo Updates, Volatility, & How to Roll with the Punches in SEO by Botify
Algo Updates, Volatility, & How to Roll with the Punches in SEOAlgo Updates, Volatility, & How to Roll with the Punches in SEO
Algo Updates, Volatility, & How to Roll with the Punches in SEO
Botify290 views
New Holiday Data Reveals Insights About Handling Seasonal Volatility - Q1 202... by Botify
New Holiday Data Reveals Insights About Handling Seasonal Volatility - Q1 202...New Holiday Data Reveals Insights About Handling Seasonal Volatility - Q1 202...
New Holiday Data Reveals Insights About Handling Seasonal Volatility - Q1 202...
Botify191 views
Living in a mobile first index world by Botify
Living in a mobile first index worldLiving in a mobile first index world
Living in a mobile first index world
Botify1.1K views
Botify Webinar - The new Version of Botify Keywords by Botify
Botify Webinar - The new Version of Botify KeywordsBotify Webinar - The new Version of Botify Keywords
Botify Webinar - The new Version of Botify Keywords
Botify665 views
Mobile-First Index: A Data-Driven Analysis & Discussion by Botify
Mobile-First Index:  A Data-Driven Analysis & DiscussionMobile-First Index:  A Data-Driven Analysis & Discussion
Mobile-First Index: A Data-Driven Analysis & Discussion
Botify2.7K views
Why auditing your rel=canonical configuration is a shrewd move by Botify
Why auditing your rel=canonical configuration is a shrewd moveWhy auditing your rel=canonical configuration is a shrewd move
Why auditing your rel=canonical configuration is a shrewd move
Botify703 views
Botify webinar Internal Linking - October 2018 by Botify
Botify webinar   Internal Linking - October 2018Botify webinar   Internal Linking - October 2018
Botify webinar Internal Linking - October 2018
Botify607 views
How Does Google Crawl the Web? by Botify
How Does Google Crawl the Web?How Does Google Crawl the Web?
How Does Google Crawl the Web?
Botify2.4K views
GSC vs Scraping: Go Beyond Rankings by Botify
GSC vs Scraping: Go Beyond RankingsGSC vs Scraping: Go Beyond Rankings
GSC vs Scraping: Go Beyond Rankings
Botify584 views
The GDPR: What, Why and How Botify is Compliant by Design by Botify
The GDPR: What, Why and How Botify is Compliant by DesignThe GDPR: What, Why and How Botify is Compliant by Design
The GDPR: What, Why and How Botify is Compliant by Design
Botify487 views
Webinar Structured Data by Botify
Webinar Structured DataWebinar Structured Data
Webinar Structured Data
Botify500 views
Mobile first index webinar by Botify
Mobile first index webinarMobile first index webinar
Mobile first index webinar
Botify196 views
Decrypt Google’s Behavior with Botify Log Analyzer by Botify
Decrypt Google’s Behavior with Botify Log AnalyzerDecrypt Google’s Behavior with Botify Log Analyzer
Decrypt Google’s Behavior with Botify Log Analyzer
Botify570 views

Recently uploaded

Marketing and Community Building in Web3 by
Marketing and Community Building in Web3Marketing and Community Building in Web3
Marketing and Community Building in Web3Federico Ast
12 views64 slides
DU Series - Day 4.pptx by
DU Series - Day 4.pptxDU Series - Day 4.pptx
DU Series - Day 4.pptxUiPathCommunity
106 views28 slides
UiPath Document Understanding_Day 3.pptx by
UiPath Document Understanding_Day 3.pptxUiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptxUiPathCommunity
105 views25 slides
PORTFOLIO 1 (Bret Michael Pepito).pdf by
PORTFOLIO 1 (Bret Michael Pepito).pdfPORTFOLIO 1 (Bret Michael Pepito).pdf
PORTFOLIO 1 (Bret Michael Pepito).pdfbrejess0410
8 views6 slides
Building trust in our information ecosystem: who do we trust in an emergency by
Building trust in our information ecosystem: who do we trust in an emergencyBuilding trust in our information ecosystem: who do we trust in an emergency
Building trust in our information ecosystem: who do we trust in an emergencyTina Purnat
100 views18 slides
IETF 118: Starlink Protocol Performance by
IETF 118: Starlink Protocol PerformanceIETF 118: Starlink Protocol Performance
IETF 118: Starlink Protocol PerformanceAPNIC
297 views22 slides

Recently uploaded(10)

Marketing and Community Building in Web3 by Federico Ast
Marketing and Community Building in Web3Marketing and Community Building in Web3
Marketing and Community Building in Web3
Federico Ast12 views
UiPath Document Understanding_Day 3.pptx by UiPathCommunity
UiPath Document Understanding_Day 3.pptxUiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptx
UiPathCommunity105 views
PORTFOLIO 1 (Bret Michael Pepito).pdf by brejess0410
PORTFOLIO 1 (Bret Michael Pepito).pdfPORTFOLIO 1 (Bret Michael Pepito).pdf
PORTFOLIO 1 (Bret Michael Pepito).pdf
brejess04108 views
Building trust in our information ecosystem: who do we trust in an emergency by Tina Purnat
Building trust in our information ecosystem: who do we trust in an emergencyBuilding trust in our information ecosystem: who do we trust in an emergency
Building trust in our information ecosystem: who do we trust in an emergency
Tina Purnat100 views
IETF 118: Starlink Protocol Performance by APNIC
IETF 118: Starlink Protocol PerformanceIETF 118: Starlink Protocol Performance
IETF 118: Starlink Protocol Performance
APNIC297 views
How to think like a threat actor for Kubernetes.pptx by LibbySchulze1
How to think like a threat actor for Kubernetes.pptxHow to think like a threat actor for Kubernetes.pptx
How to think like a threat actor for Kubernetes.pptx
LibbySchulze15 views

BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget

  • 1. Crawl Budget is dead, please welcome Rendering Budget Robin Eisenberg // Botify //
  • 2. Javascript is complex and here to stay. Rendering problems ? Looks a lot like Crawl Budget to me What is already available to optimise your JS? Agenda
  • 3. Crawl Budget Google has finite power Google doesn’t have the time to crawl everything Google needs to allocate What if Google doesn’t have enough time to crawl your whole website in 24 hours?
  • 4. Crawl Budgets affect your bottom line Search engines are visibly allocating resources Direct correlation: page speed vs crawl volume You might not be showing your whole website!
  • 5. The modern web relies on JavaScript Today is different
  • 6. Render Budgets JavaScript is Complex JavaScript is Asynchronous JavaScript is Slow What if Google doesn’t have enough time to render your whole website in 24 hours?
  • 7. We’ve seen disasters like these: JavaScript Release Drop Rollback to HTML 1 2 3 1 3 2
  • 8. JavaScript complexities Content changes over the page’s lifecycle Rendering time is relative JavaScript is not easily readable/debuggable These difficulties apply to Google, not just you.
  • 10. Javascript rendering resources JavaScript Crawl Cost Management Rendering is actually deferred until Google has the resources ready to render the client side content. Once resources are available, Google index the content further. Googlebot waits for available resources to render Javascript by itself.
  • 11. A new Step in the Search Process Please welcome rendering budget Crawl Index (HTML) Render Index (JS+HTML) SAME SAME NEW! NEW!
  • 12. Despite this, we should not duck out but rather embrace JavaScript JS is not going away
  • 13. Help us help you JavaScript is an opportunity -Google
  • 14. Rendering Budget Google still has finite power Google doesn’t have the time to render everything, everyday Google still needs to allocate Allocation has changed though, because the cost and complexity has changed
  • 15. Helping Google Google has trouble with rendering Anything you can do to help will be rewarded Or risk not being competitive anymore And of what you already know about Crawl Budget applies to Rendering Budget.
  • 16. Analyzing your JavaScript There’s nothing new under the sun. You already know how to do that!
  • 17. Page Delay We counted the time for the server to send the page Time to Interactive BEFORE: We count the time for the crawler to render the page NOW:
  • 18. Crawl Ratio We parsed log files and counted unique urls Render Ratio BEFORE: We parse log files and count unique referrers ...from googlebot, on API calls NOW:
  • 19. Test a URL & content that should be Indexed Search in Google: site:<your_url> <unique phrase>
  • 20. We know what important LINKS and CONTENT are rendered client-side Can we find this content in Google’s index? Is it relevant to assess JavaScript rendering for this website ? Is Google indexing my important content? TEST: site:https://reverb.com/marketplace/ acoustic-guitars "Martin D-18 Dreadnought Acoustic Guitar" RESULTS: Googlebot fetched the JS The rendering was DONE The 2-pass indexing is OK
  • 21. Find an expression that is always there in the HTML-only version: "used from" What proportion of my pages are rendered?
  • 22. Find an expression that is always there in the rendered version (but not in HTML-only): "Make an Offer" What proportion of my pages are rendered?
  • 23. Use site: in Google with the first expression, then the second, to get the number of indexed results and calculate the rendering ratio. + What proportion of my pages are rendered? 79% 2-pass indexed (rendered)13,700 / 17,400 =
  • 24. SEO.php We added server-side SEO tracking SEO.js BEFORE: We add crawler-side JS SEO tracking NOW:
  • 25. SEO.php Simple Example ...Before logfile analysis, we all used this trick to monitor SEs crawls.
  • 26. SEO.js Simple Example Use Google’s Index to understand its passage, and fetch vs. render.
  • 27. Staying on top of your JavaScript SEO Preparing for the future
  • 28. STEP 1 Have a coffee with your developers SHARING IS CARING Set up regular discussions Stay informed of their roadmap Have dedicated points of contact Help them understand your issues - most are unaware Your devs are your best allies in this endeavour
  • 29. STEP 2 Make your own targets MONITOR THE RIGHT KPIS Identify the right resources to allow/exclude: Timing - FCP, TTI Quality - Size, N* of requests Metrics - Lighthouse scores Start with regular manual monitoring - automate it if possible
  • 30. STEP 3 Invest in a platform for JS optimisation at scale
  • 31. Crawl Budget is (almost) dead, long live Rendering Budget Budgets are here to stay You need to optimise for them The method has not changed Key Takeaways Visit our Booth to discuss!