Crawl Budget is (almost) dead long live Rendering Budget. Relive Botify's BirghtonSEO talk on rendering budget.

1. Crawl Budget is dead, please
welcome Rendering Budget
Robin Eisenberg // Botify //

2. Javascript is complex and here
to stay.
Rendering problems ? Looks a
lot like Crawl Budget to me
What is already available to
optimise your JS?
Agenda

3. Crawl Budget
Google has finite power
Google doesn’t have the time to crawl
everything
Google needs to allocate
What if Google doesn’t have enough time
to crawl your whole website in 24 hours?

4. Crawl Budgets affect your bottom line
Search engines are visibly allocating resources
Direct correlation: page speed vs crawl volume
You might not be showing your whole website!

5. The modern web relies on JavaScript
Today is different

6. Render Budgets
JavaScript is Complex
JavaScript is Asynchronous
JavaScript is Slow
What if Google doesn’t have enough time
to render your whole website in 24 hours?

7. We’ve seen disasters like these:
JavaScript
Release
Drop
Rollback to
HTML
1
2
3
1 3
2

8. JavaScript complexities
Content changes over the page’s lifecycle
Rendering time is relative
JavaScript is not easily readable/debuggable
These difficulties apply to Google, not just you.

10. Javascript rendering resources
JavaScript Crawl Cost Management
Rendering is actually deferred until Google has the resources ready to render the client side content.
Once resources are available, Google index the content further.
Googlebot waits
for available
resources to
render Javascript
by itself.

11. A new Step in the Search Process
Please welcome rendering budget
Crawl Index
(HTML)
Render Index
(JS+HTML)
SAME SAME NEW! NEW!

12. Despite this, we should not duck out but rather
embrace JavaScript
JS is not going away

13. Help us help you
JavaScript is an
opportunity
-Google

14. Rendering Budget
Google still has finite power
Google doesn’t have the time to render
everything, everyday
Google still needs to allocate
Allocation has changed though, because
the cost and complexity has changed

15. Helping Google
Google has trouble with rendering
Anything you can do to help will be
rewarded
Or risk not being competitive anymore
And of what you already know about Crawl
Budget applies to Rendering Budget.

16. Analyzing your JavaScript
There’s nothing new under the sun.
You already know how to do that!

17. Page Delay
We counted the time for the
server to send the page
Time to Interactive
BEFORE:
We count the time for the
crawler to render the page
NOW:

18. Crawl Ratio
We parsed log files and
counted unique urls
Render Ratio
BEFORE:
We parse log files and count
unique referrers
...from googlebot, on API calls
NOW:

19. Test a URL & content that should be Indexed
Search in Google:
site:<your_url> <unique phrase>

20. We know what important LINKS and CONTENT are rendered client-side
Can we find this content in Google’s index?
Is it relevant to assess JavaScript rendering for this website ?
Is Google indexing my important content?
TEST:
site:https://reverb.com/marketplace/
acoustic-guitars "Martin D-18
Dreadnought Acoustic Guitar"
RESULTS:
Googlebot fetched the JS
The rendering was DONE
The 2-pass indexing is OK

21. Find an expression that is always there in the
HTML-only version: "used from"
What proportion of my pages are rendered?

22. Find an expression that is always there in the
rendered version (but not in HTML-only):
"Make an Offer"
What proportion of my pages are rendered?

23. Use site: in Google with the first expression, then the second, to get the number of indexed results and
calculate the rendering ratio.
+
What proportion of my pages are rendered?
79% 2-pass indexed (rendered)13,700 / 17,400 =

24. SEO.php
We added server-side
SEO tracking
SEO.js
BEFORE:
We add crawler-side
JS SEO tracking
NOW:

25. SEO.php Simple Example
...Before logfile analysis, we all used this trick to monitor SEs crawls.

26. SEO.js Simple Example
Use Google’s Index to understand its passage, and fetch vs. render.

27. Staying on top of your JavaScript SEO
Preparing for the future

28. STEP 1
Have a coffee with your developers
SHARING IS CARING
Set up regular discussions
Stay informed of their roadmap
Have dedicated points of contact
Help them understand your issues - most are unaware
Your devs are your best allies in this endeavour

29. STEP 2
Make your own targets
MONITOR THE RIGHT KPIS
Identify the right resources to allow/exclude:
Timing - FCP, TTI
Quality - Size, N* of requests
Metrics - Lighthouse scores
Start with regular manual monitoring - automate it if possible

30. STEP 3
Invest in a platform for JS
optimisation at scale

31. Crawl Budget is (almost) dead, long live Rendering Budget
Budgets are here to stay
You need to optimise for them
The method has not changed
Key Takeaways
Visit our Booth to discuss!