Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget


Published on

Crawl Budget is (almost) dead long live Rendering Budget. Relive Botify's BirghtonSEO talk on rendering budget.

Published in: Internet

BrightonSEO 2019 - Crawl Budget is dead, please welcome Rendering Budget

  1. 1. Crawl Budget is dead, please welcome Rendering Budget Robin Eisenberg // Botify //
  2. 2. Javascript is complex and here to stay. Rendering problems ? Looks a lot like Crawl Budget to me What is already available to optimise your JS? Agenda
  3. 3. Crawl Budget Google has finite power Google doesn’t have the time to crawl everything Google needs to allocate What if Google doesn’t have enough time to crawl your whole website in 24 hours?
  4. 4. Crawl Budgets affect your bottom line Search engines are visibly allocating resources Direct correlation: page speed vs crawl volume You might not be showing your whole website!
  5. 5. The modern web relies on JavaScript Today is different
  6. 6. Render Budgets JavaScript is Complex JavaScript is Asynchronous JavaScript is Slow What if Google doesn’t have enough time to render your whole website in 24 hours?
  7. 7. We’ve seen disasters like these: JavaScript Release Drop Rollback to HTML 1 2 3 1 3 2
  8. 8. JavaScript complexities Content changes over the page’s lifecycle Rendering time is relative JavaScript is not easily readable/debuggable These difficulties apply to Google, not just you.
  9. 9. Javascript rendering resources JavaScript Crawl Cost Management Rendering is actually deferred until Google has the resources ready to render the client side content. Once resources are available, Google index the content further. Googlebot waits for available resources to render Javascript by itself.
  10. 10. A new Step in the Search Process Please welcome rendering budget Crawl Index (HTML) Render Index (JS+HTML) SAME SAME NEW! NEW!
  11. 11. Despite this, we should not duck out but rather embrace JavaScript JS is not going away
  12. 12. Help us help you JavaScript is an opportunity -Google
  13. 13. Rendering Budget Google still has finite power Google doesn’t have the time to render everything, everyday Google still needs to allocate Allocation has changed though, because the cost and complexity has changed
  14. 14. Helping Google Google has trouble with rendering Anything you can do to help will be rewarded Or risk not being competitive anymore And of what you already know about Crawl Budget applies to Rendering Budget.
  15. 15. Analyzing your JavaScript There’s nothing new under the sun. You already know how to do that!
  16. 16. Page Delay We counted the time for the server to send the page Time to Interactive BEFORE: We count the time for the crawler to render the page NOW:
  17. 17. Crawl Ratio We parsed log files and counted unique urls Render Ratio BEFORE: We parse log files and count unique referrers ...from googlebot, on API calls NOW:
  18. 18. Test a URL & content that should be Indexed Search in Google: site:<your_url> <unique phrase>
  19. 19. We know what important LINKS and CONTENT are rendered client-side Can we find this content in Google’s index? Is it relevant to assess JavaScript rendering for this website ? Is Google indexing my important content? TEST: site: acoustic-guitars "Martin D-18 Dreadnought Acoustic Guitar" RESULTS: Googlebot fetched the JS The rendering was DONE The 2-pass indexing is OK
  20. 20. Find an expression that is always there in the HTML-only version: "used from" What proportion of my pages are rendered?
  21. 21. Find an expression that is always there in the rendered version (but not in HTML-only): "Make an Offer" What proportion of my pages are rendered?
  22. 22. Use site: in Google with the first expression, then the second, to get the number of indexed results and calculate the rendering ratio. + What proportion of my pages are rendered? 79% 2-pass indexed (rendered)13,700 / 17,400 =
  23. 23. SEO.php We added server-side SEO tracking SEO.js BEFORE: We add crawler-side JS SEO tracking NOW:
  24. 24. SEO.php Simple Example ...Before logfile analysis, we all used this trick to monitor SEs crawls.
  25. 25. SEO.js Simple Example Use Google’s Index to understand its passage, and fetch vs. render.
  26. 26. Staying on top of your JavaScript SEO Preparing for the future
  27. 27. STEP 1 Have a coffee with your developers SHARING IS CARING Set up regular discussions Stay informed of their roadmap Have dedicated points of contact Help them understand your issues - most are unaware Your devs are your best allies in this endeavour
  28. 28. STEP 2 Make your own targets MONITOR THE RIGHT KPIS Identify the right resources to allow/exclude: Timing - FCP, TTI Quality - Size, N* of requests Metrics - Lighthouse scores Start with regular manual monitoring - automate it if possible
  29. 29. STEP 3 Invest in a platform for JS optimisation at scale
  30. 30. Crawl Budget is (almost) dead, long live Rendering Budget Budgets are here to stay You need to optimise for them The method has not changed Key Takeaways Visit our Booth to discuss!