Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rendering: Or why your perfectly optimized content doesn't rank

It's a hard fact that Google cannot index what it cannot render. We're not talking about the efficacy of content hidden in an accordion tab or behind a read more. This talk is a hands-on look at how to identify rendering issues, best practices, and tactics for improving.
Join this talk to understand:
1. How rendering works.
2. How to spot rendering issues
3. Happy paths for effective rendering
4. Tactics for iteratively improving rendering by collaborating with developers

Tune in to Jamie's session and get ready to give your rankings a serious boost!

  • Be the first to comment

  • Be the first to like this

Rendering: Or why your perfectly optimized content doesn't rank

  1. 1. Rendering: Or why your perfectly optimized content doesn't rank
  2. 2. Websites are made of code. Code is written in languages. 3 languages are the foundation of websites.
  3. 3. HTML Content CSS layout, design, & visual effects JavaScript web version of programming code
  4. 4. How it is then so many of our optimization efforts slips between the cracks?
  5. 5. 75% of product pages for an ecommerce site were not indexed in a case study by @TomkeRudzki
  6. 6. Rendering
  7. 7. “When we crawl your page with Googlebot, we go fetch the content and then we give it to chrome. Then Chrome runs all the scripts. It loads additional content. Once everything's loaded we take a snapshot of the page and that's the content that actually gets indexed.” - Erik Hendriks, Software Engineer at Google Rendering (WMConf MTV '19)
  8. 8. Initial HTML Rendered HTML Rendering
  9. 9. The Mechanics of Google's Web Rendering Service
  10. 10. Google uses Chromium as its web rendering service.
  11. 11. =
  12. 12. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering Engine
  13. 13. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering Engine Chromium Headless Browser
  14. 14. Chromium, headless browser ● Headless means that there is no GUI (visual representation) ● Used to load web pages and extract metadata ● reading from and writing to the DOM ● observing network events ● capturing screenshots ● inspecting worker scripts ● recording Chrome Traces
  15. 15. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering Engine Blink Browser Engine
  16. 16. Blink, browser engine ● Allows for querying and manipulating the rendering engine settings (ex: mobile vs. desktop) ● responsible for 2 major elements: 1. Memory heap: stores the result of script execution (Memory Heap results are added to DOM.) 2. Call stack: queue of sequential next steps (Each entry in the call stack is called a Stack Frame.) Local storage and Session storage are key-value pairs that can store any JS objects and functions in the browser These keys are a weak point in your rendering offense against a stateless Googlebot.
  17. 17. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering EngineV8 Rendering Engine
  18. 18. V8, JavaScript engine Each thread will runs through a process of: 1. Loading 2. Parsing 3. Compiling 4. Executing ● open-source JavaScript engine and WebAssembly engine ● developed by Google & The Chromium Project ● Use in Node.js, Google Chrome, and Chromium web browsers
  19. 19. Let’s look at how Google renders.
  20. 20. 1. A URL is pulled from the crawl queue Crawl Queue Crawler
  21. 21. 2. The URI is requested using the Googlebot user-agent Crawl Queue Crawler URI
  22. 22. 3. Read HTTP response and headers Crawl Queue Crawler URI Q. Does the thing I asked for exist? A. HTTP Status Codes Q. Anything I should know before looking at this? A. Cache-Control, and Directives
  23. 23. 4. Download and parse the HTML Crawl Queue Crawler Initial HTML Use View Page Source to see the initial HTML Crawled DOM
  24. 24. 5. The Crawled DOM is passed to the index processor Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML
  25. 25. 6. The processor extracts resources links from the crawled DOM and places them in the crawl queue Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML
  26. 26. 7. The index processor passes valuable content to the index Crawl Queue Crawler Crawled DOM Index Processor Initial HTML Index Resources
  27. 27. Crawler 8. Once resources are crawled, the page queues for rendering Crawl Queue Scripts from Crawled DOM
  28. 28. Crawler Crawled DOM Render 9. When resources become available, the URI moves from the render queue to the renderer Crawl Queue Resources
  29. 29. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML
  30. 30. Crawler Crawled DOM Render Rendered DOM 10. Renderer passes the rendered HTML to the index processor Crawl Queue Resources Index Processor
  31. 31. Crawler Crawled DOM Render Rendered DOM 11. The processor extracts resources links from the Rendered DOM and places them in the crawl queue Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources
  32. 32. Crawler Crawled DOM Render Rendered DOM 12. The index processor passes valuable rendered content to the index Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources Index
  33. 33. Great! Now let's break it!
  34. 34. 1. A URL is pulled from the crawl queue Crawl Queue Crawler Create a SPA that doesn't update URLs with history.pushState()
  35. 35. 2. The URI is requested using the Googlebot user-agent Crawl Queue Crawler URI Forget to pay your prerendering service
  36. 36. 3. Read HTTP response and headers Crawl Queue Crawler URI Use whichever status code whenever
  37. 37. 3. Read HTTP response and headers Crawl Queue Crawler URI Contradictory signals
  38. 38. 4. Download and parse the HTML Crawl Queue Crawler Initial HTML Store content in the ShadowDOM <!doctype html> <head> <script type="text/javascript" src="/main.js?ver=1.1.0"> </script> </head> <body> <app-root></app-root> </body> </html>
  39. 39. 5. The Crawled DOM is passed to the index processor Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML
  40. 40. 6. The processor extracts resources links from the crawled DOM and places them in the crawl queue Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML <a href="/good link">Will be crawled</a> <span onclick="changePage('bad-link')">Not crawled</span> <a onclick="changePage('bad-link')">Not crawled</a> <a href="/good-link" onclick="changePage ('good-link')">Will be crawled</a> Undiscoverable links
  41. 41. 7. The index processor passes valuable content to the index Crawl Queue Crawler Crawled DOM Index Processor Initial HTML Index Resources Don't answer user intent with the Crawled DOM
  42. 42. Crawler 8. Once resources are crawled, the page queues for rendering Crawl Queue Scripts from Crawled DOM Blocked Resources / CORS Errors
  43. 43. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML iframes, SPOFs in <head> / Render blocking resources
  44. 44. 3 types of rendering-blocking resources A <script> tag that: ● Is in the <head> of the document. ● Does not have a defer attribute. ● Does not have an async attribute. A <link rel="import"> tag that: ● Does not have an async attribute. A <link rel="import"> tag that: ● Does not have an async attribute.
  45. 45. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Look for knock-on effects
  46. 46. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Don't prioritize resources
  47. 47. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Dependencies in the critical rendering path
  48. 48. JQuery script1.js script2.js script3.js script4.js
  49. 49. JQuery script1.js script2.js script3.js script4.js script1.js {Product name} {Product sku} {Product price}
  50. 50. JQuery script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs}
  51. 51. JQuery script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs}
  52. 52. JQuery script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs}
  53. 53. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Rely on Client-Side Rendering <!doctype html> <head> <script type="text/javascript" src="/main.js?ver=1.1.0"> </script> </head> <body> <app-root></app-root> </body> </html>
  54. 54. <!doctype html> <head> <script type="text/javascript" src="/main.js?ver=1.1.0"> </script> </head> <body> <app-root></app-root> </body> </html> script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs} Each script has a probability to fail.
  55. 55. JQuery script1.js script2.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} script3.js {Product description} {Product images} {Product specs}
  56. 56. <rel="buyproduct.js"> = 99.5% success rate
  57. 57. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Ship every script on every request
  58. 58. Everyone has to wait while JS does it’s thing.
  59. 59. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Ignore performance
  60. 60. ● Google will limit CPU consumption ● Google will interrupt scripts ● Excessive CPU consumption negatively impacts indexing
  61. 61. Crawler Crawled DOM Render Rendered DOM 10. Renderer passes the rendered HTML to the index processor Crawl Queue Resources Index Processor Require permissions
  62. 62. Crawler Crawled DOM Render Rendered DOM 11. The processor extracts resources links from the Rendered DOM and places them in the crawl queue Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources Wreck your crawl budget with indexable API endpoints
  63. 63. Crawler Crawled DOM Render Rendered DOM 12. The index processor passes valuable rendered content to the index Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources Index
  64. 64. Let's make it better
  65. 65. ● Visibility Can I see and reproduce what ishappening? SEO for JavaScript frameworks requires
  66. 66. ● Visibility Can I see and reproduce what is happening? ● Diagnostics Can I measure what is happening? SEO for JavaScript frameworks requires
  67. 67. ● Visibility Can I see and reproduce what is happening? ● Diagnostics Can I measure what is happening? ● Iteration Can take meaningful steps to improve it? SEO for JavaScript frameworks requires
  68. 68. / / / '._ (_/) _.'/ |.''._ '--(o.o)--'_ .''.| _ / `;=/ " =;` _/ `__| ___/ |__/` (_|_)/ " ` " / | ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄| WEAR A MASK AND BE KIND TO EACH OTHER |__________|
  69. 69. Thank you! Any questions? @Jammer_Volts

×