Recommended
More Related Content
What's hot
What's hot (20)
Similar to Rendering: Or why your perfectly optimized content doesn't rank
Similar to Rendering: Or why your perfectly optimized content doesn't rank (20)
More from WeLoveSEO
More from WeLoveSEO (7)
Recently uploaded
Recently uploaded (20)
Rendering: Or why your perfectly optimized content doesn't rank
- 1. Rendering: Or why your perfectly optimized content doesn't rank
- 2. Websites are made of code. Code is written in languages. 3 languages are the foundation of websites.
- 3. HTML Content CSS layout, design, & visual effects JavaScript web version of programming code
- 4. How it is then so many of our optimization efforts slips between the cracks?
- 5. 75% of product pages for an ecommerce site were not indexed in a case study by @TomkeRudzki
- 6. Rendering
- 7. “When we crawl your page with Googlebot, we go fetch the content and then we give it to chrome. Then Chrome runs all the scripts. It loads additional content. Once everything's loaded we take a snapshot of the page and that's the content that actually gets indexed.” - Erik Hendriks, Software Engineer at Google Rendering (WMConf MTV '19)
- 9. The Mechanics of Google's Web Rendering Service
- 10. Google uses Chromium as its web rendering service.
- 11. =
- 12. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering Engine
- 13. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering Engine Chromium Headless Browser
- 14. Chromium, headless browser ● Headless means that there is no GUI (visual representation) ● Used to load web pages and extract metadata ● reading from and writing to the DOM ● observing network events ● capturing screenshots ● inspecting worker scripts ● recording Chrome Traces
- 15. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering Engine Blink Browser Engine
- 16. Blink, browser engine ● Allows for querying and manipulating the rendering engine settings (ex: mobile vs. desktop) ● responsible for 2 major elements: 1. Memory heap: stores the result of script execution (Memory Heap results are added to DOM.) 2. Call stack: queue of sequential next steps (Each entry in the call stack is called a Stack Frame.) Local storage and Session storage are key-value pairs that can store any JS objects and functions in the browser These keys are a weak point in your rendering offense against a stateless Googlebot.
- 17. Web Rendering Service (WRS) Blink Browser Engine Ignition TurboFan Liftoff Display backend GoogleMagic Chromium Headless Browser V8 Rendering EngineV8 Rendering Engine
- 18. V8, JavaScript engine Each thread will runs through a process of: 1. Loading 2. Parsing 3. Compiling 4. Executing ● open-source JavaScript engine and WebAssembly engine ● developed by Google & The Chromium Project ● Use in Node.js, Google Chrome, and Chromium web browsers
- 19. Let’s look at how Google renders.
- 20. 1. A URL is pulled from the crawl queue Crawl Queue Crawler
- 21. 2. The URI is requested using the Googlebot user-agent Crawl Queue Crawler URI
- 22. 3. Read HTTP response and headers Crawl Queue Crawler URI Q. Does the thing I asked for exist? A. HTTP Status Codes Q. Anything I should know before looking at this? A. Cache-Control, and Directives
- 23. 4. Download and parse the HTML Crawl Queue Crawler Initial HTML Use View Page Source to see the initial HTML Crawled DOM
- 24. 5. The Crawled DOM is passed to the index processor Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML
- 25. 6. The processor extracts resources links from the crawled DOM and places them in the crawl queue Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML
- 26. 7. The index processor passes valuable content to the index Crawl Queue Crawler Crawled DOM Index Processor Initial HTML Index Resources
- 27. Crawler 8. Once resources are crawled, the page queues for rendering Crawl Queue Scripts from Crawled DOM
- 28. Crawler Crawled DOM Render 9. When resources become available, the URI moves from the render queue to the renderer Crawl Queue Resources
- 29. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML
- 30. Crawler Crawled DOM Render Rendered DOM 10. Renderer passes the rendered HTML to the index processor Crawl Queue Resources Index Processor
- 31. Crawler Crawled DOM Render Rendered DOM 11. The processor extracts resources links from the Rendered DOM and places them in the crawl queue Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources
- 32. Crawler Crawled DOM Render Rendered DOM 12. The index processor passes valuable rendered content to the index Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources Index
- 33. Great! Now let's break it!
- 34. 1. A URL is pulled from the crawl queue Crawl Queue Crawler Create a SPA that doesn't update URLs with history.pushState()
- 35. 2. The URI is requested using the Googlebot user-agent Crawl Queue Crawler URI Forget to pay your prerendering service
- 36. 3. Read HTTP response and headers Crawl Queue Crawler URI Use whichever status code whenever
- 37. 3. Read HTTP response and headers Crawl Queue Crawler URI Contradictory signals
- 38. 4. Download and parse the HTML Crawl Queue Crawler Initial HTML Store content in the ShadowDOM <!doctype html> <head> <script type="text/javascript" src="/main.js?ver=1.1.0"> </script> </head> <body> <app-root></app-root> </body> </html>
- 39. 5. The Crawled DOM is passed to the index processor Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML
- 40. 6. The processor extracts resources links from the crawled DOM and places them in the crawl queue Crawl Queue Crawler Resources Crawled DOM Index Processor Initial HTML <a href="/good link">Will be crawled</a> <span onclick="changePage('bad-link')">Not crawled</span> <a onclick="changePage('bad-link')">Not crawled</a> <a href="/good-link" onclick="changePage ('good-link')">Will be crawled</a> Undiscoverable links
- 41. 7. The index processor passes valuable content to the index Crawl Queue Crawler Crawled DOM Index Processor Initial HTML Index Resources Don't answer user intent with the Crawled DOM
- 42. Crawler 8. Once resources are crawled, the page queues for rendering Crawl Queue Scripts from Crawled DOM Blocked Resources / CORS Errors
- 43. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML iframes, SPOFs in <head> / Render blocking resources
- 44. 3 types of rendering-blocking resources A <script> tag that: ● Is in the <head> of the document. ● Does not have a defer attribute. ● Does not have an async attribute. A <link rel="import"> tag that: ● Does not have an async attribute. A <link rel="import"> tag that: ● Does not have an async attribute.
- 45. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Look for knock-on effects
- 46. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Don't prioritize resources
- 47. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Dependencies in the critical rendering path
- 49. JQuery script1.js script2.js script3.js script4.js script1.js {Product name} {Product sku} {Product price}
- 50. JQuery script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs}
- 51. JQuery script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs}
- 52. JQuery script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs}
- 53. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Rely on Client-Side Rendering <!doctype html> <head> <script type="text/javascript" src="/main.js?ver=1.1.0"> </script> </head> <body> <app-root></app-root> </body> </html>
- 54. <!doctype html> <head> <script type="text/javascript" src="/main.js?ver=1.1.0"> </script> </head> <body> <app-root></app-root> </body> </html> script1.js script2.js script3.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} {Product description} {Product images} {Product specs} Each script has a probability to fail.
- 55. JQuery script1.js script2.js script4.js {Product name} {Product sku} {Product price} {Product category} {Product availability} {Shipping times} {Breadcrumbs} script3.js {Product description} {Product images} {Product specs}
- 56. <rel="buyproduct.js"> = 99.5% success rate
- 57. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Ship every script on every request
- 58. Everyone has to wait while JS does it’s thing.
- 59. Crawler Crawled DOM Render Rendered DOM 9. WRS assembles the page using the Crawled DOM and resources Crawl Queue Resources Use Inspect to see rendered HTML Ignore performance
- 60. ● Google will limit CPU consumption ● Google will interrupt scripts ● Excessive CPU consumption negatively impacts indexing
- 61. Crawler Crawled DOM Render Rendered DOM 10. Renderer passes the rendered HTML to the index processor Crawl Queue Resources Index Processor Require permissions
- 62. Crawler Crawled DOM Render Rendered DOM 11. The processor extracts resources links from the Rendered DOM and places them in the crawl queue Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources Wreck your crawl budget with indexable API endpoints
- 63. Crawler Crawled DOM Render Rendered DOM 12. The index processor passes valuable rendered content to the index Crawl Queue Crawled DOM Resources Index Processor Rendered DOM Resources Index
- 65. ● Visibility Can I see and reproduce what ishappening? SEO for JavaScript frameworks requires
- 66. ● Visibility Can I see and reproduce what is happening? ● Diagnostics Can I measure what is happening? SEO for JavaScript frameworks requires
- 67. ● Visibility Can I see and reproduce what is happening? ● Diagnostics Can I measure what is happening? ● Iteration Can take meaningful steps to improve it? SEO for JavaScript frameworks requires
- 68. / / / '._ (_/) _.'/ |.''._ '--(o.o)--'_ .''.| _ / `;=/ " =;` _/ `__| ___/ |__/` (_|_)/ " ` " / |￣￣￣￣￣￣￣￣￣￣| WEAR A MASK AND BE KIND TO EACH OTHER |＿＿＿＿＿＿＿＿＿＿|