The New Renaissance of JavaScript - SMX Advanced 2019
•Download as PPTX, PDF•
1 like•112 views
Examples of two JavaScript-dependent websites and their SEO related challenges. This talk was presented at SMX Advanced 2019 in Seattle.
Recommended
More Related Content
Similar to The New Renaissance of JavaScript - SMX Advanced 2019
Similar to The New Renaissance of JavaScript - SMX Advanced 2019 (20)
Recently uploaded
Recently uploaded (20)
The New Renaissance of JavaScript - SMX Advanced 2019
- 1. @RobinRozhon
- 3. @RobinRozhon Grey.comWhite.com All content is server-side rendered with some JavaScript features.
- 4. @RobinRozhon Grey.com All content is loaded via JavaScript in the browser (SPA). White.com All content is server-side rendered with some JavaScript features.
- 5. @RobinRozhon Chrome 41 Latest Chromium
- 6. @RobinRozhon Crawl Index Render Instant, first wave of indexing New links to be crawled As rendering resources become available Second wave of indexing Source: Google
- 7. @RobinRozhon White.com All content is server-side rendered with some JavaScript features.
- 8. @RobinRozhon
- 9. @RobinRozhon Opacity is a CSS property that adds transparency to elements. body { opacity: 0; }
- 10. @RobinRozhon Opacity is a CSS property that adds transparency to elements. body { opacity: 1; }
- 11. @RobinRozhon Grey.com All content is loaded via JavaScript in the browser (SPA).
- 12. @RobinRozhon
- 14. @RobinRozhon Server Crawler Static HTML Source: Google Renderer Browser SPA SPA Dynamic Rendering
- 20. @RobinRozhon Server Render Static HTML Source: Google Renderer Browser SPA SPA Dynamic Rendering
- 22. @RobinRozhon MonitoringTesting Checking code updates before deploying to the production site.
- 23. @RobinRozhon Monitoring Checking the production site on a regular basis (i.e. daily). Testing Checking code updates before deploying to the production site.
- 24. @RobinRozhon Request URLs for each page type. Monitoring
- 25. @RobinRozhon Request URLs for each page type. Monitoring Ensure that JavaScript is disabled.
- 26. @RobinRozhon Request URLs for each page type. Monitoring Ensure that JavaScript is disabled. Set the user agent to Googlebot Smartphone.
- 27. @RobinRozhon Request URLs for each page type. Monitoring Ensure that JavaScript is disabled. Set the user agent to Googlebot Smartphone. Verify the values, not only if an element is present.
- 29. @RobinRozhon 62.7% 73.6% 99.6% Period 1 – Period 3 Initial state Period 4 – Period 7 Extending JS timeout Period 8 – Period 10 Allowing the site to load with cookies and storage turned off Rendering Success Rate
- 30. @RobinRozhon
- 31. @RobinRozhon
- 32. @RobinRozhon 62.7% 73.6% 99.6% Period 1 – Period 3 Initial state Period 4 – Period 7 Extending JS timeout Period 8 – Period 10 Allowing the site to load with cookies and storage turned off Rendering Success Rate
- 33. @RobinRozhon 62.7% 73.6% 99.6% Period 1 – Period 3 Initial state Period 4 – Period 7 Extending JS timeout Period 8 – Period 10 Allowing the site to load with cookies and storage turned off Rendering Success Rate
- 34. @RobinRozhon 62.7% 73.6% 99.6% Period 1 – Period 3 Initial state Period 4 – Period 7 Extending JS timeout Period 8 – Period 10 Allowing the site to load with cookies and storage turned off Rendering Success Rate
- 36. @RobinRozhon Detect search engines and social media crawlers.
- 37. @RobinRozhon Set up daily monitoring.
- 38. @RobinRozhon Watch rendering success rate over time.
- 39. @RobinRozhon Dynamic rendering is a workaround.
- 40. @RobinRozhon
Editor's Notes
- I don’t want to spend the next 30 minutes lecturing you about what you can read in Developer guides published by Google. I want to share my experiences. I want to show you examples of what I’ve encountered. I’ve heard on numerous occasions that JS is evil … pause … I’m not sure if I agree with that …. Pause … Yes, JS makes SEO more difficult but that’s simply how the Internet evolves. JS is here to stay so we need to learn how to deal with it. We need to make our websites accessible.
- I’d like to introduce our heroes for today.
- To do a quick testing, you can use your Chrome browser because Google upgraded its web rendering service to Evergreen Googlebot which is basically the latest version of Chromium (give and take a few weeks). The other tools such as Google URL Inspection tool still use Chrome 41. ES6 and newer JavaScript features IntersectionObserver for lazy-loading This means Google is able to more accurately replicate what the users are seeing on JavaScript websites than in the past but it doesn’t solve all the problems.
- Because of this diagram that we all have seen. Google has tremendously improved their JS rendering capabilities but other search engines and social media crawlers are still behind. Also, Google doesn't execute JS immediately, it's with a delay. The delay is usually on the order of days to a few weeks (John Mueller). Google has plans to render JS while crawling but there's no date for that. If your website changes often, waiting for the second wave of indexing may not be the best idea. (e.g. think of car dealerships, news websites, etc.) Also, only canonical tags available during the fist wave of indexing are considered by Google. Do you want to risk your revenue while hoping Google can render your website properly? I don’t! I want to make sure that search engines and other web crawlers don’t need to execute JS to understand the website. JS disabled – let’s go.
- We’re going to start with the white one
- This looks awful but actually it’s not that bad. All the content is there (server-side rendered) is there. We just can’t see it because the opacity of the entire body tag is set to 0.
- Once all the critical resources are loaded (fonts, images etc), the JS changes it to 1. It’s not ideal but it‘s not mission critical. The website still ranks very well and drives tons of traffic from search engines.
- Client side rendered SPA always means you will have some fun. What SPA is? A single-page application is a web application or web site that interacts with the user by dynamically rewriting the current page rather than loading entire new pages from a server Single-Page Applications (SPAs) are Web apps that load a single HTML page and dynamically update that page as the user interacts with the app. SPAs use AJAX and HTML5 to create fluid and responsive Web apps, without constant page reloads.
- Another beautiful page. You can see that I was very creative when coming up with the names for the sites. We don’t want to rely on Google to render JS. We want to serve static HTML to search engines and other crawlers. The solution used for this website is dynamic rendering.
- Dynamic rendering means switching between client-side rendered and pre-rendered content for specific user agents.
- These are three popular options. All of them use a headless Chrome. A headless browser is a web browser without a graphical user interface. They do what your browser does – Execute the JS into HTML
- User agent is a string that identifies who makes the request for a page. Google and Bing are two most logical choices
- If you care about international markets, you may want to consider likes of Yandex, Baidu, Naver or Seznam
- Don’t forget about social media crawlers from Facebook, Twitter, LinkedIn and others
- User agent detection done. The prerendering solution is up and running. You’re all set to receive a phenomenal visibility in search results! Is it party time? Not yet.
- Dynamic rendering is tricky because it’s hard to notice prerendering problems as long as the site is fully functional for users. How do you notice that something is wrong? Decrease of traffic from search engines? Drop in rankings? That already too late and the business is losing money! Before you label the implementation of dynamic rendering as finished, set automated monitoring of the rendering process.
- Let’s define those two terms to make sure we all think of them in the same way.
- Testing We could talk about unit tests, integration tests, end-to-end tests for quite long. Testing needs to be baked into your development process which is more challenging to do. Especially, if you work for an external agency. This is very useful stuff but let’s leave it for the next time.
- Monitoring You want to make sure that the rendering is checked on a daily basis. You can use third party tools or write your own script.
- Monitoring Test SEO elements against predefined rules. Don’t check only if an element is present in the code. Check if it includes the right value. It’s nice to know that meta robots tag is present in the code but it’s useless if you don’t know if the value is “noindex, follow” or “index, follow”. Testing and monitoring ensures you discover a problem before you’re feeling it’s impacts.
- Monitoring of the grey.com over time helped us discover problems with the prerendering process we wouldn’t have discovered because the site worked perfectly fine for the users. This tells me how often a page was successfully rendered. The higher the number is the happier I am.
- The rendering success rate was only 62.7%. Which means we returned static HTML to Googlebot only for 6 pages out of 10. The other 4 times we returned this…
- There was absolutely no content. It was just this empty layout. That wasn’t good at all.
- Timeline 62.7% success rate before implementing any fixes (Period 1 – Period 3)
- 73.6% success rate after extending JS timeout (Period 4 – Period 7)
- 99.6% success rate after allowing the site to load with cookies and storage turned off (Period 8 – Period 10) I want to use the monitoring to ensure rendering success rate stays this high and it doesn’t drop down.
- Verify that content that is supposed to be visible is actually visible.
- Use universal JS when possible.
- Don’t see JS as something that’s bad for a site.