Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Too Long; Didn’t Render - The State of JS and HTML Indexing

1,830 views

Published on

This is the deck as presented at Digital Growth Unleashed on Wednesday, October 16, 2019.

Bartosz Goralewicz’s JavaScript experiments and research have not only shaped JavaScript SEO as a whole but changed our understanding of how Google works. But his most recent research has revealed a widespread problem that goes beyond JS websites: there are hundreds of thousands of domains with unindexed HTML. With his cutting-edge data, Bartosz breaks down why this is happening, why rendering will be an important factor in the 2020 SEO landscape, and how this will have profound implications for large e-commerce websites, major news websites, and even your favorite little pizza shop’s website.

Published in: Internet
  • You can hardly find a student who enjoys writing a college papers. Among all the other tasks they get assigned in college, writing essays is one of the most difficult assignments. Fortunately for students, there are many offers nowadays which help to make this process easier. The best service which can help you is ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! I can recommend a site that has helped me. It's called ⇒ www.WritePaper.info ⇐ So make sure to check it out!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Too Long; Didn’t Render - The State of JS and HTML Indexing

  1. 1. Bartosz Góralewicz @bart_goralewicz
  2. 2. HTML HTMLHTML Everything is JS
  3. 3. We found out that for non-JavaScript/HTML websites. is a problem
  4. 4. SEO and post factum learning
  5. 5. THE PROBLEM of 2019 CHANGE
  6. 6. Medium.com Case Study
  7. 7. The irony
  8. 8. v v
  9. 9. v v v v v v
  10. 10. v v
  11. 11. v v v v v v
  12. 12. Timeframe (of JS indexing) or the lack thereof
  13. 13. Even after… Months of publishing content. It is 2019 and our research shows that there are hundreds of thousands of domains not fully indexed.
  14. 14. How rendering works with Google So, you look at the difference between the initial HTML, and, then, if after rendering you see extra content? Yeah.
  15. 15. There are certain heuristics, that, if we see after a while, like, oh, this page, actually, the renderer does not diff as much or doesn’t diff, it looks the way before The Heuristics I still haven’t fully, like, grasped what exactly triggers the heuristics
  16. 16. are FAR from perfect though. …those heuristics
  17. 17. Every website, when we see them for the first time, goes to rendering. ALL NEW SITES GET RENDERED
  18. 18. But what is a new website exactly? What is a new website?
  19. 19. We decided to experiment with how good Google is with their heuristics. Our own JS Budget Experiment (2019) Crawler budget experiment 2019
  20. 20. LET’S GO BACK TO 2017
  21. 21. Crawler budget experiment 2017
  22. 22. Crawler budget experiment 2019
  23. 23. GOOGLE is rendering and indexing new JS websites without any issues
  24. 24. The State of JavaScript Indexing Experiment 2019
  25. 25. National Geographic - with JS v
  26. 26. National Geographic - NO JS v
  27. 27. National Geographic - NO JS v 100%of JS content indexed
  28. 28. ASOS - with JS
  29. 29. ASOS - NO JS
  30. 30. ASOS - NO JS 100%of JS content indexed
  31. 31. But not every website is lucky enough
  32. 32. 82% 0% 0% 0% 42% 73% Percentage of JavaScript content indexed
  33. 33. Let’s compare H&M’s crawl with and without JS rendering
  34. 34. Let’s compare H&M’s crawl with and without JS rendering
  35. 35. What usually relies on JavaScript? 01 Pagination 02 “You might be also interested in” (internal linking) 03 Top products 04 Reviews 05 Comments 06 Main content (rarely)
  36. 36. Web development evolved. Most websites use ready-to-go components Slider Faceted navigation Related Products Comments Main content Menu The roots of the problem
  37. 37. Slider Faceted navigation Related Products Comments Main content Menu So when Google isn’t rendering your website, it may see…well, this:
  38. 38. 2 completely different websites and 2 different website graphs. Rendering delay leads to
  39. 39. Category Ladies/Men/DividedCategory Category H&M.comHOMEPAGE Sub-Category Sub-Category Sub-Category Jeans Products Products Products Products Shaping Skinny High Jeans JavaScript
  40. 40. Category Ladies/Men/DividedCategory Category H&M.comHOMEPAGE Sub-Category Sub-Category Sub-Category Jeans Products Products Products Products Shaping Skinny High Jeans JavaScript
  41. 41. 2 WAVES O F I N D E X I N G TIMEFRAME
  42. 42. Percentage of JavaScript content indexed after 14 days 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 100% 96% 70% 34% 0,5% New York Post Eventbrite Target The Guardian CNBC
  43. 43. Blame Google or say that JS is evil
  44. 44. Blame Google or say that JS is evil Every JavaScript SEO issue we saw was 100% self-induced
  45. 45. How to avoid those issues? Enter - OMFG When we realized how serious this issue is, we found out that there are no tools to diagnose this problem.
  46. 46. Onely Made for Geeks – free toolsetOMFG www.onely.com/tools
  47. 47. www.onely.com/tools WWJD - What Would JavaScript Do www.onely.com/tools
  48. 48. WWJD - What Would JavaScript Do
  49. 49. WWJD – BBC & meta tags vs. rendering
  50. 50. WWJD – BBC & meta tags vs. rendering
  51. 51. WWJD – BBC & meta tags vs. rendering
  52. 52. WWJD – BBC & meta tags vs. rendering
  53. 53. WWJD – Links added by JS
  54. 54. WWJD – Links removed by JS
  55. 55. TL;DR – Too Long; Didn’t Render
  56. 56. TGIF - The Google Indexing Forecast
  57. 57. HTML vs. JavaScript indexing
  58. 58. HTML vs. JavaScript indexing
  59. 59. A LOT of cool FREE tools coming soon!
  60. 60. JSLet's talk about HTML
  61. 61. But other parts of technical SEO are important too. JavaScript SEO is important. Sample of 1300 URLs 100%
  62. 62. But other parts of technical SEO are important too. JavaScript SEO is important. Sample of 1300 URLs 100% Number of pages indexed after 1 day: 98.5%
  63. 63. But other parts of technical SEO are important too. JavaScript SEO is important. Sample of 1300 URLs 100% Number of pages indexed after 1 day: 98.5% What about the rest?
  64. 64. Some websites aren't as lucky as The Guardian. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks The Guardian Target Reuters Eventbrite 98.5% 3.5% 68.5% 32% 99.9% 33% 88% 47% 99.9% 78% 91% 57% 99.9% 84% 91% 58%
  65. 65. Parts of your website depend on JS 01 Webmaster updates with new products
  66. 66. Parts of your website depend on JS 01 Webmaster updates with new products 02 Google crawls your website without seeing all the links
  67. 67. Parts of your website depend on JS 01 Webmaster updates with new products 02 Google crawls your website without seeing all the links 03 Only crawls a part of the domain without finding products (valuable content) 05 Budget is too low to render JS 04 Google indexer is confused - crawler budget falls
  68. 68. Which leads to this… 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks The Guardian Target Reuters Eventbrite 98.5% 3.5% 68.5% 32% 99.9% 33% 88% 47% 99.9% 78% 91% 57% 99.9% 84% 91% 58%
  69. 69. TO DO: Go to onely.com/tools and check your website (100% free) Crawl your website with and without JS and compare two crawls
  70. 70. More data is coming soon
  71. 71. Cloaking in 2019 www.onely.com @bart_goralewicz THANK YOU

×