This is Onely's Bartosz Goralewicz's deck "Too Long; Didn’t Render - The State of JS and HTML Indexing" as presented at Digital Growth Unleashed in London on October 16, 2019.

1. Bartosz Góralewicz
@bart_goralewicz

2. HTML HTMLHTML
Everything is JS

3. We found out that
for non-JavaScript/HTML
websites.
is a problem

4. SEO and post factum learning

5. THE PROBLEM of 2019
CHANGE

6. Medium.com
Case Study

7. The
irony

8. v
v

9. v
v
v v
v
v

10. v v

11. v
v
v v
v
v

12. Timeframe (of JS indexing)
or the lack thereof

13. Even after… Months of
publishing content.
It is 2019 and our
research shows that
there are hundreds
of thousands of
domains not fully
indexed.

14. How rendering works with Google
So, you look at the difference
between the initial HTML, and,
then, if after rendering you
see extra content?
Yeah.

15. There are certain heuristics, that,
if we see after a while, like, oh, this
page, actually, the renderer does not
diff as much or doesn’t diff, it looks
the way before
The Heuristics
I still haven’t fully, like, grasped
what exactly triggers the
heuristics

16. are FAR from
perfect though.
…those heuristics

17. Every website, when we see
them for the first time, goes
to rendering.
ALL NEW SITES
GET RENDERED

18. But what is
a new website exactly?
What is a new website?

19. We decided to experiment
with how good Google is
with their heuristics.
Our own JS
Budget
Experiment
(2019)
Crawler
budget
experiment
2019

20. LET’S GO BACK TO 2017

21. Crawler budget experiment 2017

22. Crawler budget experiment 2019

23. GOOGLE
is rendering and indexing
new JS websites
without any issues

24. The State of JavaScript Indexing
Experiment 2019

25. National Geographic - with JS
v

26. National Geographic - NO JS
v

27. National Geographic - NO JS
v
100%of JS content indexed

28. ASOS - with JS

29. ASOS - NO JS

30. ASOS - NO JS
100%of JS content indexed

31. But not every website
is lucky enough

32. 82%
0%
0%
0%
42%
73%
Percentage
of
JavaScript
content
indexed

33. Let’s compare H&M’s crawl
with and without JS rendering

34. Let’s compare H&M’s crawl
with and without JS rendering

35. What usually relies on JavaScript?
01
Pagination
02
“You might be also
interested in” (internal linking)
03
Top products
04
Reviews
05
Comments
06
Main content (rarely)

36. Web development evolved. Most websites use ready-to-go components
Slider
Faceted navigation
Related Products
Comments
Main content
Menu
The roots of the problem

37. Slider
Faceted navigation
Related Products
Comments
Main content
Menu
So when Google isn’t rendering your website,
it may see…well, this:

38. 2 completely different websites
and 2 different website graphs.
Rendering delay leads to

39. Category Ladies/Men/DividedCategory Category
H&M.comHOMEPAGE
Sub-Category Sub-Category Sub-Category Jeans
Products Products Products Products Shaping Skinny High Jeans
JavaScript

40. Category Ladies/Men/DividedCategory Category
H&M.comHOMEPAGE
Sub-Category Sub-Category Sub-Category Jeans
Products Products Products Products Shaping Skinny High Jeans
JavaScript

41. 2 WAVES
O F I N D E X I N G
TIMEFRAME

42. Percentage of
JavaScript
content
indexed
after 14 days
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
100%
96%
70%
34%
0,5%
New York Post Eventbrite Target The Guardian CNBC

43. Blame Google or say that JS is evil

44. Blame Google or say that JS is evil
Every JavaScript SEO issue we saw
was 100% self-induced

45. How to avoid those issues?
Enter - OMFG
When we realized how serious
this issue is, we found out that
there are no tools to diagnose
this problem.

46. Onely Made for Geeks
– free toolsetOMFG www.onely.com/tools

47. www.onely.com/tools
WWJD - What Would JavaScript Do
www.onely.com/tools

48. WWJD - What Would JavaScript Do

49. WWJD – BBC & meta tags vs. rendering

50. WWJD – BBC & meta tags vs. rendering

51. WWJD – BBC & meta tags vs. rendering

52. WWJD – BBC & meta tags vs. rendering

53. WWJD – Links added by JS

54. WWJD – Links removed by JS

55. TL;DR – Too Long; Didn’t Render

58. TGIF - The Google Indexing Forecast

59. HTML vs. JavaScript indexing

60. HTML vs. JavaScript indexing

61. A LOT of cool FREE tools coming soon!

62. JSLet's talk about
HTML

63. But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%

64. But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
Number of pages indexed after 1 day:
98.5%

65. But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
Number of pages indexed after 1 day:
98.5%
What about
the rest?

66. Some websites
aren't as lucky as The Guardian.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%

67. Parts of
your website
depend
on JS
01
Webmaster updates
with new products

68. Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links

69. Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links
03
Only crawls a part of the
domain without finding
products (valuable content)
05
Budget is too
low to render JS
04
Google indexer is
confused - crawler
budget falls

70. Which leads to this…
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%

71. TO DO:
Go to onely.com/tools
and check your website
(100% free)
Crawl your website with
and without JS and
compare two crawls

72. More data is coming soon

73. Cloaking in 2019
www.onely.com
@bart_goralewicz
THANK YOU