Too Long; Didn’t Render - The State of JS and HTML Indexing | Digital Growth Unleashed 2019

Onely
OnelyOnely
Bartosz Góralewicz
@bart_goralewicz
HTML HTMLHTML
Everything is JS
We found out that
for non-JavaScript/HTML
websites.
is a problem
SEO and post factum learning
THE PROBLEM of 2019
CHANGE
Medium.com
Case Study
The
irony
v
v
v
v
v v
v
v
v v
v
v
v v
v
v
Timeframe (of JS indexing)
or the lack thereof
Even after… Months of
publishing content.
It is 2019 and our
research shows that
there are hundreds
of thousands of
domains not fully
indexed.
How rendering works with Google
So, you look at the difference
between the initial HTML, and,
then, if after rendering you
see extra content?
Yeah.
There are certain heuristics, that,
if we see after a while, like, oh, this
page, actually, the renderer does not
diff as much or doesn’t diff, it looks
the way before
The Heuristics
I still haven’t fully, like, grasped
what exactly triggers the
heuristics
are FAR from
perfect though.
…those heuristics
Every website, when we see
them for the first time, goes
to rendering.
ALL NEW SITES
GET RENDERED
But what is
a new website exactly?
What is a new website?
We decided to experiment
with how good Google is
with their heuristics.
Our own JS
Budget
Experiment
(2019)
Crawler
budget
experiment
2019
LET’S GO BACK TO 2017
Crawler budget experiment 2017
Crawler budget experiment 2019
GOOGLE
is rendering and indexing
new JS websites
without any issues
The State of JavaScript Indexing
Experiment 2019
National Geographic - with JS
v
National Geographic - NO JS
v
National Geographic - NO JS
v
100%of JS content indexed
ASOS - with JS
ASOS - NO JS
ASOS - NO JS
100%of JS content indexed
But not every website
is lucky enough
82%
0%
0%
0%
42%
73%
Percentage
of
JavaScript
content
indexed
Let’s compare H&M’s crawl
with and without JS rendering
Let’s compare H&M’s crawl
with and without JS rendering
What usually relies on JavaScript?
01
Pagination
02
“You might be also
interested in” (internal linking)
03
Top products
04
Reviews
05
Comments
06
Main content (rarely)
Web development evolved. Most websites use ready-to-go components
Slider
Faceted navigation
Related Products
Comments
Main content
Menu
The roots of the problem
Slider
Faceted navigation
Related Products
Comments
Main content
Menu
So when Google isn’t rendering your website,
it may see…well, this:
2 completely different websites
and 2 different website graphs.
Rendering delay leads to
Category Ladies/Men/DividedCategory Category
H&M.comHOMEPAGE
Sub-Category Sub-Category Sub-Category Jeans
Products Products Products Products Shaping Skinny High Jeans
JavaScript
Category Ladies/Men/DividedCategory Category
H&M.comHOMEPAGE
Sub-Category Sub-Category Sub-Category Jeans
Products Products Products Products Shaping Skinny High Jeans
JavaScript
2 WAVES
O F I N D E X I N G
TIMEFRAME
Percentage of
JavaScript
content
indexed
after 14 days
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
100%
96%
70%
34%
0,5%
New York Post Eventbrite Target The Guardian CNBC
Blame Google or say that JS is evil
Blame Google or say that JS is evil
Every JavaScript SEO issue we saw
was 100% self-induced
How to avoid those issues?
Enter - OMFG
When we realized how serious
this issue is, we found out that
there are no tools to diagnose
this problem.
Onely Made for Geeks
– free toolsetOMFG www.onely.com/tools
www.onely.com/tools
WWJD - What Would JavaScript Do
www.onely.com/tools
WWJD - What Would JavaScript Do
WWJD – BBC & meta tags vs. rendering
WWJD – BBC & meta tags vs. rendering
WWJD – BBC & meta tags vs. rendering
WWJD – BBC & meta tags vs. rendering
WWJD – Links added by JS
WWJD – Links removed by JS
TL;DR – Too Long; Didn’t Render
Too Long; Didn’t Render - The State of JS and HTML Indexing | Digital Growth Unleashed 2019
Too Long; Didn’t Render - The State of JS and HTML Indexing | Digital Growth Unleashed 2019
TGIF - The Google Indexing Forecast
HTML vs. JavaScript indexing
HTML vs. JavaScript indexing
A LOT of cool FREE tools coming soon!
JSLet's talk about
HTML
But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
Number of pages indexed after 1 day:
98.5%
But other parts of technical SEO are important too.
JavaScript SEO is important.
Sample of
1300 URLs
100%
Number of pages indexed after 1 day:
98.5%
What about
the rest?
Some websites
aren't as lucky as The Guardian.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%
Parts of
your website
depend
on JS
01
Webmaster updates
with new products
Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links
Parts of
your website
depend
on JS
01
Webmaster updates
with new products
02
Google crawls your
website without
seeing all the links
03
Only crawls a part of the
domain without finding
products (valuable content)
05
Budget is too
low to render JS
04
Google indexer is
confused - crawler
budget falls
Which leads to this…
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Indexed after 1 day Indexed after 2 days Indexed after 1 week Indexed after 2 weeks
The Guardian Target Reuters Eventbrite
98.5%
3.5%
68.5%
32%
99.9%
33%
88%
47%
99.9%
78%
91%
57%
99.9%
84%
91%
58%
TO DO:
Go to onely.com/tools
and check your website
(100% free)
Crawl your website with
and without JS and
compare two crawls
More data is coming soon
Cloaking in 2019
www.onely.com
@bart_goralewicz
THANK YOU
1 of 73

More Related Content

Similar to Too Long; Didn’t Render - The State of JS and HTML Indexing | Digital Growth Unleashed 2019(20)

Www snapdeal com-reportWww snapdeal com-report
Www snapdeal com-report
MahipSingh1379 views
How to  Make React SEO-friendlyHow to  Make React SEO-friendly
How to Make React SEO-friendly
ultroNeous Technologies | Best Web App Development Company18 views
The Web Wonder In 40 MinutesThe Web Wonder In 40 Minutes
The Web Wonder In 40 Minutes
Great Marketing Works2.2K views
Basic SEO Structural IssuesBasic SEO Structural Issues
Basic SEO Structural Issues
R2integrated490 views
About SEMAbout SEM
About SEM
Yco Van Nieuwenhove23 views
Mark Munroe - Advanced Search Summit Napa 2019Mark Munroe - Advanced Search Summit Napa 2019
Mark Munroe - Advanced Search Summit Napa 2019
Digital Marketers Organization117 views
Google Tag Manager for EcommerceGoogle Tag Manager for Ecommerce
Google Tag Manager for Ecommerce
Daytodayebay396 views

Recently uploaded(20)

Too Long; Didn’t Render - The State of JS and HTML Indexing | Digital Growth Unleashed 2019