How to Improve Your Website's Indexation - Sean Butcher Brighton SEO Presentation from September 2018

How to improve your
website’s “indexation”?
@seanbutcher #BrightonSEO

How to improve your
website’s indexing

WTF does that mean?

Increase volume of
pages indexed

Increase speed of
pages indexed

Most importantly...

Ensuring most
important pages
are being indexed

Why?

Without indexing, your
web pages have no
way of ranking

Being indexed
does not mean it
will rank
https://www.seroundtable.com/confusing-ranking-with-indexing-in-google-seo-25173.html

Crawling

Crawling
V
Indexing

Crawling
V
Indexing
V
Ranking

Indexed = Page saved
in Google’s databases

“The Google Search index
contains hundreds of
billions of web pages and is
well over 100 million
gigabytes in size.”
https://www.google.com/search/howsearchworks/crawling-indexing/

We need to refine
what Google has
access to

“Google doesn't crawl all
the pages on the web,
and we don't index all
the pages we crawl.”
https://support.google.com/webmasters/answer/34441?hl=en

Key takeaway of
today...

#BrightonSEO@seanbutcher
Align your
indexing
signals

Some basic
examples...

No inclusion in XML
sitemap

Not linking to URL
enough on site

Internal links not
pointing to canonical

Internal links not
pointing to URL in
sitemap

We’d probably best
touch on...

Mobile
First
Index

Responsive:
No change if parity
achieved

Dynamic/Mobile Site:
Check for consistency
inc...

Navigation & internal
linking

Content & anchor text

Speed

Tags inc. meta,
structured data,
hreflang etc.

https://developers.google.
com/search/mobile-sites/
mobile-first-indexing

@seanbutcher
Checking
index
status
#BrightonSEO

Used to be pain in the
a**e

@seanbutcher #BrightonSEO#BrightonSEO@seanbutcher

Used to be very manual

Thankfully - now far
easier (& less manual)

URL inspection
tool

@seanbutcher #BrightonSEO#BrightonSEO@seanbutcher https://search.google.com/search-console/index

Index coverage
report

No longer required to
have a submitted XML
Sitemap

@seanbutcher
Fixing
current
issues
#BrightonSEO

Some of
the red
errors
include...

“Submitted URL
marked ‘noindex’”

Inconsistent signal:
Submitted = Index me
NoIndex = Don’t index
me

“Submitted URL
blocked by robots.txt”

Inconsistent signal:
Submitted = Index me
Robots.txt = Don’t
crawl me

“Submitted URL
seems to be a Soft
404”

Improve content

Return 404 server
response

“Submitted URL not
found (404)”

Remove from sitemap

Update old internal
links

“Redirect error”
(chains/loops)

Google doesn’t like
more than 3 hops -
Won’t follow more
than 5

Submit primary URL
in sitemap (i.e. not a
redirect)

Internally link to
correct URL

‘Warnings’
(ambers)

Only one warning
now flagged...

“Indexed, though
blocked by robots.txt”

@seanbutcher #BrightonSEOhttps://searchengineland.com/google-the-search-console-index-coverage-report-is-n
ow-more-accurate-303369

If intentionally
blocked, use NoIndex

@seanbutcher #BrightonSEOhttps://support.google.com/webmasters/answer/6062608

If unintentionally
blocked,
review/remove from
robots.txt

‘Valids’
(greens)

Not all rosy...

“Indexed, not
submitted in sitemap”

Review - add to
sitemap if important

“Indexed, consider
marking as canonical”

Canonicals define
primary URL

It does however
include...

#BrightonSEO
“Submitted
and
indexed”
@seanbutcher

‘Excluded’ URLs?
https://support.google.com/webmasters/answer/7440203?hl=en

“Submitted URL not
selected as canonical”

Use in conjunction
with URL Inspection
Tool

Validate issues when
fixed!
https://support.google.com/
webmasters/answer/74402
03?hl=en

@seanbutcher
Other
ideas for
improving
indexing
#BrightonSEO

Make use of Fetch &
Render

Test rendering if
using Javascript

Pages easily
accessible / logical
structure

External signals (links
& shares)

Use supported file
type

https://support.google.com/webmasters/answer/
35287?hl=en

Excluding
indexing?

“Ensuring most
important pages
are being indexed”

Of course - NoIndex
less important stuff

Large websites using
filters/facets

Make use of
NoIndex/Canonicals

Some “theoretical”
examples

“Televisions”

“50 inch
televisions”

“Panasonic
50 inch
televisions”

“Panasonic
50 inch
televisions
under £500”

Two main options:

1) NoIndex the page

Page still there for
users

Prevents issues with
‘thin’ content

2) Canonicalise back
‘up’ to ‘important’ level

Conduct keyword
research

● Televisions - 550,000 (0 filters)
● 50 inch televisions - 90,500 (1 filter)
● Panasonic 50 inch televisions -
6,600 (2 filters)
● Panasonic 50 inch televisions under
£500 - 0 (3 filters)

Provide signals of
importance

Duplication caused by
multiple filters

https://www.domain.co.uk/men/mens-footwear/trai
ners/brand/nike/size/8-10,8-11,8,7-8,8-
9/colour/black/

https://www.domain.co.uk/men/mens-footwear/t
rainers/brand/nike/colour/bl
ack/size/8-10,8-11,8,7-8,8-9/

1
2

@seanbutcher #BrightonSEO#BrightonSEO@seanbutcher https://www.seroundtable.com/google-noindex-rel-canonical-confusion-
26079.html

“DYK the
rel-canonical is a
hint, not a directive.
It’s a very strong hint,
but may still be
outweighed by other
signals.”
Gary Illyes - Mar 23rd 2017
@seanbutcher

Canonical = you want
referring URL indexed

NoIndex = you don’t
want URL indexed

Summary

Need to make it as
easy as possible for
Google

Provide aligned, clear
& consistent signals

Make use of the tools
available to you for
diagnosing issues

Exclude (refine) less
important content via
canonicals & NoIndex

https://www.slideshare
.net/seanbutcher1

Thanks for listening!

How to Improve Your Website's Indexation - Sean Butcher Brighton SEO Presentation from September 2018

How to Improve Your Website's Indexation - Sean Butcher Brighton SEO Presentation from September 2018

Recommended

More Related Content

What's hot

What's hot(20)

Similar to How to Improve Your Website's Indexation - Sean Butcher Brighton SEO Presentation from September 2018

Similar to How to Improve Your Website's Indexation - Sean Butcher Brighton SEO Presentation from September 2018(20)

Recently uploaded

Recently uploaded(20)

How to Improve Your Website's Indexation - Sean Butcher Brighton SEO Presentation from September 2018