#loganalysis at #DTDConf by @aleyda from @orainti
Discovering SEO
Opportunities through
Log Analysis

#loganalysis at #DTDConf by @aleyda from @orainti
“SEO crawling simulations
are great, but there’s
nothing like seeing your
site “real” search crawling
behaviour”
#loganalysis at #DTDConf by @aleyda from @orainti

#loganalysis at #DTDConf by @aleyda from @orainti
This can be done w/ Web server log files, that record
all requests, including search bots accesses and errors
https://www.portent.com/blog/analytics/how-to-read-a-web-site-log-file.htm

#loganalysis at #DTDConf by @aleyda from @orainti
They’re critical to monitor Web activity - Eg: Structural
changes or validating GSC errors you can’t replicate
You can’t find these URLs in your own crawls

#loganalysis at #DTDConf by @aleyda from @orainti
I’m (sadly) not Morpheus, however today I’m going to
show you what’s beyond search crawling simulations
Aleyda Solis
* International SEO Consultant & Founder.
* Conference Speaker in +100 Events in +20 Countries.
* Author “SEO. Las Claves Esenciales.”
* Blogger in SEL & SEJ.
#loganalysis at #DTDConf by @aleyda from @orainti

#loganalysis at #DTDConf by @aleyda from @orainti
Getting initial access will be the hardest part: You will
need to coordinate with your Server Admin or DevOps
Tip: Become their BFF to make your SEO life easier

#loganalysis at #DTDConf by @aleyda from @orainti
You can use the terminal to access them
directly from your Web server
Command line in Terminal

#loganalysis at #DTDConf by @aleyda from @orainti
You can also export & go through the data with a code
editor, however this might be unpractical for analysis
Sublime or Brackets

#loganalysis at #DTDConf by @aleyda from @orainti
For “manual” analysis you can also import it into Excel,
that will facilitate data organisation and filtering
https://builtvisible.com/log-file-analysis/

#loganalysis at #DTDConf by @aleyda from @orainti
To make analysis practical for larger sites, there are
log monitoring solutions with dashboards & filtering
Use Loggly or Splunk

#loganalysis at #DTDConf by @aleyda from @orainti
And now also SEO focused log analysers that directly
offer reports and segments to facilitate SEO analysis
Use Screaming Frog Log Analyser, OnCrawl

#loganalysis at #DTDConf by @aleyda from @orainti
Some of them can be directly integrated to the
sites whether with dedicated or shared hosting
SEOlyzer

#loganalysis at #DTDConf by @aleyda from @oraintiDeepcrawl
You can also use SEO crawlers integrating log files
data along other data sources to show existing gaps

#loganalysis at #DTDConf by @aleyda from @orainti
Let’s see some of the most
critical SEO insights we
can get through log
analysis by asking certain
questions…
#loganalysis at #DTDConf by @aleyda from @orainti

#loganalysis at #DTDConf by @aleyda from @orainti
Are search bots effectively accessing & frequently
crawling your most important pages to index & rank?

#loganalysis at #DTDConf by @aleyda from @orainti
Do the same by checking which are your most
crawled site areas and categories

#loganalysis at #DTDConf by @aleyda from @orainti
SEO log analysers will also allow to specify any filtering
criteria to obtain crawl behaviour for any site section

#loganalysis at #DTDConf by @aleyda from @orainti
What’s the share of your crawl activity
going to error or redirected URLs?

#loganalysis at #DTDConf by @aleyda from @orainti
Are your meaningful pages returning a 200 OK HTTP
status or are they showing errors?

#loganalysis at #DTDConf by @aleyda from @orainti
Are you returning any unintended redirects? To which
bots? Is this behaviour consistent over time?

#loganalysis at #DTDConf by @aleyda from @orainti
Are you returning different HTTP status
for the same URLs?

#loganalysis at #DTDConf by @aleyda from @orainti
Are there any gaps between your own crawlings
vs. the log file data? Are there any orphan URLs?

#loganalysis at #DTDConf by @aleyda from @orainti
Is your crawl budget wasted in not meant to be
accessed, non-relevant resources or non-indexed URLs?

#loganalysis at #DTDConf by @aleyda from @orainti
Which are the largest and slowest crawled pages?
Can they be optimised for speed?

#loganalysis at #DTDConf by @aleyda from @orainti
Do you have non-search engines bots consuming
your resources that you need to delay or block?

#loganalysis at #DTDConf by @aleyda from @orainti
Be mindful of those “spoofing” search crawlers
which you can validate too

#loganalysis at #DTDConf by @aleyda from @orainti
What's your desktop vs. smartphone googlebot crawl
share? Verify if you’re in the mobile first index

#loganalysis at #DTDConf by @aleyda from @orainti#loganalysis at #DTDConf by @aleyda from @orainti
And this is only the start

#loganalysis at #DTDConf by @aleyda from @orainti#loganalysis at #DTDConf by @aleyda from @orainti
Thanks!