Mining the SERPs for SEO,
Content & Customer
Insights
Rory Truesdale // Conductor
http://cndr.co/brighton
@RoryT11
About Me
Rory Truesdale
•SEO Strategist at Conductor
•EMEA SEO lead for WeWork
Get In Touch
brightonseo@conductor.com
@RoryT11
@RoryT11
Get The Slides
@RoryT11
http://cndr.co/brighton
•SERPs are a great resource to learn what Google
‘thinks’ our customers want
•Workflows that will help you understand the intent of
the people you want to reach
•How to use these insights to improve the quality of your
on-page optimisation
What To Expect
@RoryT1
That’s how often Google rewrites the SERP
displayed meta description
WHY?
To make
SEOs sad?
Just for a
laugh?
Nope…
It’s because Google thinks
it is smarter than us
Intriguing…
Can we use that to our advantage?
Yes, we
can!
(sorry, that was
the last puppy pic)
How?
@RoryT11
We can deconstruct &
analyse the language in
SERP displayed content
to learn what Google
thinks our customers
are interested in
@RoryT1
Curious?
This is important
because we are
in the age of
semantic search
@RoryT1
Google isn’t ranking a page based on how
it uses a keyword.
Google provides accurate results based on
intent, query context & word relationships.
On-page Optimisation
@RoryT1
• User intent
• Query context
•Topical relevance
• Word relationships
Target the keyword, but optimise for this.
On-page Optimisation
@RoryT1
Understand
customer intent
& desire to better
tailor your
messaging
@RoryT1
Structure landing
pages to help
Google understand
context & how it
meets the needs
of the searcher
@RoryT1
Build more
meaningful
online
experiences that
better convert
website visitors
@RoryT1
Your
Toolkit
@RoryT11
You need
SERP content
There are
three ways
you can get
this.
@RoryT1
Scrape at scale with
Screaming Frog
Follow these instructions
@RoryT1
Option
A
Option
B
Get SERP content via an
API
Option
C
Get SERP content using the
Scraper Chrome extension
Get
Scraper
There are four
ways you can
get this.
You need
Jupyter Notebook
What is
that?
The Jupyter Notebook is an open-source web
application that allows you to create and share
documents that contain live code, equations,
visualizations and narrative text. Uses include: data
cleaning and transformation, numerical simulation,
statistical modeling, data visualization, machine
learning, and much more.
Jupyter.org
@RoryT1
Stumped?
Me too…
Here’s my definition
Jupyter Notebook is an environment on my laptop
where I can learn Python by copying scripts created
by people significantly smarter than me and
breaking them or making them do something
slightly different.
RoryTruesdale
Python Charlatan
@RoryT1
Resources to get started…
Jupyter
Notebook –
Getting
Started Guide
Robin Lord
Find
scripts
Paul Shapiro JR Oakes Hamlet
Batista
Find
scripts
Find
scripts
You’ll end up
with…
@RoryT11
Your SERP content in a CSV@RoryT1
Imported into Jupyter Notebook
@RoryT1
You’re
ready to
use Python
to analyse
the SERPs!
@RoryT1
There’s a
treat for you.
I’ll share a link to a Dropbox
with everything you need to get
you started
@RoryT11
Before we
dive in…
Start by
cleaning
your SERP
content
@RoryT1
Lower case avoids
duplication &
punctuation adds
no value to this
analysis
Lower Case &
Remove
Punctuation
@RoryT1
Stop words are
commonly
occurring words
that don’t improve
our analysis
Remove
Stop
Words
@RoryT1
The process of
chopping up a
sentence into
individual pieces,
called ‘tokens’
Tokenization
@RoryT1
The process of
converting a word
to its root (i.e.
“playing” becomes
“play”)
Lemmatization
(optional)
@RoryT1
@RoryT11
@RoryT11
How many times
does a word or
combination of
words appear in
your SERP
content?
Co-occurrence
@RoryT1
Co-
occurrence
Snapshot of
phrases
frequently
occurring in
the SERPs
@RoryT1
Co-
occurrence
Demonstrates
the topics
competitors
cover on landing
pages
@RoryT1
Co-
occurrence
Understand the
types of phrases
that Google sees
as semantically
relevant to a target
keyword set
•Additional source of data for keyword research
•Identify topical content gaps on landing pages
•Optimise landing page content by incorporating
semantically relevant phrases
HOW CAN WE APPLY THIS?
@RoryT1
Cost:
Range:
Time to Charge:
Battery Size/Capacity:
All Wheel Drive:
Towing Capacity:
Semi-Conductor SERP XLT: Product Page
£
44,360 MSV
9,620 MSV
7,470 MSV
380 MSV
3,040 MSV
180 MSV
What are the most
frequently
occurring nouns,
verbs &
adjectives in a
SERP?
Part of Speech
Tagging
@RoryT1
PoS
Tagging
Uncover the
phrases or topics
you should include
in your landing
pages to rank for a
term
Nouns (people, place, thing)
@RoryT1
PoS
Tagging
Get clues around
how Google is
interpreting the
context and intent
of a search
Verbs (action or state)
@RoryT1
PoS
Tagging
Understand the
language and
tone that might
resonate with a
searcher
Adjectives (descriptive word)
@RoryT1
PoS
Tagging
Credit Card Example – P1 Verbs
Intent Clues: What is the specific motivation
our searcher has?
PoS
Tagging
Credit Card Example - P1 Nouns
Context Clues: Words that clarify meaning &
help us understand what a searcher wants
@RoryT1
PoS
Tagging
Credit Card Example - P1 Adjectives
Context Clues: Words that clarify meaning &
help us understand what a searcher wants
@RoryT1
•Create landing pages that are aligned with the
intent of a searcher
•Help copywriters understand the language and
desires of a target audience
•Tactically incorporate more semantically relevant
phrases into landing pages
HOW CAN WE APPLY THIS?
@RoryT1
Can we use NLP
to uncover topical
trends in the
SERPs to help us
with content
ideation?
Topic
Modelling
@RoryT1
Topic
Modelling
Topic modelling is an NLP method that assumes a
corpus contains a mixture of topics. It looks at how
words and phrases co-occur in a corpus and attempts
to group them in coherent themes or topics.
@RoryT1
Topic
Modelling
OK, computer. Here’s some words. Group them.
@RoryT1
RoryTruesdale
Cheapening machine
learning since 2019
Topic
Modelling
Each bubble
represents a
topic
@RoryT1
Topic
Modelling
The bigger
the bubble
the more
prominent
the topic
@RoryT1
Topic
Modelling
The further
away the
bubbles are,
the more
distinct those
topic are
Topic
Modelling
Get a
breakdown of
the terms our
topics consist
of
@RoryT1
Topic
Modelling
The output is an
interactive visual
on topical trends
that can be easily
shared with other
teams
@RoryT1
Topic
Modelling
Use Google’s
algorithm to help
us identify areas
of interest for our
audience
Topic
Modelling
Uncover topical
trends hidden in
the language of
the SERPs that
can inform
content ideation
@RoryT1
•Valuable data point to reference for content
ideation
•Inform internal linking and content
recommendations across a website
•Incorporate topically relevant phrases into existing
pages to improve semantic relevance
HOW CAN WE APPLY THIS?
@RoryT1
How can we make
our scripts work
across other data
sources to
understand our
customers?
Other
Useful
Applications
@RoryT1
Product
Reviews
@RoryT11
Product Reviews
@RoryT11
GMB
Reviews
@RoryT11
GMB Reviews
@RoryT11
Reddit
@RoryT11
Reddit
@RoryT11
YouTube
Captions
@RoryT11
YouTube Captions
@RoryT11
Competitors & Top
Ranking Pages
@RoryT11
Competitors & Top Ranking Pages
@RoryT11
With some minor
tweaks we can
make our scripts
work across a huge
corpus of user-
centric content
Pretty cool, right?
@RoryT1
Potential to ramp up and apply sentiment analysis
to these sources for useful visualisations
@RoryT11
Deconstruct product reviews to find out what really
matters to customers
•Simple
•Easy to use
•Intuitive
•Buggy
•Slow
@RoryT1
A lot to
take
in…what
does it all
mean?
@RoryT11
SERPs give us
amazing insight
into what
customers want
@RoryT1
Python makes
getting these
insights at scale
accessible
@RoryT1
Use these insights
to align landing
pages with intent
and semantic
relevance
@RoryT1
Scripts we create
allow us to get these
insights from lots of
other user-centric
sources beyond the
SERPs
@RoryT1
http://cndr.co/jupyter
Python Dropbox Link
@RoryT1
Get The Slides
@RoryT11
http://cndr.co/brighton
• https://www.searchenginejournal.com/scrape-google-serp-custom-extractions/267211/
• https://www.searchenginejournal.com/mine-serps-seo-content-customer-insights/311137/
• https://www.seerinteractive.com/blog/user-testing-serps-an-audience-first-approach-to-seo/
• https://www.dropbox.com/sh/vl5miyt6sgbvmkl/AAC5365YcWTun_EzkQLtixe1a?dl=0 (Jupyter
Notebook tutorial)
• http://www.blindfiveyearold.com/algorithm-analysis-in-the-age-of-embeddings
• https://www.searchenginejournal.com/semantic-search-seo/264037/#close
• https://www.slideshare.net/DawnFitton/natural-language-processing-and-search-intent-
understanding-c3-conductor-2019-dawn-anderson
• https://moz.com/blog/what-is-semantic-search
• https://www.slideshare.net/paulshapiro/redefining-technical-seo-mozcon-2019-by-paul-shapiro
Useful Resources
@RoryT1
Thanks
For
Listening!
Conductor.com
@RoryT11
brightonseo@conductor.com

BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights