SlideShare a Scribd company logo
1 of 33
Download to read offline
How does the Hero do it?
Extended Version
Table of Content
1. How it Doesn’t Work
2. Understanding the Domain
3. Generating Keywords
4. Clustering and Classifying Keywords
5. Teaching the Hero
6. The First Computation
7. The Final Output
1. How it does not work
For Clarification Purposes
There are no unique keys that would allow you to map a keyword with
a session. For this very reason we use several data sources to reduce the
number of possible keywords for a specific session to the minimum.
Key Keyword Key Session
2. Understanding the Domain
First Things First
Once a user signs up, the Hero analyzes the history of
the GA account, looking for:
1. patterns e.g. site structure, URLs that pull traffic; clusters among
organic traffic; device categories; time / date; locations; …
2. type and content e.g. # of direct sessions attributed to
organic; news or ecomm; one-pager or many product sites;
semantical topics; …
3. Generating Keywords
What could it be???
In the first step we only generate a large set of
possible keywords per URL.
There is no classification or attribution in this step.
There are two types of data sources we use for this:
1. 3rd party data sources e.g. rank monitoring services;
browser extensions data; Bing search API; ....
2. 1st party user data Search Console; remaining keywords in
GA
→ the result is one big bucket with
possible keywords per URL
Keyword 1
Keyword 2
Keyword 3
Keyword 4
...
YOURDOMAIN.COM/FOLDER
4. Clustering and Classifying
Keywords
It’s getting tricky...
Using external APIs we look for anormal behaviour in
a keyword’s history
were there changes in
traffic in the last 12
months
(per country / device)
is this a new keyword?
did spikes occur?
is there a reference
article on Wikipedia and
how does it behave?
→ cool link
Semantics
After the Hero created a keyword set and checked for
potential spikes and anormalities, he needs to
“understand” the keyword.
is this a brand or a
non-brand keyword?
is it a person / brand
name? (important for
ngram analysis)
is the keyword
transactional /
navigational or
informational?
category clusters
→ the result is one big dataframe
where certain parameters are
attributed to the keywords
(looks sort of like this thing above)
In the next step we analyze differences in the
performance of organic traffic due to keyword
fluctuation in the SERPs.
suppose we have a site that ranked on position 13 - 15 in calendar week X for the term
“shoes”. At this point we have the following metrics for this landing page:
in calendar week X + 1 the site now ranks on position 5 for shoes and metrics changed to:
in this simple example we assume ceteris paribus, that nothing else has changed, so almost
all new sessions can be attributed to “shoes”.
these differences tell us a couple of things:
1. “shoes” on position 5 brings in 1000 more daily sessions compared to position 13 - 15
2. the bounce rate of the keyword is identical to other keywords that rank for this site
3. the conversion rate of this keyword seems to be significantly lower than the CR of the
others. It has brought only 5 more conversions with 1000 sessions, compared to the 10
conversions with the 500 sessions we had before.
→ when analyzing this, we take a couple of factors into consideration, most importantly:
- seasonal fluctuations
- overall site performance during this time frame
- keyword performance → spike due to trends?
Luckily the SERPs are changing quite a lot, otherwise this would not be
possible.
5. Training Classifications
This is where his brains kick in...
A first step towards matching keywords
and sessions
Using the aforementioned generated data, the Hero tries to calculate a
first probability of a certain keyword matching a cluster of sessions.
Sessions that were captured thru extensions and those where the
keyword is still visibly in GA (= “hard data”) can be matched with 100%
certainty, they will be left out.
→ after the first probability is calculated, the data will again be
compared against the “hard data”. This happens on the cluster level to
adjust the classifications.
The second step towards matching
After the first iteration the Hero constructs strings from the GA data.
These strings contain between 5 and 25 dimensions.
(It's interesting how few sessions are left if you query a big string - even
if you have huge sites. You can check this for yourself using the Core
Reporting API, where you get up to 7 dimensions in one string)
Now: the keywords have been clustered really well and you get only a
handful of sessions back from one string, leaving very few potential
keywords per cluster of session (the cluster has gotten smaller, too)
An unfair visual representation
Keyword A
Keyword A might now be matched with one or more of those sessions.
This is a session
cluster, containing
four sessions.
This results from the
Hero requesting all
sessions with a certain
string
Session 1
Session
2
Session
3
Session
4
Training the classifications
What happened now: the Hero created an adjusted unique algorithm
for your account, that tries to match sessions with the keywords.
He applies this algorithm to the entire history of the GA account and
then matches the results with the “hard data”, the remaining visible
keywords and the keywords captured by browser extensions.
After this last step the algo is ready to go for each page.
In most cases the historic data is not enough as some data sources are
not available retrospectively, which is why the Hero works far better
after a month.
The learning curve
MatchingCertainty
Time
Entire algorithm
Your algorithm
6. The First Computation
The magic happens...
After all that learning, the Hero can start
The process is essentially the same as the training
process:
1. Computing possible keywords
2. Checking the results against the “hard data”
3. Adjust classifications based on results
Actually, quite straightforward!
7. The Final Output
What are you giving me here?
What data do you get?
Obviously the matching does not happen thru CIDs.
Looking in that mirrored account, you’ll see that
some dimensions are missing. This is because the
matching happens based on dimension strings and
not sessions.
Session vs. string-based matching
If the data was session based, you’d get all info:
but what it looks like is this:
{{Keyword}} GA-CID =
All Dimensions and
Metrics for this CID
{{Keyword}}
GA-CID =
GA-CID =
similar
metrics
same cluster
of
dimensions
What do the 83% certainty mean?
This is just the average threshold but easier to
communicate.
Really, the certainty threshold for a keyword to
match with a session varies between 80% and 85%.
The certainty level is based on the assumption that
95% of all possible keywords have been found in the
first bucket in page 9.
This keeps the Hero from top shape:
- you had a major redesign / relaunch / big change
of your website resulting in very different metrics.
- the Hero can’t recognize your brand or all of your
(misspelled) brand terms. We adjust this manually.
- your GA account is in bad shape.
We hope that helped. We are certain about the
validity of the mathematical concepts involved here.
We have to remain quite top level mostly, as the Hero
tries to keep some secrets :-)
However, if you have any further questions about
some of the concepts involved, we try our best to
answer them. Just let us know @
tech-questions@keyword-hero.com
Appendix
Really???
How does the semantic clustering
work?
After all possible keywords that rank were clustered
and analyzed, the classification will be adjusted to
what the Hero knows from analyzing the browser
extension data and their users.
→
An example:
Search phrase: “How tall is the Eiffel Tower?”
Is the data set from the browser extensions large
enough to attribute metrics to this keyword?
YES: how deep can you go? can you even attribute
on country / device level?
NO: can we get an approximation of the
performance from the columns in the dataframe?
Some stats about the browser
extensions
We buy data from some browser extension conglomerates across the
globe. Depending on vertical and country the ratio of desktop searches
that we catch is between 0.5% and 8%.
For many URLs, however, we don’t have a matching search phrase
captured by a plugin. Still the data is very important as it trains the
overall algorithm keyword related stats, such as:
“clicks on paid terms vs. organic”
“intention vs. vertical based click behaviour”

More Related Content

What's hot

Google Tag Manager Can Do What
Google Tag Manager Can Do WhatGoogle Tag Manager Can Do What
Google Tag Manager Can Do Whatpatrickstox
 
Advanced Content Analysis in Google Analytics
Advanced Content Analysis in Google AnalyticsAdvanced Content Analysis in Google Analytics
Advanced Content Analysis in Google Analyticsdirefuleyesight87
 
Advanced Content Analysis in Google Analytics
Advanced Content Analysis in Google AnalyticsAdvanced Content Analysis in Google Analytics
Advanced Content Analysis in Google Analyticsabandonedjurist69
 
11 Advanced Uses of Screaming Frog Nov 2019 DMSS
11 Advanced Uses of Screaming Frog Nov 2019 DMSS11 Advanced Uses of Screaming Frog Nov 2019 DMSS
11 Advanced Uses of Screaming Frog Nov 2019 DMSSOliver Brett
 
SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'
SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'
SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'Distilled
 
Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...
Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...
Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...Rachel Costello
 
Location-Free Local SEO
Location-Free Local SEOLocation-Free Local SEO
Location-Free Local SEOTom Capper
 
Paywall SEO: Digital First Print Second, From 0 to 35k subscribers in a year
Paywall SEO: Digital First Print Second, From 0 to 35k subscribers in a yearPaywall SEO: Digital First Print Second, From 0 to 35k subscribers in a year
Paywall SEO: Digital First Print Second, From 0 to 35k subscribers in a yearDaniel Smullen
 
Brighton SEO July 2021 - Google Discover - optimisation, measurement, tech seo
Brighton SEO July 2021 - Google Discover - optimisation, measurement, tech seoBrighton SEO July 2021 - Google Discover - optimisation, measurement, tech seo
Brighton SEO July 2021 - Google Discover - optimisation, measurement, tech seoTom Capper
 
A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019
A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019
A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019patrickstox
 
Utilizing the natural langauage toolkit for keyword research
Utilizing the natural langauage toolkit for keyword researchUtilizing the natural langauage toolkit for keyword research
Utilizing the natural langauage toolkit for keyword researchErudite
 
What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015Will Critchlow
 
TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...
TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...
TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...Catalyst
 
Brighton SEO April 2018 Craig Campbell
Brighton SEO April 2018 Craig CampbellBrighton SEO April 2018 Craig Campbell
Brighton SEO April 2018 Craig CampbellCraig Campbell
 
SEO for Large/Enterprise Websites - Data & Tech Side
SEO for Large/Enterprise Websites - Data & Tech SideSEO for Large/Enterprise Websites - Data & Tech Side
SEO for Large/Enterprise Websites - Data & Tech SideDominic Woodman
 
BrightonSEO: Leveraging information architecture for Ecommerce SEO
BrightonSEO: Leveraging information architecture for Ecommerce SEOBrightonSEO: Leveraging information architecture for Ecommerce SEO
BrightonSEO: Leveraging information architecture for Ecommerce SEOMichael Curtis
 
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...Hamlet Batista
 
Advanced data-driven technical SEO - SMX London 2019
Advanced data-driven technical SEO - SMX London 2019Advanced data-driven technical SEO - SMX London 2019
Advanced data-driven technical SEO - SMX London 2019Bastian Grimm
 
Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...
Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...
Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...Jamie Indigo
 

What's hot (19)

Google Tag Manager Can Do What
Google Tag Manager Can Do WhatGoogle Tag Manager Can Do What
Google Tag Manager Can Do What
 
Advanced Content Analysis in Google Analytics
Advanced Content Analysis in Google AnalyticsAdvanced Content Analysis in Google Analytics
Advanced Content Analysis in Google Analytics
 
Advanced Content Analysis in Google Analytics
Advanced Content Analysis in Google AnalyticsAdvanced Content Analysis in Google Analytics
Advanced Content Analysis in Google Analytics
 
11 Advanced Uses of Screaming Frog Nov 2019 DMSS
11 Advanced Uses of Screaming Frog Nov 2019 DMSS11 Advanced Uses of Screaming Frog Nov 2019 DMSS
11 Advanced Uses of Screaming Frog Nov 2019 DMSS
 
SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'
SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'
SearchLove London | 'Jono Alderson', Turbocharging your Wordpress Website'
 
Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...
Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...
Conflicting Website Signals & Confused Search Engines - Rachel Costello, Tech...
 
Location-Free Local SEO
Location-Free Local SEOLocation-Free Local SEO
Location-Free Local SEO
 
Paywall SEO: Digital First Print Second, From 0 to 35k subscribers in a year
Paywall SEO: Digital First Print Second, From 0 to 35k subscribers in a yearPaywall SEO: Digital First Print Second, From 0 to 35k subscribers in a year
Paywall SEO: Digital First Print Second, From 0 to 35k subscribers in a year
 
Brighton SEO July 2021 - Google Discover - optimisation, measurement, tech seo
Brighton SEO July 2021 - Google Discover - optimisation, measurement, tech seoBrighton SEO July 2021 - Google Discover - optimisation, measurement, tech seo
Brighton SEO July 2021 - Google Discover - optimisation, measurement, tech seo
 
A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019
A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019
A Crash Course in Technical SEO from Patrick Stox - Beer & SEO Meetup May 2019
 
Utilizing the natural langauage toolkit for keyword research
Utilizing the natural langauage toolkit for keyword researchUtilizing the natural langauage toolkit for keyword research
Utilizing the natural langauage toolkit for keyword research
 
What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015What does Google want? Future of Digital Marketing 2015
What does Google want? Future of Digital Marketing 2015
 
TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...
TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...
TechSEO Boost 2017: Working Smarter: SEO Automation to Increase Efficiency & ...
 
Brighton SEO April 2018 Craig Campbell
Brighton SEO April 2018 Craig CampbellBrighton SEO April 2018 Craig Campbell
Brighton SEO April 2018 Craig Campbell
 
SEO for Large/Enterprise Websites - Data & Tech Side
SEO for Large/Enterprise Websites - Data & Tech SideSEO for Large/Enterprise Websites - Data & Tech Side
SEO for Large/Enterprise Websites - Data & Tech Side
 
BrightonSEO: Leveraging information architecture for Ecommerce SEO
BrightonSEO: Leveraging information architecture for Ecommerce SEOBrightonSEO: Leveraging information architecture for Ecommerce SEO
BrightonSEO: Leveraging information architecture for Ecommerce SEO
 
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
How to scale SEO work NOBODY wants to do (including your competitors) to rapi...
 
Advanced data-driven technical SEO - SMX London 2019
Advanced data-driven technical SEO - SMX London 2019Advanced data-driven technical SEO - SMX London 2019
Advanced data-driven technical SEO - SMX London 2019
 
Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...
Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...
Crafting Expertise, Authority and Trust with Entity-Based Content Strategy - ...
 

Similar to Uncovering 'not provided' keyword data

How to successfully predict traffic on your website with high accuracy converted
How to successfully predict traffic on your website with high accuracy convertedHow to successfully predict traffic on your website with high accuracy converted
How to successfully predict traffic on your website with high accuracy convertedDigitalberge
 
Persona Driven Keyword Research
Persona Driven Keyword ResearchPersona Driven Keyword Research
Persona Driven Keyword ResearchMichael King
 
You Don't Know SEO
You Don't Know SEOYou Don't Know SEO
You Don't Know SEOMichael King
 
Croud Presents: How to Build a Data-driven SEO Strategy Using NLP
Croud Presents: How to Build a Data-driven SEO Strategy Using NLPCroud Presents: How to Build a Data-driven SEO Strategy Using NLP
Croud Presents: How to Build a Data-driven SEO Strategy Using NLPDaniel Liddle
 
Amazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOAmazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOWill Critchlow
 
Search Analytics at Enterprise Search Summit Fall 2011
Search Analytics at Enterprise Search Summit Fall 2011Search Analytics at Enterprise Search Summit Fall 2011
Search Analytics at Enterprise Search Summit Fall 2011Sematext Group, Inc.
 
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' Distilled
 
Keyword tools.pptx
Keyword tools.pptxKeyword tools.pptx
Keyword tools.pptxASHAVI2
 
How to Scale and Grow your Enterprise Technical SEO Strategy
How to Scale and Grow your Enterprise Technical SEO StrategyHow to Scale and Grow your Enterprise Technical SEO Strategy
How to Scale and Grow your Enterprise Technical SEO StrategySearch Engine Journal
 
Web Marketing Week3
Web Marketing Week3Web Marketing Week3
Web Marketing Week3cghb1210
 
Lessons from SEO split-testing
Lessons from SEO split-testingLessons from SEO split-testing
Lessons from SEO split-testingWill Critchlow
 
The Future of Search Engine Rankings
The Future of Search Engine RankingsThe Future of Search Engine Rankings
The Future of Search Engine Rankingsgstuncay
 
The Future of Search Engine Rankings
The Future of Search Engine RankingsThe Future of Search Engine Rankings
The Future of Search Engine Rankingsffats1
 
5 must have seo tools that you can't miss
5 must have seo tools that you can't miss5 must have seo tools that you can't miss
5 must have seo tools that you can't missOrbit Informatics
 
C-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO StrategiesC-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO StrategiesIzzi Smith
 
Seo questions for 2013
Seo questions for 2013Seo questions for 2013
Seo questions for 2013Lalit Kant
 
Using SEO to Build Your Business
Using SEO to Build Your BusinessUsing SEO to Build Your Business
Using SEO to Build Your BusinessKatie Spence
 
How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...
How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...
How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...Matthew Woodward
 

Similar to Uncovering 'not provided' keyword data (20)

How to successfully predict traffic on your website with high accuracy converted
How to successfully predict traffic on your website with high accuracy convertedHow to successfully predict traffic on your website with high accuracy converted
How to successfully predict traffic on your website with high accuracy converted
 
Persona Driven Keyword Research
Persona Driven Keyword ResearchPersona Driven Keyword Research
Persona Driven Keyword Research
 
Gaps in the algorithm
Gaps in the algorithmGaps in the algorithm
Gaps in the algorithm
 
You Don't Know SEO
You Don't Know SEOYou Don't Know SEO
You Don't Know SEO
 
Croud Presents: How to Build a Data-driven SEO Strategy Using NLP
Croud Presents: How to Build a Data-driven SEO Strategy Using NLPCroud Presents: How to Build a Data-driven SEO Strategy Using NLP
Croud Presents: How to Build a Data-driven SEO Strategy Using NLP
 
Amazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOAmazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEO
 
Search Analytics at Enterprise Search Summit Fall 2011
Search Analytics at Enterprise Search Summit Fall 2011Search Analytics at Enterprise Search Summit Fall 2011
Search Analytics at Enterprise Search Summit Fall 2011
 
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
 
Keyword tools.pptx
Keyword tools.pptxKeyword tools.pptx
Keyword tools.pptx
 
How to Scale and Grow your Enterprise Technical SEO Strategy
How to Scale and Grow your Enterprise Technical SEO StrategyHow to Scale and Grow your Enterprise Technical SEO Strategy
How to Scale and Grow your Enterprise Technical SEO Strategy
 
Web Marketing Week3
Web Marketing Week3Web Marketing Week3
Web Marketing Week3
 
Lessons from SEO split-testing
Lessons from SEO split-testingLessons from SEO split-testing
Lessons from SEO split-testing
 
The Future of Search Engine Rankings
The Future of Search Engine RankingsThe Future of Search Engine Rankings
The Future of Search Engine Rankings
 
The Future of Search Engine Rankings
The Future of Search Engine RankingsThe Future of Search Engine Rankings
The Future of Search Engine Rankings
 
5 must have seo tools that you can't miss
5 must have seo tools that you can't miss5 must have seo tools that you can't miss
5 must have seo tools that you can't miss
 
C-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO StrategiesC-T-R-You Ready for 2021?! - On-SERP SEO Strategies
C-T-R-You Ready for 2021?! - On-SERP SEO Strategies
 
Getting started-with-similar web-api
Getting started-with-similar web-apiGetting started-with-similar web-api
Getting started-with-similar web-api
 
Seo questions for 2013
Seo questions for 2013Seo questions for 2013
Seo questions for 2013
 
Using SEO to Build Your Business
Using SEO to Build Your BusinessUsing SEO to Build Your Business
Using SEO to Build Your Business
 
How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...
How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...
How To Get Maximum Links Per Minute Using GSA Search Engine Ranker In 5 Simpl...
 

Recently uploaded

Understand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMMUnderstand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMMsearchextensionin
 
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...CIO Business World
 
Understanding the Affiliate Marketing Channel; the short guide
Understanding the Affiliate Marketing Channel; the short guideUnderstanding the Affiliate Marketing Channel; the short guide
Understanding the Affiliate Marketing Channel; the short guidePartnercademy
 
McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)DEVARAJV16
 
Storyboards for my Final Major Project Video
Storyboards for my Final Major Project VideoStoryboards for my Final Major Project Video
Storyboards for my Final Major Project VideoSineadBidwell
 
social media optimization complete indroduction
social media optimization complete indroductionsocial media optimization complete indroduction
social media optimization complete indroductioninfoshraddha747
 
Introduction to marketing Management Notes
Introduction to marketing Management NotesIntroduction to marketing Management Notes
Introduction to marketing Management NotesKiranTiwari42
 
SEO and Digital PR - How to Connect Your Teams to Maximise Success
SEO and Digital PR - How to Connect Your Teams to Maximise SuccessSEO and Digital PR - How to Connect Your Teams to Maximise Success
SEO and Digital PR - How to Connect Your Teams to Maximise SuccessLiv Day
 
The power of SEO-driven market intelligence
The power of SEO-driven market intelligenceThe power of SEO-driven market intelligence
The power of SEO-driven market intelligenceHinde Lamrani
 
(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...
(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...
(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...Hugues Rey
 
Fiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview AssignmentFiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview AssignmentFarrel Brest
 
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local LeadsSearch Engine Journal
 
Digital Marketing Courses In Pune- school Of Internet Marketing
Digital Marketing Courses In Pune- school Of Internet MarketingDigital Marketing Courses In Pune- school Of Internet Marketing
Digital Marketing Courses In Pune- school Of Internet MarketingShauryaBadaya
 
15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing Strategy15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing StrategyBlue Atlas Marketing
 
Unlocking Passive Income: The Power of Affiliate Marketing
Unlocking Passive Income: The Power of Affiliate MarketingUnlocking Passive Income: The Power of Affiliate Marketing
Unlocking Passive Income: The Power of Affiliate MarketingDaniel
 
Prezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media TrendsPrezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media TrendsCristian Manafu
 
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...Ahrefs
 
Exploring Web 3.0 Growth marketing: Navigating the Future of the Internet
Exploring Web 3.0 Growth marketing: Navigating the Future of the InternetExploring Web 3.0 Growth marketing: Navigating the Future of the Internet
Exploring Web 3.0 Growth marketing: Navigating the Future of the Internetnehapardhi711
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfVWO
 
Best digital marketing e-book form bignners
Best digital marketing e-book form bignnersBest digital marketing e-book form bignners
Best digital marketing e-book form bignnersmuntasibkhan58
 

Recently uploaded (20)

Understand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMMUnderstand the Key differences between SMO and SMM
Understand the Key differences between SMO and SMM
 
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
Most Impressive Construction Leaders in Tech, Making Waves in the Industry, 2...
 
Understanding the Affiliate Marketing Channel; the short guide
Understanding the Affiliate Marketing Channel; the short guideUnderstanding the Affiliate Marketing Channel; the short guide
Understanding the Affiliate Marketing Channel; the short guide
 
McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)McDonald's: A Journey Through Time (PPT)
McDonald's: A Journey Through Time (PPT)
 
Storyboards for my Final Major Project Video
Storyboards for my Final Major Project VideoStoryboards for my Final Major Project Video
Storyboards for my Final Major Project Video
 
social media optimization complete indroduction
social media optimization complete indroductionsocial media optimization complete indroduction
social media optimization complete indroduction
 
Introduction to marketing Management Notes
Introduction to marketing Management NotesIntroduction to marketing Management Notes
Introduction to marketing Management Notes
 
SEO and Digital PR - How to Connect Your Teams to Maximise Success
SEO and Digital PR - How to Connect Your Teams to Maximise SuccessSEO and Digital PR - How to Connect Your Teams to Maximise Success
SEO and Digital PR - How to Connect Your Teams to Maximise Success
 
The power of SEO-driven market intelligence
The power of SEO-driven market intelligenceThe power of SEO-driven market intelligence
The power of SEO-driven market intelligence
 
(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...
(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...
(Generative) AI & Marketing: - Out of the Hype - Empowering the Marketing M...
 
Fiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview AssignmentFiverr's Product Marketing Interview Assignment
Fiverr's Product Marketing Interview Assignment
 
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads2024's Top PPC Tactics: Triple Your Google Ads Local Leads
2024's Top PPC Tactics: Triple Your Google Ads Local Leads
 
Digital Marketing Courses In Pune- school Of Internet Marketing
Digital Marketing Courses In Pune- school Of Internet MarketingDigital Marketing Courses In Pune- school Of Internet Marketing
Digital Marketing Courses In Pune- school Of Internet Marketing
 
15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing Strategy15 Tactics to Scale Your Trade Show Marketing Strategy
15 Tactics to Scale Your Trade Show Marketing Strategy
 
Unlocking Passive Income: The Power of Affiliate Marketing
Unlocking Passive Income: The Power of Affiliate MarketingUnlocking Passive Income: The Power of Affiliate Marketing
Unlocking Passive Income: The Power of Affiliate Marketing
 
Prezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media TrendsPrezentare Brandfluence 2023 - Social Media Trends
Prezentare Brandfluence 2023 - Social Media Trends
 
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
What I learned from auditing over 1,000,000 websites - SERP Conf 2024 Patrick...
 
Exploring Web 3.0 Growth marketing: Navigating the Future of the Internet
Exploring Web 3.0 Growth marketing: Navigating the Future of the InternetExploring Web 3.0 Growth marketing: Navigating the Future of the Internet
Exploring Web 3.0 Growth marketing: Navigating the Future of the Internet
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdfResearch and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
 
Best digital marketing e-book form bignners
Best digital marketing e-book form bignnersBest digital marketing e-book form bignners
Best digital marketing e-book form bignners
 

Uncovering 'not provided' keyword data

  • 1. How does the Hero do it? Extended Version
  • 2. Table of Content 1. How it Doesn’t Work 2. Understanding the Domain 3. Generating Keywords 4. Clustering and Classifying Keywords 5. Teaching the Hero 6. The First Computation 7. The Final Output
  • 3. 1. How it does not work For Clarification Purposes
  • 4. There are no unique keys that would allow you to map a keyword with a session. For this very reason we use several data sources to reduce the number of possible keywords for a specific session to the minimum. Key Keyword Key Session
  • 5. 2. Understanding the Domain First Things First
  • 6. Once a user signs up, the Hero analyzes the history of the GA account, looking for: 1. patterns e.g. site structure, URLs that pull traffic; clusters among organic traffic; device categories; time / date; locations; … 2. type and content e.g. # of direct sessions attributed to organic; news or ecomm; one-pager or many product sites; semantical topics; …
  • 8. In the first step we only generate a large set of possible keywords per URL. There is no classification or attribution in this step. There are two types of data sources we use for this: 1. 3rd party data sources e.g. rank monitoring services; browser extensions data; Bing search API; .... 2. 1st party user data Search Console; remaining keywords in GA
  • 9. → the result is one big bucket with possible keywords per URL Keyword 1 Keyword 2 Keyword 3 Keyword 4 ... YOURDOMAIN.COM/FOLDER
  • 10. 4. Clustering and Classifying Keywords It’s getting tricky...
  • 11. Using external APIs we look for anormal behaviour in a keyword’s history were there changes in traffic in the last 12 months (per country / device) is this a new keyword? did spikes occur? is there a reference article on Wikipedia and how does it behave? → cool link
  • 12. Semantics After the Hero created a keyword set and checked for potential spikes and anormalities, he needs to “understand” the keyword. is this a brand or a non-brand keyword? is it a person / brand name? (important for ngram analysis) is the keyword transactional / navigational or informational? category clusters
  • 13. → the result is one big dataframe where certain parameters are attributed to the keywords (looks sort of like this thing above)
  • 14. In the next step we analyze differences in the performance of organic traffic due to keyword fluctuation in the SERPs. suppose we have a site that ranked on position 13 - 15 in calendar week X for the term “shoes”. At this point we have the following metrics for this landing page: in calendar week X + 1 the site now ranks on position 5 for shoes and metrics changed to: in this simple example we assume ceteris paribus, that nothing else has changed, so almost all new sessions can be attributed to “shoes”.
  • 15. these differences tell us a couple of things: 1. “shoes” on position 5 brings in 1000 more daily sessions compared to position 13 - 15 2. the bounce rate of the keyword is identical to other keywords that rank for this site 3. the conversion rate of this keyword seems to be significantly lower than the CR of the others. It has brought only 5 more conversions with 1000 sessions, compared to the 10 conversions with the 500 sessions we had before. → when analyzing this, we take a couple of factors into consideration, most importantly: - seasonal fluctuations - overall site performance during this time frame - keyword performance → spike due to trends? Luckily the SERPs are changing quite a lot, otherwise this would not be possible.
  • 16. 5. Training Classifications This is where his brains kick in...
  • 17. A first step towards matching keywords and sessions Using the aforementioned generated data, the Hero tries to calculate a first probability of a certain keyword matching a cluster of sessions. Sessions that were captured thru extensions and those where the keyword is still visibly in GA (= “hard data”) can be matched with 100% certainty, they will be left out. → after the first probability is calculated, the data will again be compared against the “hard data”. This happens on the cluster level to adjust the classifications.
  • 18. The second step towards matching After the first iteration the Hero constructs strings from the GA data. These strings contain between 5 and 25 dimensions. (It's interesting how few sessions are left if you query a big string - even if you have huge sites. You can check this for yourself using the Core Reporting API, where you get up to 7 dimensions in one string) Now: the keywords have been clustered really well and you get only a handful of sessions back from one string, leaving very few potential keywords per cluster of session (the cluster has gotten smaller, too)
  • 19. An unfair visual representation Keyword A Keyword A might now be matched with one or more of those sessions. This is a session cluster, containing four sessions. This results from the Hero requesting all sessions with a certain string Session 1 Session 2 Session 3 Session 4
  • 20. Training the classifications What happened now: the Hero created an adjusted unique algorithm for your account, that tries to match sessions with the keywords. He applies this algorithm to the entire history of the GA account and then matches the results with the “hard data”, the remaining visible keywords and the keywords captured by browser extensions. After this last step the algo is ready to go for each page. In most cases the historic data is not enough as some data sources are not available retrospectively, which is why the Hero works far better after a month.
  • 22. 6. The First Computation The magic happens...
  • 23. After all that learning, the Hero can start The process is essentially the same as the training process: 1. Computing possible keywords 2. Checking the results against the “hard data” 3. Adjust classifications based on results Actually, quite straightforward!
  • 24. 7. The Final Output What are you giving me here?
  • 25. What data do you get? Obviously the matching does not happen thru CIDs. Looking in that mirrored account, you’ll see that some dimensions are missing. This is because the matching happens based on dimension strings and not sessions.
  • 26. Session vs. string-based matching If the data was session based, you’d get all info: but what it looks like is this: {{Keyword}} GA-CID = All Dimensions and Metrics for this CID {{Keyword}} GA-CID = GA-CID = similar metrics same cluster of dimensions
  • 27. What do the 83% certainty mean? This is just the average threshold but easier to communicate. Really, the certainty threshold for a keyword to match with a session varies between 80% and 85%. The certainty level is based on the assumption that 95% of all possible keywords have been found in the first bucket in page 9.
  • 28. This keeps the Hero from top shape: - you had a major redesign / relaunch / big change of your website resulting in very different metrics. - the Hero can’t recognize your brand or all of your (misspelled) brand terms. We adjust this manually. - your GA account is in bad shape.
  • 29. We hope that helped. We are certain about the validity of the mathematical concepts involved here. We have to remain quite top level mostly, as the Hero tries to keep some secrets :-) However, if you have any further questions about some of the concepts involved, we try our best to answer them. Just let us know @ tech-questions@keyword-hero.com
  • 31. How does the semantic clustering work? After all possible keywords that rank were clustered and analyzed, the classification will be adjusted to what the Hero knows from analyzing the browser extension data and their users. →
  • 32. An example: Search phrase: “How tall is the Eiffel Tower?” Is the data set from the browser extensions large enough to attribute metrics to this keyword? YES: how deep can you go? can you even attribute on country / device level? NO: can we get an approximation of the performance from the columns in the dataframe?
  • 33. Some stats about the browser extensions We buy data from some browser extension conglomerates across the globe. Depending on vertical and country the ratio of desktop searches that we catch is between 0.5% and 8%. For many URLs, however, we don’t have a matching search phrase captured by a plugin. Still the data is very important as it trains the overall algorithm keyword related stats, such as: “clicks on paid terms vs. organic” “intention vs. vertical based click behaviour”