SlideShare a Scribd company logo
@dawnieando
Google BERT
What SEOs and Marketers Need to Know
@dawnieando
Here’s another Bert, & I am Dawn Anderson
Managing Director
of BerteyBert, my Pomeranian
@dawnieando
• A Google algorithmic
update
• Google announce BERT to the
organic search world in a VERY
geeky way
• Mentions of the 15% of new
queries every day
• Touches on ‘The Vocabulary
Problem’ (many ways of querying
the same thing)
October 2019 - Welcome To Search, BERT
@dawnieando
• Probably the biggest
improvement in search EVER
• The biggest change in search
in five years, since RankBrain
Fundamentally… Google BERT is
@dawnieando
!Layman’s Terms: it can be
used to help Google better
understand the context of
words in search queries &
content
So, just what is Google BERT update?
@dawnieando
• Used globally in all
languages on featured
snippets
• BERT to impact rankings for 1
in 10 queries
• Initially for English language
queries in US
The bottom line search announcement
@dawnieando
Dec 2019 – BERT expands internationally
• Over 70 languages
• Still only impacts 10% of
queries despite the
considerable expansion
• Still all featured snippets
globally
@dawnieando
• BERT deals with
ambiguity & ‘nuance’ in
queries & content
• Unlikely to impact short
queries
• More likely to impact
conversational queries
• Unlikely to impact
branded queries
Why just 10% of Google Queries Impacted?
@dawnieando
• The SEO community is
abuzz
• BERT is a big deal
• Likened to ‘Rank Brain’ in
some of the ‘interesting’
interpretations
• Some confusions around
‘What BERT is and what it
means for search’
SEO’s React
@dawnieando
!A neural network-based
technique for natural language
processing pre-training
!An anagram of Bi-Directional
Encoder Representations from
Transformers
BERT in Geek Speak
@dawnieando
!Bi-directional
!Encoder
!Representations
From
!Transformers
Let’s Visit The B – E – R – T Explanations Later
@dawnieando
• Search algorithm update
• Open source pre-trained model / framework for
natural language understanding
• Academic research paper
• Evolving tool for computational linguistics efficiency
• Beginning of MANY BERT’ish language models
Important: BERT is Many Things
@dawnieando
So What’s The Backstory?
Where%did%BERT%come%from?
Where%did%the%need%for%BERT%arise?
The$Impact$of$BERT$for$SEO$&$beyond?
What%next?
@dawnieando
• Academic Paper
• Research Project by Devlin et al
• Published a year before the
update in October 2018
• Bert: Pre-training of deep
bidirectional transformers for
language understanding
BERT started as a research paper in 2018
@dawnieando
• Open sourced so anyone can
build a BERT
• BERT created a sea-change
leap-forward in natural language
understanding in information
retrieval very quickly
• Provided a pre-trained language
model which required only fine-
tuning
BERT Open Sourced in 2018
@dawnieando
The whole of the English
Wikipedia & The Books
Corpus combined.
Over 2,500 million words
BERT Has Been Pre-Trained On Many Words
@dawnieando
Vanilla BERT provides a pre-
trained starting point layer for
neural networks in machine
learning & natural language
diverse tasks
The machine learning community got very
excited about BERT
@dawnieando
• BERT is fine-tuned on a variety of
downstream NLP tasks, including
question and answer datasets
BERT Can Be Fine-Tuned in A Short Space of Time
@dawnieando
• Vanilla BERT can be used ‘out of the box’
or fine-tuned
• Provides a great starting point & saves
huge amounts of time & money
• Those wishing to, ‘can build upon’, and
improve BERT
BERT Saves Researchers Time AND Money
@dawnieando
• Microsoft – MT-DNN
• Facebook – RoBERTa
• XLNet
• ERNIE – Baidu
• Lots of other
contenders
Since 2018 Major tech companies extend BERT
@dawnieando
Training Datasets Like MSMARCO Are Used To Fine
Tune With Question & Answer Datasets
@dawnieando
Real Bing Questions Feed MSMARCO
Microsoft Machine
Reading
Comprehension
Dataset. Real Bing
User Queries for NLU
Research
@dawnieando
You think SEOs are competitive? ML Engineers are
more so
• GLUE
• SuperGLUE
• MSMARCO
• SQuAD
…And Leaderboards
@dawnieando
SuperGLUE
was created
because
GLUE got too
easy
Progress was phenomenal with many new SOTAs
@dawnieando
Language models like
BERT help machines
understand the nuance
in word’s context and
surrounding text
cohesion
What Purpose Does BERT Serve & How?
@dawnieando
• Dates back over 60 years old to the Turing Test paper
• Aims at understanding the way words fit together with
structure and meaning.
• NLU is Connected to the field of linguistics (computational
linguistics)
• Over time, increasingly computational linguistics
overflows to a growing online web of content
What is Natural Language Understanding?
@dawnieando
• Natural language
understanding
requires:
• Word’s context
• Common sense reasoning
Natural Language Recognition is NOT
Understanding
@dawnieando
Humans mostly
understand nuance and
jargon from multiple
meanings in written and
spoken word because
of ‘context’
Humans ‘Naturally’ Understand Context
@dawnieando
• Synonymous
• Polysemous
• Homonymous
But Words Can Be VERY Problematic for
Machines & Sometimes Even for Humans
@dawnieando
“The meaning of a word is its use in a
language” (Ludwig, Wittgenstein,
Philosopher, 1953)
Image attribution: Mortiz, Nahr
(Public domain)
Single Words Have No Meaning
@dawnieando
The word ‘like’ in this sentence, is both a:
!(VBP) : (‘verb’ (non 3rd-person, singular,
present) )
!(IN) : (Preposition or subordinating
conjunction)
An Example of Word’s Meaning Changing
• I -> PRP
• Like -> VBP
• That -> IN
• He -> PRP
• Is -> VBZ
• Like -> IN
• That -> DT
@dawnieando
Linguists
Tag ‘Parts
of Speech’
@dawnieando
E.g. Verbs, nouns, adjectives
• Penn-treebank tagger -> 36
different parts of speech
• CLAWS7 (C7) -> 146 different
parts of speech
• Brown Corpus Tagger -> 81
different parts of speech
Words Are ‘Part of Speech’ When Combined
@dawnieando
• He kicked the bucket
• I have yet to tick that off
my bucket list
• The bucket was filled with
water
The Meaning of The Word ‘Bucket’ Changes
@dawnieando
Words Need ’Text Cohesion’
The$‘Glue’$which$adds$meaning
May$historically$be$‘stop$words’
Surrounding)words)can)change)‘intent’
They%add%‘context’
@dawnieando
”Ambiguity is the greatest bottleneck to computational
knowledge acquisition, the killer problem of all natural
language processing.”
(Stephen Clark, formerly of Cambridge University & now a full-
time research scientist with Google Deep Mind)
Ambiguity Is Problematic
@dawnieando
• Words with a similar meaning to something else
• Example: humorous, comical, hilarious, hysterical are ALL
synonyms of funny
Synonymous (Synonyms)
@dawnieando
Ambiguity & Polysemy
• Ambiguity is at a sentence level
• Polysemous words are arguably the
most problematic due to ‘nuanced’
nature
@dawnieando
• Words usually with the
same root and multiple
meanings
• Example: “Run” has 396
Oxford English Dictionary
definitions
Polysemous (Polysemy)
@dawnieando
•Over%40%%of%English%words%are%
polysemous)(McCarthy,)1997;)
Durkin'&'Manning,'1989)
@dawnieando
• Words spelt the same but with very different ‘root’ of word
meanings
• Example: pen (writing implement), pen (pig pen)
• Example: rose (stood up / ascended), rose (flower)
• Example: bark (dog sound), bark (tree bark)
Homonyms
@dawnieando
Spelt differently with
VERY different
meanings but
sound exactly the
same
• Draft, draught
• Dual, duel
• Made, maid
• For, fore, four
• To, too, two
• There, their
• Where, wear, were
Homophones – Difficult To Disambiguate Verbally
@dawnieando
Fork handles
Four candles
Very difficult to
disambiguate in spoken
word
Worse When Words are Joined Together
@dawnieando
Did you want four candles or fork
handles?
Much Comedy Comes From ‘Play
on Words’
@dawnieando
Which Does Not Bode Well For Voice Search
@dawnieando
EXAMPLES
• Zipfian Distribution
• Firthian Linguistics
• Treebanks
• Language can be tied back to
mathematical spaces & algorithms
Language Has Natural Patterns & Phenomena
@dawnieando
Example: Zipfian Distribution (Power Law)
• The frequency of any
word in a collection is
inversely proportional to
its rank in the frequency
table
• Applies to any word
frequency ANYWHERE
• Image is 30 Wikipedias
@dawnieando
To illustrate Zipfian Distribution (Most used Words):
Rank Word Frequency)of)Use)in)a)Corpus
1 the
2 be 1/2
3 to 1/3
4 of 1/4
5 and 1/5
6 a 1/6
7 in 1/7
8 that 1/8
9 have 1/9
10 I 1/10
@dawnieando
“You shall know a word by the
company it keeps” (Firth, 1957)
Firthian Linguistics
One Such Phenomenon is Co-occurrence
@dawnieando
Words with similar meaning tend
to live near each other in a body
of text
Word’s ‘nearness’ can be
measured in mathematical vector
spaces – a context vector is
‘word’s company’
Distributional Relatedness & Firthian Linguistics
@dawnieando
Co-occurrence, Similarity & Relatedness
• Language models
are trained on
large bodies of
text to learn
‘distributional
similarity’ (co-
occurrence)
@dawnieando
Context Vectors & Word Embeddings
• And build vector
space models for word
embeddings
• Models learn the
weights of similarity &
relatedness distances
@dawnieando
Context-Free Word Embeddings
• Past models have been context-free
embeddings
• They lacked the ‘text-cohesion necessary to
understand a word in context
@dawnieando
• He kicked the bucket
• I have yet to tick that off
my bucket list
• The bucket was filled with
water
Remember ‘bucket’ Without Text Cohesion?
@dawnieando
Word’s Context Still Needed Gaps Filling
• Past models used
context-free
embeddings
• A moving
‘context window’
was used to gain
word’s context
@dawnieando
But Even Then True Context Needs Both Sides of a
Word
• Past models were
‘uni-directional’
• The context
window moved
from left to right
or right to left
@dawnieando
They Didn’t Look At Words On Either Side
Simultaneously
@dawnieando
!Bi-directional
!Encoder
!Representations
From
!Transformers
So What About That B – E – R – T explanation?
@dawnieando
• BERT can see the
word’s context on
both sides of a
word in a context
window
Bi-Directional is The B in BERT
@dawnieando
!Encoder Representations
relates to the input and
output process of ‘word’s
context’ & embeddings
What About Encoder Representations?
@dawnieando
!Transformer is a big
deal
!Derived from a 2017
paper called ‘Attention
is all you Need’ (Vaswani, A.,
Shazeer, N., Parmar, N., Uszkoreit, J., Jones,
L., Gomez, A.N., Kaiser, Ł. and Polosukhin,
I., 2017)
What About The Transformer Part?
@dawnieando
Transformer & Attention
Works out how important words are to each
other in a given context & focuses attention
@dawnieando
!Bi-directional
!Encoder
!Representations
From
!Transformers
This Technology Provides Word’s Context
@dawnieando
River Bank or Financial Bank?
By identifying ‘cheque’ or
‘deposit’ in the company
of ‘bank’ BERT can
disambiguate from a ‘river’
bank
@dawnieando
So Where is BERT’s Value in Google Search
• Named entity determination
• Textual entailment (next sentence prediction)
• Coreference resolution
• Question answering
• Word sense disambiguation
• Automatic summarization
• Polysemy resolution
@dawnieando
BERT recognizes the
word ‘to’ makes all
the difference to the
intent of the query
BERT and Disambiguating Nuance
@dawnieando
BERT recognizes
the ambiguous
word ‘stand’s
meaning and
importance in the
context of the
query
BERT and Disambiguating Nuance
@dawnieando
!A single word can change the whole intent of a query
!Conversational queries particularly so
!The ‘stop words’ are actually part of text-cohesion
!Historically ‘stop-words’ were often ignored
!The next sentence matters
BERT and Intent Understanding
@dawnieando
Example:
“I remember what my
Grandad said just
before he kicked the
bucket.”
Next Sentence Prediction (Textual Entailment)
Often the next sentence REALLY matters
@dawnieando
“How far do you
reckon I can kick this
bucket?”
Not What You Expected?
@dawnieando
• There have been lots of improvement by others upon
BERT
• Google have likely improved dramatically on BERT too
• There were some issues with next-sentence prediction
• Facebook built RoBERTa
BERT Probably Doesn’t Resemble The Original BERT
Paper
@dawnieando
• Named entity determination
• Coreference resolution
• Question answering
• Word sense disambiguation
• Automatic summarization
• Polysemy resolution
Featured Snippets
Knowledge Graph & Web Page Extraction
Together
@dawnieando
!BERT is multilingual from mono-lingual
!Other language specific BERTs are being built
!Transformer was trained on international translations
!Language has transferrable phenomena
BERT and International SEO
Expect Big Things
@dawnieando
• Deepset – German BERT
• CamemBERT – French BERT
• AlBERTo – Italian BERT
• RobBERT - Dutch RoBERTa model
BERT & International SEO
@dawnieando
!The challenges of Pygmalion
!Conversational search can now ‘scale’
!BERT takes away some of the human
labelling effort necessary
!Next sentence prediction could impact
assistants and clarifying questions
BERT and Conversational Search
Expect Big Things
@dawnieando
Semantic Heterogeneity Issues in Entity Oriented
Search (Semantic Search)
!Helps with anaphora & cataphora
resolution (resolving pronouns of entities)
!Helps with coreference resolution
!Helps with named entity determination
!Next sentence prediction could impact
assistants and clarifying questions
@dawnieando
Bing has been BERTing since April 2019
• Impacts ALL Bing
queries globally
@dawnieando
• It’s supposed to be natural
• In the same way you can’t optimize for Rank
Brain you can’t optimize for BERT
• BERT is a tool / learning process in search for
disambiguation & contextual understanding of
words
• BERT is a ‘black-box’ algorithm
Why can’t you optimize for BERT?
@dawnieando
• Black-box algorithm
• Hugging Face coined the phrase
BERTology
• Now a field of study exploring why
BERT makes choices
• Some concerns over bias &
responsible AI
Black Box Algorithms & BERTology
@dawnieando
!Cluster together content and interlink well on topic & nuance
!Avoid ‘too-similar’ completing categories - merge
!Consider not just the content in the page but the content in
the linked pages & sections
!Consider the content of the ‘whole domain’ as everything
contributes in co-occurrence
!Be extra vigilant when ‘pruning
Utilising Co-Occurrence Strategically
Employ Relatedness
@dawnieando
Categorisation & Subcategorisation Are King
• Employ strong
conceptual logic in your
site architecture
• Be careful with random
blogs
• If you must ‘tag’, tag
thoughfully
@dawnieando
Anyone can build a BERT to train their own
language processing system for a variety of
natural language understanding downstream
tasks.
Fine-tuning can be carried out in a short time
BERT represents a union of data science and SEO
Anyone Can Use BERT – BERT is a Tool
@dawnieando
• Automatic categorization & subcategorization of
content
• Automatic generation of meta-descriptions
• Automatic summarization of extracts & teasers
• Categorising user-generated content / posts
probably better than humans
How Could BERT Be Harnessed For Efficiency
in SEO? A Few Examples
@dawnieando
• J R Oakes - @jroakes
• Hamlet Batista - @hamletbatista
• Andrea Volpini - @cyberandy
• Gefen Hermesh - @ghermesh
SEOs Are Getting Busy With BERTishness
@dawnieando
Efficiency is Also A Focus
DistilBERT (Hugging'Face)
ALBERT'(Google)
Fast%BERT
@dawnieando
• Original BERT was computationally expensive to
run
• ALBERT stands for A Lite BERT
• Increased efficiency
• ALBERT is BERT’s natural successor
• ALBERT much leaner whilst providing similar
results
• A joint research work between Google & Toyota
ALBERT – BERT’s Successor
@dawnieando
Reformer (Google) – Transformer’s Successor
Understands word’s context
from the perspective of a
‘whole novel’.
https://venturebeat.com/2020/01/16/goog
les-ai-language-model-reformer-can-
process-the-entirety-of-novels/
@dawnieando
Growth has been huge in the natural language
processing community – Current Superglue
Leaderboard
BERT Was Just The Start
• Google T5 is winning
• Even more
advanced
technology
• Transfer-learning
• Expect big things
@dawnieando
SEE YOU AT THE NEXT SMX!

More Related Content

What's hot

Intro to nlp
Intro to nlpIntro to nlp
Intro to nlp
Rutu Mulkar-Mehta
 
SearchLove London - Analysing the SERPs for SEO, Content & Customer Insights
SearchLove London - Analysing the SERPs for SEO, Content & Customer InsightsSearchLove London - Analysing the SERPs for SEO, Content & Customer Insights
SearchLove London - Analysing the SERPs for SEO, Content & Customer Insights
Rory Truesdale
 
Intellectual theft
Intellectual theftIntellectual theft
Intellectual theft
tinamac25
 
The User is the Query - The Rise of Predictive Proactive Search
The User is the Query - The Rise of Predictive Proactive SearchThe User is the Query - The Rise of Predictive Proactive Search
The User is the Query - The Rise of Predictive Proactive Search
Dawn Anderson MSc DigM
 
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Edureka!
 
Microblog-genre noise and its impact on semantic annotation accuracy
Microblog-genre noise and its impact on semantic annotation accuracyMicroblog-genre noise and its impact on semantic annotation accuracy
Microblog-genre noise and its impact on semantic annotation accuracy
Leon Derczynski
 
Natural language procssing
Natural language procssing Natural language procssing
Natural language procssing
Rajnish Raj
 
Natural language-processing
Natural language-processingNatural language-processing
Natural language-processing
Hareem Naz
 
#5 Predicting Machine Translation Quality
#5 Predicting Machine Translation Quality#5 Predicting Machine Translation Quality
#5 Predicting Machine Translation Quality
Berlin Language Technology
 
Natural lanaguage processing
Natural lanaguage processingNatural lanaguage processing
Natural lanaguage processing
gulshan kumar
 
Introduction to Natural Language Processing
Introduction to Natural Language ProcessingIntroduction to Natural Language Processing
Introduction to Natural Language Processing
Pranav Gupta
 
Effective Data Science: The Art of Delivering and Demonstrating Value
Effective Data Science: The Art of Delivering and Demonstrating ValueEffective Data Science: The Art of Delivering and Demonstrating Value
Effective Data Science: The Art of Delivering and Demonstrating Value
IDEAS - Int'l Data Engineering and Science Association
 
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your AudienceSearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
Distilled
 
Datascope: Designing your Data Viz - The (Iterative) Process
Datascope: Designing your Data Viz - The (Iterative) ProcessDatascope: Designing your Data Viz - The (Iterative) Process
Datascope: Designing your Data Viz - The (Iterative) Process
Mollie Pettit
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
Ila Group
 
You've Got (Big) Data! Now What?
You've Got (Big) Data! Now What?You've Got (Big) Data! Now What?
You've Got (Big) Data! Now What?
Jess Freaner
 
Webinar: Natural Language Search with Solr
Webinar: Natural Language Search with SolrWebinar: Natural Language Search with Solr
Webinar: Natural Language Search with Solr
Lucidworks
 

What's hot (17)

Intro to nlp
Intro to nlpIntro to nlp
Intro to nlp
 
SearchLove London - Analysing the SERPs for SEO, Content & Customer Insights
SearchLove London - Analysing the SERPs for SEO, Content & Customer InsightsSearchLove London - Analysing the SERPs for SEO, Content & Customer Insights
SearchLove London - Analysing the SERPs for SEO, Content & Customer Insights
 
Intellectual theft
Intellectual theftIntellectual theft
Intellectual theft
 
The User is the Query - The Rise of Predictive Proactive Search
The User is the Query - The Rise of Predictive Proactive SearchThe User is the Query - The Rise of Predictive Proactive Search
The User is the Query - The Rise of Predictive Proactive Search
 
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
 
Microblog-genre noise and its impact on semantic annotation accuracy
Microblog-genre noise and its impact on semantic annotation accuracyMicroblog-genre noise and its impact on semantic annotation accuracy
Microblog-genre noise and its impact on semantic annotation accuracy
 
Natural language procssing
Natural language procssing Natural language procssing
Natural language procssing
 
Natural language-processing
Natural language-processingNatural language-processing
Natural language-processing
 
#5 Predicting Machine Translation Quality
#5 Predicting Machine Translation Quality#5 Predicting Machine Translation Quality
#5 Predicting Machine Translation Quality
 
Natural lanaguage processing
Natural lanaguage processingNatural lanaguage processing
Natural lanaguage processing
 
Introduction to Natural Language Processing
Introduction to Natural Language ProcessingIntroduction to Natural Language Processing
Introduction to Natural Language Processing
 
Effective Data Science: The Art of Delivering and Demonstrating Value
Effective Data Science: The Art of Delivering and Demonstrating ValueEffective Data Science: The Art of Delivering and Demonstrating Value
Effective Data Science: The Art of Delivering and Demonstrating Value
 
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your AudienceSearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
 
Datascope: Designing your Data Viz - The (Iterative) Process
Datascope: Designing your Data Viz - The (Iterative) ProcessDatascope: Designing your Data Viz - The (Iterative) Process
Datascope: Designing your Data Viz - The (Iterative) Process
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
You've Got (Big) Data! Now What?
You've Got (Big) Data! Now What?You've Got (Big) Data! Now What?
You've Got (Big) Data! Now What?
 
Webinar: Natural Language Search with Solr
Webinar: Natural Language Search with SolrWebinar: Natural Language Search with Solr
Webinar: Natural Language Search with Solr
 

Similar to Google BERT - What SEOs and Marketers Need to Know

New Words New Toys
New Words New ToysNew Words New Toys
New Words New Toys
resec
 
Natural Language Processing: L01 introduction
Natural Language Processing: L01 introductionNatural Language Processing: L01 introduction
Natural Language Processing: L01 introduction
ananth
 
Metaphic or the art of looking another way.
Metaphic or the art of looking another way.Metaphic or the art of looking another way.
Metaphic or the art of looking another way.
Suresh Manian
 
Using Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL PronunicationUsing Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL Pronunication
Rebecca Allen
 
TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...
TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...
TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...
TAUS - The Language Data Network
 
Why Study Linguistics
Why Study LinguisticsWhy Study Linguistics
Why Study Linguistics
Dr. Cupid Lucid
 
Marriage of speech, vision and natural language processing
Marriage of speech, vision and natural language processingMarriage of speech, vision and natural language processing
Marriage of speech, vision and natural language processing
Yaman Kumar
 
Whystudylinguistics 1225478191520703 8
Whystudylinguistics 1225478191520703 8Whystudylinguistics 1225478191520703 8
Whystudylinguistics 1225478191520703 8
Student at Shah Abdul University Dept of English
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Saurabh Kaushik
 
1 Introduction.ppt
1 Introduction.ppt1 Introduction.ppt
1 Introduction.ppt
tanishamahajan11
 
Proposal presentation.pptx
Proposal presentation.pptxProposal presentation.pptx
Proposal presentation.pptx
Nhlakanipho Majola
 
Writing for extension audiences 2021
Writing for extension audiences  2021Writing for extension audiences  2021
Writing for extension audiences 2021
Amy Cole
 
BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights
BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer InsightsBrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights
BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights
Rory Truesdale
 
Extra credit webinar 1 LIST 4373
Extra credit webinar 1  LIST 4373Extra credit webinar 1  LIST 4373
Extra credit webinar 1 LIST 4373
Peggy Semingson
 
5 complicatedproblems
5 complicatedproblems5 complicatedproblems
5 complicatedproblems
Hywel Evans
 
Isabelle jones-lost in translation
Isabelle jones-lost in translationIsabelle jones-lost in translation
Isabelle jones-lost in translation
Isabelle Jones
 
Speech recognition
Speech recognitionSpeech recognition
Speech recognition
fluffyemily
 
Surname 5Chang QiuLinguistic1112018Machine and Human.docx
Surname 5Chang QiuLinguistic1112018Machine and Human.docxSurname 5Chang QiuLinguistic1112018Machine and Human.docx
Surname 5Chang QiuLinguistic1112018Machine and Human.docx
mabelf3
 
Natural Language Processing - Unit 1
Natural Language Processing - Unit 1Natural Language Processing - Unit 1
Natural Language Processing - Unit 1
Mithun B N
 
M3 l16 translation at facebook
M3 l16 translation at facebookM3 l16 translation at facebook
M3 l16 translation at facebook
BoPeng76
 

Similar to Google BERT - What SEOs and Marketers Need to Know (20)

New Words New Toys
New Words New ToysNew Words New Toys
New Words New Toys
 
Natural Language Processing: L01 introduction
Natural Language Processing: L01 introductionNatural Language Processing: L01 introduction
Natural Language Processing: L01 introduction
 
Metaphic or the art of looking another way.
Metaphic or the art of looking another way.Metaphic or the art of looking another way.
Metaphic or the art of looking another way.
 
Using Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL PronunicationUsing Corpus Linguistics to Teach ESL Pronunication
Using Corpus Linguistics to Teach ESL Pronunication
 
TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...
TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...
TaaS Workshop 2014, Terminology Trends- First-hand Experience as a Blogger, M...
 
Why Study Linguistics
Why Study LinguisticsWhy Study Linguistics
Why Study Linguistics
 
Marriage of speech, vision and natural language processing
Marriage of speech, vision and natural language processingMarriage of speech, vision and natural language processing
Marriage of speech, vision and natural language processing
 
Whystudylinguistics 1225478191520703 8
Whystudylinguistics 1225478191520703 8Whystudylinguistics 1225478191520703 8
Whystudylinguistics 1225478191520703 8
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
 
1 Introduction.ppt
1 Introduction.ppt1 Introduction.ppt
1 Introduction.ppt
 
Proposal presentation.pptx
Proposal presentation.pptxProposal presentation.pptx
Proposal presentation.pptx
 
Writing for extension audiences 2021
Writing for extension audiences  2021Writing for extension audiences  2021
Writing for extension audiences 2021
 
BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights
BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer InsightsBrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights
BrightonSEO 2019 - Mining the SERP for SEO, Content & Customer Insights
 
Extra credit webinar 1 LIST 4373
Extra credit webinar 1  LIST 4373Extra credit webinar 1  LIST 4373
Extra credit webinar 1 LIST 4373
 
5 complicatedproblems
5 complicatedproblems5 complicatedproblems
5 complicatedproblems
 
Isabelle jones-lost in translation
Isabelle jones-lost in translationIsabelle jones-lost in translation
Isabelle jones-lost in translation
 
Speech recognition
Speech recognitionSpeech recognition
Speech recognition
 
Surname 5Chang QiuLinguistic1112018Machine and Human.docx
Surname 5Chang QiuLinguistic1112018Machine and Human.docxSurname 5Chang QiuLinguistic1112018Machine and Human.docx
Surname 5Chang QiuLinguistic1112018Machine and Human.docx
 
Natural Language Processing - Unit 1
Natural Language Processing - Unit 1Natural Language Processing - Unit 1
Natural Language Processing - Unit 1
 
M3 l16 translation at facebook
M3 l16 translation at facebookM3 l16 translation at facebook
M3 l16 translation at facebook
 

More from Dawn Anderson MSc DigM

Human vs AI Quality Raters for Search Engines.pdf
Human vs AI Quality Raters for Search Engines.pdfHuman vs AI Quality Raters for Search Engines.pdf
Human vs AI Quality Raters for Search Engines.pdf
Dawn Anderson MSc DigM
 
Life of An SEO - Surfing The Waves of Googles Many Algorithmic Updates
Life of An SEO - Surfing The Waves of Googles Many Algorithmic UpdatesLife of An SEO - Surfing The Waves of Googles Many Algorithmic Updates
Life of An SEO - Surfing The Waves of Googles Many Algorithmic Updates
Dawn Anderson MSc DigM
 
Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...
Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...
Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...
Dawn Anderson MSc DigM
 
Passage indexing is likely more important than you think
Passage indexing is likely more important than you thinkPassage indexing is likely more important than you think
Passage indexing is likely more important than you think
Dawn Anderson MSc DigM
 
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
Dawn Anderson MSc DigM
 
Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020
Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020
Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020
Dawn Anderson MSc DigM
 
Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...
Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...
Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...
Dawn Anderson MSc DigM
 
Planning an SEO Strategy for a New Website - SMXL Milan 2019
Planning an SEO Strategy for a New Website - SMXL Milan 2019Planning an SEO Strategy for a New Website - SMXL Milan 2019
Planning an SEO Strategy for a New Website - SMXL Milan 2019
Dawn Anderson MSc DigM
 
SEO in a Mobile First World
SEO in a Mobile First WorldSEO in a Mobile First World
SEO in a Mobile First World
Dawn Anderson MSc DigM
 
Modern Ecommerce SEO
Modern Ecommerce SEOModern Ecommerce SEO
Modern Ecommerce SEO
Dawn Anderson MSc DigM
 
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...
Dawn Anderson MSc DigM
 
The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...
The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...
The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...
Dawn Anderson MSc DigM
 
SEO and The Mobile-First Paradigm Shift
SEO and The Mobile-First Paradigm ShiftSEO and The Mobile-First Paradigm Shift
SEO and The Mobile-First Paradigm Shift
Dawn Anderson MSc DigM
 
Pubcon florida 2018 logs dont lie dawn anderson
Pubcon florida 2018 logs dont lie dawn andersonPubcon florida 2018 logs dont lie dawn anderson
Pubcon florida 2018 logs dont lie dawn anderson
Dawn Anderson MSc DigM
 
Voice Search Challenges For Search and Information Retrieval and SEO
Voice Search Challenges For Search and Information Retrieval and SEOVoice Search Challenges For Search and Information Retrieval and SEO
Voice Search Challenges For Search and Information Retrieval and SEO
Dawn Anderson MSc DigM
 
Digital Olympus Technical SEO Findings Whilst Taming An SEO Beast
Digital Olympus Technical SEO Findings Whilst Taming An SEO BeastDigital Olympus Technical SEO Findings Whilst Taming An SEO Beast
Digital Olympus Technical SEO Findings Whilst Taming An SEO Beast
Dawn Anderson MSc DigM
 
Cruft busting technical debt code smell and refactoring for seo - state of ...
Cruft busting   technical debt code smell and refactoring for seo - state of ...Cruft busting   technical debt code smell and refactoring for seo - state of ...
Cruft busting technical debt code smell and refactoring for seo - state of ...
Dawn Anderson MSc DigM
 
Duplicate Content Myths Types and Ways To Make It Work For You
Duplicate Content Myths Types and Ways To Make It Work For YouDuplicate Content Myths Types and Ways To Make It Work For You
Duplicate Content Myths Types and Ways To Make It Work For You
Dawn Anderson MSc DigM
 
SEO - The Rise of Persona Modelled Intent Driven Contextual Search
SEO - The Rise of Persona Modelled Intent Driven Contextual SearchSEO - The Rise of Persona Modelled Intent Driven Contextual Search
SEO - The Rise of Persona Modelled Intent Driven Contextual Search
Dawn Anderson MSc DigM
 
Technical SEO - Generational cruft in SEO - there is never a new site when th...
Technical SEO - Generational cruft in SEO - there is never a new site when th...Technical SEO - Generational cruft in SEO - there is never a new site when th...
Technical SEO - Generational cruft in SEO - there is never a new site when th...
Dawn Anderson MSc DigM
 

More from Dawn Anderson MSc DigM (20)

Human vs AI Quality Raters for Search Engines.pdf
Human vs AI Quality Raters for Search Engines.pdfHuman vs AI Quality Raters for Search Engines.pdf
Human vs AI Quality Raters for Search Engines.pdf
 
Life of An SEO - Surfing The Waves of Googles Many Algorithmic Updates
Life of An SEO - Surfing The Waves of Googles Many Algorithmic UpdatesLife of An SEO - Surfing The Waves of Googles Many Algorithmic Updates
Life of An SEO - Surfing The Waves of Googles Many Algorithmic Updates
 
Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...
Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...
Natural Semantic SEO - Surfacing Walnuts in Densely Represented, Every Increa...
 
Passage indexing is likely more important than you think
Passage indexing is likely more important than you thinkPassage indexing is likely more important than you think
Passage indexing is likely more important than you think
 
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
Zipfs Law & Zipfian Distribution in SEO - Pubcon Virtual Fall 2020 - Dawn And...
 
Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020
Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020
Disambiguating Equiprobability in SEO Dawn Anderson Friends of Search 2020
 
Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...
Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...
Connecting The Worlds of Information Retrieval & SEO - Search solutions 2019 ...
 
Planning an SEO Strategy for a New Website - SMXL Milan 2019
Planning an SEO Strategy for a New Website - SMXL Milan 2019Planning an SEO Strategy for a New Website - SMXL Milan 2019
Planning an SEO Strategy for a New Website - SMXL Milan 2019
 
SEO in a Mobile First World
SEO in a Mobile First WorldSEO in a Mobile First World
SEO in a Mobile First World
 
Modern Ecommerce SEO
Modern Ecommerce SEOModern Ecommerce SEO
Modern Ecommerce SEO
 
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...
 
The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...
The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...
The Iceberg Approach - Power from what lies beneath in SEO for a mobile-first...
 
SEO and The Mobile-First Paradigm Shift
SEO and The Mobile-First Paradigm ShiftSEO and The Mobile-First Paradigm Shift
SEO and The Mobile-First Paradigm Shift
 
Pubcon florida 2018 logs dont lie dawn anderson
Pubcon florida 2018 logs dont lie dawn andersonPubcon florida 2018 logs dont lie dawn anderson
Pubcon florida 2018 logs dont lie dawn anderson
 
Voice Search Challenges For Search and Information Retrieval and SEO
Voice Search Challenges For Search and Information Retrieval and SEOVoice Search Challenges For Search and Information Retrieval and SEO
Voice Search Challenges For Search and Information Retrieval and SEO
 
Digital Olympus Technical SEO Findings Whilst Taming An SEO Beast
Digital Olympus Technical SEO Findings Whilst Taming An SEO BeastDigital Olympus Technical SEO Findings Whilst Taming An SEO Beast
Digital Olympus Technical SEO Findings Whilst Taming An SEO Beast
 
Cruft busting technical debt code smell and refactoring for seo - state of ...
Cruft busting   technical debt code smell and refactoring for seo - state of ...Cruft busting   technical debt code smell and refactoring for seo - state of ...
Cruft busting technical debt code smell and refactoring for seo - state of ...
 
Duplicate Content Myths Types and Ways To Make It Work For You
Duplicate Content Myths Types and Ways To Make It Work For YouDuplicate Content Myths Types and Ways To Make It Work For You
Duplicate Content Myths Types and Ways To Make It Work For You
 
SEO - The Rise of Persona Modelled Intent Driven Contextual Search
SEO - The Rise of Persona Modelled Intent Driven Contextual SearchSEO - The Rise of Persona Modelled Intent Driven Contextual Search
SEO - The Rise of Persona Modelled Intent Driven Contextual Search
 
Technical SEO - Generational cruft in SEO - there is never a new site when th...
Technical SEO - Generational cruft in SEO - there is never a new site when th...Technical SEO - Generational cruft in SEO - there is never a new site when th...
Technical SEO - Generational cruft in SEO - there is never a new site when th...
 

Recently uploaded

Compitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptxCompitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptx
SauravDey45
 
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
omywaf
 
Advanced Storytelling Concepts for Marketers
Advanced Storytelling Concepts for MarketersAdvanced Storytelling Concepts for Marketers
Advanced Storytelling Concepts for Marketers
Ed Shimp
 
DCC Technology Intro Slides June 2024.pptx
DCC Technology Intro Slides June 2024.pptxDCC Technology Intro Slides June 2024.pptx
DCC Technology Intro Slides June 2024.pptx
vishc81
 
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
5ys5mvlp
 
How to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.etHow to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.et
Y
 
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptxINTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
Giorgio Chiesa
 
一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理
一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理
一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理
oeveu
 
一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理
一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理
一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理
dqvc9xf1
 
Digital Marketing Company in India - DIGI Brooks
Digital Marketing Company in India -  DIGI BrooksDigital Marketing Company in India -  DIGI Brooks
Digital Marketing Company in India - DIGI Brooks
DIGI Brooks
 
What is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive GuideWhat is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive Guide
V-tech Marketing
 
From Subreddits To Search: Maximizing Your Brand's Impact On Reddit
From Subreddits To Search: Maximizing Your Brand's Impact On RedditFrom Subreddits To Search: Maximizing Your Brand's Impact On Reddit
From Subreddits To Search: Maximizing Your Brand's Impact On Reddit
Search Engine Journal
 
Digital Marketing Services Agency in USA
Digital Marketing Services Agency in USADigital Marketing Services Agency in USA
Digital Marketing Services Agency in USA
akramulislamtarek007
 
AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024
Amanda Farrell
 
Practical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven KingpdfPractical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven Kingpdf
william charnock
 
How to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social CommerceHow to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social Commerce
Vbout.com
 
Advertising and Promotion of whisper by Sakthi Sundar
Advertising and Promotion of whisper by Sakthi SundarAdvertising and Promotion of whisper by Sakthi Sundar
Advertising and Promotion of whisper by Sakthi Sundar
sakthisundar2001
 
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
Jorge Calmett
 
3 Best “Add to Calendar” Link Generator Tools (2024)
3 Best  “Add to Calendar”  Link Generator Tools (2024)3 Best  “Add to Calendar”  Link Generator Tools (2024)
3 Best “Add to Calendar” Link Generator Tools (2024)
Y
 
SEO in the AI Era - Trust, Quality and Content Discovery - Andy Crestodina
SEO in the AI Era - Trust, Quality and Content Discovery - Andy CrestodinaSEO in the AI Era - Trust, Quality and Content Discovery - Andy Crestodina
SEO in the AI Era - Trust, Quality and Content Discovery - Andy Crestodina
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 

Recently uploaded (20)

Compitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptxCompitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptx
 
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
 
Advanced Storytelling Concepts for Marketers
Advanced Storytelling Concepts for MarketersAdvanced Storytelling Concepts for Marketers
Advanced Storytelling Concepts for Marketers
 
DCC Technology Intro Slides June 2024.pptx
DCC Technology Intro Slides June 2024.pptxDCC Technology Intro Slides June 2024.pptx
DCC Technology Intro Slides June 2024.pptx
 
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
在线办理(英国UWS毕业证书)西苏格兰大学毕业证学位证一模一样
 
How to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.etHow to Generate Add to Calendar Link using Cal.et
How to Generate Add to Calendar Link using Cal.et
 
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptxINTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
INTRODUCTION TO SEARCH ENGINE OPTIMIZATION (SEO).pptx
 
一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理
一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理
一比一原版(TMU毕业证)多伦多都会大学毕业证如何办理
 
一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理
一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理
一比一原版英国南安普顿大学毕业证(Soton学位证)如何办理
 
Digital Marketing Company in India - DIGI Brooks
Digital Marketing Company in India -  DIGI BrooksDigital Marketing Company in India -  DIGI Brooks
Digital Marketing Company in India - DIGI Brooks
 
What is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive GuideWhat is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive Guide
 
From Subreddits To Search: Maximizing Your Brand's Impact On Reddit
From Subreddits To Search: Maximizing Your Brand's Impact On RedditFrom Subreddits To Search: Maximizing Your Brand's Impact On Reddit
From Subreddits To Search: Maximizing Your Brand's Impact On Reddit
 
Digital Marketing Services Agency in USA
Digital Marketing Services Agency in USADigital Marketing Services Agency in USA
Digital Marketing Services Agency in USA
 
AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024AI Best Practices for Marketing HUG June 2024
AI Best Practices for Marketing HUG June 2024
 
Practical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven KingpdfPractical Progress from a Theory by Steven Kingpdf
Practical Progress from a Theory by Steven Kingpdf
 
How to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social CommerceHow to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social Commerce
 
Advertising and Promotion of whisper by Sakthi Sundar
Advertising and Promotion of whisper by Sakthi SundarAdvertising and Promotion of whisper by Sakthi Sundar
Advertising and Promotion of whisper by Sakthi Sundar
 
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
 
3 Best “Add to Calendar” Link Generator Tools (2024)
3 Best  “Add to Calendar”  Link Generator Tools (2024)3 Best  “Add to Calendar”  Link Generator Tools (2024)
3 Best “Add to Calendar” Link Generator Tools (2024)
 
SEO in the AI Era - Trust, Quality and Content Discovery - Andy Crestodina
SEO in the AI Era - Trust, Quality and Content Discovery - Andy CrestodinaSEO in the AI Era - Trust, Quality and Content Discovery - Andy Crestodina
SEO in the AI Era - Trust, Quality and Content Discovery - Andy Crestodina
 

Google BERT - What SEOs and Marketers Need to Know

  • 1. @dawnieando Google BERT What SEOs and Marketers Need to Know
  • 2. @dawnieando Here’s another Bert, & I am Dawn Anderson Managing Director of BerteyBert, my Pomeranian
  • 3. @dawnieando • A Google algorithmic update • Google announce BERT to the organic search world in a VERY geeky way • Mentions of the 15% of new queries every day • Touches on ‘The Vocabulary Problem’ (many ways of querying the same thing) October 2019 - Welcome To Search, BERT
  • 4. @dawnieando • Probably the biggest improvement in search EVER • The biggest change in search in five years, since RankBrain Fundamentally… Google BERT is
  • 5. @dawnieando !Layman’s Terms: it can be used to help Google better understand the context of words in search queries & content So, just what is Google BERT update?
  • 6. @dawnieando • Used globally in all languages on featured snippets • BERT to impact rankings for 1 in 10 queries • Initially for English language queries in US The bottom line search announcement
  • 7. @dawnieando Dec 2019 – BERT expands internationally • Over 70 languages • Still only impacts 10% of queries despite the considerable expansion • Still all featured snippets globally
  • 8. @dawnieando • BERT deals with ambiguity & ‘nuance’ in queries & content • Unlikely to impact short queries • More likely to impact conversational queries • Unlikely to impact branded queries Why just 10% of Google Queries Impacted?
  • 9. @dawnieando • The SEO community is abuzz • BERT is a big deal • Likened to ‘Rank Brain’ in some of the ‘interesting’ interpretations • Some confusions around ‘What BERT is and what it means for search’ SEO’s React
  • 10. @dawnieando !A neural network-based technique for natural language processing pre-training !An anagram of Bi-Directional Encoder Representations from Transformers BERT in Geek Speak
  • 12. @dawnieando • Search algorithm update • Open source pre-trained model / framework for natural language understanding • Academic research paper • Evolving tool for computational linguistics efficiency • Beginning of MANY BERT’ish language models Important: BERT is Many Things
  • 13. @dawnieando So What’s The Backstory? Where%did%BERT%come%from? Where%did%the%need%for%BERT%arise? The$Impact$of$BERT$for$SEO$&$beyond? What%next?
  • 14. @dawnieando • Academic Paper • Research Project by Devlin et al • Published a year before the update in October 2018 • Bert: Pre-training of deep bidirectional transformers for language understanding BERT started as a research paper in 2018
  • 15. @dawnieando • Open sourced so anyone can build a BERT • BERT created a sea-change leap-forward in natural language understanding in information retrieval very quickly • Provided a pre-trained language model which required only fine- tuning BERT Open Sourced in 2018
  • 16. @dawnieando The whole of the English Wikipedia & The Books Corpus combined. Over 2,500 million words BERT Has Been Pre-Trained On Many Words
  • 17. @dawnieando Vanilla BERT provides a pre- trained starting point layer for neural networks in machine learning & natural language diverse tasks The machine learning community got very excited about BERT
  • 18. @dawnieando • BERT is fine-tuned on a variety of downstream NLP tasks, including question and answer datasets BERT Can Be Fine-Tuned in A Short Space of Time
  • 19. @dawnieando • Vanilla BERT can be used ‘out of the box’ or fine-tuned • Provides a great starting point & saves huge amounts of time & money • Those wishing to, ‘can build upon’, and improve BERT BERT Saves Researchers Time AND Money
  • 20. @dawnieando • Microsoft – MT-DNN • Facebook – RoBERTa • XLNet • ERNIE – Baidu • Lots of other contenders Since 2018 Major tech companies extend BERT
  • 21. @dawnieando Training Datasets Like MSMARCO Are Used To Fine Tune With Question & Answer Datasets
  • 22. @dawnieando Real Bing Questions Feed MSMARCO Microsoft Machine Reading Comprehension Dataset. Real Bing User Queries for NLU Research
  • 23. @dawnieando You think SEOs are competitive? ML Engineers are more so • GLUE • SuperGLUE • MSMARCO • SQuAD …And Leaderboards
  • 24. @dawnieando SuperGLUE was created because GLUE got too easy Progress was phenomenal with many new SOTAs
  • 25. @dawnieando Language models like BERT help machines understand the nuance in word’s context and surrounding text cohesion What Purpose Does BERT Serve & How?
  • 26. @dawnieando • Dates back over 60 years old to the Turing Test paper • Aims at understanding the way words fit together with structure and meaning. • NLU is Connected to the field of linguistics (computational linguistics) • Over time, increasingly computational linguistics overflows to a growing online web of content What is Natural Language Understanding?
  • 27. @dawnieando • Natural language understanding requires: • Word’s context • Common sense reasoning Natural Language Recognition is NOT Understanding
  • 28. @dawnieando Humans mostly understand nuance and jargon from multiple meanings in written and spoken word because of ‘context’ Humans ‘Naturally’ Understand Context
  • 29. @dawnieando • Synonymous • Polysemous • Homonymous But Words Can Be VERY Problematic for Machines & Sometimes Even for Humans
  • 30. @dawnieando “The meaning of a word is its use in a language” (Ludwig, Wittgenstein, Philosopher, 1953) Image attribution: Mortiz, Nahr (Public domain) Single Words Have No Meaning
  • 31. @dawnieando The word ‘like’ in this sentence, is both a: !(VBP) : (‘verb’ (non 3rd-person, singular, present) ) !(IN) : (Preposition or subordinating conjunction) An Example of Word’s Meaning Changing • I -> PRP • Like -> VBP • That -> IN • He -> PRP • Is -> VBZ • Like -> IN • That -> DT
  • 33. @dawnieando E.g. Verbs, nouns, adjectives • Penn-treebank tagger -> 36 different parts of speech • CLAWS7 (C7) -> 146 different parts of speech • Brown Corpus Tagger -> 81 different parts of speech Words Are ‘Part of Speech’ When Combined
  • 34. @dawnieando • He kicked the bucket • I have yet to tick that off my bucket list • The bucket was filled with water The Meaning of The Word ‘Bucket’ Changes
  • 35. @dawnieando Words Need ’Text Cohesion’ The$‘Glue’$which$adds$meaning May$historically$be$‘stop$words’ Surrounding)words)can)change)‘intent’ They%add%‘context’
  • 36. @dawnieando ”Ambiguity is the greatest bottleneck to computational knowledge acquisition, the killer problem of all natural language processing.” (Stephen Clark, formerly of Cambridge University & now a full- time research scientist with Google Deep Mind) Ambiguity Is Problematic
  • 37. @dawnieando • Words with a similar meaning to something else • Example: humorous, comical, hilarious, hysterical are ALL synonyms of funny Synonymous (Synonyms)
  • 38. @dawnieando Ambiguity & Polysemy • Ambiguity is at a sentence level • Polysemous words are arguably the most problematic due to ‘nuanced’ nature
  • 39. @dawnieando • Words usually with the same root and multiple meanings • Example: “Run” has 396 Oxford English Dictionary definitions Polysemous (Polysemy)
  • 41. @dawnieando • Words spelt the same but with very different ‘root’ of word meanings • Example: pen (writing implement), pen (pig pen) • Example: rose (stood up / ascended), rose (flower) • Example: bark (dog sound), bark (tree bark) Homonyms
  • 42. @dawnieando Spelt differently with VERY different meanings but sound exactly the same • Draft, draught • Dual, duel • Made, maid • For, fore, four • To, too, two • There, their • Where, wear, were Homophones – Difficult To Disambiguate Verbally
  • 43. @dawnieando Fork handles Four candles Very difficult to disambiguate in spoken word Worse When Words are Joined Together
  • 44. @dawnieando Did you want four candles or fork handles? Much Comedy Comes From ‘Play on Words’
  • 45. @dawnieando Which Does Not Bode Well For Voice Search
  • 46. @dawnieando EXAMPLES • Zipfian Distribution • Firthian Linguistics • Treebanks • Language can be tied back to mathematical spaces & algorithms Language Has Natural Patterns & Phenomena
  • 47. @dawnieando Example: Zipfian Distribution (Power Law) • The frequency of any word in a collection is inversely proportional to its rank in the frequency table • Applies to any word frequency ANYWHERE • Image is 30 Wikipedias
  • 48. @dawnieando To illustrate Zipfian Distribution (Most used Words): Rank Word Frequency)of)Use)in)a)Corpus 1 the 2 be 1/2 3 to 1/3 4 of 1/4 5 and 1/5 6 a 1/6 7 in 1/7 8 that 1/8 9 have 1/9 10 I 1/10
  • 49. @dawnieando “You shall know a word by the company it keeps” (Firth, 1957) Firthian Linguistics One Such Phenomenon is Co-occurrence
  • 50. @dawnieando Words with similar meaning tend to live near each other in a body of text Word’s ‘nearness’ can be measured in mathematical vector spaces – a context vector is ‘word’s company’ Distributional Relatedness & Firthian Linguistics
  • 51. @dawnieando Co-occurrence, Similarity & Relatedness • Language models are trained on large bodies of text to learn ‘distributional similarity’ (co- occurrence)
  • 52. @dawnieando Context Vectors & Word Embeddings • And build vector space models for word embeddings • Models learn the weights of similarity & relatedness distances
  • 53. @dawnieando Context-Free Word Embeddings • Past models have been context-free embeddings • They lacked the ‘text-cohesion necessary to understand a word in context
  • 54. @dawnieando • He kicked the bucket • I have yet to tick that off my bucket list • The bucket was filled with water Remember ‘bucket’ Without Text Cohesion?
  • 55. @dawnieando Word’s Context Still Needed Gaps Filling • Past models used context-free embeddings • A moving ‘context window’ was used to gain word’s context
  • 56. @dawnieando But Even Then True Context Needs Both Sides of a Word • Past models were ‘uni-directional’ • The context window moved from left to right or right to left
  • 57. @dawnieando They Didn’t Look At Words On Either Side Simultaneously
  • 59. @dawnieando • BERT can see the word’s context on both sides of a word in a context window Bi-Directional is The B in BERT
  • 60. @dawnieando !Encoder Representations relates to the input and output process of ‘word’s context’ & embeddings What About Encoder Representations?
  • 61. @dawnieando !Transformer is a big deal !Derived from a 2017 paper called ‘Attention is all you Need’ (Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I., 2017) What About The Transformer Part?
  • 62. @dawnieando Transformer & Attention Works out how important words are to each other in a given context & focuses attention
  • 64. @dawnieando River Bank or Financial Bank? By identifying ‘cheque’ or ‘deposit’ in the company of ‘bank’ BERT can disambiguate from a ‘river’ bank
  • 65. @dawnieando So Where is BERT’s Value in Google Search • Named entity determination • Textual entailment (next sentence prediction) • Coreference resolution • Question answering • Word sense disambiguation • Automatic summarization • Polysemy resolution
  • 66. @dawnieando BERT recognizes the word ‘to’ makes all the difference to the intent of the query BERT and Disambiguating Nuance
  • 67. @dawnieando BERT recognizes the ambiguous word ‘stand’s meaning and importance in the context of the query BERT and Disambiguating Nuance
  • 68. @dawnieando !A single word can change the whole intent of a query !Conversational queries particularly so !The ‘stop words’ are actually part of text-cohesion !Historically ‘stop-words’ were often ignored !The next sentence matters BERT and Intent Understanding
  • 69. @dawnieando Example: “I remember what my Grandad said just before he kicked the bucket.” Next Sentence Prediction (Textual Entailment) Often the next sentence REALLY matters
  • 70. @dawnieando “How far do you reckon I can kick this bucket?” Not What You Expected?
  • 71. @dawnieando • There have been lots of improvement by others upon BERT • Google have likely improved dramatically on BERT too • There were some issues with next-sentence prediction • Facebook built RoBERTa BERT Probably Doesn’t Resemble The Original BERT Paper
  • 72. @dawnieando • Named entity determination • Coreference resolution • Question answering • Word sense disambiguation • Automatic summarization • Polysemy resolution Featured Snippets Knowledge Graph & Web Page Extraction Together
  • 73. @dawnieando !BERT is multilingual from mono-lingual !Other language specific BERTs are being built !Transformer was trained on international translations !Language has transferrable phenomena BERT and International SEO Expect Big Things
  • 74. @dawnieando • Deepset – German BERT • CamemBERT – French BERT • AlBERTo – Italian BERT • RobBERT - Dutch RoBERTa model BERT & International SEO
  • 75. @dawnieando !The challenges of Pygmalion !Conversational search can now ‘scale’ !BERT takes away some of the human labelling effort necessary !Next sentence prediction could impact assistants and clarifying questions BERT and Conversational Search Expect Big Things
  • 76. @dawnieando Semantic Heterogeneity Issues in Entity Oriented Search (Semantic Search) !Helps with anaphora & cataphora resolution (resolving pronouns of entities) !Helps with coreference resolution !Helps with named entity determination !Next sentence prediction could impact assistants and clarifying questions
  • 77. @dawnieando Bing has been BERTing since April 2019 • Impacts ALL Bing queries globally
  • 78. @dawnieando • It’s supposed to be natural • In the same way you can’t optimize for Rank Brain you can’t optimize for BERT • BERT is a tool / learning process in search for disambiguation & contextual understanding of words • BERT is a ‘black-box’ algorithm Why can’t you optimize for BERT?
  • 79. @dawnieando • Black-box algorithm • Hugging Face coined the phrase BERTology • Now a field of study exploring why BERT makes choices • Some concerns over bias & responsible AI Black Box Algorithms & BERTology
  • 80. @dawnieando !Cluster together content and interlink well on topic & nuance !Avoid ‘too-similar’ completing categories - merge !Consider not just the content in the page but the content in the linked pages & sections !Consider the content of the ‘whole domain’ as everything contributes in co-occurrence !Be extra vigilant when ‘pruning Utilising Co-Occurrence Strategically Employ Relatedness
  • 81. @dawnieando Categorisation & Subcategorisation Are King • Employ strong conceptual logic in your site architecture • Be careful with random blogs • If you must ‘tag’, tag thoughfully
  • 82. @dawnieando Anyone can build a BERT to train their own language processing system for a variety of natural language understanding downstream tasks. Fine-tuning can be carried out in a short time BERT represents a union of data science and SEO Anyone Can Use BERT – BERT is a Tool
  • 83. @dawnieando • Automatic categorization & subcategorization of content • Automatic generation of meta-descriptions • Automatic summarization of extracts & teasers • Categorising user-generated content / posts probably better than humans How Could BERT Be Harnessed For Efficiency in SEO? A Few Examples
  • 84. @dawnieando • J R Oakes - @jroakes • Hamlet Batista - @hamletbatista • Andrea Volpini - @cyberandy • Gefen Hermesh - @ghermesh SEOs Are Getting Busy With BERTishness
  • 85. @dawnieando Efficiency is Also A Focus DistilBERT (Hugging'Face) ALBERT'(Google) Fast%BERT
  • 86. @dawnieando • Original BERT was computationally expensive to run • ALBERT stands for A Lite BERT • Increased efficiency • ALBERT is BERT’s natural successor • ALBERT much leaner whilst providing similar results • A joint research work between Google & Toyota ALBERT – BERT’s Successor
  • 87. @dawnieando Reformer (Google) – Transformer’s Successor Understands word’s context from the perspective of a ‘whole novel’. https://venturebeat.com/2020/01/16/goog les-ai-language-model-reformer-can- process-the-entirety-of-novels/
  • 88. @dawnieando Growth has been huge in the natural language processing community – Current Superglue Leaderboard BERT Was Just The Start • Google T5 is winning • Even more advanced technology • Transfer-learning • Expect big things
  • 89. @dawnieando SEE YOU AT THE NEXT SMX!