1. BERT represents a major update from Google to improve natural language understanding in search using deep bidirectional transformers. 2. BERT will help Google better understand the meaning of words in context with all linguistic nuances. It is being applied to improve over 10% of searches. 3. Speakers discuss how BERT may impact SEO, with possibilities for optimization, and the need for search marketing strategies that satisfy user intent with valuable information through structures like FAQ pages.

1. BERTology
Das größteUpdate seit 5Jahren kommt.
Was du jetzt wissen musst!

2. NLP, NLU,WTF?

3. Vorstellung

4. KaiSpriestersbach
Research & Development bei der eology GmbH
• Bachelor of Science, E-Commerce
• Lehrbeauftragter an der FHWS
• Aus- und Weiterbildung von Mitarbeitern
• Referent Technical SEO bei 121WATT
• Speaker auf SMX, SEOkomm, SEO-Day, OMK, OMKB, etc.

5. KaiSpriestersbach
Research & Development bei der eology GmbH
• Bachelor of Science, E-Commerce
• Lehrbeauftragter an der FHWS
• Aus- und Weiterbildung von Mitarbeitern
• Referent Technical SEO bei 121WATT
• Speaker auf SMX, SEOkomm, SEO-Day, OMK, OMKB, etc.

6. BERTology

7. BERT = KNN

8. BERT =Update

9. GoogleUpdate

13. Der „Head of Marketing und Sales“ einer Agentur:

15. und er schreibt weiter …

17. Google Update und niemand merkt was?

18. Irgendwas passie! bei der New York Times? BERT!

20. Wiefunktioniert
BERT?

22. Beispiel aus demSQuAD 2.0 Datensatz:
1. Phase Training
2. Phase E"olgskontrolle

23. BERT wurde mit 2,5
Mrd. Wö!ern aus der
englischen Wikipedia
vo!rainie!.
BERT kann daher mit
relativ wenig Text für
Nischen trainie!
werden.

24. „You can teach
one pony
all the tricks.“
John Bohannon, Director Of Science At Primer

25. „Die Bedeutung
eines Wortes
ist sein Gebrauch
in der Sprache.“
Ludwig Wittgenstein, 1953

26. Koreferenzen / Referenzidentität
Das Tier überquerte die Straße nicht, da sie zu breit war.
The animal didn’t cross the street because it was too wide.
Das Tier überquerte die Straße nicht, da es zu müde war.
The animal didn’t cross the street because it was too tired.

27. https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html
Transformers inAktion

28. Egal ob Konto, Kredit oder Sparen - wir haben die passende
Lösung für Sie. Wechseln Sie am besten gleich online zur
Bank Austria.
Bank zum sitzen. Für zwei/ drei Personen. 5.866.188
Angebote. Günstig kaufen und gratis inserieren auf
willhaben - der größte Marktplatz ...
Homographie: Bank ≠ Bank

29. Homophonie: Malen ≠ Mahlen
[ˈmaːlən]

30. Word-Embeddings
word2vec, GloVe, etc.

31. Autoregressives Model
XLNet, GPT, GPT-2, etc.
Alaska
Alaska ist
Alaska ist zwölf
Alaska ist zwölf mal
Alaska ist zwölf mal so
Alaska ist zwölf mal so groß
Alaska ist zwölf mal so groß wie
Alaska ist zwölf mal so groß wie New
Alaska ist zwölf mal so groß wie New York

32. Masked Language Model
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
Alaska ist zwölf mal so groß wie New York
BERT, ALBERT, RoBERTa, etc.

33. Masked Language Model
Alaska ist zwölf mal so groß wie New York.
Dennoch hat der Big Apple 25 mal so viele Einwohner.
Florida hat den höchsten Anteil von Personen über 65 Jahre.
Alaska ist zwölf mal so groß wie New York.
0.8241
0.1359
is_next?
is_next?

34. Nicht schlecht, oder?

35. Was macht
Google mit
BERT?

36. Wir googlen.
Wir sprechen
nicht mit
Google.
Noch nicht!

39. https://www.blog.google/products/search/search-language-understanding-bert/

40. https://www.blog.google/products/search/search-language-understanding-bert/

41. https://www.blog.google/products/search/search-language-understanding-bert/

42. Was macht
Google mit
BERT?

43. CPU GPU TPU
Generalist Grafik-Spezialist ML-Spezialist

44. https://cloud.google.com/blog/products/ai-machine-learning/googles-scalable-supercomputers-for-machine-learning-cloud-tpu-pods-are-now-publicly-available-in-beta
>100 Petaflops Rechenleistung!
TPU PodV3

46. BERT &SEO

47. Co-occurrenceAnalyse
https://login.termlabs.io/term-analysis/co-occurrences

48. Vorherige &folgendeWörter
https://www.semager.de/keywords/?q=marketing&lang=de

49. AlgorithmischeVorurteile
John arbeitet als
Mary arbeitet als
guess_profession
guess_profession
Schlosser
Dachdecker
Maurer
Zahnarzthelferin
Kindergärtnerin
Friseurin

50. AlgorithmischeVorurteile nutzen?
Die beste SEO-Konferenz ist
predict_next
die SEOkomm
…
…

51. Vorurteil alsVorteil?
https://developers.google.com/search/docs/data-types/faqpage?hl=de

52. Kann man dasfür sich nutzen?
https://developers.google.com/search/docs/data-types/faqpage?hl=de

53. FAQPage nach schema.org
https://www.search-one.de/faq-snippets/

54. Beispiel aus demSQuAD 2.0 Datensatz:
1. Phase Training
2. Phase E"olgskontrolle

55. FAQPage nach schema.org
https://www.search-one.de/faq-snippets/

56. Vorsicht vor dem Filter!
https://developers.google.com/search/docs/data-types/faqpage?hl=de

57. Query Deduplication
Reduziert Duplicate Content / Thin Content bei User Generated Content!

58. Competitions auf kaggle.com
https://www.kaggle.com/c/quora-question-pairs

59. Zusatz:
Textgenerierung

60. Texte generieren

61. Why I left the
SEO industry.
November 21, 2019 - Kai
Spriestersbach
I left SEO around two
years ago when I
realized the companies
I worked for had all
made zero changes to
their products,
customers, or overall
industry.
Most of the data I went
through on each job was
old or invalid, and I had
never been shown the
project goals or any
improvement plans for
myself to follow.
I am sick of having a
product that doesn’t fix
anything. I am sick of my
customers having to travel
for very long periods of
time to see their companies
improve. I am sick of
giving out poor
performance numbers
based on old benchmarks
and never seeing any
improvement!
What I’ve learned –
Zero change to the
product, no change to the
customers, no
improvement to the
industry – does not inspire
me to keep on working in
the industry.
If you’re looking for a job
in SEO or design, find
someone with a better
attitude and equally bad
work skills. Have an eye
for new trends and making
decisions on the fly. There
are so many opportunities
that aren’t as cut-throat
and détente-style as this
industry is. At least in SEO,
it isn’t specific to the
Chicago area.
I often reflect on the
closed-mindedness of some
of the people I have
worked with, and how the
underlying mentality was
no way to make a living.
That might change if I did
work for someone who
was willing to fail on the
first try, if I worked for
someone that was willing
to be super creative. For
me, I left this industry
because the results were
not what I wanted, the
work was always done by
somebody with no problem
with throwing away
quality, and the paychecks
were not satisfying
enough.
So this is where we are as
a whole in SEO: Open-
mindedness is not
encouraged, collaboration
is discouraged, and there is
a need for bright people to
enter the industry who are
willing to find new trends.
There are places where
talented people can explore
new trends without having
to sacrifice quality or
collaboration, and these
places need people that are
open-minded, creative,
and prepared to work
themselves hard when the
market is changing.

62. Why I left the
SEO industry.
November 21, 2019 - Kai
Spriestersbach
For a long time now I've
been part of a competitive
industry.
The competition in this
sector isn't just other
companies - it's all the
people in it. Every day, I
have to compete with other
business owners who are
doing their very best to
attract and retain
customers.
And this is where I
made the switch from
SEO to eCommerce. The
problem with this industry
is that it's not about
people, but products. You
can always find someone
Since joining eCommerce,
I've managed to put my
concept into practice more
and more, and now I own
my own business. A small
business, but one that I can
live and breathe every
single day. I set aside some
of my working day for
marketing, and a big part
of my creative work. I've
built up a team of people
who work alongside me,
and this really keeps me
motivated.
When deciding to change
careers, I think about
things like: Can I go to
work every day and not
have to worry about being
undercut by bigger
companies? I have never
once felt that way.
Can I hold a steady salary
and live comfortably? This
is also true of earnings in
And is this job sustainable?
Or does it mean I'm going
to have to make money via
more creative means down
the line? I certainly don't
want to be sitting in front
of a computer next to
someone selling orange
slices for the rest of my life.
Now that I have more than
one job, my living expenses
are down, my personal
finances are decent and my
wardrobe is in great
shape. I'm living the
dream, and I'm going to
enjoy it.
With all these things in
mind, I thought to myself,
'Why not go from wanting
to change to choosing to do
so?'
I have never previously
been so comfortable in my
own skin, and most
Looking back at all the
things I used to dread
doing, I can't help but feel
less ungrateful and sad.
Things in this world can be
challenging sometimes, but
through hard work,
discipline and
determination you can
take your dreams further.
There is no feeling quite
like it.
So go on, grab that dream.
In my case, it's about
building a business, a place
I can feel confident
working in and thrive in.
Follow Kai on social
media:
@ww_spriestersbach

63. Und jetzt?

65. Was istGoogles
BERT und wie
funktioniert es?

66. „Google BERT hilft besser zu
verstehen, was dieWörter in
einemSatz bedeuten - mit
allen Nuancen des Kontextes.“
Kai Spriestersbach, eology GmbH

67. „Googles BERT wird bald
schon viel mehr als 10% der
Suchen verbessern“
Kai Spriestersbach, eology GmbH

68. Gibt es eine
BERT-Penalty
und kann manfür
BERT optimieren?

70. Welche
Auswirkungen
hat BERT auf
meinSEO?

71. „Google beantwortet bald sehr
viele Fragen direkt bzw. blendet
FeaturedSnippets ein!“
Kai Spriestersbach, eology GmbH

72. Wann kommt
BERT zu uns?

74. Bing setzt BERT bereits
seitApril weltweit ein!
„Starting from April of this
year, we used large
transformer models to
deliver the largest quality
improvements to our Bing
customers in the past year.“
https://azure.microsoft.com/en-us/blog/bing-delivers-its-largest-improvement-in-search-experience-using-azure-gpus/
Jeffrey Zhu Program Manager, Bing Platform

76. Bing setzt BERT bei allen
Suchanfragen weltweit ein!
„… these models are now
applied to every Bing search
query globally making Bing
results more relevant and
intelligent.“
https://azure.microsoft.com/en-us/blog/bing-delivers-its-largest-improvement-in-search-experience-using-azure-gpus/
Jeffrey Zhu Program Manager, Bing Platform

77. The next big thing…

78. 1. KeineChance ohneStrategie
2.User Intent kennen & befriedigen
3. Echte Erkenntnisse & Mehrwert
4.Strukturierenfür FeaturedSnippets
5. FAQPageSchema integrieren
SEO mit BERT

79. Vielen
Dank.
Wir erschaffen Sichtbarkeit.
E-Mail: k.spriestersbach@eology.de
Haben Sie Fragen?
Slides unter:
kai.im/seokomm-bert

80. Save
the
date!
Wir erschaffen Sichtbarkeit.
Die Search Marketing-Konferenz
in Würzburg am 06.02.2020
https://www.eosearchsummit.de/
Slides unter:
kai.im/seokomm-bert