SearchLove San Diego 2019 - Will Critchlow - SEO + CRO: When ‘what’s good for users’ isn’t good for Google

When “what’s good for users”
isn’t good for Google
Will Critchlow -- @willcritchlow
SearchLove San Diego 2019

UX is a ranking
factorHow users see
your site

UX is a ranking
factorAffects organic
search
performance

Not everyone agrees
Don’t worry. I have seen the reddit threads and twitter arguments

Later, I’ll get onto why I don’t care

Round up
UX ranking
factors
Make it
actionable

CTR for algorithm
evaluation
This has been
confirmed
repeatedly.

Paul Haahr
Worth following @haahr
His online bio dramatically
understates his influence:

“I’ve talked explicitly about using
clickthrough rate for experiments”

https://youtu.be/eGSGZMI4Z_I?t=979

CTR for
personalization
Slightly more
controversial as
we don’t know
exactly what they
mean.

“We use it [CTR] for personalization”

Human UX
ratings
Qualitative ratings
of pages that rank:
used to evaluate
algorithms

“... make this page a frustratingly
poor user experience”
-- reason for page quality rating
Search quality rater guidelines [PDF]

We also got to see the “leaked” UX playbooks come out of Google recently across a
range of industries - getting deep into implementation details. [H/T @gfiorelli1]

ML models
trained on ratings
E.g. Panda -
trained to like the
things humans
like

Back in 2011, I was suggesting we run our own
Panda-like quality surveys (WBF here, instructions here)

Sidenote: totally worth running
these surveys for executive buy-in
Since Panda went real-time, quality issues don’t necessarily cause
obvious drops correlated with algorithm history dates

Client site 1 Client site 2
Would you trust information from this website? 72% 64%

Client site 1 Client site 2 Key competitor
Would you trust information from this website? 72% 64% 81%

Is this website written by experts? 50% 46% 65%

Would you give this site your credit card details? 29% 21% 43%

Are there any noticeable errors on this page? 6% 4% 1%

Does this page provide original content or info? 76% 72% 85%

Would you recognize this site as an authority? 44% 33% 58%

Does this website contain insightful analysis? 72% 62% 81%

Would you consider bookmarking pages on this site? 44% 38% 56%

Are there excessive adverts on this website? 2% 2% 8%

Are there excessive adverts on this website? 2% 2% 8%
Could pages from this site appear in print? 54% 54% 59%

We also asked for free-text feedback and
found some surprising priorities from
non-SEOs

“The reviews seem fake”
Trust is a huge deal for real-world users

“There's not enough information
about the company and why I
should use their products”
On a micro-site that doesn’t have an “about” page

“In this day and age every page that
has anything at all to do with
business should be https”
Security is a big deal in B2B - even without on-site purchases

“The pictures were of low quality
and blurry”
We know this matters to users. It’s at the easier end of ML detection

Site speed
measured
directly
Real User Metrics
including site
speed from
Chrome

https://moz.com/blog/google-chrome-usage-data-measure-site-speed

https://moz.com/blog/google-chrome-usage-data-measure-site-speed
First announced at SearchLove San Diego last year

“The Chrome User Experience Report is powered by
real user measurement of key user experience
metrics across the public web, aggregated from
users who have opted-in to syncing their browsing
history, have not set up a Sync passphrase, and have
usage statistic reporting enabled.”
Chrome User Experience Report documentation and announcement

Soft denial by Gary Illyes in his AMA
Q: Is CrUX data being used to assess page
speed as a ranking factor?
A: “I don't remember where the data is
coming from, but not our publicly
available tools”

CTR as direct
ranking factor
This has been
explicitly denied
by Google reps.

https://www.slideshare.net/randfish/mad-science-experiments-in-seo-social-media

We’ve seen similar things in-person
with experiments at conferences
[Note: others have tried to reproduce results more recently including this failure to do so by Bartosz]

More:
https://sparktoro.com/blog/queries-clicks-influence-googles-results/

https://www.reddit.com/r/TechSEO/comments/ao3fmk/i_am_gary_illyes_googles_chief_of_sunshine_an

“In the most general, broadest way - without
attempting to skirt, skip, duck or dive it, does
Google use interactions of searchers to alter
what position certain results may hold? Those
actions could include, but aren't limited to; CTR,
return and click another listing, adjusting the
query, repeat search and repeat visit etc.”
-- /user/Darth_Autocrat

“we primarily use these things in
evaluations.”

“Can you please confirm/deny whether
[RankBrain] uses UX signals of any kind?”
-- /user/Darth_Autocrat

“Dwell time, CTR, whatever
Fishkin's new theory is, those are
generally made up crap”

A little devil’s advocacy: are there any charitable interpretations?

Technically correct
Not exactly clickthrough rate
[Something more like “difference between expected and observed CTR”?]

Technically correct
Not exactly a “ranking factor”
[They call it something more like a “post-evaluation modifier” or something?]

Dodging the question
“primarily use”
[and maybe limiting the “crap” comments to RankBrain?]

1. They aren’t marketers
2. Their interests aren’t
aligned with yours

aligned with yours
3. They “don’t
understand what it’s
doing exactly”(*)
(*) Haahr on RankBrain

I gave a
presentation here
in SD last year:
“From the horse’s
mouth”
Slides & Video

But grrrr:
a) their job is
supposed to be
to help
webmasters

and
b) calling it
“made up crap” is
unprofessional
and discourages
the vulnerable

aligned with yours
3. They “don’t
understand what it’s
doing exactly”(*)
(*) Haahr on RankBrain
Remember:

Some SERP
interaction as
some kind of
ranking factor
My theory.

We sell custom cigar humidors. Our
custom cigar humidors are handmade. If
you’re thinking of buying a custom cigar
humidor, please contact our custom
cigar humidor specialists at
custom.cigar.humidors@example.com
Keyword stuffing: Doesn’t mean keyword density is the correct metric

Controversial Accepted
Future
More intent
measurement?

Future
More intent
measurement?
[“Needs met” is
already the
primary metric]

Future
More intent
measurement?
Measure task
completion?

Future
More intent
measurement?
Measure task
completion?
Consider
conversion rate?

So, given all of this, ALL we need to
do is optimize for UX then?

Haahr talked about evaluating
algorithms on “next page click rate”

Adding white
space gamed
that metric

It’s also really hard, and people are
confusing

“There are so many experiments
that we’ve done that have very
misleading live metrics that you
really need to dig into”

“We had a long-running experiment
that swapped results two and four”

“My guess is more people would
scroll down and click on position
three”

“...actually more people click on
[position] one”

It is easy to think you have shown
something, but find it full of
confounding details when you dig
into the individual results
We have seen the same thing with our testing

We just need to know that UX is aligned with Google’s goals
But we don’t need the futurist
perspectives or the tinfoil hat conspiracy
theories for my purposes here

We just need to know that UX is aligned with Google’s goals
1. Panda (ML-trained on UX signals)
2. Quant and qual measurement of algo
changes using UX metrics

We should be building SEO
hypotheses from UX fundamentals

If we’re talking UX, we’re talking CRO:
Conversion Rate Optimization

Most organisations treat them entirely separately
SEO CRO
Search Engine
Optimization
Conversion Rate
Optimization

But in truth there is a lot of overlap
SEO CRO

While some SEO changes don’t impact CRO
SEO
Meta info -
invisible to UX CRO

And some CRO changes don’t impact SEO
SEO
Non-indexed
pagesCRO

An awful lot of changes affect both

“How do we know the tests won’t
hurt conversion rate?”

Never: “how do we know our CRO
tests aren’t hurting rankings?”

Anyway. You ought to worry about BOTH

I wrote a post at moz.com about how big an impact this can have

You think this is what happens as you roll out winning CRO tests

But you don’t realise that some of them are bad for search

What if there was a way to GET the best
of both?

You can do better if you only roll out the true winners

SEO split testing is template basedOur solution: full funnel testing

Conversion rate
optimization focuses on
converting more of your
existing traffic (bottom of
the funnel)
Conversion Rate
Optimization

Conversion rate
the funnel)
Search engine
getting additional traffic to
your website (top of
funnel)
Conversion Rate
Optimization
Search Engine
Optimization

Conversion rate
the funnel)
Search engine
getting additional traffic to
your website (top of
funnel)
Full funnel testing
combines these
approaches to do both at
the same time resulting in
higher ROI from testing
Conversion Rate
Optimization
Search Engine
Optimization
Full Funnel Testing

SEO split testing is template basedHere’s how it works - on an example website
Home
Animals
Cats Dogs Unicorns Badgers

CatsCats
We want to test two templates for our animals pages

CRO test - different users see different templates on the same pages
1

CRO test - different users see different templates on the same pages
1
2
Cats BadgersDogs Unicorns

Cats Badgers
SEO test - different users see the same templates on DIFFERENT pages
1
2
Cats Badgers
Cats Badgers

Cats Badgers Unicorns Dogs
SEO test - different users see the same templates on DIFFERENT pages
1
2

The site looks like this during the test
Home
Animals
Cats Dogs
Countries
Scotland England Ireland WalesUnicorns Badgers

Animals
A B A B A B A B
Animals
A B A B
SEO testingCRO testing vs.

Google’s view Animals
AB BA

User view Animals
AB BA
?????

Animals
AB BA
Landing page
established
What do unicorns eat

SEO test
complete
Animals
AB BA

Cookied
AA AA
Animals

Second page
view
Animals
AA AA

Third page view Animals
AA AA

Sidenote:
This just makes better SEO tests
anyway BTW
By avoiding an inconsistent user experience as they browse the site

Not everyone agrees

To know that we should be building
SEO hypotheses from UX
fundamentals

BUT: then we affect users and search

SEO split testing is template basedWhich is why I believe the future of SEO is testing

Thank you
Will Critchlow, CEO Distilled
@willcritchlow
With help, ideas, slides and more from @craigbradford,
@dom_woodman and @tomanthonyseo

Image credits
● Hornet
● Beach
● Paul Haahr
● Google
● Smile
● Pug
● Chrome
● Tinfoil hat
● Broken plate
● Confused monkey
● Unicyclist
● Horse
● Phone
● Devil cat
● Coloured smoke
● San Diego

SearchLove San Diego 2019 - Will Critchlow - SEO + CRO: When ‘what’s good for users’ isn’t good for Google

More Related Content

What's hot

Similar to SearchLove San Diego 2019 - Will Critchlow - SEO + CRO: When ‘what’s good for users’ isn’t good for Google

More from Distilled

Recently uploaded

SearchLove San Diego 2019 - Will Critchlow - SEO + CRO: When ‘what’s good for users’ isn’t good for Google