SearchLove Boston 2019 - Will Critchlow - SEO + CRO: When ‘What’s Good for Users’ Isn’t Good for Google
•
2 likes•1,000 views
The document discusses Google's use of user experience (UX) signals as ranking factors. It notes that while Google representatives have denied that click-through rate (CTR) is directly used as a ranking factor, some evidence suggests it may influence rankings through personalization or as a post-evaluation modifier. The document also discusses Google's use of human UX ratings to evaluate search algorithms, as well as training machine learning models like Panda using such ratings. In the future, Google may measure user intent and task completion more directly. However, optimizing only for UX is not sufficient, as Google's goals are aligned with but not defined by UX alone.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to SearchLove Boston 2019 - Will Critchlow - SEO + CRO: When ‘What’s Good for Users’ Isn’t Good for Google
Similar to SearchLove Boston 2019 - Will Critchlow - SEO + CRO: When ‘What’s Good for Users’ Isn’t Good for Google (20)
More from Distilled
More from Distilled (20)
Recently uploaded
Recently uploaded (20)
SearchLove Boston 2019 - Will Critchlow - SEO + CRO: When ‘What’s Good for Users’ Isn’t Good for Google
- 1. When “what’s good for users” isn’t good for Google Will Critchlow -- @willcritchlow SearchLove Boston 2019
- 2. UX is a ranking factor
- 3. UX is a ranking factor
- 4. UX is a ranking factorHow users see your site
- 5. UX is a ranking factor
- 6. UX is a ranking factor
- 7. UX is a ranking factorAffects organic search performance
- 8. Not everyone agrees Don’t worry. I have seen the reddit threads and twitter arguments
- 9. Later, I’ll get onto why I don’t care
- 12. Round up UX ranking factors Make it actionable
- 14. CTR for algorithm evaluation This has been confirmed repeatedly.
- 15. Paul Haahr Worth following @haahr His online bio dramatically understates his influence:
- 19. “I’ve talked explicitly about using clickthrough rate for experiments”
- 22. CTR for personalization Slightly more controversial as we don’t know exactly what they mean.
- 23. “We use it [CTR] for personalization”
- 26. Human UX ratings Qualitative ratings of pages that rank: used to evaluate algorithms
- 29. “... make this page a frustratingly poor user experience” -- reason for page quality rating Search quality rater guidelines [PDF]
- 30. We also got to see the “leaked” UX playbooks come out of Google recently across a range of industries - getting deep into implementation details. [H/T @gfiorelli1]
- 33. ML models trained on ratings E.g. Panda - trained to like the things humans like
- 34. Back in 2011, I was suggesting we run our own Panda-like quality surveys (WBF here, instructions here)
- 35. Sidenote: totally worth running these surveys for executive buy-in Since Panda went real-time, quality issues don’t necessarily cause obvious drops correlated with algorithm history dates
- 36. Client site 1 Client site 2 Would you trust information from this website? 72% 64%
- 37. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81%
- 38. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65%
- 39. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43%
- 40. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1%
- 41. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85%
- 42. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85% Would you recognize this site as an authority? 44% 33% 58%
- 43. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85% Would you recognize this site as an authority? 44% 33% 58% Does this website contain insightful analysis? 72% 62% 81%
- 44. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85% Would you recognize this site as an authority? 44% 33% 58% Does this website contain insightful analysis? 72% 62% 81% Would you consider bookmarking pages on this site? 44% 38% 56%
- 45. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85% Would you recognize this site as an authority? 44% 33% 58% Does this website contain insightful analysis? 72% 62% 81% Would you consider bookmarking pages on this site? 44% 38% 56% Are there excessive adverts on this website? 2% 2% 8%
- 46. Client site 1 Client site 2 Key competitor Would you trust information from this website? 72% 64% 81% Is this website written by experts? 50% 46% 65% Would you give this site your credit card details? 29% 21% 43% Are there any noticeable errors on this page? 6% 4% 1% Does this page provide original content or info? 76% 72% 85% Would you recognize this site as an authority? 44% 33% 58% Does this website contain insightful analysis? 72% 62% 81% Would you consider bookmarking pages on this site? 44% 38% 56% Are there excessive adverts on this website? 2% 2% 8% Could pages from this site appear in print? 54% 54% 59%
- 47. We also asked for free-text feedback and found some surprising priorities from non-SEOs
- 48. “The reviews seem fake” Trust is a huge deal for real-world users
- 49. “There's not enough information about the company and why I should use their products” On a micro-site that doesn’t have an “about” page
- 50. “In this day and age every page that has anything at all to do with business should be https” Security is a big deal in B2B - even without on-site purchases
- 51. “The pictures were of low quality and blurry” We know this matters to users. It’s at the easier end of ML detection
- 53. Site speed measured directly Real User Metrics including site speed from Chrome
- 55. https://moz.com/blog/google-chrome-usage-data-measure-site-speed First announced at SearchLove San Diego last year
- 56. “The Chrome User Experience Report is powered by real user measurement of key user experience metrics across the public web, aggregated from users who have opted-in to syncing their browsing history, have not set up a Sync passphrase, and have usage statistic reporting enabled.” Chrome User Experience Report documentation and announcement
- 57. Soft denial by Gary Illyes in his AMA Q: Is CrUX data being used to assess page speed as a ranking factor? A: “I don't remember where the data is coming from, but not our publicly available tools”
- 59. CTR as direct ranking factor This has been explicitly denied by Google reps.
- 61. MozCon 2014
- 65. We’ve seen similar things in-person with experiments at conferences [Note: others have tried to reproduce results more recently including this failure to do so by Bartosz]
- 67. But: not everyone agrees
- 68. https://www.reddit.com/r/TechSEO/comments/ao3fmk/i_am_gary_illyes_googles_chief_of_sunshine_and/
- 69. “In the most general, broadest way - without attempting to skirt, skip, duck or dive it, does Google use interactions of searchers to alter what position certain results may hold? Those actions could include, but aren't limited to; CTR, return and click another listing, adjusting the query, repeat search and repeat visit etc.” -- /user/Darth_Autocrat
- 70. “we primarily use these things in evaluations.”
- 71. “Can you please confirm/deny whether [RankBrain] uses UX signals of any kind?” -- /user/Darth_Autocrat
- 72. “Dwell time, CTR, whatever Fishkin's new theory is, those are generally made up crap”
- 73. “Dwell time, CTR, whatever Fishkin's new theory is, those are generally made up crap”
- 74. A little devil’s advocacy: are there any charitable interpretations?
- 75. Technically correct Not exactly clickthrough rate [Something more like “difference between expected and observed CTR”?]
- 76. Technically correct Not exactly a “ranking factor” [They call it something more like a “post-evaluation modifier” or something?]
- 77. Dodging the question “primarily use” [and maybe limiting the “crap” comments to RankBrain?]
- 78. Dodging the question “primarily use” [and maybe limiting the “crap” comments to RankBrain?]
- 79. Dodging the question “primarily use” [and maybe limiting the “crap” comments to RankBrain?]
- 80. 1. They aren’t marketers
- 81. 1. They aren’t marketers 2. Their interests aren’t aligned with yours
- 82. 1. They aren’t marketers 2. Their interests aren’t aligned with yours 3. They “don’t understand what it’s doing exactly”(*) (*) Haahr on RankBrain
- 83. I gave a presentation here last year: “From the horse’s mouth” Slides & Video
- 84. But grrrr: a) their job is supposed to be to help webmasters
- 85. and b) calling it “made up crap” is unprofessional and discourages the vulnerable
- 86. 1. They aren’t marketers 2. Their interests aren’t aligned with yours 3. They “don’t understand what it’s doing exactly”(*) (*) Haahr on RankBrain Remember:
- 87. Some SERP interaction as some kind of ranking factor My theory.
- 88. We sell custom cigar humidors. Our custom cigar humidors are handmade. If you’re thinking of buying a custom cigar humidor, please contact our custom cigar humidor specialists at custom.cigar.humidors@example.com Keyword stuffing: Doesn’t mean keyword density is the correct metric
- 93. Controversial Accepted Future More intent measurement? [“Needs met” is already the primary metric]
- 94. Controversial Accepted Future More intent measurement? Measure task completion?
- 95. Controversial Accepted Future More intent measurement? Measure task completion? Consider conversion rate?
- 96. So, given all of this, ALL we need to do is optimize for UX then?
- 99. Haahr talked about evaluating algorithms on “next page click rate”
- 100. Adding white space gamed that metric
- 101. It’s also really hard, and people are confusing
- 102. “There are so many experiments that we’ve done that have very misleading live metrics that you really need to dig into”
- 103. “We had a long-running experiment that swapped results two and four”
- 104. “My guess is more people would scroll down and click on position three”
- 105. “...actually more people click on [position] one”
- 107. It is easy to think you have shown something, but find it full of confounding details when you dig into the individual results We have seen the same thing with our testing
- 108. We just need to know that UX is aligned with Google’s goals But we don’t need the futurist perspectives or the tinfoil hat conspiracy theories for my purposes here
- 109. We just need to know that UX is aligned with Google’s goals 1. Panda (ML-trained on UX signals) 2. Quant and qual measurement of algo changes using UX metrics
- 110. We should be building SEO hypotheses from UX fundamentals
- 111. If we’re talking UX, we’re talking CRO: Conversion Rate Optimization
- 112. Most organisations treat them entirely separately SEO CRO Search Engine Optimization Conversion Rate Optimization
- 113. But in truth there is a lot of overlap SEO CRO
- 114. While some SEO changes don’t impact CRO SEO Meta info - invisible to UX CRO
- 115. And some CRO changes don’t impact SEO SEO Non-indexed pagesCRO
- 116. An awful lot of changes affect both
- 117. “How do we know the tests won’t hurt conversion rate?”
- 118. Never: “how do we know our CRO tests aren’t hurting rankings?”
- 121. Anyway. You ought to worry about BOTH
- 122. When CRO messes up SEO
- 123. When CRO messes up SEO -25%!
- 124. I wrote a post at moz.com about how big an impact this can have
- 125. You think this is what happens as you roll out winning CRO tests
- 126. But you don’t realise that some of them are bad for search
- 127. What if there was a way to GET the best of both?
- 128. You can do better if you only roll out the true winners
- 129. SEO split testing is template basedOur solution: full funnel testing
- 130. Conversion rate optimization focuses on converting more of your existing traffic (bottom of the funnel) Conversion Rate Optimization
- 131. Conversion rate optimization focuses on converting more of your existing traffic (bottom of the funnel) Search engine optimization focuses on getting additional traffic to your website (top of funnel) Conversion Rate Optimization Search Engine Optimization
- 132. Conversion rate optimization focuses on converting more of your existing traffic (bottom of the funnel) Search engine optimization focuses on getting additional traffic to your website (top of funnel) Full funnel testing combines these approaches to do both at the same time resulting in higher ROI from testing Conversion Rate Optimization Search Engine Optimization Full Funnel Testing
- 133. How does it work?
- 134. SEO split testing is template basedHere’s how it works - on an example website Home Animals Cats Dogs Unicorns Badgers
- 135. CatsCats We want to test two templates for our animals pages
- 136. Cats Dogs Unicorns Badgers CRO test - different users see different templates on the same pages 1
- 137. Cats Dogs Unicorns Badgers CRO test - different users see different templates on the same pages 1 2 Cats BadgersDogs Unicorns
- 138. Cats Badgers SEO test - different users see the same templates on DIFFERENT pages 1 2 Cats Badgers Cats Badgers
- 139. Cats Badgers Unicorns Dogs SEO test - different users see the same templates on DIFFERENT pages Cats Badgers Unicorns Dogs Cats Badgers Unicorns Dogs 1 2
- 140. The site looks like this during the test Home Animals Cats Dogs Countries Scotland England Ireland WalesUnicorns Badgers
- 141. Animals A B A B A B A B Animals A B A B SEO testingCRO testing vs.
- 142. Google’s view Animals Cats Dogs Unicorns Badgers AB BA
- 143. User view Animals Cats Dogs Unicorns Badgers AB BA ?????
- 144. Animals Cats Dogs Unicorns Badgers AB BA Landing page established What do unicorns eat
- 145. SEO test complete Animals Cats Dogs Unicorns Badgers AB BA What do unicorns eat
- 146. Cookied AA AA What do unicorns eat Animals Cats Dogs Unicorns Badgers
- 147. Second page view Animals Cats Dogs Unicorns Badgers AA AA What do unicorns eat
- 148. Third page view Animals Cats Dogs Unicorns Badgers AA AA What do unicorns eat
- 149. Sidenote: This just makes better SEO tests anyway BTW By avoiding an inconsistent user experience as they browse the site
- 151. In summary
- 152. UX is a ranking factor?
- 153. Not everyone agrees Controversial Accepted
- 154. We just need to know that UX is aligned with Google’s goals But we don’t need the futurist perspectives or the tinfoil hat conspiracy theories for my purposes here
- 155. We just need to know that UX is aligned with Google’s goals 1. Panda (ML-trained on UX signals) 2. Quant and qual measurement of algo changes using UX metrics
- 156. To know that we should be building SEO hypotheses from UX fundamentals
- 157. BUT: then we affect users and search
- 158. SEO split testing is template basedWhich is why I believe the future of SEO is testing
- 159. Thank you Will Critchlow, CEO Distilled @willcritchlow With help, ideas, slides and more from @craigbradford, @dom_woodman and @tomanthonyseo
- 161. Image credits ● Hornet ● Beach ● Paul Haahr ● Google ● Smile ● Pug ● Chrome ● Tinfoil hat ● Broken plate ● Confused monkey ● Unicyclist ● Horse ● Phone ● Devil cat ● Coloured smoke ● Boston