Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Evolution of SEO: Split-Testing for Search, by Rob Ousbey

4,183 views

Published on

Presented live at Mozcon 2019:
Google's algorithms have undergone significant changes in recent years. Traditional ranking signals don't hold the same sway they used to, and they're being usurped by factors like UX and brand that are becoming more important than ever before. What's an SEO to do? The answer lies in testing.

Sharing original data and results from clients, Rob will highlight the necessity of testing, learning, and iterating your work, from traditional UX testing to weighing the impact of technical SEO changes, tweaking on-page elements, and changing up content on key pages. Actionable processes and real-world results abound in this thoughtful presentation on why you should be testing SEO changes, how and where to run them, and what kinds of tests you ought to consider for your circumstances.

Published in: Marketing
  • Since using Brain Training for Dogs my dogs are more excited to learn. They actively want to take part and figure out what they need to do! It's not about "commanding" that they perform a behavior, it's about getting them thinking. I really love this approach to dog training and my dogs do too! ●●● https://tinyurl.com/rrpbzfr
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

The Evolution of SEO: Split-Testing for Search, by Rob Ousbey

  1. 1. The Evolution of SEO: Split-Testing For Search @RobOusbey
  2. 2. Everything Changes When you’re newer to an industry / an organization / a department, you have the opportunity to ask questions like “why do we do this thing?” and “why do we do it this way?” and you don’t put up with the answer “because that’s the way we’ve always done it..”
  3. 3. @RobOusbey @Distilled Everything Changes
  4. 4. SEO is Changing 15 years ago, the ranking factors we talked about were almost exclusively around term-relevance and the link-based authority of the site & page.
  5. 5. SEO is Changing Is the page relevant to the term? Is the page/site authoritative?
  6. 6. SEO is Changing The ‘ranking factors’ were determined and tweaked by a team of humans inside Google.
  7. 7. SEO is Changing In response, we ran correlation studies to try to reverse engineer what they were doing.
  8. 8. Google’s Goal: Portal to the Internet Google’s aspiration was to be your jumping-off point onto the rest of the web. If you clicked through to a site and had a good internet experience, it reflected well on them. And if you had a bad experience, it reflected poorly on them for sending you to that site.
  9. 9. Malware & viruses Paywall / registration-wall on landing page Out of stock products Out of date information Pages that make it hard to find information Pages that are full of advertising Slow Website Bad User Experiences
  10. 10. Measuring User Satisfaction They tried assessing the users’ satisfaction with results by directly asking us what we thought.
  11. 11. Google Has More and More Data But Google ended up being able to collect more data about users and their paths than anyone ever imagined.
  12. 12. Google Has More and More Data (Though we don’t know which of these data sources they’ve ever used as direct or indirect inputs.)
  13. 13. Google is Comfortable with Machine Learning https://www.wired.com/2016/06/how-google-is-remaking-itself-as-a-machine-learning-first-company/ They also become much more amenable to using machine learning to process all of this data. This has better allowed user satisfaction to be measured directly. A good example would be if a site ranked number one, but everyone who clicked on it then clicked back to the results and selected something else, then maybe it shouldn’t be ranking there.
  14. 14. @RobOusbey @Distilled What We’re Seeing Now
  15. 15. Contemporary SEO
  16. 16. Contemporary SEO Page 1 Page 2+ For any search, most of the clicks are on page 1, rather than page 2 and beyond.
  17. 17. Contemporary SEO Page 1 Page 2+ Very little user data Loads of user data So there’s a lot more user data available to Google about the engagement with page 1. And there’s very little data about the deeper results.
  18. 18. Contemporary SEO Page 1 Page 2+ Some correlation with links Data via: distilled.net On pages 2+, where there’s not much user data, we see a pretty reasonable correlation between links and rankings.
  19. 19. Contemporary SEO Page 1 Page 2+ Some correlation with links Less correlation with links Data via: distilled.net But at the bottom of page 1, the correlation is weaker.
  20. 20. Contemporary SEO Page 1 Page 2+ Some correlation with links Weakest correlation with links Data via: distilled.net And at the top of the first page, there’s barely any correlation between rankings & links! Less correlation with links
  21. 21. Contemporary SEO Page 1 And… this correlation is getting weaker every year. Data via: distilled.net Less correlation with links Weakest correlation with links
  22. 22. Contemporary SEO SearchVolume Very little user data Loads of user data Data via: distilled.net When we think about the head & long tail of keywords, we know that there is much more user data available for head terms than then long tail.
  23. 23. Contemporary SEO SearchVolume Rankings more based on user engagement metrics Rankings are more link-based Data via: distilled.net And again, we see that the long tail rankings are more link based. High volume head terms are more dictated by user engagement, brand strength, etc.
  24. 24. Contemporary SEO Larry Kim’s research into click-through-rate’s influence on rankings: moz.com/blog/higher-organic-click-through-conversion-rates-rankings
  25. 25. Contemporary SEO Brian Dean’s conclusions about the impact of bounce rate on rankings: backlinko.com/search-engine-ranking
  26. 26. Contemporary SEO Searchmetrics’ analysis of time on site’s correlation with rankings: www.searchmetrics.com/ranking-factors
  27. 27. Contemporary SEO Google have even been somewhat open about the fact that they are at least looking at these kinds of metrics.
  28. 28. Contemporary SEO “The team ran through various data points, like what percent of users clicked through a picture-link and then quickly clicked back (a bad sign), or whether there was a significant increase in the time until they made their first interaction with the results (also bad).” https://www.cnbc.com/2018/09/17/google-tests-changes-to-its-search-algorithm-how-search-works.html
  29. 29. User Metrics as Ranking Factors When we see a site like this, with pop-ups and ads obscuring the content, we would expect to see search visibility declining.
  30. 30. User Metrics as Ranking Factors Which is exactly what’s happened.
  31. 31. User Metrics as Ranking Factors This site added a registration form before you could see any of the content.
  32. 32. User Metrics as Ranking Factors And they have been losing search visibility.
  33. 33. @RobOusbey Forbes is on another level.
  34. 34. @RobOusbey
  35. 35. @RobOusbey
  36. 36. @RobOusbey
  37. 37. @RobOusbey
  38. 38. @RobOusbey
  39. 39. @RobOusbey
  40. 40. @RobOusbey I would imagine that people are clicking back away from Forbes, to get back to the search results. Or possibly even avoiding clicking on a Forbes result in the first place.
  41. 41. SEO is Changing 700 new linking sites, every day And even though they’re still earning a lot of links….
  42. 42. SEO is Changing 35% y/y decline … they’ve lost 35% search visibility over the last year.
  43. 43. SEO is Changing https://moz.com/blog/how-often-does-google-update-its-algorithm It’s still not a cliché to say that SEO is changing.
  44. 44. @RobOusbey @Distilled The SEO Process But I don’t think our standard SEO processes have kept up with all of this.
  45. 45. The SEO Process Traditionally, we’d read a bunch of articles with ideas for things we needed to do.
  46. 46. The SEO Process
  47. 47. The SEO Process
  48. 48. The SEO Process 2018 20162014 Or watch these “so-called experts” on stage at conferences, talking about what worked for them.
  49. 49. The SEO Process Then you come up with a bunch of ideas.
  50. 50. The SEO Process And – hopefully – get them all implemented.
  51. 51. The SEO Process Outcome: 14% traffic increase With any luck, traffic goes up!
  52. 52. The SEO Process +2% +2% +2% +2% +2% Outcome: 14% traffic increase +2% +2% But do you know if each change contributed an equal amount?
  53. 53. The SEO Process 0% 0% 0% 0% 0% Outcome: 14% traffic increase +8% +6% Or did only some of the changes have an impact?
  54. 54. The SEO Process 0% 0% -1% -3% -4% Outcome: 14% traffic increase +12% +10% Or – worse still – what if some of the changes actually harmed traffic? (But the success of the other changes masked this effect.)
  55. 55. The SEO Process 0% 0% -1% -3% -4% Outcome: 22% traffic increase +12% +10% If you’d known what the impact of each change was going to be, you would have only applied the successful ones! You’d have saved time & effort in implementation, and had MORE traffic! The SEO process of the future would let us isolate every single change, and measure the impact of them separately.
  56. 56. Testing the Impact of SEO Changes Before 500k visits /month How do we test the impact on any one change to a website? Here’s how you might go about it…
  57. 57. Testing the Impact of SEO Changes Before 500k visits /month
  58. 58. Testing the Impact of SEO Changes Before 500k visits /month After 600k visits /month You could look at how much traffic you had before and after making the change to your site.
  59. 59. Testing the Impact of SEO Changes But that is nonsense.
  60. 60. Testing the Impact of SEO Changes If your site is very seasonal, this won’t work.
  61. 61. Testing the Impact of SEO Changes If your industry has a general trend, it won’t work.
  62. 62. Testing the Impact of SEO Changes If there are any UX or general SERP changes during the period of your test, it will throw off your data.
  63. 63. We need to test against a parallel universe You really need to be able to compare two parallel universes: one where we made the change, and one where we didn’t do anything.
  64. 64. We need to test against a parallel universe
  65. 65. Testing the Impact of SEO Changes Animals Dogs UnicornsCats Badgers Let’s use this as the section of a demo site.
  66. 66. Testing the Impact of SEO Changes H1 Badgers Old Design Intro Copy Image
  67. 67. Badgers Testing the Impact of SEO Changes H1 Badgers Old Design Intro Copy Image H1 Proposed Design Video Bullet List
  68. 68. Testing the Impact of SEO Changes Animals Dogs UnicornsCats Badgers We’ll keep half the site as it is at the moment…
  69. 69. Testing the Impact of SEO Changes Animals Cats Dogs Unicorns Badgers … and apply our proposed changes to the other half of the site.
  70. 70. Testing the Impact of SEO Changes Control group Variant group 1000 - 5000 - 4000 - 3000 - 2000 - 6000 - 7000 - Before the change, traffic to each group will be able the same.
  71. 71. Testing the Impact of SEO Changes Control group Variant group 1000 - 5000 - 4000 - 3000 - 2000 - 6000 - 7000 - Test start date Then we roll those changes out to the variant group.
  72. 72. Testing the Impact of SEO Changes Control group Variant group 1000 - 5000 - 4000 - 3000 - 2000 - 6000 - 7000 - Test start date Then we wait for Google to crawl/index the updated pages.
  73. 73. Testing the Impact of SEO Changes Control group Variant group Test start date 1000 - 5000 - 4000 - 3000 - 2000 - 6000 - 7000 - We’re not looking for the change in traffic to the variant group, but to see if there’s a gap between the variant and control group.
  74. 74. Testing the Impact of SEO Changes 1000 - 5000 - 4000 - 3000 - 2000 - 6000 - 7000 - In this case, traffic to the variant group moved above the control group, so we know the change was an effective one.
  75. 75. Testing the Impact of SEO Changes Positive Test Roll out to whole site There are three potential outcomes from a test.
  76. 76. Testing the Impact of SEO Changes Positive Test Roll out to whole site Revert the changes Negative Test
  77. 77. Testing the Impact of SEO Changes Positive Test Roll out to whole site Null Test Make an informed decision Revert the changes Negative Test
  78. 78. Testing the Impact of SEO Changes Split pages into equal groups Apply changes to variant group Create predictive model for traffic to variant group Analyze impact of the changes This is the process we are running.
  79. 79. Distilled has developed a tool to do all of this for you. We call it our “Optimization Delivery Network”: DistilledODN.com
  80. 80. Testing the Impact of SEO Changes
  81. 81. @RobOusbey @Distilled What We’ve Learned
  82. 82. I’m going to share some examples from clients who are under NDA, but I’m going to use BestBuy.com as the site to take screenshots from for these demos.
  83. 83. Testing SEO Best Practices 1
  84. 84. Testing SEO Best Practices Image Alt Tags
  85. 85. HYPOTHESIS: ALT attributes on images are good for SEO
  86. 86. Control: Product page images had no alt text Image Alt Text Test
  87. 87. Control: Product page images had no alt text Variant: Product name added as image alt text Image Alt Text Test
  88. 88. Control: Product page images had no alt text Variant: Product name added as image alt text Result: no change in traffic Image Alt Text Test
  89. 89. “Best practices” are not so clear cut.
  90. 90. Revisiting Old SEO Practices 2
  91. 91. HYPOTHESIS: Text content on category pages is goodCategory Page SEO Content Test
  92. 92. Category Page SEO Content Test
  93. 93. Control: Category pages still have ‘SEO content’ Category Page SEO Content Test
  94. 94. Control: Category pages still have ‘SEO content’ Variant: ‘SEO content’ removed from these pages Category Page SEO Content Test
  95. 95. Control: Category pages still have ‘SEO content’ Variant: ‘SEO content’ removed from these pages Result: no change in traffic Category Page SEO Content Test
  96. 96. Control: Category pages still have ‘SEO content’ Variant: ‘SEO content’ removed from these pages Result: 3.1% increase in traffic Category Page SEO Content Test
  97. 97. Control: Category pages still have ‘SEO content’ Variant: ‘SEO content’ removed from these pages Result: 3.8% decrease in traffic Category Page SEO Content Test
  98. 98. No change has the same impact on any two sites
  99. 99. Structured Markup Tests 3
  100. 100. Structured Markup Tests
  101. 101. Structured data before Structured data after Structured Markup Tests
  102. 102. Control: Category pages had no structured data Structured Markup Tests
  103. 103. Control: Category pages had no structured data Variant: Structured data added to pages Structured Markup Tests
  104. 104. Control: Category pages had no structured data Variant: Structured data added to pages Result: 11% increase in traffic Structured Markup Tests
  105. 105. $1.9 million in annual revenue from the additional traffic to these category pages. Structured Markup Tests
  106. 106. Control: Category pages had no structured data Variant: Structured data added to pages Result: 11% increase in traffic Structured Markup Tests
  107. 107. Your Competitors Hate This! ! These nerds found one weird trick that can add 11% organic traffic to any website’s category page! LEARN THE TRICK
  108. 108. Control: Category pages had no structured data Variant: Structured data added to pages Result: No change in traffic Structured Markup Tests
  109. 109. Control: Category pages had no structured data Variant: Structured data added to pages Result: No change in traffic Structured Markup Tests
  110. 110. Control: Category pages had no structured data Variant: Structured data added to pages Result: No change in traffic Structured Markup Tests
  111. 111. Periodically challenge your beliefs.
  112. 112. Learning About SEO Practicalities 4
  113. 113. Title Tag / Meta Description Tests Before: After:
  114. 114. Control: Title tags remain focusses on main topic Variant: Title tag mentions other topics covered on page Result: 10% decrease in traffic Title Tag / Meta Description Tests
  115. 115. Title Tag / Meta Description Tests 6%: Positive Tests 37%: Null Tests 57%: Negative Tests
  116. 116. Typical results: 5% - 15% Title Tag / Meta Description Tests
  117. 117. Typical timeline: 2-4 days Title Tag / Meta Description Tests
  118. 118. Google can discover and & index changes quickly
  119. 119. Learning the Risk of Testing 5
  120. 120. Buy Dresses Buy Hats Buy Belts More Title Tag Tests Online Online Online
  121. 121. Buy Dresses Buy Hats Buy Belts More Title Tag Tests Online Online Online
  122. 122. More Title Tag Tests Removed “online”
  123. 123. More Title Tag Tests Removed “online” Re-added “online”
  124. 124. The effects of changes are reversible
  125. 125. Learning More About Google 6
  126. 126. Javascript Enabled Javascript Disabled Page rendering before Javascript Visibility Test
  127. 127. MAY 2014 “our indexing system has been rendering a substantial number of web pages more like an average user’s browser with JavaScript turned on." OCT 2015 “We are generally able to render and understand your web pages like modern browsers.” Javascript Visibility Test
  128. 128. Javascript Enabled Javascript Disabled Page rendering before Javascript Visibility Test
  129. 129. Control: Pages still require JS to render Javascript Visibility Test
  130. 130. Control: Pages still require JS to render Variant: Content is loaded directly in HTML Javascript Visibility Test
  131. 131. Control: Pages still require JS to render Variant: Content is loaded directly in HTML Result: 6.2% increase in traffic Javascript Visibility Test
  132. 132. Google is not the most reliable source of truth
  133. 133. Fresh Content Tests
  134. 134. Fresh Content Tests
  135. 135. Fresh Content Tests
  136. 136. Fresh Content Tests <time itemprop="dateModified" datetime="2016-06-28 00:16:00"> Updated on: June 28th, 2016 </time>
  137. 137. Fresh Content Tests <time itemprop="dateModified" datetime="{{ today }}"> Updated on: {{ today }} </time>
  138. 138. Control: Pages had honest dateModified dates Fresh Content Test
  139. 139. Control: Pages had honest dateModified dates Variant: Pages had dateModified set to today’s date Fresh Content Test
  140. 140. Control: Pages had honest dateModified dates Variant: Pages had dateModified set to today’s date Result: 8% increase in traffic Fresh Content Test
  141. 141. [ not provided ]
  142. 142. @RobOusbey @Distilled Getting Started
  143. 143. Do It Yourself Read one of our very early posts about how to run these SEO split tests, here on the Moz Blog.
  144. 144. Free Data Analysis Software Read how to do everything yourself, using our data analysis tool, here on the Distilled blog.
  145. 145. Interdisciplinary Team Analytics SEO Stats Front EndBack End
  146. 146. DistilledODN.com Did I mention we have a tool that does all this for you? DistilledODN.com
  147. 147. @RobOusbey @Distilled Let’s Do This
  148. 148. Split-testing is becoming more necessary SearchVolume Rankings more based on user engagement metrics
  149. 149. Split-testing is becoming more established
  150. 150. Split-testing makes you more credible 0% 0% -1% -3% -4%+12% +10%
  151. 151. Split-testing is already accessible
  152. 152. DISTILLED.NET SEE ALL OUR FREE VIDEOS AND BLOG POSTS: CHECK OUT OUR SEO A/B TESTING PLATFORM: DISTILLEDODN.COM @RobOusbey
  153. 153. Photo Credits Film Rolls: Colby Stopa on Visual hunt / CC BY Face Icons: Icon made by Freepik from www.flaticon.com Google and the Google logo are registered trademarks of Google LLC, used with permission. @RobOusbey @Distilled

×