Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons from SEO split-testing

4,869 views

Published on

Computers might already be better at your job than you are. Are you ready to partner with them to keep your job?

This presentation shows you how hard our jobs have become, gives you the results of some of our testing, and outlines a plan to keep our jobs.

Published in: Marketing
  • The Insider's Edge You've Been Looking For.... ➤➤ http://t.cn/A6hP86vM
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The Insider's Edge You've Been Looking For.... ■■■ http://t.cn/A6hP86vM
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hey! Stop when you hit £700 profit on the week! Want to get started and make £700 this week? He's running a special introductory offer for the start of 2020 but this ends soon. CLICK HERE TO JOIN.. ♥♥♥ http://t.cn/A6vAxKsh
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/3iKLy ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • This is Amazing! He Won the Lotto Jackpot 7 Times, and Doesn't Mind Revealing His Secrets? ♣♣♣ http://t.cn/Airfq84N
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Lessons from SEO split-testing

  1. 1. Lessons from SEO split-testing Including evidence that we don’t know what we’re talking about @willcritchlow
  2. 2. We used to have a pretty good understanding of ranking factors
  3. 3. My mental model for ~2009 ranking factors had three different modes:
  4. 4. One in the hyper-competitive head My mental model for ~2009 ranking factors had three different modes: One in the competitive mid-tail ...and one in the long-tail
  5. 5. One in the hyper-competitive head
  6. 6. Tons of perfectly on-topic pages to choose from One in the hyper-competitive head
  7. 7. So pick only perfectly-on-topic pages One in the hyper-competitive head
  8. 8. (*) Page authority, but the domain inevitably factors into that calculation. This is why so many homepages ranked One in the hyper-competitive head ...and rank by authority (*)
  9. 9. This resulted in a mix of homepages of mid-size sites, and inner pages on huge sites One in the hyper-competitive head
  10. 10. But the general way to move up was through increased authority One in the hyper-competitive head
  11. 11. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Long-tail
  12. 12. One in the hyper-competitive head One in the competitive mid-tail
  13. 13. Wealth of ROUGHLY on-topic pages to choose from One in the competitive mid-tail
  14. 14. PERFECTLY on-topic could do well even on a relatively weak site One in the competitive mid-tail
  15. 15. Rank the roughly on-topic pages by authority x “on-topicness” One in the competitive mid-tail
  16. 16. Move up with better targeting or more authority One in the competitive mid-tail
  17. 17. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail
  18. 18. One in the hyper-competitive head One in the competitive mid-tail ...and one in the long-tail
  19. 19. In the long-tail, a site of arbitrary weakness could rank if it was the most relevant ...and one in the long-tail
  20. 20. Otherwise, massive sites rank with off-topic pages that mention something similar ...and one in the long-tail
  21. 21. Generally, move up with better targeting ...and one in the long-tail
  22. 22. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting.
  23. 23. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting. So that was ~2009
  24. 24. It’s not so simple any more. Google is harder to understand these days.
  25. 25. PageRank (the first algorithm to use the link structure of the web) We know how we got to ~2009...
  26. 26. Information retrieval PageRank
  27. 27. Information retrieval PageRank Original research
  28. 28. Information retrieval PageRank Original research TWEAKS ...with growing complexity in subsequent years
  29. 29. Particularly this comment from a user called Kevin Lacker (@lacker):
  30. 30. I was thinking about it like it was a math puzzle and if I just thought really hard it would all make sense. -- Kevin Lacker (@lacker)
  31. 31. Hey why don't you take the square root? -- Amit Singhal according to Kevin Lacker (@lacker)
  32. 32. oh... am I allowed to write code that doesn't make any sense? -- Kevin Lacker (@lacker)
  33. 33. -- Amit Singhal according to Kevin Lacker (@lacker) Multiply by 2 if it helps, add 5, whatever, just make things work and we can make it make sense later.
  34. 34. No, but I’m still pretty good at this You’re thinking this to yourself right now.
  35. 35. I can beat you with a simple machine Well. It turns out all we needed was a coin to flip. You’re all fired.
  36. 36. It’s only going to get worse under Sundar Pichai
  37. 37. So we need to test. Here’s how SEO split tests work
  38. 38. You might have seen @TomAnthonySEO tweeting about the platform we’ve built to make this easy
  39. 39. Excuse a brief diversion into geeky details
  40. 40. Instead of comparing the performance of the control pages directly with the variant pages, we build a forecast of what’s called the counterfactual which is an estimate of what would have happened if we hadn’t made the change. We use the control group to make a counterfactual forecast that takes into account seasonality and site-wide changes. The black line on the chart above is the actual organic traffic to the variant pages. The blue line is the counterfactual. More: Distilled blog post and free forecasting tool
  41. 41. It’s easiest to analyse the results by looking at the cumulative difference over time between the actual organic traffic and the counterfactual. The pale blue area is the 95% confidence interval. We can see a (statistically) zero effect for an initial time while Google crawls and indexes the test, followed by steady growth. A couple of weeks in, the confidence interval goes above zero and we have a winning test. More: Distilled blog
  42. 42. It’s easiest to analyse the results by looking at the cumulative difference over time between the actual organic traffic and the counterfactual. The pale blue area is the 95% confidence interval. We can see a (statistically) zero effect for an initial time while Google crawls and indexes the test, followed by steady growth. A couple of weeks in, the confidence interval goes above zero and we have a winning test. More: Distilled blog Hashtag winning
  43. 43. Further reading for those interested: ● Predicting the present with Bayesian structural time series [PDF] ● Inferring causal impact using Bayesian structural time series [PDF] ● CausalImpact R package ● Finding the ROI of title tag changes More: Distilled blog
  44. 44. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  45. 45. Thanks Dom @dom_woodman
  46. 46. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  47. 47. Category pages have lots of images and not much text
  48. 48. Adding structured data to category pages
  49. 49. Organic sessions increased by 11%
  50. 50. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  51. 51. Established wisdom would say ALT tags on images are good for SEO
  52. 52. We found adding ALT tags to images had no effect
  53. 53. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  54. 54. Title tag before: Which TV should I buy? - Argos Title tag after: Which TV to buy? - Argos What happens when you match title tags to the greatest search volume?
  55. 55. Organic sessions decreased by an average of 8%
  56. 56. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  57. 57. We can render Javascript!
  58. 58. What happens if your content is only visible with Javascript? Javascript EnabledJavascript Disabled
  59. 59. Making it visible increased organic sessions by ~ 6.2%
  60. 60. Read more on our blog: early results from split-testing JS for SEO
  61. 61. 1. Adding structured data 2. Adding ALT attributes 3. Setting exact match title tags 4. Using JS to show content 5. Removing SEO category text
  62. 62. How does SEO text on category pages perform?
  63. 63. E-commerce site number 1 ~ 3.1% increase in organic sessions
  64. 64. E-commerce site number 2 - No effect/negative effect
  65. 65. This is why we have been investing so much in split-testing Check out odn.distilled.net if you haven’t already. The team will be happy to demo for you. We served ~5 billion requests last quarter and recently published everything from response times to our +£100k / month split test.
  66. 66. But I’m also seeing more subtle impacts on my recommendations: ● You can recommend small tweaks and see the benefits compound ● You can test wild hypotheses with unknown upsides ● You can try things that might have a downside (more focused targeting, less copy, etc.) And that’s even before you get the benefits of testing clickthrough rate, and the benefits of pretty charts to show the boss highlighting the impact of your work! More: blog post
  67. 67. Oh, and one more thing
  68. 68. Can we build a better machine than a coin flip? Seems like a low bar
  69. 69. The goal is a winning combination of human and machine Human + computer beats computer (for now)
  70. 70. Questions: @willcritchlow
  71. 71. ● Mobius strip ● Confusion ● Signal box ● Cigar ● Discontinuity ● Confidence ● Burt Totaro ● Sundar Pichai ● John Giannandrea ● Chuck Norris ● Jeff Dean ● Fencing ● Keyboard Image credits ● Go ● Robot ● Leopard print sofa ● Leopard ● Bug ● Lego robots ● Iron Man ● Leeds

×