Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us

3,497 views

Published on

There is an increasing necessity to properly test SEO changes. Here Tom outlines how Google's use of user signals and the increasing complexity of their algorithm make a purely 'best practice' approach no longer viable.

Published in: Internet

SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us

  1. 1. SEO TESTS ON BIG SITES & SMALL WHAT ETSY, PINTEREST AND OTHERS CAN TEACH US Tom Anthony
  2. 2. WOW OPENINGNBED 2 NBED SOME WOWNESS
 A COUPLE OF IDEAS BELOW FIRST DEMONSTRATES IT CAN BE DANGEROUS TO TRY SOMETHING WITHOUT KNOWING SECOND IS LIKE PEOPLE WHO BLOG STUFF AND CONVINCE OTHERS - BUT ARE WILDLY WRONG
  3. 3. 3
  4. 4. @TomAnthonySEO #SMX THIS IS WHY MEDICINE HAS CONTROLLED STUDIES
  5. 5. @TomAnthonySEO #SMX INTRO ODN WE RUN TESTS NBED
  6. 6. @TomAnthonySEO #SMX USERS ARE SPLIT INTO TWO GROUPS ‘TRADITIONAL’ CRO A/B TESTING Visitors A & B: control version of a page Visitors C & D: test version of the same page
  7. 7. @TomAnthonySEO #SMX SPLIT A GROUP OF SIMILAR PAGES INTO TWO GROUPS SEO SPLIT-TESTING Pages 1, 2, 3: Control Layout Pages 4, 5, 6: Test Layout All users see the same as each other
  8. 8. @TomAnthonySEO #SMX IMPORTANT - GOOGLEBOT IS TREATED NO DIFFERENTLY SEO SPLIT-TESTING Pages 1, 2, 3: Control Layout Pages 4, 5, 6: Test Layout Googlebot sees the same as users
  9. 9. @TomAnthonySEO #SMX LASTLY - MEASURE WHICH SET OF PAGES GOOGLE PREFERS SEO SPLIT-TESTING
  10. 10. @TomAnthonySEO #SMX A 12% UPLIFT FOR THIS UK RETAILER ADDING SCHEMA.ORG PRODUCT REVIEWS
  11. 11. @TomAnthonySEO #SMX Early on we found many SEO ‘best practice’ changes don’t help.
  12. 12. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. Dark Ages
  13. 13. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. Dark Ages Middle Ages
  14. 14. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. Dark Ages Middle Ages Renaissance
  15. 15. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. Dark Ages Middle Ages Renaissance Ranking signals become external. Knowing about your site is not enough.
  16. 16. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 WMT/GSC. Link indexes. Rank tracking. Dark Ages Middle Ages Renaissance Industrial Revolution
  17. 17. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Dark Ages 2 Middle Ages Pre-Google. Keyword stuffing reigns. 3 Renaissance Google arrives. PageRank. Links. 4 Industrial Revolution Pre-search engine, big list of sites. WMT/GSC. Link indexes. Rank tracking. The rise of data… Cannot be effective without these external data sources.
  18. 18. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. The Google algo becomes more complex: • Machine Learning • Explosion of Signals • User Signals
  19. 19. @TomAnthonySEO #SMX SHEER SCALE AND THE INTRODUCTION OF MACHINE LEARNING HAVE CHANGED IT… GOOGLE’S ALGO IS NO LONGER PREDICTABLE GOOGLE ALGO USED TO BE LOTS OF INDEPENDENT LEVERS BUT NOW IT IS A COMPLEX SYSTEM. IMPOSSIBLE TO PREDICT. Keyword in H1 # of Links Domain Age + + + Relevancy Score Site Speed Social Signals Domain Age Domain Trust User Signals Text above fold Structured Data # of Links Ad Density Keyword in Title
  20. 20. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. So, we have data, but it is hard to understand how to turn that data into decisions.
  21. 21. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking.
  22. 22. @TomAnthonySEO #SMX FROM ALTAVISTA TO PRESENT DAY. SEO TIMELINE 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. Since 2014, we’ve been headed into this stage.
  23. 23. @TomAnthonySEO #SMX LOTS OF DATA - BUT HARD TO KNOW THE BEST DECISIONS TO MAKE WITH IT So many companies have started split-testing…
  24. 24. @TomAnthonySEO #SMX A GUIDE TO SPLIT-TESTING THE NEXT ERA OF SEO
  25. 25. @TomAnthonySEO #SMX
  26. 26. USER SIGNALS ARE RANKING SIGNALS1 2 RANKING SIGNALS ARE NO LONGER LINEAR 3 DATA TO HYPOTHESIS TO ACTION
  27. 27. USER SIGNALS ARE RANKING SIGNALS1
  28. 28. @TomAnthonySEO #SMX https://medium.com/@Pinterest_Engineering/demystifying-seo-with-experiments-a183b325cf4c
  29. 29. @TomAnthonySEO #SMX OBSERVATION Search Console detected many duplicate title tags on board pages: e.g.  “{board_name} on Pinterest” Test #1
  30. 30. @TomAnthonySEO #SMX ION tected many n board pages: on Pinterest” TEST IDEA Reduce duplicate title tags by including Pin counts in the title tag. e.g.  “{board_name} on Pinterest | ({number} Pins) Test #1
  31. 31. @TomAnthonySEO #SMX RESULT No statistically significant improvement in traffic. EA tags by including e title tag. est | ({number} Pins) Test #1
  32. 32. @TomAnthonySEO #SMX OBSERVATION “Pinterest is filled with tens of billions of Pins, but the text descriptions can be lacking” Test #2
  33. 33. @TomAnthonySEO #SMX ION tens of billions descriptions king” TEST IDEA Add a better description, written by users of other Pins of the same image. Test #2
  34. 34. @TomAnthonySEO #SMX RESULT A 30% increase in organic traffic. EA tion, written by the same image. Test #2
  35. 35. @TomAnthonySEO #SMX RESEARCH PAPER FROM GOOGLE IN 2010 GOOGLE IS USING USER SIGNALS http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36363.pdf
  36. 36. @TomAnthonySEO #SMX GOOGLE IS USING USER SIGNALS “Another direction for future work is to incorporate active learning in order to gather a more representative sample of user preferences.” RESEARCH PAPER FROM GOOGLE IN 2010
  37. 37. @TomAnthonySEO #SMX BLOG POST FROM DAN PETROVIC — 3YEARS AGO GOOGLE IS USING USER SIGNALS https://moz.com/blog/user-behaviour-data-as-a-ranking-signal
  38. 38. @TomAnthonySEO #SMX EXPERIMENT BY RAND FISHKIN SHORT TERM CTR AFFECTS RANKINGS https://twitter.com/randfish/status/612730826615230464
  39. 39. Reduce duplicate title tags by including Pin counts in the title tag. Test #1 No statistically significant improvement in traffic. Why? Add a better description, written by users of other Pins of the same image. Test #2 A 30% increase in organic traffic. Why?
  40. 40. Reduce duplicate title tags by including Pin counts in the title tag. Test #1 No statistically significant improvement in traffic. Titles matter as they help users by making it clear what a page is about. The change didn’t help users. Add a better description, written by users of other Pins of the same image. Test #2 A 30% increase in organic traffic. Better descriptions help users and bots better understand contents of a pin. The change helped users.
  41. 41. Understand why you are seeing warnings. Try to address the underlying issue. TAKEAWAY Doing this is more likely to have a real impact.
  42. 42. Users are unpredictable, and so you must test whenever they are involved. TAKEAWAY Google is using user signals as a ranking factor.
  43. 43. 2 RANKING SIGNALS ARE NO LONGER LINEAR
  44. 44. “By running a large number of experiments, we found some well-known strategies for SEO didn’t work for us, while certain tactics we weren’t confident about worked like a charm.”
  45. 45. Best Practices weren’t having an impact. But they were wasting time and resource trying them. TAKEAWAY
  46. 46. @TomAnthonySEO #SMX TEST RESULT: ALT ATTRIBUTES HAD NO IMPACT WE HAVE FOUND THE SAME MANY TIMES OVERALT ATTRIBUTE TEST FROM LASTYEAR
  47. 47. @TomAnthonySEO #SMX https://codeascraft.com/2017/01/25/optimizing-meta-descriptions-h1s-and-title-tags-lessons-from-multivariate-seo-testing-at-etsy/
  48. 48. @TomAnthonySEO #SMX
  49. 49. Shorter title tags, consisting only of the target search keyword, appear more relevant/enticing to search users. “ ” Shorter title tags can improve CTRs. TAKEAWAY
  50. 50. H1 tags are still important and still effect SEO performance, but… TAKEAWAY We found that an H1 change can have a statistically significant impact on organic search traffic. “ ”
  51. 51. @TomAnthonySEO #SMX Title uplift: 5% H1 test uplift: 5% Uplift from both: 7% TESTS SHOW THAT UPLIFTS ARE NOT INDEPENDENTEXAMPLE NUMBERS TO ILLUSTRATE
  52. 52. All SEO changes affect one another. TAKEAWAY Don’t think one change won’t impact others.
  53. 53. 3 DATA TO HYPOTHESIS TO ACTION
  54. 54. Some of our earliest DistilledODN tests…
  55. 55. @TomAnthonySEO #SMX
  56. 56. @TomAnthonySEO #SMX
  57. 57. @TomAnthonySEO #SMX BUT BUT BUT… BEST PRACTICE SAID USP IS A GOOD IDEA?! NOTHING HAPPENED
  58. 58. Before After query search volume which tv should i buy 250 query search volume which tv to buy 1,300
  59. 59. @TomAnthonySEO #SMX MAYBE EVEN A BIT WORSE NO REAL IMPACT
  60. 60. <img src="product1.jpg" alt="Home 6 Piece Towel Bale - Geo Grey / Zest">
  61. 61. @TomAnthonySEO #SMX WE’VE NEVER HAD AN IMPACT FROM ALT ATTRIBUTE NADA… NOTHING… NO IMPACT…
  62. 62. “Stand back guys, I got this.”
  63. 63. Before After
  64. 64. @TomAnthonySEO #SMX WORST TEST RESULT WE EVER HAD OH GOD…
  65. 65. COULD BE OUR RESULTS AREN’T ACCURATE ?
  66. 66. @TomAnthonySEO #SMX THE RESULTS ARE CONSISTENT THE RESULT WAS ACCURATE — WE SUCK
  67. 67. @TomAnthonySEO #SMX Most SEO changes have a no or a negative impact.
  68. 68. @TomAnthonySEO #SMX IT LEADS TO MORE EFFECTIVE TESTING CRO EXPERTS KNOW - YOU NEED A HYPOTHESIS dis.tl/cro-hypothesis
  69. 69. @TomAnthonySEO #SMX A HYPOTHESISES SHOULD BE SPECIFIC Changing <abc> will have an impact on <xyz> & thus improve organic search performance. Based on <evidence>, we hypothesise…
  70. 70. @TomAnthonySEO #SMX HYPOTHESIS EXAMPLE Changing <title formats, by…> will improve <CTR> & thus improve organic search performance. Based on <survey data>, we hypothesise…
  71. 71. @TomAnthonySEO #SMX CRO HYPOTHESISES FRAMEWORK https://conversion.com/blog/introducing-hypothesis-framework/
  72. 72. @TomAnthonySEO #SMX EXAMPLE OF HYPOTHESIS APPROACH
  73. 73. CATEGORY PAGE HTML ISVERBOSE, AND CONTAINS A LOT OF JUNK. ADDING A LIST OF PRODUCTS AS STRUCTURED MARKUP WILL MAKE THE INTENT CLEARER. HYPOTHESIS
  74. 74. @TomAnthonySEO #SMX ADD STRUCTURED MARKUP
  75. 75. @TomAnthonySEO #SMX AN 11% UPLIFT IN ORGANIC TRAFFICOUR HYPOTHESIS WAS ACCURATE
  76. 76. @TomAnthonySEO #SMX REPEATED IN ANOTHER REGION - 11% AGAINWE SEE THIS OFTEN IN DIFFERENT REGIONS
  77. 77. #StateOfSearch@TomAnthonySEO
  78. 78. #StateOfSearch@TomAnthonySEO
  79. 79. THE HYPOTHESIS DID NOT FIT THESE OTHER SITES. !
  80. 80. Hypothseses ensure you understand why you are making a change. A hypothesis approach leads to more effective changes, and helps avoid wasting time on ineffective changes. DATAPOINT TAKEAWAY
  81. 81. @TomAnthonySEO #SMX HAVE A HYPOTHESIS - YOU WILL MAKE BETTER RECOMMENDATIONS
  82. 82. SUMMARY
  83. 83. @TomAnthonySEO #SMX TO BE COMPETITIVE TESTING IS BECOMING A NECESSITY WE ARE IN A NEW ERA OF SEO 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. Just as tactics from here… …wouldn’t work well here…
  84. 84. @TomAnthonySEO #SMX TO BE COMPETITIVE TESTING IS BECOMING A NECESSITY WE ARE IN A NEW ERA OF SEO 1 Pre-search engine, big list of sites. 2 Pre-Google. Keyword stuffing reigns. 3 Google arrives. PageRank. Links. 4 5 Information Age Data-driven SEO via Split-Testing. Dark Ages Middle Ages Renaissance Industrial Revolution WMT/GSC. Link indexes. Rank tracking. Just as tactics from here… …wouldn’t work well here… … being competitive here means testing!
  85. 85. SEO must be user-centric, which makes testing necessary. TAKEAWAY Google includes user signals. Users are unpredictable.
  86. 86. TAKEAWAY Realise you are not at optimised as you think you are. Many SEO tests get uplifts from updating titles, meta descriptions & H1s etc.
  87. 87. Ranking signals are no longer ‘summed’ together.They effect one another. TAKEAWAY This also means that what worked for someone else, may not work for you.
  88. 88. TAKEAWAY Have a hypothesis for what you expect from any SEO change. You will make more effective recommendations.
  89. 89. Testing allows you to stop chasing ranking factors. TAKEAWAY
  90. 90. @TomAnthonySEO #SMX FREE TOOL TO DO AN ANALYSIS OF TESTS BLOG POST - HOW YOU CAN TEST https://www.distilled.net/resources/do-it-yourself-seo-split-testing-tool-with-causal-impact/
  91. 91. @TomAnthonySEO #SMX SMX MUNICH 2017 DECK MORE TEST EXAMPLES https://www.slideshare.net/TomAnthony/next-era-of-seo-a-guide-to-seo-splittesting A GUIDE TO SPLIT-TESTING THE NEXT ERA OF SEO
  92. 92. DANKE! @TomAnthonySEO http://dis.tl/smx-testing-2018

×