Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons from another year in SEO A/B Testing - SearchLove London 2019

3,932 views

Published on

Talk from SearchLove London 2019 sharing case studies from DistilledODN that reveal on-page best practices that are harming your organic traffic.

Published in: Marketing
  • can direct copy link Create DOWNLOAD for free book ===http://dayna.club/8836816053-Case-di-campagna-inglesi-Ediz-a-colori-.html
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Lessons from another year in SEO A/B Testing - SearchLove London 2019

  1. 1. On-Page Best Practices That Harm Your Traffic Lessons from another year of SEO A/B testing Emily Potter - DistilledODN Consultant @e_mpotter
  2. 2. SEO A/B testing?
  3. 3. SEO A/B testing in depth Case studies to apply understanding Takeaways
  4. 4. SEO A/B testing in depth Case studies to apply understanding Takeaways
  5. 5. What is SEO A/B Testing?
  6. 6. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template
  7. 7. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template www.croabtesting.com/cats www.croabtesting.com/cats
  8. 8. Variant UsersControl Users
  9. 9. Variant UsersControl Users 50% of visits 50% of visits
  10. 10. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template www.croabtesting.com/cats www.croabtesting.com/cats
  11. 11. CR control (conversion rate) CR variant (conversion rate) Positive Test
  12. 12. CR variant (conversion rate) Negative Test CR control (conversion rate)
  13. 13. CR control (conversion rate) CR variant (conversion rate) Neutral Test
  14. 14. But there is only one Googlebot
  15. 15. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template www.croabtesting.com/cats www.croabtesting.com/cats
  16. 16. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template www.croabtesting.com/cats www.croabtesting.com/cats
  17. 17. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template www.croabtesting.com/cats www.croabtesting.com/cats
  18. 18. Cats H1 Intro copy Featured image Cats H1 Featured video Variant TemplateControl Template www.croabtesting.com/cats www.croabtesting.com/cats
  19. 19. We can’t split Googlebot
  20. 20. We can’t split Googlebot
  21. 21. Googlebot would see something like this
  22. 22. https://www.croabtesting.com › Cats cro test All Shopping Images News About 150,000 results (0.35 seconds) More SERP on Experiment Day One Welcome to our page on cats with nice static text and a beautiful image of your favorite female friends. Control Page Title - Template A Same url
  23. 23. https://www.croabtesting.com › Cats cro test All Shopping Images News About 150,000 results (0.35 seconds) More SERP on Experiment Day Two Welcome to our new updated page on cats with a beautiful and informative video on your favourite feline friends. Variant Page Title - Template B Same url
  24. 24. https://www.croabtesting.com › Cats cro test All Shopping Images News About 150,000 results (0.35 seconds) More SERP on Experiment Day Three Welcome to our page on cats with nice static text and a beautiful image of your favorite female friends. Control Page Title - Template A Same url
  25. 25. So how do we test for Googlebot?
  26. 26. DistilledODN A/B testing for a scientific approach to SEO Get a demo
  27. 27. Variant UsersControl Users 50% of visits 50% of visits
  28. 28. Example website with an “animals” category Animals Cats Dogs Unicorns Badgers
  29. 29. Cats H1 Intro copy Featured image Dogs H1 Featured video Variant TemplateControl Template
  30. 30. Cats H1 Intro copy Featured image Dogs H1 Featured video 50% of pages 50% of pages Variant TemplateControl Template
  31. 31. Cats H1 Intro copy Featured image Dogs H1 Featured video 50% of pages 50% of pages Control organic traffic Variant organic traffic Variant TemplateControl Template
  32. 32. Now our website looks like this Animals Cats Dogs Unicorns Badgers
  33. 33. Now our website looks like this Animals Cats Dogs Unicorns Badgers
  34. 34. Everyone (inc. Googlebot) sees the same pages Animals Cats Dogs Unicorns Badgers
  35. 35. Organic traffic Control Organic traffic Variant Positive Test
  36. 36. Organic traffic Control Organic traffic Variant Negative Test
  37. 37. Organic traffic Control Organic traffic Variant Neutral Test
  38. 38. What is SEO?
  39. 39. 3 levers to increase organic traffic
  40. 40. Users click something because it’s cool
  41. 41. Or because it’s there
  42. 42. Look cool in SERP to improve organic CTR
  43. 43. Improve rankings so you’re “there” more Not ranking :(
  44. 44. These are our three levers
  45. 45. Rank for new keywords so you’re “there” more
  46. 46. How do you look more cool in the SERP?
  47. 47. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Update title Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Optimise titles
  48. 48. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Optimise meta descriptions Update meta description
  49. 49. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Rating: 4.9 - 1,000,000 votes - Price range: Premium Add schema for rich results Implement schema markup
  50. 50. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Rating: 4.9 - 1,000,000 votes - Price range: Premium Add schema for breadcrumbs Implement schema markup
  51. 51. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net > About > People > Emily Potter Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Rating: 4.9 - 1,000,000 votes - Price range: Premium Add schema for breadcrumbs Implement schema markup
  52. 52. when did emily potter join the ODN team? All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting Rating: 4.9 - 1,000,000 votes - Price range: Premium In March 2019, Emily joined the DistilledODN team full time as a consultant and began speaking at conferences, including BrightonSEO, SearchLeeds, and SeachLove. Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Win featured snippets Win featured snippets
  53. 53. CTR changes will change your appearance in the SERPs
  54. 54. How do you get “there” more? Not ranking :(
  55. 55. How do you get “there” more?
  56. 56. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the University of California, Santa Barbara in 2016 where she studied creative writing, sociology and also dabbled in mathematics and economics for fun. Update H1 Update H1s
  57. 57. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. She now works full time on the Distilled ODN team, running lots and lots and lots of split tests. Update on-page content Write new copy
  58. 58. Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. She now works full time on the Distilled ODN team, running lots and lots and lots of split tests. Emily has also been seen speaking at conferences like BrightonSEO, SearchLeeds, and SearchLove. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Update on-page content Add new content
  59. 59. Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. She now works full time on the Distilled ODN team, running lots and lots and lots of split tests. Emily has also been seen speaking at conferences like BrightonSEO, SearchLeeds, and SearchLove. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Add internal links Add internal links
  60. 60. When results get fuzzy
  61. 61. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Update title Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Optimise titles
  62. 62. Google CTR history - Advanced Web Ranking Full study here
  63. 63. CTR might be a ranking factor
  64. 64. This change improved organic traffic
  65. 65. This change hurt organic traffic
  66. 66. This change had no measurable impact on organic traffic
  67. 67. This allows us to...
  68. 68. Conversion Rate Revenue Increase Make business cases Gained Sessions
  69. 69. Rescue sessions & Get a competitive advantage
  70. 70. wordpress templates All Shopping Images News About 150,000 results (0.35 seconds) More Include quantity of products in titles... https://colorlib.com › Home › Free Themes 9 Sep 2019 - The best free WordPress theme you can currently get on the market, all responsive, mobile-ready and easy to use. 60+ Best Free Responsive WordPress Themes 2019 - Colorlib https://themeisle.com › wordpress-themes 30+ WordPress Themes - 2019's Best WordPress Templates https://athemes.com › collections › free-wordpress-themes 3 May 2019 - So, without further ado, keep reading to learn more about the best free WordPress themes on the market. Airi. Sydney. Neve. Talon. Zakra. Hestia. Flash. Astrid. Best Free Responsive WordPress Themes 2019 - aThemes
  71. 71. wordpress templates All Shopping Images News About 150,000 results (0.35 seconds) More Include quantity of products in titles... https://colorlib.com › Home › Free Themes 9 Sep 2019 - The best free WordPress theme you can currently get on the market, all responsive, mobile-ready and easy to use. 60+ Best Free Responsive WordPress Themes 2019 - Colorlib https://themeisle.com › wordpress-themes 30+ WordPress Themes - 2019's Best WordPress Templates https://athemes.com › collections › free-wordpress-themes 3 May 2019 - So, without further ado, keep reading to learn more about the best free WordPress themes on the market. Airi. Sydney. Neve. Talon. Zakra. Hestia. Flash. Astrid. 70+ Best Free Responsive WordPress Themes 2019 - aThemes
  72. 72. wordpress templates All Shopping Images News About 150,000 results (0.35 seconds) More Include quantity of products in titles... https://colorlib.com › Home › Free Themes 9 Sep 2019 - The best free WordPress theme you can currently get on the market, all responsive, mobile-ready and easy to use. 60+ Best Free Responsive WordPress Themes 2019 - Colorlib https://themeisle.com › wordpress-themes 30+ WordPress Themes - 2019's Best WordPress Templates https://athemes.com › collections › free-wordpress-themes 3 May 2019 - So, without further ado, keep reading to learn more about the best free WordPress themes on the market. Airi. Sydney. Neve. Talon. Zakra. Hestia. Flash. Astrid. 70+ Best Free Responsive WordPress Themes 2019 - aThemes
  73. 73. wordpress templates All Shopping Images News About 150,000 results (0.35 seconds) More Include quantity of products in titles... https://colorlib.com › Home › Free Themes 9 Sep 2019 - The best free WordPress theme you can currently get on the market, all responsive, mobile-ready and easy to use. 60+ Best Free Responsive WordPress Themes 2019 - Colorlib https://themeisle.com › wordpress-themes 30+ WordPress Themes - 2019's Best WordPress Templates https://athemes.com › collections › free-wordpress-themes 3 May 2019 - So, without further ado, keep reading to learn more about the best free WordPress themes on the market. Airi. Sydney. Neve. Talon. Zakra. Hestia. Flash. Astrid. 70+ Best Free Responsive WordPress Themes 2019 - aThemes
  74. 74. Most people are satisfied with these answers
  75. 75. Wait...but why?
  76. 76. wordpress templates All Shopping Images News About 150,000 results (0.35 seconds) More Let’s revisit this https://colorlib.com › Home › Free Themes 9 Sep 2019 - The best free WordPress theme you can currently get on the market, all responsive, mobile-ready and easy to use. 60+ Best Free Responsive WordPress Themes 2019 - Colorlib https://themeisle.com › wordpress-themes 30+ WordPress Themes - 2019's Best WordPress Templates https://athemes.com › collections › free-wordpress-themes 3 May 2019 - So, without further ado, keep reading to learn more about the best free WordPress themes on the market. Airi. Sydney. Neve. Talon. Zakra. Hestia. Flash. Astrid. 70+ Best Free Responsive WordPress Themes 2019 - aThemes
  77. 77. This change hurt organic traffic
  78. 78. Adding the quantity to the title hurt organic traffic
  79. 79. But why?
  80. 80. Did click-through-rate go down?
  81. 81. :’( Did rankings drop?
  82. 82. Did we stop ranking for keywords we had been ranking for?
  83. 83. wordpress templates All Shopping Images News About 150,000 results (0.35 seconds) More CTR is the likely cause...but what if it wasn’t? https://colorlib.com › Home › Free Themes 9 Sep 2019 - The best free WordPress theme you can currently get on the market, all responsive, mobile-ready and easy to use. 60+ Best Free Responsive WordPress Themes 2019 - Colorlib https://themeisle.com › wordpress-themes 30+ WordPress Themes - 2019's Best WordPress Templates https://athemes.com › collections › free-wordpress-themes 3 May 2019 - So, without further ado, keep reading to learn more about the best free WordPress themes on the market. Airi. Sydney. Neve. Talon. Zakra. Hestia. Flash. Astrid. 70+ Best Free Responsive WordPress Themes 2019 - aThemes
  84. 84. 9.81 m/s^2 * time = ?
  85. 85. SEO testing relieves us from all that noise
  86. 86. Three levers inform our hypothesis
  87. 87. This change improved organic traffic This change hurt organic traffic This change had no measurable impact on organic traffic
  88. 88. The result either supports or disproves your hypothesis
  89. 89. To see how this works, we’ll cover four case studies
  90. 90. SEO A/B testing in depth Case studies to apply understanding Takeaways
  91. 91. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net > About > People > Emily Potter Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Rating: 4.9 - 1,000,000 votes - Price range: Premium Add schema for breadcrumbs Case Study #1: Fixing breadcrumbs
  92. 92. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Update title Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Case Study #2: Price in titles
  93. 93. Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. She now works full time on the Distilled ODN team, running lots and lots and lots of split tests. Emily has also been seen speaking at conferences like BrightonSEO, SearchLeeds, and SearchLove. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Case Study #3: Internal linking Add internal links
  94. 94. £60 Shop our over 600 different styles of women’s shoes to complete your wardrobe. We have heels, boots, trainers, and any other type of shoe you could imagine. All our shoes come with free shipping and handling. Dresses Jackets Shoes Activewear Accessories Women's Shoes 654 results New In! Red heels £55 Teal heels Black heels £65 Case Study #4: New product page template
  95. 95. Case study #1: Overview
  96. 96. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Emily Potter - Consultant - Distilled https://www.distilled.net > About > People > Emily Potter Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Rating: 4.9 - 1,000,000 votes - Price range: Premium Add schema for breadcrumbs Case Study #1: Fixing breadcrumbs
  97. 97. Had run this test previously, overwhelmingly positive
  98. 98. Mens Swimwear | Shorts, Beach Towels & Swimming ... Men's Swimwear: Swim Shorts, Swimming Trunks & Briefs ... Men's Swimwear | Swim Shorts & Speedos for Men | ASOS mens swimwear All Shopping Images News About 150,000 results (0.35 seconds) More https://www.asos.com › Men › Swimwear › Category Later, breadcrumb schema broke https://www.next.co.uk/shop/men/clothing/swimwear Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next day ... https://www.sportsdirect.com › Swimming › Mens Swimwear Discover the range of men's swimwear at ASOS. Explore swim shorts, board shorts and speedos for men in a variety of colours and lengths. Shop now at ASOS.
  99. 99. Case study #1: Hypothesis
  100. 100. What are we changing? ● Fixing breadcrumb schema
  101. 101. What levers will we pull? ● Breadcrumbs in SERP will increase CTR
  102. 102. ● We will measure organic traffic How will we know?
  103. 103. Case study #1: Control vs. Variant
  104. 104. Define your beach style with women's swimwear in printed & plain, while beach towels & swimming accessories complete your holiday bag. Next da ... Control Template Variant Template Womens Swimwear | Shorts, Beach Towels & Swimm ... https://www.next.co.uk/shop/women/clothing/swimwear Mens Swimwear | Shorts, Beach Towels & Swimming ... https://www.next.co.uk › Shop › Men › Clothing Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next da ...
  105. 105. womens swimwear Womens Swimwear | Shorts, Beach Towels & Swimming ... Women's Swimwear: Swim Bikinis, Swimming Suits & To ... Women's Swimwear | Swim Bikinis & Suits for Women ... All Shopping Images News About 150,000 results (0.35 seconds) https://www.asos.com › Women › Swimwear › Category More https://www.next.co.uk/shop/women/clothing/swimwear Define your beach style with women's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next day ... https://www.sportsdirect.com › Swimming › Mens Swimwear Discover the range of women's swimwear at ASOS. Explore swim shorts, board shorts and bikinis for women in a variety of colours and lengths. Shop now at ASOS. Mens Swimwear | Shorts, Beach Towels & Swimming ... Men's Swimwear: Swim Shorts, Swimming Trunks & Briefs ... Men's Swimwear | Swim Shorts & Speedos for Men | ASOS https://www.asos.com › Women › Swimwear › Category https://www.next.co.uk › Shop › Men › Clothing Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next day ... https://www.sportsdirect.com › Swimming › Mens Swimwear Discover the range of men's swimwear at ASOS. Explore swim shorts, board shorts and speedos for men in a variety of colours and lengths. Shop now at ASOS. mens swimwear All Shopping Images News About 150,000 results (0.35 seconds) More Control Template Variant Template
  106. 106. Case study #1: Results
  107. 107. Define your beach style with women's swimwear in printed & plain, while beach towels & swimming accessories complete your holiday bag. Next da ... Control Template Variant Template Womens Swimwear | Shorts, Beach Towels & Swimm ... https://www.next.co.uk/shop/women/clothing/swimwear Mens Swimwear | Shorts, Beach Towels & Swimming ... https://www.next.co.uk › Shop › Men › Clothing Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next da ...
  108. 108. So what the f*** happened?
  109. 109. Why did control have a higher CTR than variant?
  110. 110. Define your beach style with women's swimwear in printed & plain, while beach towels & swimming accessories complete your holiday bag. Next da ... Control Template Variant Template Womens Swimwear | Shorts, Beach Towels & Swimm ... https://www.next.co.uk/shop/women/clothing/swimwear Mens Swimwear | Shorts, Beach Towels & Swimming ... https://www.next.co.uk › Shop › Men › Clothing Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next da ...
  111. 111. Mens Swimwear | Shorts, Beach Towels & Swimming ... https://www.next.co.uk › Shop › Men › Clothing Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next da ... Define your beach style with women's swimwear in printed & plain, while beach towels & swimming accessories complete your holiday bag. Next da ... Control Template Variant Template Womens Swimwear | Shorts, Beach Towels & Swimm ... https://www.next.co.uk/shop/women/clothing/swimwear
  112. 112. https://www.next.co.uk › Shop › Men › Clothing Mens Swimwear | Shorts, Beach Towels & Swimming ... Men's Swimwear: Swim Shorts, Swimming Trunks & Briefs ... Men's Swimwear | Swim Shorts & Speedos for Men | ASOS mens swimwear All Shopping Images News About 150,000 results (0.35 seconds) More https://www.asos.com › Men › Swimwear › Category Variant Template Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next day ... https://www.sportsdirect.com › Swimming › Mens Swimwear Discover the range of men's swimwear at ASOS. Explore swim shorts, board shorts and speedos for men in a variety of colours and lengths. Shop now at ASOS.
  113. 113. Conclusion ● Don’t blindly follow “best practice” approaches
  114. 114. Conclusion ● Consider why you’re implementing a change, and how it gets you there
  115. 115. Define your beach style with women's swimwear in printed & plain, while beach towels & swimming accessories complete your holiday bag. Next da ... Control Template Variant Template Womens Swimwear | Shorts, Beach Towels & Swimm ... https://www.next.co.uk/shop/women/clothing/swimwear Mens Swimwear | Shorts, Beach Towels & Swimming ... https://www.next.co.uk › Shop › Men › Clothing Define your beach style with men's swimwear in printed & plain, while shorts, beach towels & swimming accessories complete your holiday bag. Next da ... How will this change improve CTRs?
  116. 116. Case study #2: Overview
  117. 117. best consultant at distilled All Shopping Images News About 150,000 results (0.35 seconds) More Update title Emily Potter - Consultant - Distilled https://www.distilled.net/about/people/emily-potter/ Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. An American abroad, Emily graduated from the ... Case Study #2: Price in titles
  118. 118. Titles can pull all three levers
  119. 119. hotel from $10 a night All Shopping Images News More But this isn’t a high volume search query...
  120. 120. cheap hotels All Shopping Images News More You’ll more likely see this
  121. 121. Therefore, we’re mostly talking about CTR
  122. 122. Case study #2: Hypothesis
  123. 123. What are we changing? ● Adding the price to title
  124. 124. What levers will we pull? ● Having the price in SERP will increase CTR
  125. 125. ● We will measure organic traffic How will we know?
  126. 126. Case study #2: Control vs. Variant
  127. 127. San Francisco Hotels from $30/night | Find & compare deals https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels | Find & compare great deals | Trivago https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. Control Template Variant Template
  128. 128. San Francisco Hotels from $30/night | Find & compare deals |Trivago https://www.trivago.com › USA › California San Francisco Hotels | Find & compare deals | Trivago https://www.trivago.com › USA › California hotels in san francisco Hotels in San Francisco from $29/night - Search on KAYAK The 30 best hotels & places to stay in San Francisco, USA ... All Shopping Images News About 150,000 results (0.35 seconds) https://www.booking.com› USA › California › Visit San Francisco More Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. https://www.kayak.com › Hotels › United States › California San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. Hotels in San Francisco from $29/night - Search on KAYAK The 30 best hotels & places to stay in San Francisco, USA ... https://www.booking.com› USA › California › Visit San Francisco Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. https://www.kayak.com › Hotels › United States › California San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More Control Template Variant Template
  129. 129. Case study #2: Results
  130. 130. San Francisco Hotels from $30/night | Find & compare gr ... Control Template Variant Template https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels | Find & compare great deals | Trivago https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk.
  131. 131. So what the f*** happened?
  132. 132. What levers will we pull? ● Having the price in SERP will increase CTR
  133. 133. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. Variant Template The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $29/night - Search on KAYAK Competitor cheaper
  134. 134. Conclusion ● Having the price in the title can hurt CTRs if it makes competitors more appealing
  135. 135. Later, we re-run the test with updated price data
  136. 136. Case study #2: Test two - Hypothesis
  137. 137. What are we changing? ● Adding price to the title
  138. 138. What levers will we pull? ● Adding price to title will increase CTR
  139. 139. ● We will measure organic traffic How will we know?
  140. 140. Case study #2: Test two - Control vs. Variant
  141. 141. San Francisco Hotels from $34/night | Find & compare deals https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels | Find & compare deals | Trivago https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. Control Template Variant Template
  142. 142. San Francisco Hotels from $30/night | Find & compare deals |Trivago https://www.trivago.com › USA › California San Francisco Hotels | Find & compare deals | Trivago https://www.trivago.com › USA › California hotels in san francisco Hotels in San Francisco from $29/night - Search on KAYAK The 30 best hotels & places to stay in San Francisco, USA ... All Shopping Images News About 150,000 results (0.35 seconds) https://www.booking.com› USA › California › Visit San Francisco More Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. https://www.kayak.com › Hotels › United States › California San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. Hotels in San Francisco from $29/night - Search on KAYAK The 30 best hotels & places to stay in San Francisco, USA ... https://www.booking.com› USA › California › Visit San Francisco Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. https://www.kayak.com › Hotels › United States › California San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More Control Template Variant Template
  143. 143. Case study #2: Test two - Results
  144. 144. Control Template Variant Template San Francisco Hotels from $34/night | Find & compare deals https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels | Find & compare deals | Trivago https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk.
  145. 145. Wait so then, what the f*** happened?!
  146. 146. Test #1 and Test #2 had the same hypothesis
  147. 147. What lever will we pull? ● Having the price in SERP will increase CTR
  148. 148. But there was a difference: Test #1 had out of date prices
  149. 149. Here’s test #1 day to day in SERPs
  150. 150. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. Variant Template The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $29/night - Search on KAYAK
  151. 151. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $35/night - Search on KAYAK Variant Template
  152. 152. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $31/night - Search on KAYAK Variant Template
  153. 153. Here’s test #2 day to day in SERPs
  154. 154. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $34/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $33/night - Search on KAYAK Variant Template
  155. 155. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $35/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $30/night - Search on KAYAK Variant Template
  156. 156. hotels in san francisco All Shopping Images News About 150,000 results (0.35 seconds) More https://www.booking.com › USA › California › Visit San Francisco San Francisco's best hotels with breakfast. Seaside Inn. Comfort Inn By the Bay Hotel San Francisco. Hyatt Regency San Francisco. Holiday Inn Express Hotel…. The 30 best hotels & places to stay in San Francisco, USA ... https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $31/night | Find & compare deals | Trivago https://www.kayak.com › Hotels › United States › California Hotels in San Francisco from $34/night - Search on KAYAK Variant Template
  157. 157. What if another lever was also being pulled?
  158. 158. Theory one
  159. 159. https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago Are negative user signals a ranking factor?
  160. 160. https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago Are negative user signals a ranking factor?
  161. 161. Book now from $40/night https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago www.trivago.com/san-francisco-hotels Are negative user signals a ranking factor?
  162. 162. Book now from $40/night https://www.trivago.com › USA › California Compare the prices of 4233 hotels in San Francisco, USA. Find the ideal rate from millions of accommodation deals and save with trivago.co.uk. San Francisco Hotels from $30/night | Find & compare deals | Trivago www.trivago.com/san-francisco-hotels Prices don’t match Are negative user signals a ranking factor?
  163. 163. Theory two
  164. 164. Can having the incorrect price directly impact rankings?
  165. 165. “ For these reasons, we regularly check your website to ensure that the information you provided matched the information on your website. ”
  166. 166. It’s plausible Google would look out for this
  167. 167. Conclusion ● Having an accurate price in your titles matters
  168. 168. Case study #3: Overview
  169. 169. Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. She now works full time on the Distilled ODN team, running lots and lots and lots of split tests. Emily has also been seen speaking at conferences like BrightonSEO, SearchLeeds, and SearchLove. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Case Study #3: Internal linking Add internal links
  170. 170. Why internal linking?
  171. 171. PageRank
  172. 172. How can we measure internal linking in SEO A/B tests?
  173. 173. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  174. 174. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads Main Category Pages
  175. 175. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads Sub-category Pages
  176. 176. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  177. 177. VariantControl Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  178. 178. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads This becomes our testing section VariantControl
  179. 179. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads This becomes our first measurement section VariantControl
  180. 180. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads This becomes our second measurement section VariantControl
  181. 181. Emily joined the Distilled team as an Analyst in February of 2018 with a background in copywriting and PR. She now works full time on the Distilled ODN team, running lots and lots and lots of split tests. Emily has also been seen speaking at conferences like BrightonSEO, SearchLeeds, and SearchLove. Consulting Services Conferences Resources About Us Contact Us Emily Potter Consultant Case Study #3: Internal linking Add internal links
  182. 182. Two examples of the same test on different sites
  183. 183. Case study #3: Hypothesis
  184. 184. What are we changing? ● Adding sub category links to main category pages
  185. 185. ● More links to sub category pages will increase link equity What levers will we pull?
  186. 186. ● More links from main category pages will improve relevancy signals What levers will we pull?
  187. 187. ● More links from main category pages may decrease link equity What levers will we pull?
  188. 188. How will we know? ● We will measure organic traffic to main categories, sub categories, & overall impact
  189. 189. Case study #3: Control vs. Variant
  190. 190. VariantControl Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  191. 191. Case study #3: Results
  192. 192. VariantControl Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  193. 193. What happened to the different page types?
  194. 194. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  195. 195. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  196. 196. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  197. 197. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  198. 198. Conclusion ● Internal links have a positive impact on rankings for pages receiving links
  199. 199. Conclusion ● Internal links have a negative impact on rankings for pages sending out links (?!?)
  200. 200. We get a chance to run this test again on a different site
  201. 201. VariantControl Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  202. 202. Case study #3: Test two - Results
  203. 203. VariantControl Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads
  204. 204. What happened to the different page types?
  205. 205. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  206. 206. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  207. 207. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  208. 208. Mammals Reptiles AmphibiansBirds Cats DogsBears Snakes TurtlesLizards Owls DucksFinches Frogs NewtsToads VariantControl
  209. 209. Wait so then, what the f*** happened?!
  210. 210. Test #1 and Test #2 had the same hypothesis
  211. 211. So we have to compare the two websites
  212. 212. Website #1: E-commerce
  213. 213. VariantControl Home Furniture Dresses Nursing & Feeding Patio Furniture Living Room Furniture Nursery Furniture Bedroom Furniture Patio Seats Patio Tables Patio Heaters Baby Formula Baby Bibs Baby Food Black Dresses Party Dresses Work Dresses
  214. 214. Website #2: Grocery
  215. 215. Fish & Seafood Cooking Ingredients Chips & Potatoes Fish Pies Fish Fillets Fish Fingers Chips & Fries WedgesPotatoes Ham BurgersTurkey Herbs & Spices Oil & Dressing Gravy & Stuffing Meats & Deli
  216. 216. Theory one
  217. 217. Nursing & Feeding Patio Furniture Patio Seats Patio Tables Patio Heaters Baby Cutlery Baby Weaning Baby Bottles Baby Formula Baby Bibs Baby Food Patio Decor Patio Sets Patio Daybeds E-commerce website included more links
  218. 218. Nursing & Feeding Patio Furniture Patio Seats Patio Tables Patio Heaters Baby Cutlery Baby Weaning Baby Bottles Baby Formula Baby Bibs Baby Food Patio Decor Patio Sets Patio Daybeds E-commerce site stuffed keywords in anchor text
  219. 219. Theory two
  220. 220. Grocery website had stronger relevancy signals Fish & Seafood Cooking Ingredients Chips & Potatoes Fish Pies Fish Fillets Fish Fingers Chips & Fries WedgesPotatoes Ham BurgersTurkey Herbs & Spices Oil & Dressing Gravy & Stuffing Meats & Deli
  221. 221. Theory three
  222. 222. Grocery website created pathways for users Fish & Seafood Cooking Ingredients Chips & Potatoes Fish Pies Fish Fillets Fish Fingers Chips & Fries WedgesPotatoes Ham BurgersTurkey Herbs & Spices Oil & Dressing Gravy & Stuffing Meats & Deli
  223. 223. Could this send positive user signals?
  224. 224. Conclusion ● Internal linking can boost rankings for pages receiving links
  225. 225. Conclusion ● Internal linking can benefit pages sending out links, but not universally
  226. 226. Case study #4: Overview
  227. 227. £60 Shop our over 600 different styles of women’s shoes to complete your wardrobe. We have heels, boots, trainers, and any other type of shoe you could imagine. All our shoes come with free shipping and handling. Dresses Jackets Shoes Activewear Accessories Women's Shoes 654 results New In! Red heels £55 Teal heels Black heels £65 Case Study #4: New product page template
  228. 228. Client had positive CRO test with new template
  229. 229. They wanted to confirm it wouldn’t hurt SEO
  230. 230. The catch: This new template used React
  231. 231. What is React? ● React is a JavaScript framework that’s popular with developers
  232. 232. Case study #4: Hypothesis
  233. 233. What are we changing? ● New product page template utilising React framework
  234. 234. What levers will we pull? ● Improved performance will increase organic rankings
  235. 235. ● We will measure organic traffic How will we know?
  236. 236. Case study #4: Control vs. Variant
  237. 237. $60 Shop our over 600 different styles of women’s shoes to complete your wardrobe. We have heels, boots, trainers, and any other type of shoe you could imagine. All our shoes come with free shipping and handling. Dresses Jackets Shoes Accessories Women's Shoes 654 results New In! Red heels $55 Teal heels $75 Black heels $60 Shop our over 400 different styles of men’s sweaters to complete your wardrobe. We have sweaters of all colours and material. All our sweaters come with free shipping and handling. Dresses Jackets Shoes Accessories Men’s Sweaters 437 results New In! Red sweater $55 Teal sweater $75 Green sweater Control Template Variant Template
  238. 238. Case study #4: Results
  239. 239. $60 Shop our over 600 different styles of women’s shoes to complete your wardrobe. We have heels, boots, trainers, and any other type of shoe you could imagine. All our shoes come with free shipping and handling. Dresses Jackets Shoes Accessories Women's Shoes 654 results New In! Red heels $55 Teal heels $75 Black heels $60 Shop our over 400 different styles of men’s sweaters to complete your wardrobe. We have sweaters of all colours and material. All our sweaters come with free shipping and handling. Dresses Jackets Shoes Accessories Men’s Sweaters 437 results New In! Red sweater $55 Teal sweater $75 Green sweater Control Template Variant Template
  240. 240. What the f*** happened?!
  241. 241. We haven’t changed the SERP
  242. 242. We haven’t changed the SERP
  243. 243. We haven’t added new content
  244. 244. We haven’t added new content
  245. 245. That means there was a ranking increase
  246. 246. How?
  247. 247. Two important points
  248. 248. $60 Shop our over 600 different styles of women’s shoes to complete your wardrobe. We have heels, boots, trainers, and any other type of shoe you could imagine. All our shoes come with free shipping and handling. Dresses Jackets Shoes Accessories Women's Shoes 654 results New In! Red heels $55 Teal heels $75 Black heels $60 Shop our over 400 different styles of men’s sweaters to complete your wardrobe. We have sweaters of all colours and material. All our sweaters come with free shipping and handling. Dresses Jackets Shoes Accessories Men’s Sweaters 437 results New In! Red sweater $55 Teal sweater $75 Green sweater Control Template Variant Template First, Googlebot could process without JS
  249. 249. $60 Shop our over 600 different styles of women’s shoes to complete your wardrobe. We have heels, boots, trainers, and any other type of shoe you could imagine. All our shoes come with free shipping and handling. Dresses Jackets Shoes Accessories Women's Shoes 654 results New In! Red heels $55 Teal heels $75 Black heels $60 Shop our over 400 different styles of men’s sweaters to complete your wardrobe. We have sweaters of all colours and material. All our sweaters come with free shipping and handling. Dresses Jackets Shoes Accessories Men’s Sweaters 437 results New In! Red sweater $55 Teal sweater $75 Green sweater Control Template Variant Template Second, the variant now loads faster
  250. 250. ● Increased page speed improves organic rankings (because it improves user signals?) Conclusion
  251. 251. ● React and other JavaScript frameworks can be positive if implemented with SEO in mind Conclusion
  252. 252. SEO A/B testing in depth Case studies to apply understanding Takeaways
  253. 253. Takeaway #1: SEO A/B testing relieves us from noise created by interactions between our 3 levers
  254. 254. Takeaway #2: Many SEO “best practices” fail. SEO A/B testing rescues would-be-lost sessions
  255. 255. Don’t be sheep.
  256. 256. Don’t be sheep.
  257. 257. Takeaway #3: Even tried and true “best practices” can go wrong if we forget why we’re doing it
  258. 258. Takeaway #4: Don’t blindly add internal links
  259. 259. Takeaway #5: Page speed impacts rankings. JS frameworks can help if done with SEO in mind
  260. 260. Emily Potter - Distilled http://bit.ly/2IQNWhI @em_potter www.linkedin.com/in/emargaretpotter/

×