Successfully reported this slideshow.
Your SlideShare is downloading. ×

TechSEO Boost 2021 - SEO Experimentation

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 49 Ad

TechSEO Boost 2021 - SEO Experimentation

Download to read offline

View the recording here: https://www.catalystdigital.com/techseoboost/#on-demand-recordings

Brian Ta, Product Manager, Coinbase

Learn how to set up an A/B testing framework, and more importantly, how you should approach, design, and run SEO experiments.

View the recording here: https://www.catalystdigital.com/techseoboost/#on-demand-recordings

Brian Ta, Product Manager, Coinbase

Learn how to set up an A/B testing framework, and more importantly, how you should approach, design, and run SEO experiments.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to TechSEO Boost 2021 - SEO Experimentation (20)

Advertisement

More from Catalyst (20)

Advertisement

TechSEO Boost 2021 - SEO Experimentation

  1. 1. Your Name | @Twitterhandle | #TechSEOBoost SEO Experimentation Brian Ta Product Manager, Coinbase
  2. 2. Brian Ta | @fanfavorite_bta | #TechSEOBoost How to run SEO experiments
  3. 3. Brian Ta | @fanfavorite_bta | #TechSEOBoost About me
  4. 4. Brian Ta | @fanfavorite_bta | #TechSEOBoost Strava Product Manager, Acquisition Airbnb SEO Lead AngelList Product Manager, Growth
  5. 5. Brian Ta | @fanfavorite_bta | #TechSEOBoost what is SEO experimentation?
  6. 6. Brian Ta | @fanfavorite_bta | #TechSEOBoost Assumptions: 1. You’re familiar with SEO 2. Organic traffic is already one of the main drivers of your business 3. You have thousands of pages
  7. 7. Brian Ta | @fanfavorite_bta | #TechSEOBoost it’s like any normal a/b test you’d run, but cooler(and different) what can you test? anything your heart desires* title tags meta descriptions image alt tags content any on-page element
  8. 8. Brian Ta | @fanfavorite_bta | #TechSEOBoost it allows you to know, with certainty, how effective your changes are
  9. 9. Brian Ta | @fanfavorite_bta | #TechSEOBoost how is it useful?
  10. 10. Brian Ta | @fanfavorite_bta | #TechSEOBoost it takes the guesswork out of SEO
  11. 11. Brian Ta | @fanfavorite_bta | #TechSEOBoost You get to learn and test the boundaries of SEO safely
  12. 12. Brian Ta | @fanfavorite_bta | #TechSEOBoost how many of you knew this? Airbnb SEO team has known since 2017 this is a competitive advantage
  13. 13. Brian Ta | @fanfavorite_bta | #TechSEOBoost when is it useful?
  14. 14. Brian Ta | @fanfavorite_bta | #TechSEOBoost 1. When you have something to learn/test 2. If leadership wants you to prove out your wins
  15. 15. Brian Ta | @fanfavorite_bta | #TechSEOBoost experiment if you have something to learn Don’t test for the sake of testing Ship often. Ship fast. Run experiments as they’re needed, not because you can. Embody a growth mindset. If you’re not learning, you’re failing.
  16. 16. Brian Ta | @fanfavorite_bta | #TechSEOBoost opportunity optimization know the difference
  17. 17. Brian Ta | @fanfavorite_bta | #TechSEOBoost How do we build it?
  18. 18. Brian Ta | @fanfavorite_bta | #TechSEOBoost By yourself. In-house. → The large majority of SEO experimentation platforms use Javascript to do bucketing. 👎 → Non-SEO specific experimentation platforms don’t let you bucket by page. → The ones that don’t ask you to route through their own CDN 👎
  19. 19. Brian Ta | @fanfavorite_bta | #TechSEOBoost Regular A/B Tests → They’re bucketed on the user level → Users will get the same experience on the entire site
  20. 20. Brian Ta | @fanfavorite_bta | #TechSEOBoost Regular A/B Tests control treatment user www.example.com/category/{pdp}
  21. 21. Brian Ta | @fanfavorite_bta | #TechSEOBoost this obviously doesn’t work for seo why doesn’t it? Inconsistent experience for Google it’s not a consistent experience for Google If Google hits the same page multiple times, it’s going to get bucketed into control or treatment, and will get an inconsistent experience. experimentation platforms leverage JS/cookies to do bucketing This is not ideal for Google, as Google doesn’t execute JS everytime it crawls a page and it doesn’t persist cookies across sessions
  22. 22. Brian Ta | @fanfavorite_bta | #TechSEOBoost SEO A/B Tests → They’re bucketed on the page level → Users will get a different experience based on the page they’re on → Google will get a consistent experience
  23. 23. Brian Ta | @fanfavorite_bta | #TechSEOBoost SEO A/B Tests control treatment Googlebot www.example.com/category/roses www.example.com/category/lilies
  24. 24. Brian Ta | @fanfavorite_bta | #TechSEOBoost Let’s get tactical
  25. 25. Brian Ta | @fanfavorite_bta | #TechSEOBoost Prerequisites → You’ll need the ability to bucket your control and treatment on the page level → You’ll need the ability to track incoming organic traffic to your site → You’ll need a decent amount of traffic to your set of pages. ~5k organic traffic a day
  26. 26. Brian Ta | @fanfavorite_bta | #TechSEOBoost bucketing your control and treatment on the page level how do I do this? Programmatically or manually Programatically Take the hash of the URL to decide whether it goes in control or treatment Manually If your site is small enough, you can manually define it yourself. Try your best to make sure that organic traffic to both buckets is roughly the same
  27. 27. Brian Ta | @fanfavorite_bta | #TechSEOBoost tracking incoming organic traffic how do I do this? Pulling from as close to the data source as possible Don’t use GA It’s sampled. I guess it’ll work in a pinch, but I wouldn’t trust GA data. How’d we do it elsewhere? Airbnb has a visitors table that logged every visitor and how they came into the website. That’s what we used. Same with Strava
  28. 28. Brian Ta | @fanfavorite_bta | #TechSEOBoost Step 1: Design and Launch → Design, implement, and launch your experiment. → Run one experiment at a time to start → Roll out your experiment to your entire site
  29. 29. Brian Ta | @fanfavorite_bta | #TechSEOBoost Step 2: Wait → Let it run between 2-4 weeks → SEO experiments take longer than normal A/B tests to show impact
  30. 30. Brian Ta | @fanfavorite_bta | #TechSEOBoost seriously. wait. why? that’s just the nature of running experiments. you need time for the results to settle.
  31. 31. Brian Ta | @fanfavorite_bta | #TechSEOBoost Step 3: Analysis → We’re going to look at the difference in expected difference of traffic. → Compare the difference in traffic pre- experiment, and the difference in traffic post-experiment
  32. 32. Brian Ta | @fanfavorite_bta | #TechSEOBoost measuring difference in difference how do I do this? Look at the difference in traffic before launch, and difference after launch
  33. 33. Brian Ta | @fanfavorite_bta | #TechSEOBoost Step 4: Conclude → You can conclude if the experiment has run for more than two weeks, and if your results are statistically significant* *if you have someone to tell you that. If not, just launch it if it looks good
  34. 34. Brian Ta | @fanfavorite_bta | #TechSEOBoost Let’s design an experiment
  35. 35. Brian Ta | @fanfavorite_bta | #TechSEOBoost Setting the stage We’re a Growth PM at Figma on the Community team, and we want to grow the organic traffic coming into the Community pages
  36. 36. Brian Ta | @fanfavorite_bta | #TechSEOBoost Current title tag: {name of file | description of file that gets truncated} <title>Figma - iOS & iPadOS 14 UI Kit for Figma | <p>Excited to share the latest iOS &amp; iPadOS 14 UI Kit for Figma!</p><p><br></p><p>---</p><p><...</title> Step 1: Designing the experiment
  37. 37. Brian Ta | @fanfavorite_bta | #TechSEOBoost Title tag to test: {name of file | tags} <title>Figma - iOS & iPadOS 14 UI Kit for Figma | 14, alert, apple, dark, emoji, ios, iphone, kit, light</title> Step 1: Designing the experiment cont’d
  38. 38. Brian Ta | @fanfavorite_bta | #TechSEOBoost We’ll do a straight A/B test with 50/50 treatment versus control to ALL Community File pages. Step 2: Rollout
  39. 39. Brian Ta | @fanfavorite_bta | #TechSEOBoost Title tag experiments are usually impactful. I’d probably wait two weeks before running an analysis. Step 3: Wait
  40. 40. Brian Ta | @fanfavorite_bta | #TechSEOBoost We’ll look at the expected difference in traffic if you hadn’t run the experiment (control group) and compare it against the difference that you see now that you’ve made the change (treatment). I’ll work with a data scientist to make sure that we’re pulling data from our visitors table. Step 4: Analysis
  41. 41. Brian Ta | @fanfavorite_bta | #TechSEOBoost If results are positive and are statistically significant, I’d launch to 100%. If it’s neutral, or inconclusive after a month, you can use your own judgment on whether or not you want to launch it. Step 5: Launch
  42. 42. Brian Ta | @fanfavorite_bta | #TechSEOBoost Document your learnings, and start thinking about other experiments you can do. Step 6: Iterate
  43. 43. Brian Ta | @fanfavorite_bta | #TechSEOBoost things to keep in mind
  44. 44. Brian Ta | @fanfavorite_bta | #TechSEOBoost impact of experiments will decay. +5% increase in traffic won’t last forever
  45. 45. Brian Ta | @fanfavorite_bta | #TechSEOBoost don’t be afraid to ship neutral experiments. experiments are to inform your decision.
  46. 46. Brian Ta | @fanfavorite_bta | #TechSEOBoost experimentation should not be slowing you down
  47. 47. Brian Ta | @fanfavorite_bta | #TechSEOBoost have a good growth/experiment culture. ideate. iterate. test. launch.
  48. 48. Brian Ta | @fanfavorite_bta | #TechSEOBoost The Growth Process Bottoms Up Ideation Grouping & Prioritizing Experiment Scoping Opportunity Scoping Experimentation & Learning Rapid Prototyping Experiment Execution Insights & Sharing Research & Analysis
  49. 49. Brian Ta | @fanfavorite_bta | #TechSEOBoost Thanks! @fanfavorite_bta

×