Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[MozCon 2021] Taking Charge of Your Indexability

Taking Charge of Your Indexability: How to Optimize and Prioritize Your Technical Work
Take charge over the indexability of your website! With a focus on aggregators and classifieds, Areej will share advice on how to best reduce index bloat for large websites. Diving into parameter handling, sitemap logic, robots directives, and more, we'll also assess how to analyze the most impactful changes, how to get sign-off from senior stakeholders, and how to prioritize work with product teams.

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

[MozCon 2021] Taking Charge of Your Indexability

  1. 1. @areej_abuali Taking Charge of Your Indexability How to Optimise and Prioritise Your Technical Work Areej AbuAli SEO Consultant @areej_abuali
  2. 2. @areej_abuali HELLO! I’m here to talk to you about something that took me a while to wrap my head around... 2
  3. 3. @areej_abuali But spending the last 18 months working on a 40M+ indexed page website forced me to finally get it... 3
  4. 4. “ We are IN CONTROL of how Google crawls and indexes our website 4
  5. 5. “ We are IN CONTROL of how Google crawls and indexes our website (but it can feel scary!) 5
  6. 6. @areej_abuali Because if we don’t do it properly, then we won’t benefit from many* things we’re doing right 6 *All the awesome content and off page work
  7. 7. @areej_abuali So if we’re recommending SEO fixes (small or large), let’s please drop the 100-page audit 7
  8. 8. @areej_abuali And focus on prioritising recommendations that will have an impact 8
  9. 9. Setting the Scene
  10. 10. @areej_abuali We work for a car aggregator website
  11. 11. @areej_abuali Our users can... ▸ Search for cars near their location 11
  12. 12. @areej_abuali ▸ Search for cars near their location ▸ Filter the search results by make, model, year, type, transmission, engine size, colour, price range (and more!) 12 Our users can...
  13. 13. @areej_abuali ▸ Search for cars near their location ▸ Filter the search results by make, model, year, type, transmission, engine size, colour, price range (and more!) ▸ View car listings and contact sellers 13 Our users can...
  14. 14. @areej_abuali ▸ Search for cars near their location ▸ Filter the search results by make, model, year, type, transmission, engine size, colour, price range (and more!) ▸ View car listings and contact sellers ▸ View car history information 14 Our users can...
  15. 15. @areej_abuali ▸ Search for cars near their location ▸ Filter the search results by make, model, year, type, transmission, engine size, colour, price range (and more!) ▸ View car listings and contact sellers ▸ View car history information ▸ Read the latest car news 15 Our users can...
  16. 16. @areej_abuali 16 Our users can...
  17. 17. @areej_abuali Most webmasters tend to set up websites in one of two ways... 17
  18. 18. @areej_abuali 1. Index everything to capture as much ranking opportunity as possible 18 They either attempt to...
  19. 19. @areej_abuali 1. Index everything to capture as much ranking opportunity as possible 2. No-index a huge chunk to avoid suffering from index bloat 19 They either attempt to...
  20. 20. @areej_abuali Index bloat is when a site has an excessive number of low-value pages indexed 20
  21. 21. @areej_abuali 21 We might end up indexing: ▸ Every possible geo-location ▸ Every possible filter combination ▸ Every possible car listing page ▸ Every possible car history information page If we attempt to index everything...
  22. 22. @areej_abuali Why is this a problem? 22
  23. 23. @areej_abuali 23 ▸ Every possible geo-location ▸ Every possible filter combination ▸ Every possible car listing page ▸ Every possible car history information page Will Google bother crawling…?
  24. 24. @areej_abuali 24 ▸ Every possible geo-location ▸ Every possible filter combination ▸ Every possible car listing page ▸ Every possible car history information page Will Google bother indexing…?
  25. 25. @areej_abuali 25 ▸ Every possible geo-location ▸ Every possible filter combination ▸ Every possible car listing page ▸ Every possible car history information page Do you even want to rank for…?
  26. 26. @areej_abuali The answer to all three questions is most probably: No 26
  27. 27. @areej_abuali 27 1. Google might not crawl all our pages 2. Google might not index all our pages 3. Our valuable pages might not be crawled and/or indexed 4. Our valuable pages might not rank If we attempt to index everything…
  28. 28. @areej_abuali I’m being generous by using the word “might” 28
  29. 29. @areej_abuali 1. Index everything to capture as much ranking opportunity as possible 2. No-index a huge chunk to avoid suffering from index bloat 29 They either attempt to...
  30. 30. @areej_abuali 30 We’d miss out on ranking potential for: ▸ Every possible geo-location ▸ Every possible filter combination ▸ Every possible car listing page ▸ Every possible car history information page If we no-index a huge chunk...
  31. 31. @areej_abuali 1. Index everything to capture as much ranking opportunity as possible 2. No-index a huge chunk to avoid suffering from index bloat 31 They either attempt to...
  32. 32. @areej_abuali So, how and where do we 32
  33. 33. @areej_abuali Before we make any decision on how to control our indexability, we need to know the answer to one question 33
  34. 34. @areej_abuali What’s the high level KPI that we’re being measured against? 34
  35. 35. @areej_abuali 1. Organic rank 35 It probably isn’t...
  36. 36. @areej_abuali 1. Organic rank 2. Organic sessions 36 It probably isn’t...
  37. 37. @areej_abuali 1. Organic rank 2. Organic sessions 3. Some third party tool’s visibility chart... 37 It probably isn’t...
  38. 38. @areej_abuali 38 It probably isn’t...
  39. 39. @areej_abuali It’s most likely organic leads or revenue
  40. 40. @areej_abuali An organic lead on our car aggregator site happens when a user contacts a seller for a potential car
  41. 41. @areej_abuali Indexed Pages 41 Organic Leads So, why don’t we tie these together?
  42. 42. @areej_abuali ▸ Search for cars near their location ▸ Filter the search results by make, model, year, type, transmission, engine size, colour, price range (and more!) ▸ View car listings and contact sellers ▸ View car history information ▸ Read the latest car news 42 Let’s go back to this list...
  43. 43. @areej_abuali I tend to understand things better when they’re in table format, so... 43
  44. 44. @areej_abuali 44 Core Templates Description Car search pages (CSP) Result pages when a user searches for cars by location (includes all search filters) Car listing pages (CLP) Listing pages when a user chooses a car to learn more about it Car history pages (CHP) Information pages that include car valuation and history Other All other pages (homepage, blog, about, etc…)
  45. 45. @areej_abuali We now need to figure out which templates provide the most value 45
  46. 46. Let’s analyse some data
  47. 47. @areej_abuali 1. # of indexed pages (and % from total) 2. # of organic sessions (and % from total) 3. # of organic leads (and % from total) 47 Sessions & Leads date range --> past 12 months What data do we need per template?
  48. 48. @areej_abuali 48 Template Indexed Pages Organic Session s Organic Leads CSP 35% 55% 75% Other 10% 20% 15% CLP 10% 15% 9% CHP 45% 10% 1%
  49. 49. @areej_abuali 49 Template Indexed Pages Organic Session s Organic Leads CSP 35% 55% 75% Other 10% 20% 15% CLP 10% 15% 9% CHP 45% 10% 1%
  50. 50. @areej_abuali Wouldn’t we be better off spreading our link equity across more valuable pages? 50
  51. 51. @areej_abuali Wouldn’t we be better off having Google crawl our more valuable pages? 51
  52. 52. @areej_abuali And most importantly: Is it worth doubling up our number of indexed pages for only 1% of organic leads? 52
  53. 53. @areej_abuali Websites do not (and most probably should not) need to have every single page indexed 53
  54. 54. @areej_abuali Instead, focus on indexing pages that have the potential to provide good search results 54
  55. 55. @areej_abuali Yes, this feels like a fairly straight forward example...but every aggregator site tends to have one of these 55
  56. 56. @areej_abuali 56 Template Indexed Pages Organic Session s Organic Leads CSP 35% 55% 75% There’s also so much more that can be dissected on a template by template basis
  57. 57. @areej_abuali ▸ Make ▸ Model ▸ Year ▸ Type 57 ▸ Transmission ▸ Engine size ▸ Colour ▸ Price range Our indexed filters
  58. 58. @areej_abuali ▸ Make (10) ▸ Model (4) ▸ Year (10) ▸ Type (4) 58 ▸ Transmission (2) ▸ Engine size (5) ▸ Colour (10) ▸ Price range (5) Our indexed filters
  59. 59. @areej_abuali That’s 800,000 potential indexed pages purely based on our search result filters 59
  60. 60. @areej_abuali 60 Template Indexed Pages Organic Session s Organic Leads CSP 35% 55% 75% So it’s fundamental that we break this down to the next level
  61. 61. @areej_abuali 61 Template Indexed Pages Organic Sessions Organic Leads CSP/Make CSP/Model CSP/Year CSP/Type
  62. 62. “ The more we slice and dice our data, the more we’ll be able to make informed decisions. 62
  63. 63. *Intermission*
  64. 64. @areej_abuali A typical meeting in the life of an SEO... 64
  65. 65. @areej_abuali Let’s say hello to a senior stakeholder named Mark 65
  66. 66. @areej_abuali “Car history information pages are critical for our consumers. They can look up any car registration number and get all the information they need.” 66 Mark
  67. 67. @areej_abuali “Yes, but they are useless for Google. They’re thin pages that provide low organic value and double up our indexed pages.” 67 SEO:
  68. 68. @areej_abuali “Nevermind Google, what does this mean for LEADS?” 68 Mark:
  69. 69. @areej_abuali “Well, I’m glad you asked because I spent the past 2 weeks doing this piece of analysis where I used BigQuery to group our page templates, calculate our organic leads per template, and…. ” 69 SEO:
  70. 70. @areej_abuali 70 “....” Mark:
  71. 71. @areej_abuali “1% of our total organic leads. ” 71 SEO:
  72. 72. @areej_abuali Dream Life Scenario: “Is that it? De-index these pages immediately!” 72 Mark:
  73. 73. @areej_abuali Real Life Scenario: “Work with Team A to build a strategy that can convince Team B to convince Team C to consider convincing Team D to potentially de-index the pages.” 73 Mark:
  74. 74. @areej_abuali 74 “....” SEO:
  75. 75. @areej_abuali Joking aside, I promise you there is a purpose to this story 75
  76. 76. “ As SEOs, we need to speak the SAME language as senior stakeholders to get sign off on technical fixes. 76
  77. 77. Let’s Recommend
  78. 78. @areej_abuali 78 Template Indexed Pages Organic Session s Organic Leads CSP 35% 55% 75% Other 10% 20% 15% CLP 10% 15% 9% CHP 45% 10% 1%
  79. 79. @areej_abuali 79 Template Indexed Pages Organic Session s Organic Leads CHP 45% 10% 1%
  80. 80. @areej_abuali I know what you’re thinking: 10% of total organic sessions is a scary thing to simply get rid of... 80
  81. 81. @areej_abuali I’m very risk-averse but in this specific case: What good is 10% traffic that doesn’t convert? 81
  82. 82. @areej_abuali Every aggregator/classified type website tends to have something similar to a car history page 82
  83. 83. @areej_abuali As Mark told us: These pages are useful for our users because they want to look up info via a car registration number 83
  84. 84. @areej_abuali So to be on the safe side, let’s do a few checks on our car history pages (CHPs) 84
  85. 85. @areej_abuali Do they take up lots of crawl budget? 85 Yes, we analysed our logs and in an average week, 50% of Googlebot’s crawl requests are spent on CHPs.
  86. 86. @areej_abuali Do they rank for important terms? 86 No, we analysed our ranking data and CHPs barely rank for any keywords we care about.
  87. 87. @areej_abuali Do they have unique content? 87 Yes, but it’s thin, only useful for users on a personalised basis and doesn’t convert.
  88. 88. @areej_abuali Do they have high quality backlinks? 88 No, they barely have backlinks pointing at them and none that are high quality.
  89. 89. @areej_abuali ▸ 45% of total indexed pages ▸ 50% of weekly crawl requests ▸ 10% of total organic sessions ▸ Thin content ▸ Minimal ranking ▸ Minimal backlinks ▸ 1% of total organic leads 89 So, to summarise:
  90. 90. @areej_abuali Now, do we feel more comfortable de-indexing them? 90
  91. 91. @areej_abuali 91 Things to remember Make sure your sitemaps are setup to automatically remove any noindex pages
  92. 92. @areej_abuali This is an example of something that might seem easy in principle but can be more complex due to legacy code 92
  93. 93. @areej_abuali 93 Things to remember Our CHPs won’t de-index overnight, it could take a few months (or more) for Google to flush them out of the index
  94. 94. @areej_abuali 94
  95. 95. @areej_abuali 95 Things to remember Blocking CHPs from robots.txt should be on the roadmap but give it 1-3 months to flush out of the index
  96. 96. Let’s Prioritise
  97. 97. @areej_abuali As an industry, let’s make a pact to drop the 100-page audit documents 97
  98. 98. @areej_abuali 98 Prioritisation is KEY
  99. 99. @areej_abuali 99 I like to use T-shirt Sizing because it’s simple
  100. 100. @areej_abuali 100 How much of an SEO impact will this recommendation likely have on our overall organic leads? SEO Impact
  101. 101. @areej_abuali 101 How much Technical Effort will this recommendation take to implement? Tech Effort
  102. 102. @areej_abuali 102 Tech Effort
  103. 103. @areej_abuali 103 For every identified issue: Prioritisation Matrix
  104. 104. @areej_abuali 104
  105. 105. @areej_abuali 105 If you want to learn more about my full framework: bit.ly/areej-matrix
  106. 106. Let’s Communicate
  107. 107. @areej_abuali So, what’s the first thing that tends to happen once an SEO recommendation gets implemented? 107
  108. 108. @areej_abuali Real Life Scenario: “Are organic leads up yet?” 108 *Yup, still Mark! Senior Stakeholder*:
  109. 109. @areej_abuali Yes, our senior stakeholder will be expecting results, and they’ll be expecting it fast 109
  110. 110. @areej_abuali ▸ Communicate before ▸ Communicate during ▸ Communicate after 110 Manage expectations
  111. 111. @areej_abuali Under-promise & Over-deliver 111
  112. 112. @areej_abuali De-indexing CHPs is simply a stepping stone* for fixing technical foundation 112 *It won’t magically increase our share of voice
  113. 113. @areej_abuali Communicating updates is one thing... 113
  114. 114. @areej_abuali Communicating updates that are backed up with data is a game changer 114
  115. 115. @areej_abuali Indexed Pages 115 Organic Leads Let’s go back here...
  116. 116. @areej_abuali ▸ Update these metrics on a weekly basis ▸ Show WoW, MoM and YoY change ▸ Make our dashboards accessible & open 116 Keep everyone informed
  117. 117. @areej_abuali Stakeholders feel better knowing the information is there (even if they don’t bother checking it) 117
  118. 118. @areej_abuali And it’ll serve as a benchmark and can be used as case studies for future sign off 118
  119. 119. One Final Thought…
  120. 120. @areej_abuali I’ve personally never worked on a car aggregator website (yet) This was just an example 120
  121. 121. @areej_abuali But this methodology can apply to all forms of aggregator sites: fashion, jobs, property... 121
  122. 122. @areej_abuali When I first pitched this talk, my aim was to go into detail about the technical SEO requirements for aggregator sites 122
  123. 123. @areej_abuali But the actual technical recommendation is the easy part... 123
  124. 124. @areej_abuali It’s the backing up our recommendation with data, the stakeholder management, the meetings for the sake of meetings… (those are the hard parts) 124
  125. 125. @areej_abuali The first time I worked on an aggregator- type site, I panicked 125
  126. 126. I felt completely out of my depth and I didn’t know where or how to start
  127. 127. @areej_abuali But they’re now the type of websites that I enjoy working on the most... 127
  128. 128. @areej_abuali Because over time, I’ve come to realise that they all have similar problems that require similar solutions 128
  129. 129. @areej_abuali And once we fix the problems that actually matter, the impact feels very rewarding 129
  130. 130. @areej_abuali Yes, rewarding in terms of SEO impact 130
  131. 131. @areej_abuali But also in terms of achieving higher level of trust with stakeholders... 131
  132. 132. @areej_abuali Which makes the process (and our life!) easier the next time round 132
  133. 133. @areej_abuali The past 132 slides were based on an example that simply touched *one* metric 133
  134. 134. @areej_abuali I tied number of indexed pages (a technical SEO metric) to organic leads (the KPI that matters to the business) 134
  135. 135. @areej_abuali This same concept can be translated across far more SEO metrics 135
  136. 136. @areej_abuali As SEOs, it’s fundamental that we don’t work in silos 136
  137. 137. @areej_abuali We can achieve far more if we measure our wins using metrics that our business cares about 137
  138. 138. @areej_abuali And please remember... 138
  139. 139. “ It’s okay to feel overwhelmed working on something new =) 139
  140. 140. @areej_abuali bit.ly/areej-mozcon 140
  141. 141. @areej_abuali 141 THANKS! Questions? You can find me on: ▸ areejabuali.com ▸ @areej_abuali
  142. 142. Thank You!

    Be the first to comment

  • AndySimpson4

    Jul. 13, 2021
  • gueamu

    Jul. 13, 2021
  • blmbmj

    Jul. 14, 2021
  • communicationsnetwork

    Jul. 14, 2021
  • guest6e29902

    Jul. 14, 2021
  • AvinashKumar1736

    Jul. 14, 2021
  • MeganAnderson41

    Jul. 14, 2021
  • Martijn11

    Jul. 14, 2021
  • MohamedNabil321

    Jul. 14, 2021
  • ongchinonn

    Jul. 16, 2021
  • MouktikSaha

    Jul. 16, 2021
  • MdSajib6

    Jul. 17, 2021
  • Yannimus

    Jul. 17, 2021
  • nkapsomenakis

    Jul. 19, 2021
  • kmcvey

    Jul. 20, 2021

Taking Charge of Your Indexability: How to Optimize and Prioritize Your Technical Work Take charge over the indexability of your website! With a focus on aggregators and classifieds, Areej will share advice on how to best reduce index bloat for large websites. Diving into parameter handling, sitemap logic, robots directives, and more, we'll also assess how to analyze the most impactful changes, how to get sign-off from senior stakeholders, and how to prioritize work with product teams.

Views

Total views

2,661

On Slideshare

0

From embeds

0

Number of embeds

388

Actions

Downloads

0

Shares

0

Comments

0

Likes

15

×