Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How To Tackle Enterprise Sites - Rachel Costello, Technical SEO, DeepCrawl

1,664 views

Published on

How do you extract meaningful insights from the largest websites, spanning millions of URLs? Digging into that amount of data to find insights, or even knowing where to start can be daunting, but at DeepCrawl we've already done the hard work for you. In this talk, Rachel will share real-world examples of how we worked with some top-tier, world-class brands to dissect and analyse their enterprise sites, and how that data was used to inform impactful changes that improved the quality of their websites.

Published in: Marketing

How To Tackle Enterprise Sites - Rachel Costello, Technical SEO, DeepCrawl

  1. 1. @rachellcostello brightonSEO TALK ABOUT WHAT YOU KNOW What we do every day: Help people to crawl websites to get insights, no matter how many URLs they have.
  2. 2. Recommended fixes for a better site. @rachellcostello brightonSEO WHAT WE’LL COVER Tips and tactics for crawling large sites. Common pitfalls of the biggest brands.
  3. 3. CRAWLING LARGE SITES Effectively & Efficiently @rachellcostello brightonSEO
  4. 4. The sheer scale of enterprise sites and knowing where to start can be daunting. @rachellcostello brightonSEO
  5. 5. That’s why you need to start smaller. @rachellcostello brightonSEO
  6. 6. Targeted crawls can run regularly, allowing even the largest sites to bypass resource and time constraints. @rachellcostello brightonSEO
  7. 7. Good news: you don’t need to crawl every URL every time. TACTICAL CRAWLING @rachellcostello brightonSEO
  8. 8. You only need enough data to validate issues. TACTICAL CRAWLING @rachellcostello brightonSEO
  9. 9. TACTICAL CRAWLING = @rachellcostello brightonSEO Getting the data you need as quickly as possible.1 Building the bigger picture from smaller parts.2
  10. 10. @rachellcostello brightonSEO
  11. 11. @rachellcostello brightonSEO
  12. 12. With tactical crawling methods, you can use segments to give a representative picture of the whole site. @rachellcostello brightonSEO
  13. 13. Learn more about a site by breaking it into manageable, predictable chunks. @rachellcostello brightonSEO
  14. 14. Depending on the vertical, each site should be able to be split into 5-7 categories: @rachellcostello brightonSEO
  15. 15. Category PagesProduct Pages Blog Pages Vanity Pages Navigational Pages Transactional Pages FAQ Pages @rachellcostello brightonSEO
  16. 16. SAMPLING SLICING @rachellcostello brightonSEO
  17. 17. SAMPLING @rachellcostello brightonSEO
  18. 18. SAMPLING This method is about collecting a percentage of the whole site. @rachellcostello brightonSEO
  19. 19. SAMPLING METHOD #1 Start a small crawl and find areas of crawl waste to exclude for whole crawl. @rachellcostello brightonSEO
  20. 20. @rachellcostello brightonSEO Homepage Category Page Subcategory Page Tracking URLs Pagination URLs Product Page Parameter URLs Social Tag URLs
  21. 21. @rachellcostello brightonSEO
  22. 22. SAMPLING METHOD #2 Use level limitation to assess site breadth without getting lost in the weeds. @rachellcostello brightonSEO
  23. 23. @rachellcostello brightonSEO Homepage Category Page Subcategory Page Category Page Category Page Subcategory Page Subcategory Page Level 1 Level 2 Level 3 Product Page Product Page Product PageLevel 4
  24. 24. @rachellcostello brightonSEO
  25. 25. SAMPLING METHOD #3 Crawl a certain number of examples of each page type. @rachellcostello brightonSEO
  26. 26. @rachellcostello brightonSEO Product Page 4Category Page 4 Blog Post 4 Category Page 3 Category Page 2 Category Page 1 Product Page 3 Product Page 2 Product Page 1 Blog Post 3 Blog Post 2 Blog Post 1
  27. 27. @rachellcostello brightonSEO
  28. 28. SLICING @rachellcostello brightonSEO
  29. 29. SLICING This method is about analysing an isolated section of the site. @rachellcostello brightonSEO
  30. 30. SLICING METHOD #1 All of the products but none of the categories (horizontal slice). @rachellcostello brightonSEO
  31. 31. @rachellcostello brightonSEO Homepage Category Page Subcategory Page Category Page Category Page Subcategory Page Subcategory Page Level 1 Level 2 Level 3 Product Page Product Page Product PageLevel 4
  32. 32. @rachellcostello brightonSEO
  33. 33. SLICING METHOD #2 Crawl a distinct section that serves a separate function (e.g. blog or separate mobile site). @rachellcostello brightonSEO
  34. 34. @rachellcostello brightonSEO Homepage Category Page Subcategory Page Category Page Blog Blog Category Subcategory Page Level 1 Level 2 Level 3 Product Page Product Page Blog PostLevel 4
  35. 35. @rachellcostello brightonSEO
  36. 36. @rachellcostello brightonSEO 1. A strong history of organic traffic. 2. Consistent levels of customer engagement. + 3. Close monitoring of all changes made. + HAVE SUBSETS WITH:
  37. 37. Because when you see the needle move it will be more meaningful. @rachellcostello brightonSEO
  38. 38. @rachellcostello brightonSEO Supplement your segments in external data sources too. Use custom properties, inclusion rules and filtered reports to pre-filter data in: Analytics tools Log files Sitemaps Google Search Console
  39. 39. @rachellcostello brightonSEO
  40. 40. Tactical crawling allows you to get to know your website and its structure better. @rachellcostello brightonSEO
  41. 41. Establishing patterns means you can crawl more efficiently in future. @rachellcostello brightonSEO
  42. 42. But, remember to also run annual or semi-annual full crawls and tech SEO audits. @rachellcostello brightonSEO
  43. 43. The takeaways from these are the primary baselines for planning tactical crawls. @rachellcostello brightonSEO
  44. 44. HOW THE BIGGEST BRANDS MEASURE UP @rachellcostello brightonSEO
  45. 45. I analysed the most recent audits we’ve completed using tactical crawling methods for our biggest enterprise clients. Here are the key takeaways of 3... @rachellcostello brightonSEO
  46. 46. FOOD DELIVERY TRANSPORTATION ONLINE CLASSIFIEDS @rachellcostello brightonSEO
  47. 47. The key overlapping problem areas... @rachellcostello brightonSEO
  48. 48. 1 2 3 4 5 Mobile Internal Linking Site Speed Indexing Internationalisation @rachellcostello brightonSEO
  49. 49. MAIN INTERNATIONALISATION FIXES @rachellcostello brightonSEO
  50. 50. @rachellcostello brightonSEO
  51. 51. Removing URLs for nonsensical locales. @rachellcostello brightonSEO
  52. 52. Removing URLs for nonsensical locales. Use location to show relevant results. @rachellcostello brightonSEO
  53. 53. Removing URLs for nonsensical locales. Use location to show relevant results. Allow users to toggle between languages. @rachellcostello brightonSEO
  54. 54. Removing URLs for nonsensical locales. Use location to show relevant results. Allow users to toggle between languages. Add x-default for non- regionalised URLs. @rachellcostello brightonSEO
  55. 55. Removing URLs for nonsensical locales. Use location to show relevant results. Allow users to toggle between languages. Add x-default for non- regionalised URLs. Translate all titles and descriptions to the correct language. @rachellcostello brightonSEO
  56. 56. MAIN INDEXING FIXES @rachellcostello brightonSEO
  57. 57. @rachellcostello brightonSEO
  58. 58. Replace 302 redirects with 301 redirects. @rachellcostello brightonSEO
  59. 59. Replace 302 redirects with 301 redirects. Fix any server errors in Google Search Console ‘Crawl Errors’ report. @rachellcostello brightonSEO
  60. 60. Replace 302 redirects with 301 redirects. Fix any server errors in Google Search Console ‘Crawl Errors’ report. Ensure all content elements are unique and relevant. @rachellcostello brightonSEO
  61. 61. Replace 302 redirects with 301 redirects. Fix any server errors in Google Search Console ‘Crawl Errors’ report. Ensure all content elements are unique and relevant. @rachellcostello brightonSEO Increase pagination to show more results per page.
  62. 62. Replace 302 redirects with 301 redirects. Fix any server errors in Google Search Console ‘Crawl Errors’ report. Ensure all content elements are unique and relevant. Increase pagination to show more results per page. Review orphaned URLs in the sitemaps. @rachellcostello brightonSEO
  63. 63. MAIN SITE SPEED FIXES @rachellcostello brightonSEO
  64. 64. @rachellcostello brightonSEO
  65. 65. Lazy-load below-the-fold images. @rachellcostello brightonSEO
  66. 66. Lazy-load below-the-fold images. Eliminate use of unnecessary plugins. @rachellcostello brightonSEO
  67. 67. Lazy-load below-the-fold images. Eliminate use of unnecessary plugins. Remove unused and/or render-blocking scripts. @rachellcostello brightonSEO
  68. 68. Lazy-load below-the-fold images. Eliminate use of unnecessary plugins. Remove unused and/or render-blocking scripts. Reduce page HTML size for high priority templates. @rachellcostello brightonSEO
  69. 69. Lazy-load below-the-fold images. Eliminate use of unnecessary plugins. Remove unused and/or render-blocking scripts. Reduce page HTML size for high priority templates. Increase caching time for static pages from minutes to days. @rachellcostello brightonSEO
  70. 70. MAIN INTERNAL LINKING FIXES @rachellcostello brightonSEO
  71. 71. @rachellcostello brightonSEO
  72. 72. Remove redirect loops and chains. @rachellcostello brightonSEO
  73. 73. Remove redirect loops and chains. Reinstate links to pages with 10+ impressions and/or visits. @rachellcostello brightonSEO
  74. 74. Remove redirect loops and chains. Reinstate links to pages with 10+ impressions and/or visits. Remove internal links to nofollow pages. @rachellcostello brightonSEO
  75. 75. Remove redirect loops and chains. Reinstate links to pages with 10+ impressions and/or visits. Remove internal links to nofollow pages. Avoid using parameter URLs for tracking. @rachellcostello brightonSEO
  76. 76. Remove redirect loops and chains. Reinstate links to pages with 10+ impressions and/or visits. Remove internal links to nofollow pages. Avoid using parameter URLs for tracking. Remove internal links to canonicalised search pages. @rachellcostello brightonSEO
  77. 77. MAIN MOBILE FIXES @rachellcostello brightonSEO
  78. 78. @rachellcostello brightonSEO
  79. 79. Avoid using ‘maximum-scale =1’ in viewport settings. @rachellcostello brightonSEO
  80. 80. Avoid using ‘maximum-scale =1’ in viewport settings. Change ‘user-scalable= no’ to ‘yes’ in viewport settings. @rachellcostello brightonSEO
  81. 81. Avoid using ‘maximum-scale =1’ in viewport settings. Change ‘user-scalable= no’ to ‘yes’ in viewport settings. Migrate all pages to responsive ASAP. @rachellcostello brightonSEO
  82. 82. Avoid using ‘maximum-scale =1’ in viewport settings. Change ‘user-scalable= no’ to ‘yes’ in viewport settings. Migrate all pages to responsive ASAP. Ensure each mobile-desktop reciprocal URL pair has 200 status. @rachellcostello brightonSEO
  83. 83. Avoid using ‘maximum-scale =1’ in viewport settings. Change ‘user-scalable= no’ to ‘yes’ in viewport settings. Migrate all pages to responsive ASAP. Ensure each mobile-desktop reciprocal URL pair has 200 status. Fix content mismatches between mobile and desktop URLs. @rachellcostello brightonSEO
  84. 84. Luckily, there are already resources on these topics! @rachellcostello brightonSEO
  85. 85. @rachellcostello brightonSEO Internationalisation - bit.ly/internationalisation-dc Indexing - bit.ly/indexing-dc Site Speed - bit.ly/site-speed-dc Internal Linking - bit.ly/internal-linking-dc Mobile - bit.ly/mobile-dc RESOURCES
  86. 86. @rachellcostello brightonSEO TO WIN IN ENTERPRISE SEO: Run targeted, agile crawls on smaller sections of your site. Better understand internationalisation, indexing, site speed, internal linking and mobile. Use the findings to deliver quicker and more impactful insights into site health. 1 2 3
  87. 87. @rachellcostello brightonSEO THANK YOU! Any questions? Send me a tweet: @DeepCrawl www.deepcrawl.com

×