Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Site Crawling: What To Do & What To Look For

3,333 views

Published on

Kate Morris presented at State of Search 2014 on Web Site Crawling: What to Do and What to Look For. One of the top maintenance items that directly impacts search performance is doing a quarterly crawl of your site for issues like duplicate content, optimized text, internal linking, errors and overall structure. This session will give you an easy process to follow along with what to look for, what the data means, and action items after a crawl.

Published in: Internet
  • Be the first to comment

Site Crawling: What To Do & What To Look For

  1. 1. Site Crawling What To Do & What To Look For @katemorris
  2. 2. The “Oh Shit” Moment @katemorris
  3. 3. The “WTF” Moment @katemorris
  4. 4. WHEN IS THE LAST TIME YOU @katemorris CRAWLED YOUR SITE?
  5. 5. CRAWLING TOOLS CRAWLING DATA CRAWLING FUN CRAWLING TIMELINES @katemorris
  6. 6. CRAWLING @katemorris TOOLS
  7. 7. @katemorris Moz Best For: • In-House Marketers • Beginning SEOs Cost: $99/month+ User Score: 1-2
  8. 8. @katemorris Raven Best For: • In-House Marketers • Small Agencies Cost: $99/month+ User Score: 3
  9. 9. Xenu Link Sleuth @katemorris Best For: • Technical Experts • Developers Cost: Free User Score: 4
  10. 10. Screaming Frog @katemorris Best For: • Technical Marketers • SEOs • Developers Cost: $160/year User Score: 5
  11. 11. @katemorris Deep Crawl Best For: • Expert SEOs • Sites with Large Footprints Cost: $80/ month+ User Score: 6
  12. 12. Crawler User Scale 1 2 3 4 @katemorris Unrelated to SEO/Marketing Novice Level SEO Knowledge, Good SEO Knowledge, Non-Technical Good Technical, Minimal SEO 5 6 7 Good SEO and Technical Knowledge Expert Level SEO Knowledge, Minimal Technical Expert Technical and SEO Knowledge
  13. 13. Crawler User Scale @katemorris Unrelated to SEO/Marketing Novice Level SEO Knowledge, Good SEO Knowledge, Non-Technical Good Technical, Minimal SEO Good SEO and Technical Knowledge Expert Level SEO Knowledge, Minimal Technical BUILD YOUR OWN Expert Technical and SEO Knowledge
  14. 14. THE CRAW @katemorris L All Examples Use Screaming Frog or DeepCrawl
  15. 15. DeepCrawl Dashboard @katemorris
  16. 16. 4XX/5XX Errors @katemorris
  17. 17. 4XX/5XX Errors @katemorris
  18. 18. Internal Redirects @katemorris
  19. 19. External Redirects @katemorris
  20. 20. Duplicate Titles @katemorris
  21. 21. Duplicate Content @katemorris
  22. 22. @katemorris URL Issues
  23. 23. Meta Descriptions @katemorris
  24. 24. Meta Descriptions @katemorris DeepCrawl breaks it down for you.
  25. 25. @katemorris Canonicals
  26. 26. Image Alt Text @katemorris
  27. 27. @katemorris PDF Files
  28. 28. HOW OFTEN DO I HAVE TO @katemorris DO THIS?
  29. 29. Every Quarter @katemorris
  30. 30. Before and After All Major Site Updates WEBSITE UPDATE WEBSITE UPDATE WEBSITE UPDATE WEBSITE UPDATE WEBSITE UP @katemorris
  31. 31. Any Major Changes in Organic Traffic @katemorris
  32. 32. FUN CRAWLER @katemorris USES
  33. 33. Testing Sitemaps @katemorris
  34. 34. Testing Sitemaps @katemorris
  35. 35. Testing Backlinks Issue: Backlink Analysis is time consuming, especially checking backlink lists for live links. Solution: Make a text list of all the backlinks and crawl. For Screaming Frog, set a custom search for the client site. @katemorris
  36. 36. Testing Backlinks @katemorris
  37. 37. Testing Backlinks @katemorris
  38. 38. Content Inventory @katemorris
  39. 39. Confirm GA Implementation @katemorris
  40. 40. @katemorris Want more? http://www.seerinteractive.com/blog/screaming-frog-guide
  41. 41. @katemorris Do This Download and find your favorite crawler. Make a calendar entry that reoccurs every three months to remind you to crawl your site. At minimum, review: – Errors – Redirects – Duplicated Titles – XML Sitemaps
  42. 42. Thanks Icons provided by: Medialoot

×