Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
HOW TO AVOID SEO’S BIGGEST 
DISASTERS 
Mark Munroe 
SEORadar.com 
mark@seoradar.com 
@markemunroe 
#smx #23A 
October, 201...
SEO DISASTERS & REAL LIFE HORROR STORIES 
(AND HOW TO PREVENT THEM) 
Warning, this presentation is not for 
the faint of h...
The Laws 
• Murphy’s law “Whatever can go wrong, will go wrong” 
• Smith’s law “Murphy was an optimist!” 
• Munroe’s law o...
Summary of problems 
Problem 
Meta Robots noindex and/ 
or nofollow 
Noindex or nofollow on important conent 
Rel- canonic...
True life disaster number 1 
• Site accidentally places a no index on every page. 
• Traffic drops to 50% before fixed. 
•...
How could that possibly happen? 
• Think that can’t happen to you? Think again! 
• With conditional no-indexes proliferati...
Robots.txt 
• Yet another way to de-index your site or sections of your 
site 
#smx #23A @markemunroe 
User-agent: * 
Disa...
Here is a really creative way to de-index 
• On a major ecommerce site: 
• Somebody went into Webmaster Tools to 
delete a...
De-indexing - Identification and 
Prevention 
Symptoms 
• Sudden drop in traffic, over a short period of time as the noind...
True life disaster #2 – the end of a 
promising startup 
• This one happened in my early SEO career 
• A steady decline ap...
Site re-architecture required new URLs 
www.site.abc 301 www.site.xyz 
• 301s makes sure that anyone clicking on the old l...
The 301 Risk 
301 Cutover 
• In the early days, if the 301s are missing, they are easily identifiable. 
• Everyone is focu...
Identification and Prevention 
Symptoms 
• Broadly impacts all pages of the site as the entire site is weakened. 
• Some t...
Missing Links or Changed Links 
• Link structure of a large website is incredibly 
complex and SEO is very much dependent ...
True Life Disaster # 3 – 9 months of lost 
traffic 
The Situation 
• A company I joined had suffered a severe loss in traf...
True life disaster # 3 – 9 months of lost 
traffic 
The Investigation 
• I tried to identify changes that occurred during ...
Missing Link Path 
Symptoms 
• Gradual drop in traffic 
• Gradual drop in indexing 
Prevention 
• Check source for meta ro...
True Life Disaster # 4 – A little bit of 
knowledge is dangerous 
• At Trulia we acquired a site which had been on a long ...
Missing Links/nofollowed links 
Symptoms 
• Gradual drop in traffic 
• Gradual drop in indexing 
• Impacts both Google and...
True-life disaster # 5 
Warning – this one is going to be really, really disturbing. 
#smx #23A @markemunroe
True-life disaster # 5 
• Several years ago I was asked to help debug this ugly 
situation. 
• Clearly an algorithmic pena...
Evil Spammers 
• Spammers figured outs how to 
use the blog to create landing 
pages. 
• Created massive amount of 
pages ...
Spam penalty 
Symptoms 
• Dramatic drop in traffic 
• Inappropriate words showing up on your keyword list in Webmaster 
To...
Quick summary of additional 
disaster scenarios 
#smx #23A @markemunroe
Rel Canonicals – the Fiat of tags 
• Rel Canonicals – always breaking! 
• Disappearing 
• Broken Syntax 
• Pointing to nul...
Accidental/intentional page changes with 
SEO impact 
SEO impacting page elements: 
#smx #23A @markemunroe 
• H1, H2, H3 
...
Mobile Mishaps 
• M.Dot issues 
• Broken canonicals on m.dot pointing 
back to desktop. 
• Broken rel alternate 
• Serving...
Ajax and the incredible disappearing 
website 
• Links in navigation 
• Search Results 
• Content 
• Navigation 
• Disappe...
Disaster Avoidance Summary 
People 
• Training 
• Education 
• Testing – regularly testing SEO impacting changes 
Tools 
•...
Upcoming SlideShare
Loading in …5
×

How To Avoid SEO's Biggest Disasters By Mark Munroe

6,038 views

Published on

From the SMX East 2014 Conference in New York City, NY. SESSION: Conquering Today's Technical SEO Challenges. PRESENTATION: How To Avoid SEO's Biggest Disasters - Given by Mark Munroe, @MarkMunroe, SEO - SEO Radar. #SMX #23A

Published in: Marketing
  • Be the first to comment

How To Avoid SEO's Biggest Disasters By Mark Munroe

  1. 1. HOW TO AVOID SEO’S BIGGEST DISASTERS Mark Munroe SEORadar.com mark@seoradar.com @markemunroe #smx #23A October, 2014
  2. 2. SEO DISASTERS & REAL LIFE HORROR STORIES (AND HOW TO PREVENT THEM) Warning, this presentation is not for the faint of heart. Please be prepared for the following: • Sudden drastic losses in traffic • Disappearing revenue • Finger pointing • The complete destruction of a once exciting startup • Lost jobs • Zombies! #smx #23A @markemunroe
  3. 3. The Laws • Murphy’s law “Whatever can go wrong, will go wrong” • Smith’s law “Murphy was an optimist!” • Munroe’s law of SEO - “Anything that can break, will break. And things that cannot possibly break, will also break!!!” #smx #23A @markemunroe
  4. 4. Summary of problems Problem Meta Robots noindex and/ or nofollow Noindex or nofollow on important conent Rel- canonicals Pointing to the wrong place, syntax, improper use Disappearing 301s Losing the value of historic links Disappearing on-site links Content drops from the index Titles, H1, content changes Removing important keywords Spammers Spam content, spam links Crawl issues Crawler spending time in the wrong paces (faceted search). Nofollow links Nofollows to important content. 302s 301s magically turning into 302s Robots.txt Blocking content Unintentional cloaking Engineers trying to be too smart #smx #23A @markemunroe
  5. 5. True life disaster number 1 • Site accidentally places a no index on every page. • Traffic drops to 50% before fixed. • Upon fixed, traffic never goes back to pre-problem level and site loses 20% of traffic and revenue. • Potential exit for the site owner evaporates #smx #23A @markemunroe
  6. 6. How could that possibly happen? • Think that can’t happen to you? Think again! • With conditional no-indexes proliferating now through a site, it’s easy for the logic to get swapped. • Templates with common code get moved, swapped. • Logic that is intended for beta, white label and other duplicated pages propagates to the live site. • Example: “If BETA == True then insert_noindex” If a bug turns the variable BETA to “True” or if somehow ‘==‘ turns into “!==“ then the site gets noindexed. #smx #23A @markemunroe
  7. 7. Robots.txt • Yet another way to de-index your site or sections of your site #smx #23A @markemunroe User-agent: * Disallow: / or User-agent: * Disallow: /blogs
  8. 8. Here is a really creative way to de-index • On a major ecommerce site: • Somebody went into Webmaster Tools to delete a URL • They intended to delete a single URL • Instead deleted a folder #smx #23A @markemunroe
  9. 9. De-indexing - Identification and Prevention Symptoms • Sudden drop in traffic, over a short period of time as the noindex propagates. • Impacts both Google and Bing, but not necessarily at exactly the same time. • Even direct traffic may be be impacted as sometimes SEO bleeds into Direct (misclassification). • Drop in indexation Prevention • Check HTML source for meta robots noindex, nofollow tag whenever pages are updated. • Make sure to test all unique templates • Verify robots.txt • Create automated test scripts • Use 3rd party audit software to help identify the problems • If only Google or Bing impacted, look into webmaster tools for possible clues, specifically removing folders. #smx #23A @markemunroe
  10. 10. True life disaster #2 – the end of a promising startup • This one happened in my early SEO career • A steady decline appeared and the site lost ~around 40% of its traffic over a 6 week period. • This came at a time that we needed more funding. • With cash running out and in decline, site was sold for a song. Eventually, resold to Groupon as part of a bigger sale, who then shut it down. • At its peak, it had the most overlapping keywords with TripAdvisor according to SEM Rush. • How did this happen? #smx #23A @markemunroe
  11. 11. Site re-architecture required new URLs www.site.abc 301 www.site.xyz • 301s makes sure that anyone clicking on the old link still get to the new page. • 301s pass juice from the historic links that are distributed around the Internet. • So what’s the hairy, dark, creepy, insidious, monster hiding here?
  12. 12. The 301 Risk 301 Cutover • In the early days, if the 301s are missing, they are easily identifiable. • Everyone is focused on testing the redirects. • If the 301s don’t work, it will be clear from search engine traffic. What about 1, 2, 3, 6 months later? • The truly insidious risk is that weeks or many months later, a simple server config can break 301s. • Nobody is watching anymore. • Site functions fine. • No problem reaching the site from search because old URLs are de-indexed. • Traffic can severely suffer because all the great historic links have been severed. #smx #23A @markemunroe
  13. 13. Identification and Prevention Symptoms • Broadly impacts all pages of the site as the entire site is weakened. • Some top pages which had lots of links coming in may demonstrate more significant drop. • Increase in 404s in Webmaster Tools to old URLs (may lag the actual problem by quite a bit). Prevention • Create scripts or use tools that verify the old URLs still redirect on a regular basis. • Tech education on the importance that the redirects persist over time. • Periodic manual testing of the redirects. • Periodic review of the 404 errors in Webmaster Tools to see if the old, unsupported URLs show up. #smx #23A @markemunroe
  14. 14. Missing Links or Changed Links • Link structure of a large website is incredibly complex and SEO is very much dependent on the linking structure. • UX re-design and product changes can remove links critical for SEO • Or bugs can cause links to disappear from paged lists and/or HTML sitemaps. • On a site with millions of pages, it’s not easy to keep track. #smx #23A @markemunroe
  15. 15. True Life Disaster # 3 – 9 months of lost traffic The Situation • A company I joined had suffered a severe loss in traffic about 6 months earlier. • They had concluded that the problem was related to Bing hitting the servers too hard and slowing down Google’s crawl which resulted in increased time to crawl a page (hurting indexing). • My analysis found something very different. #smx #23A @markemunroe
  16. 16. True life disaster # 3 – 9 months of lost traffic The Investigation • I tried to identify changes that occurred during the time in question. This was extremely difficult and release notes were poor. • I finally isolated the loss to a particular page type (long tail pages, a million plus). • I found only about 30% had a click-path. • We then were to identify the specific bug that caused the problem. • Within 30 days of the fix, we started to regain traffic. #smx #23A @markemunroe
  17. 17. Missing Link Path Symptoms • Gradual drop in traffic • Gradual drop in indexing Prevention • Check source for meta robots noindex, nofollow tag whenever pages are updated. • Make sure to test all unique templates • Create automated test scripts • Use 3rd party software to help identify the problems • If only Google or Bing impacted, look into webmaster tools for possible clues, specifically removing folders. #smx #23A @markemunroe
  18. 18. True Life Disaster # 4 – A little bit of knowledge is dangerous • At Trulia we acquired a site which had been on a long downward trend. • On investigation, an engineer was trying to make page 1 rank better on a paged list and didn’t want the other pages to compete. So they nofollowed the paged links. • Other links were removed to make it a ‘cleaner’ site. • Oops - 6 months of traffic down 50% (and they blamed Panda). All the paged links were nofollowed. #smx #23A @markemunroe
  19. 19. Missing Links/nofollowed links Symptoms • Gradual drop in traffic • Gradual drop in indexing • Impacts both Google and Bing (not necessarily in step) • Only impacts a subset of content Prevention • Education so that everyone touching product knows the importance of on-site links. • Make sure to test all unique templates with a Chrome plugin that highlights nofollow. • Create automated test scripts or use 3rd party software that alerts user to disappearing persistent links and nofollow links. #smx #23A @markemunroe
  20. 20. True-life disaster # 5 Warning – this one is going to be really, really disturbing. #smx #23A @markemunroe
  21. 21. True-life disaster # 5 • Several years ago I was asked to help debug this ugly situation. • Clearly an algorithmic penalty which caused the site to bounce back and forth out of the index. #smx #23A @markemunroe
  22. 22. Evil Spammers • Spammers figured outs how to use the blog to create landing pages. • Created massive amount of pages focused on online drug sales (viagra, cialis and others). • Generated massive amounts of links to these landing pages with rich anchor text. • Cleanup and recovery involved manually removing thousands of pages with Webmaster Tools. #smx #23A @markemunroe
  23. 23. Spam penalty Symptoms • Dramatic drop in traffic • Inappropriate words showing up on your keyword list in Webmaster Tools. • UGC spam landing pages showing up your top pages report. • Good news today is that you may get a UGC spam warning from Google. Prevention • Build and train Bayesian filters to stop spam from getting posted. • Moderation to identify and delete spam. • Regularly examine keywords and landing pages in Webmaster Tools. • Periodically search site for likely spam words with site command. #smx #23A @markemunroe
  24. 24. Quick summary of additional disaster scenarios #smx #23A @markemunroe
  25. 25. Rel Canonicals – the Fiat of tags • Rel Canonicals – always breaking! • Disappearing • Broken Syntax • Pointing to null • Pointing to the home page • Pointing to broken URLs • Pointing the sites product cancellation page (yes I saw that) • Cloaked rel canonicals – (really, I couldn’t make that up!) Potentially very, very destructive - allowing all sorts of duplicate content to get indexed. We need scripts, people and tools testing them to make sure they don’t break. #smx #23A @markemunroe
  26. 26. Accidental/intentional page changes with SEO impact SEO impacting page elements: #smx #23A @markemunroe • H1, H2, H3 • Titles • Alt Text • Anchor text • On page text Prevention of issues: • No Surprises • Training and education throughout the organization • Automated Scripts, audit and test tools to identify problems • Regular post release audit
  27. 27. Mobile Mishaps • M.Dot issues • Broken canonicals on m.dot pointing back to desktop. • Broken rel alternate • Serving incorrect version to mobile spiders • Regularly view pages and check source. • Canonicals on m.dot is particularly important. #smx #23A @markemunroe
  28. 28. Ajax and the incredible disappearing website • Links in navigation • Search Results • Content • Navigation • Disappearing Pages Ajax can work but must be implemented in a way that crawlers can find the content. Ultimately it can results all the content and link related problems I’ve already discussed. #smx #23A @markemunroe
  29. 29. Disaster Avoidance Summary People • Training • Education • Testing – regularly testing SEO impacting changes Tools • Custom Scripts • SEORadar – designed specifically to identify and alert users for potential SEO disasters (change analysis). Disclaimer: my company. • Audit Tools – ScreamingFrog, Raven, Moz, BrightEdge, SearchMetrics, Conductor #smx #23A @markemunroe

×