More Related Content
Similar to Roland Dunn - What DO Searchbots Actually Do? - ionSearch 2012
Similar to Roland Dunn - What DO Searchbots Actually Do? - ionSearch 2012 (20)
More from ionSearch Conference
More from ionSearch Conference (13)
Roland Dunn - What DO Searchbots Actually Do? - ionSearch 2012
- 5. Do you provide what your potential customers want?
Straightforward for humans and searchbots to find your content?
© 2007-2012 Roland Dunn
- 6. Do you stand out? Are you compelling, engaging, useful,
unique, the best?
Do you provide what your potential customers want?
Straightforward for humans and searchbots to find your content?
© 2007-2012 Roland Dunn
- 7. Do you stand out? Are you compelling, engaging, useful,
unique, the best?
Do you provide what your potential customers want?
Straightforward for humans and searchbots to find your content?
© 2007-2012 Roland Dunn
- 9. “Googlebot ... may be unable to completely index
all the content on your site”
© 2007-2012 Roland Dunn
- 23. Over 2 months, 40% of all Googlebot’s requests
were to …. just 2 URL’s (on-site search URLs)
© 2007-2012 Roland Dunn
- 24. Over 2 months, 40% of all Googlebot’s requests
were to …. just 2 URL’s (on-site search URLs)
On a website with approximately 40,000 URLs in
Google’s index (allegedly, according to site:…)
© 2007-2012 Roland Dunn
- 25. Over 2 months, 40% of all Googlebot’s requests
were to …. just 2 URL’s (on-site search URLs)
On a website with approximately 40,000 URLs in
Google’s index (allegedly, according to site:…)
On a website serving approx. 150-200K unique
natural search visits/month
© 2007-2012 Roland Dunn
- 26. Over 2 months, 40% of all Googlebot’s requests
were to …. 2 URL’s.
On a website with approximately 40,000 URLs in
Google’s index.
On a website serving approx. 150-200K unique
natural search visits/month
© 2007-2012 Roland Dunn
- 27. Over 2 months, 40% of all Googlebot’s requests
were to …. 2 URL’s.
On a website with approximately 40,000 URLs in
Google’s index.
On a website serving approx. 150-200K unique
natural search visits/month
© 2007-2012 Roland Dunn
- 31. Categorise URLs by “top level section” – perhaps more
useful than just URLs ...
© 2007-2012 Roland Dunn
- 32. Table of “top level sections” requested by Googlebot
© 2007-2012 Roland Dunn
- 33. Chart of “top level sections” requested by Gbot – 6 months
© 2007-2012 Roland Dunn
- 43. Googlebot can become distracted, obsessed, or even lost:
• On-site search
• Additive filters/faceted URLs e.g.
“shoes?size=3&colour=green&price=10&brand=smith”
• Sections with thin and/or very similar content
© 2007-2012 Roland Dunn
- 44. Googlebot may not spend its time efficiently
It needs help focusing on what we value
© 2007-2012 Roland Dunn
- 46. SMX West, March 2011, Matt Cutts:
(http://goo.gl/ZZz7E):
“... if Google determines a site isn’t as useful to
users, they may not crawl it as frequently"
© 2007-2012 Roland Dunn
- 50. Suggestions:
• Embrace logfiles – full of useful information
• Periodically check your searchbot behaviour
• Especially if the “Googlebot found a high number” message
keeps appearing
© 2007-2012 Roland Dunn
- 52. If searchbots appear distracted, unfocused, lost:
• Alter internal navigation & linking (e.g. flatten hierarchy)
© 2007-2012 Roland Dunn
- 53. If searchbots appear distracted, unfocused, lost:
• Alter internal navigation & linking (e.g. flatten hierarchy)
• Robots.txt out (blunt approach)
© 2007-2012 Roland Dunn
- 54. If searchbots appear distracted, unfocused, lost:
• Alter internal navigation & linking (e.g. flatten hierarchy)
• Robots.txt out (blunt approach)
• Alter URL construction (expensive!)
© 2007-2012 Roland Dunn
- 55. If searchbots appear distracted, unfocused, lost:
• Alter internal navigation & linking (e.g. flatten hierarchy)
• Robots.txt out (blunt approach)
• Alter URL construction (expensive!)
• Check “sitemaps” – all URLs? What priorities expressed?
© 2007-2012 Roland Dunn
- 63. Online Branding: Natural Search:
http://www.refinedpractice.com/ http://www.cloudshapes.co.uk/
T: @RefinedPractice T: @roland_dunn
Slides Available At:
http://www.cloudshapes.co.uk/talks/
© 2007-2012 Roland Dunn
