Blocking Search Engines from Indexing your SharePoint Site
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Blocking Search Engines from Indexing your SharePoint Site

on

  • 1,142 views

There are many occasions where you may want to exclude a website or portion of a site from search engine crawling and indexing.

There are many occasions where you may want to exclude a website or portion of a site from search engine crawling and indexing.

Statistics

Views

Total Views
1,142
Views on SlideShare
1,142
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Blocking Search Engines from Indexing your SharePoint Site Presentation Transcript

  • 1. BLOCKING SEARCH ENGINES FROM INDEXING YOUR SHAREPOINT SITE Ahmed Madany Senior SharePoint Consultant http://eg.linkedin.com/pub/ahmed-madany/35/80/2b6 http://ahmedmadany.wordpress.com/ https://twitter.com/ahmed_madany
  • 2. AGENDA  Introduction  What is Robots.txt ?  Location of Robots.txt in SharePoint Site.  Creating Robots File.  Integrate Robots.txt with SharePoint Site.
  • 3. INTRODUCTION  There are many occasions where you may want to exclude a website or portion of a site from search engine crawling and indexing.  There are several ways to prevent Google, Yahoo!, Bing or Ask from indexing a site’s pages.  robots.txt files to prevent sites from being indexed and thus showing up in the search engines.
  • 4. WHAT IS ROBOTS.TXT  Robots.txt is a text (not html) file placed in the root of your site to tell search robots which pages should and should not be visited/indexed.  It is not mandatory for search engines to adhere to the instructions found in the robots.txt but generally search engines obey what they are asked not to do
  • 5. WHAT IS ROBOTS.TXT  It is important to note that a robots.txt does not completely prevent search engines from crawling your site (i.e. it is not a firewall) and the fact that you may have a robots.txt file on your site is something like putting a note “Please, do not enter” on your unlocked front door. Put simply, it will not prevent thieves from coming in but the good guys will not open to door and enter.It goes without saying therefore, if you have sensitive data, you cannot rely 100% on a robots.txt to protect it from being indexed and displayed in search results.
  • 6. LOCATION OF ROBOTS.TXT IN SHAREPOINT SITE  The location of robots.txt is very important  It must be in the main directory because otherwise user agents (search engines) will not be able to find it.  Search engines look first in the main directory (i.e.http://www.sitename.com/robots.txt) and if they don’t find it there, they simply assume that this site does not have a robots.txt file
  • 7. CREATING ROBOTS FILE  Creating a Robots.txt  Launch Notepad  Put the following in your robots.txt file to refuse your entire site from indexing : User-agent: * Disallow: /  Save the file as: robots.txt Note : Make sure understand the robots.txt is case sensitive for the urls
  • 8. INTEGRATE ROBOTS.TXT WITH SHAREPOINT SITE  Adding a robots.txt file to the root of your SharePoint site.  Open up your root site in SharePoint Designer.  Double Click the folder All Files  Drag and drop the newly created robots.txt to the All Files folder.  Exit SharePoint Designer. Alternatively you can create the robots.txt from within SharePoint Designer itself.
  • 9. REFERENCES  http://ahmedmadany.wordpress.com/2012/0 3/15/how-to-block-search-engines-from- indexing-your-site/  http://blog.mastykarz.nl/generating-robots-txt- files-mavention-robots-txt/  http://blog.drisgill.com/2009/01/adding- robotstxt-to-sharepoint.html
  • 10. CONTACTS  If you have any questions plz Contact Me on Prog_madany@hotmail.com