Deep-Hidden-Invisible Web

Deep-Hidden-Invisible Web Prepared by Prof.K.Prabhakar Assisted by P.Subha [email_address]

Background of the Invisible Web
" Invisible ” is used in the context of world wide web. It is not Invisible in the sense that it cannot be “ seen ”, the content is not available while we are searching using the most commonly used search engines such as Google or MSN or Yahoo. That is reason why many prefer using the word DEEP WEB rather than INVISIBLE WEB.

The size of Invisible web
The size is difficult to measure and no estimation is possible due to dynamic nature of creation of content and fast pace with which the search engines index WebPages.
However its size as well as content is too large to ignore.

Will the situation change?
Google said that it is dedicated to indexing the world's content, however long it takes. Also, more previously invisible pages are getting indexed because of manually-added links to them from visible pages.

Why “Invisibility”?
Invisible does not mean that it is inaccessible. It means that it is not indexed by a search engine and is invisible to the person who is searching the net. If you find no results in the search engine that does not mean that the content is not available.

Reasons for Invisibility
Dynamic URLs : If web pages have long string of parameters and equal signs and question marks, such that they get duplicated what is in their data base.
Form controlled Entry : Pages are displayed only when some actions are taken by human.
Hidden Pages :Hidden means there is simply no sequence of hyperlink clicks that could take you to such a page. The pages are accessible, but only people who know of their existence know how to view them.
New Pages: If the pages are new then they will not be indexed by any engine.

Reasons for Invisibility
Flash Presentations: Text content in Flash presentations is not indexed.
Geo-Tagged: Computers from certain regions may be blocked out. That may include blocking of search engines also. Many of the American TV broadcasters are showing TV online, however, they are not available for searching.
There may be other reasons for not invisibility.

Some Ways to Make Invisible Content Visible by site owner
There are ways to make deep web visible. We will study what a site owner could do and then what a searcher can do. Let us consider the what the site owner can do
Link it to a visible or indexed page . If some content is available to you , you may put it on a static HTML page, with relevant formatting and necessary hyperlinks, then link to this static page from an already "visible" (indexed) page.

Convert formats . For flash and other files transcribe them in to words and add as text.
For Audio :A udio content such as a pod cast may be transcribed and published as supplementary text.

Build links . Link to your own pages from other related pages. If you write about, say, trees on page A, then write about trees again on page B, link from page B to page A to give A more relevance. If page A hasn't been indexed, it will be after B is indexed. Points 6-9 are alternate ways to build links, hence helping make content visible.

Build a topic pyramid . This is a specialized form of sitemap that actually spans many pages. The apex (top-most) page has general topics and links to the next layer of pages, which have more specific topics and links to the next layer. The bottom-most layer of the topic pyramid are your original Web pages or blog posts, which have the most specific content. This method builds page relevance via the serial linking, which induces spiders to want to visit and index.

Socially bookmark it . If you find something, say a book at The Gutenberg Project, that you like, bookmark the URL at a social bookmaking site such as Del.icio.us with a brief description.
Remove access restrictions . Get rid of the need to login, or don't apply time-limits.

What user can do
Use a site's search engine . Some times the site search engine may provide better information.
Use site archive navigation . On web logs in particular, you can use the archive links to find info, albeit through manual searching.

What user can do
Using the word "database" in regular search engine query will find information that is difficult to find. For example, if you are looking for a database of images, you can type the search string images database into Google or one of the other engines. Somewhere down the results list in Google, you'll find Full-Text Database Images from the USPTO (US Patent and Trademark Office). You can then use the Quick or Advanced search forms to find patents relating to one or more terms. If there are images to be seen, there will be links to them.

What user can do
We can use an "invisible Web" directory, portal or specialized search engine such as Google Book Search , Google Scholar , Librarian's Internet Index , or BrightPlanet's Complete Planet (70,000 searchable databases and specialty search engines).

Invisible web search tools
Deep Web Search Engine — Clusty .
Art — Musie du Louvre .
Books Online — The Online Books Page .
Business — Explorit Now! .
Consumer — US Consumer Products Safety Commission Recalled Products .
Economic and Job Data — FreeLunc.com — A searchable directory of free economic data.
Finance and Investing — Bankrate.com .

Invisible web search tools
General Research — GPO's Catalog of US Government Publications .
Government Data — Copyright Records (LOCIS) .
International — International Data Base (IDB) .
Law and Politics — THOMAS (Library of Congress) .
Library of Congress — Library of Congress .
Medical and Health — PubMed .
Science — ScienceResearch.com .
Transportation — FAA Flight Delay Information .

Further research tools
About WebSearch — Christmas 2006 web search guide .
About Websearch — The deep web — find out more about the deep web — deep web search .
ALA — American Library Association .
BrightPlanet — FAQ .

Further Research
Deep Web Research — A gigantic list of resources.
Deep Web Technologies .
Ellipsis — Metadata, Google, and the Invisible Web .
Envisional .

Further Research
Google Librarian Center .
Google Library Project .
Lifehacker — How to search the invisible web .
MediaBistro — Some resources for freelancers .

Further Research
MetaQuerier — Exploring and integrating the deep web .
QProber — Classifying and searching hidden-web text databases .
The Invisible Web Weblog .
University of California, Berkeley — Invisible or deep web .

One of the most important site
Please go through http://oedb.org/library
This is an online education data base that will provide you information on various areas relating to career and education.

http://southwest.mpls.k12.mn.us	55
http://www.slideshare.net	13
http://web2097.blogspot.com	12
http://accessibility_checker.siteimprove.com	11
http://mjokhadze.blogspot.com	9
http://perso.gratisweb.com	9
http://philosophyrecycled.blogspot.com	6
http://www.health.medicbd.com	2
http://www.mjokhadze.blogspot.com	1
http://www.lmodules.com	1

Deep-Hidden-Invisible Web

by Prabhakar Krishnamurthy on Feb 10, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

10 Embeds 119

Statistics

Deep-Hidden-Invisible Web — Presentation Transcript