Digging into the Deep Web

Dig into the Deep Web Going on a treasure hunt… Wendy DeGroat MW Library, November 2009 By cameronparkins

How do sources get buried?
Requires log-on
Accessed by query
Protected from robots
Public access expired
Can’t be bookmarked (scripted, dynamic, etc.)
More than 3 clicks deep
(He, et al., 2007)

How big is the deep web?
Conflicting answers…
91,000 terabytes (Deep Web, n.d.)
500x larger than surface (Sullivan as cited in He, et al., 2007)
63% (He, et al., 2007) - 95% of web
(Dash, 2007)

What’s in the treasure chest?
Images and multimedia files, including the content of many online exhibits
Diaries, letters, historic maps
Patents, laws, bills, and treaties
Articles licensed/protected by copyright
“ Old” newspaper articles (esp. pre-1980’s)
Collections of papers, preprints (conference, etc.)
Declassified documents, Congressional testimony
Government reports, data sets
And much more…

Strategies for finding an X, and then digging for… By merfam

Why search for it? By brittanyg

Treasure Hunt
Mapping
Planning
Selecting the proper tools
Digging

(Concept) mapping
Browse for broader concepts
Categories of web directories
Sections of archives
Subject headings in databases
Note related/narrower concepts
Amend your map as you search

Preparing for the journey
Reference book articles
PowerSearch – Books tab
U.S. and World History RC’s – Reference tab
Biography, Science, and Literature RC’s
Google Book/Worldcat
NetLibrary (note: not all reference, some biased)

Preparing for the journey
Finding reference sources, overviews, etc.
Recommended sources from Wikipedia/EB
Literature reviews
Reference lists
Further/recommended reading lists
Briefs/reports from Congressional Research Service, State Department, think tanks, etc.
Finding compilations of sources
Search subject + bibliography, research guide, resource guide, study guide, handbook…

Google Book

Tools for the dig

Digging tips
Gale
Reference sources
Search by broad concept
See Reference / Books result tabs
Refine by document type
Go beyond PowerSearch
Follow leads within and between databases
Journal/mag/news articles & primary sources
Start broad, then narrow
Use Advanced search to combine keywords and subject headings

Digging tips
JSTOR and Project MUSE
Use Advanced Search
In general, limit to articles
If looking for books, limit to reviews
JSTOR
Refine by limiting by discipline and/or date range
Project MUSE
Refine by combining key word and subject searching
May also refine by limiting to All Except Text

Digging tips
LexisNexis
Supreme court decisions
Laws
News
General
Regional
Foreign news sources
Wires
Transcripts

Tools for the dig
VCU databases
Browse by topic
Search by journal title or article citation
Follow to find in other databases
Not in a database? Check online catalog for copies in print, and on microfilm or microfiche
VCU is also a government repository
VCU library’s resource guides

Tools for the dig
Federated search tools
e.g. Google Scholar, USAsearch.gov
Finding tools = portals in library’s bookmarks
Directories
e.g. Infomine, Intute
Finding tools = directories in library’s bookmarks
Metasearch engines
e.g. Clusty, Dogpile
Finding tools = metasearch in library’s bookmarks

Tools for the dig
Search for search tools
Finding tools bundle in MW Library’s bookmarks = portals, directories, metasearch
Search subject + directory, portal , or search
Social bookmarking sites
del.icio.us
digg
stumbleupon

Choose the right Google
Google Book
Google Scholar
Google News & News Archive
Save customizations
Specify news source location
Available for other countries
Set up news alerts
Google by country
Google custom search engine
By shawnbot

Surveying the landscape
Who cares about this topic?
Where do these people live/gather?
Research centers
Regulating/monitoring agencies
Professional/stakeholder organizations
Where would their work be collected?
Key publications, conferences, databases
Historical societies, museums, galleries

Staying on track
Stop periodically
See where you are
Reflect on what you’ve found/learned
Decide where to look next
Follow leads
Amend your (concept) map
Take notes (interacting vs. pasting)
Document your sources

By ninjaneil902

What questions do you have?
MW Library web site
http://mwlibrary.wordpress.com (or select Media Center on school’s home page)
Navigation menus at top and on left
MW Library bookmark account
http://del.icio.us/dragonlibrary/wilkes_koreanwar
MW Library research wiki
http://mwlibrary.wetpaint.com
See Dig into the Deep Web

By brittanyg

References
Dash, R. (2007, April 24). Exposing the invisible web to search engines. Search Engine Journal . Retrieved March 8, 2008, from http://www.searchenginejournal.com/exposing-the-invisible-web-to-search-engines/4771/
Deep web. (n.d.). Wikipedia . Retrieved March 8, 2008, from http://en.wikipedia.org/wiki/Deep_web
He, B., Patel, M., Zhang, Z., & Chang, K. (2007). Accessing the deep web. Communications of the ACM 50 (5), 94-101. Retrieved March 8, 2008, from ACM Digital Library.

Digging into the Deep Web

by Wendy DeGroat on Mar 11, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 8

Statistics

Digging into the Deep Web — Presentation Transcript

http://www.slideshare.net	7
http://www.ccsu.edu	1