Dig into the Deep Web Going on a treasure hunt… Wendy DeGroat MW Library, November 2009 By cameronparkins

How do sources get buried? Requires log-on Accessed by query Protected from robots Public access expired Can’t be bookmarked (scripted, dynamic, etc.) More than 3 clicks deep (He, et al., 2007)

Requires log-on

Accessed by query

Protected from robots

Public access expired

Can’t be bookmarked (scripted, dynamic, etc.)

More than 3 clicks deep

How big is the deep web? Conflicting answers… 91,000 terabytes (Deep Web, n.d.) 500x larger than surface (Sullivan as cited in He, et al., 2007) 63% (He, et al., 2007) - 95% of web (Dash, 2007)

Conflicting answers…

91,000 terabytes (Deep Web, n.d.)

500x larger than surface (Sullivan as cited in He, et al., 2007)

63% (He, et al., 2007) - 95% of web

= 1/3

What’s in the treasure chest? Images and multimedia files, including the content of many online exhibits Diaries, letters, historic maps Patents, laws, bills, and treaties Articles licensed/protected by copyright “ Old” newspaper articles (esp. pre-1980’s) Collections of papers, preprints (conference, etc.) Declassified documents, Congressional testimony Government reports, data sets And much more…

Images and multimedia files, including the content of many online exhibits

Diaries, letters, historic maps

Patents, laws, bills, and treaties

Articles licensed/protected by copyright

“ Old” newspaper articles (esp. pre-1980’s)

Collections of papers, preprints (conference, etc.)

Declassified documents, Congressional testimony

Government reports, data sets

And much more…

Strategies for finding an X, and then digging for… By merfam

Why search for it? By brittanyg

Treasure Hunt Mapping Planning Selecting the proper tools Digging

Mapping

Planning

Selecting the proper tools

Digging

(Concept) mapping Browse for broader concepts Categories of web directories Sections of archives Subject headings in databases Note related/narrower concepts Amend your map as you search

Browse for broader concepts

Categories of web directories

Sections of archives

Subject headings in databases

Note related/narrower concepts

Amend your map as you search

Preparing for the journey Reference book articles PowerSearch – Books tab U.S. and World History RC’s – Reference tab Biography, Science, and Literature RC’s Google Book/Worldcat NetLibrary (note: not all reference, some biased)

Reference book articles

PowerSearch – Books tab

U.S. and World History RC’s – Reference tab

Biography, Science, and Literature RC’s

Google Book/Worldcat

NetLibrary (note: not all reference, some biased)

Preparing for the journey Finding reference sources, overviews, etc. Recommended sources from Wikipedia/EB Literature reviews Reference lists Further/recommended reading lists Briefs/reports from Congressional Research Service, State Department, think tanks, etc. Finding compilations of sources Search subject + bibliography, research guide, resource guide, study guide, handbook…

Finding reference sources, overviews, etc.

Recommended sources from Wikipedia/EB

Literature reviews

Reference lists

Further/recommended reading lists

Briefs/reports from Congressional Research Service, State Department, think tanks, etc.

Finding compilations of sources

Search subject + bibliography, research guide, resource guide, study guide, handbook…

Google Book

Tools for the dig

Digging tips Gale Reference sources Search by broad concept See Reference / Books result tabs Refine by document type Go beyond PowerSearch Follow leads within and between databases Journal/mag/news articles & primary sources Start broad, then narrow Use Advanced search to combine keywords and subject headings

Gale

Reference sources

Search by broad concept

See Reference / Books result tabs

Refine by document type

Go beyond PowerSearch

Follow leads within and between databases

Journal/mag/news articles & primary sources

Start broad, then narrow

Use Advanced search to combine keywords and subject headings

Digging tips JSTOR and Project MUSE Use Advanced Search In general, limit to articles If looking for books, limit to reviews JSTOR Refine by limiting by discipline and/or date range Project MUSE Refine by combining key word and subject searching May also refine by limiting to All Except Text

JSTOR and Project MUSE

Use Advanced Search

In general, limit to articles

If looking for books, limit to reviews

JSTOR

Refine by limiting by discipline and/or date range

Project MUSE

Refine by combining key word and subject searching

May also refine by limiting to All Except Text

Digging tips LexisNexis Supreme court decisions Laws News General Regional Foreign news sources Wires Transcripts

LexisNexis

Supreme court decisions

Laws

News

General

Regional

Foreign news sources

Wires

Transcripts

Tools for the dig VCU databases Browse by topic Search by journal title or article citation Follow to find in other databases Not in a database? Check online catalog for copies in print, and on microfilm or microfiche VCU is also a government repository VCU library’s resource guides

VCU databases

Browse by topic

Search by journal title or article citation

Follow to find in other databases

Not in a database? Check online catalog for copies in print, and on microfilm or microfiche

VCU is also a government repository

VCU library’s resource guides

Tools for the dig Federated search tools e.g. Google Scholar, USAsearch.gov Finding tools = portals in library’s bookmarks Directories e.g. Infomine, Intute Finding tools = directories in library’s bookmarks Metasearch engines e.g. Clusty, Dogpile Finding tools = metasearch in library’s bookmarks

Federated search tools

e.g. Google Scholar, USAsearch.gov

Finding tools = portals in library’s bookmarks

Directories

e.g. Infomine, Intute

Finding tools = directories in library’s bookmarks

Metasearch engines

e.g. Clusty, Dogpile

Finding tools = metasearch in library’s bookmarks

Tools for the dig Search for search tools Finding tools bundle in MW Library’s bookmarks = portals, directories, metasearch Search subject + directory, portal , or search Social bookmarking sites del.icio.us digg stumbleupon

Search for search tools

Finding tools bundle in MW Library’s bookmarks = portals, directories, metasearch

Search subject + directory, portal , or search

Social bookmarking sites

del.icio.us

digg

stumbleupon

Choose the right Google Google Book Google Scholar Google News & News Archive Save customizations Specify news source location Available for other countries Set up news alerts Google by country Google custom search engine By shawnbot

Google Book

Google Scholar

Google News & News Archive

Save customizations

Specify news source location

Available for other countries

Set up news alerts

Google by country

Google custom search engine

Surveying the landscape Who cares about this topic? Where do these people live/gather? Research centers Regulating/monitoring agencies Professional/stakeholder organizations Where would their work be collected? Key publications, conferences, databases Historical societies, museums, galleries

Who cares about this topic?

Where do these people live/gather?

Research centers

Regulating/monitoring agencies

Professional/stakeholder organizations

Where would their work be collected?

Key publications, conferences, databases

Historical societies, museums, galleries

Staying on track Stop periodically See where you are Reflect on what you’ve found/learned Decide where to look next Follow leads Amend your (concept) map Take notes (interacting vs. pasting) Document your sources

Stop periodically

See where you are

Reflect on what you’ve found/learned

Decide where to look next

Follow leads

Amend your (concept) map

Take notes (interacting vs. pasting)

Document your sources

By ninjaneil902

What questions do you have? MW Library web site http://mwlibrary.wordpress.com (or select Media Center on school’s home page) Navigation menus at top and on left MW Library bookmark account http://del.icio.us/dragonlibrary/wilkes_koreanwar MW Library research wiki http://mwlibrary.wetpaint.com See Dig into the Deep Web

MW Library web site

http://mwlibrary.wordpress.com (or select Media Center on school’s home page)

Navigation menus at top and on left

MW Library bookmark account

http://del.icio.us/dragonlibrary/wilkes_koreanwar

MW Library research wiki

http://mwlibrary.wetpaint.com

See Dig into the Deep Web

By brittanyg

References Dash, R. (2007, April 24). Exposing the invisible web to search engines. Search Engine Journal . Retrieved March 8, 2008, from http://www.searchenginejournal.com/exposing-the-invisible-web-to-search-engines/4771/ Deep web. (n.d.). Wikipedia . Retrieved March 8, 2008, from http://en.wikipedia.org/wiki/Deep_web He, B., Patel, M., Zhang, Z., & Chang, K. (2007). Accessing the deep web. Communications of the ACM 50 (5), 94-101. Retrieved March 8, 2008, from ACM Digital Library.

Dash, R. (2007, April 24). Exposing the invisible web to search engines. Search Engine Journal . Retrieved March 8, 2008, from http://www.searchenginejournal.com/exposing-the-invisible-web-to-search-engines/4771/

Deep web. (n.d.). Wikipedia . Retrieved March 8, 2008, from http://en.wikipedia.org/wiki/Deep_web

He, B., Patel, M., Zhang, Z., & Chang, K. (2007). Accessing the deep web. Communications of the ACM 50 (5), 94-101. Retrieved March 8, 2008, from ACM Digital Library.