Digging into the Deep Web


Published on

Strategies for searching the Deep Web in the context of Social Studies.

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Digging into the Deep Web

  1. 1. Dig into the Deep Web Going on a treasure hunt… Wendy DeGroat MW Library, November 2009 By cameronparkins
  2. 2. How do sources get buried? <ul><li>Requires log-on </li></ul><ul><li>Accessed by query </li></ul><ul><li>Protected from robots </li></ul><ul><li>Public access expired </li></ul><ul><li>Can’t be bookmarked (scripted, dynamic, etc.) </li></ul><ul><li>More than 3 clicks deep </li></ul>(He, et al., 2007)
  3. 3. How big is the deep web? <ul><li>Conflicting answers… </li></ul><ul><ul><li>91,000 terabytes (Deep Web, n.d.) </li></ul></ul><ul><ul><li>500x larger than surface (Sullivan as cited in He, et al., 2007) </li></ul></ul><ul><ul><li>63% (He, et al., 2007) - 95% of web </li></ul></ul>(Dash, 2007)
  4. 4. = 1/3
  5. 5. What’s in the treasure chest? <ul><li>Images and multimedia files, including the content of many online exhibits </li></ul><ul><li>Diaries, letters, historic maps </li></ul><ul><li>Patents, laws, bills, and treaties </li></ul><ul><li>Articles licensed/protected by copyright </li></ul><ul><li>“ Old” newspaper articles (esp. pre-1980’s) </li></ul><ul><li>Collections of papers, preprints (conference, etc.) </li></ul><ul><li>Declassified documents, Congressional testimony </li></ul><ul><li>Government reports, data sets </li></ul><ul><li>And much more… </li></ul>
  6. 6. Strategies for finding an X, and then digging for… By merfam
  7. 7. Why search for it? By brittanyg
  8. 8. Treasure Hunt <ul><li>Mapping </li></ul><ul><li>Planning </li></ul><ul><li>Selecting the proper tools </li></ul><ul><li>Digging </li></ul>
  9. 9. (Concept) mapping <ul><li>Browse for broader concepts </li></ul><ul><ul><li>Categories of web directories </li></ul></ul><ul><ul><li>Sections of archives </li></ul></ul><ul><ul><li>Subject headings in databases </li></ul></ul><ul><li>Note related/narrower concepts </li></ul><ul><li>Amend your map as you search </li></ul>
  10. 10. Preparing for the journey <ul><li>Reference book articles </li></ul><ul><ul><li>PowerSearch – Books tab </li></ul></ul><ul><ul><li>U.S. and World History RC’s – Reference tab </li></ul></ul><ul><ul><li>Biography, Science, and Literature RC’s </li></ul></ul><ul><ul><li>Google Book/Worldcat </li></ul></ul><ul><ul><li>NetLibrary (note: not all reference, some biased) </li></ul></ul>
  11. 11. Preparing for the journey <ul><li>Finding reference sources, overviews, etc. </li></ul><ul><ul><li>Recommended sources from Wikipedia/EB </li></ul></ul><ul><ul><li>Literature reviews </li></ul></ul><ul><ul><li>Reference lists </li></ul></ul><ul><ul><li>Further/recommended reading lists </li></ul></ul><ul><ul><li>Briefs/reports from Congressional Research Service, State Department, think tanks, etc. </li></ul></ul><ul><li>Finding compilations of sources </li></ul><ul><ul><li>Search subject + bibliography, research guide, resource guide, study guide, handbook… </li></ul></ul>
  12. 12. Google Book
  13. 13. Tools for the dig
  14. 14. Digging tips <ul><li>Gale </li></ul><ul><ul><li>Reference sources </li></ul></ul><ul><ul><ul><li>Search by broad concept </li></ul></ul></ul><ul><ul><ul><li>See Reference / Books result tabs </li></ul></ul></ul><ul><ul><ul><li>Refine by document type </li></ul></ul></ul><ul><ul><ul><li>Go beyond PowerSearch </li></ul></ul></ul><ul><ul><li>Follow leads within and between databases </li></ul></ul><ul><ul><li>Journal/mag/news articles & primary sources </li></ul></ul><ul><ul><ul><li>Start broad, then narrow </li></ul></ul></ul><ul><ul><ul><li>Use Advanced search to combine keywords and subject headings </li></ul></ul></ul>
  15. 15. Digging tips <ul><li>JSTOR and Project MUSE </li></ul><ul><ul><li>Use Advanced Search </li></ul></ul><ul><ul><li>In general, limit to articles </li></ul></ul><ul><ul><li>If looking for books, limit to reviews </li></ul></ul><ul><ul><li>JSTOR </li></ul></ul><ul><ul><ul><li>Refine by limiting by discipline and/or date range </li></ul></ul></ul><ul><ul><li>Project MUSE </li></ul></ul><ul><ul><ul><li>Refine by combining key word and subject searching </li></ul></ul></ul><ul><ul><ul><li>May also refine by limiting to All Except Text </li></ul></ul></ul>
  16. 16. Digging tips <ul><li>LexisNexis </li></ul><ul><ul><li>Supreme court decisions </li></ul></ul><ul><ul><li>Laws </li></ul></ul><ul><ul><li>News </li></ul></ul><ul><ul><ul><li>General </li></ul></ul></ul><ul><ul><ul><li>Regional </li></ul></ul></ul><ul><ul><ul><li>Foreign news sources </li></ul></ul></ul><ul><ul><ul><li>Wires </li></ul></ul></ul><ul><ul><ul><li>Transcripts </li></ul></ul></ul>
  17. 17. Tools for the dig <ul><li>VCU databases </li></ul><ul><ul><li>Browse by topic </li></ul></ul><ul><ul><li>Search by journal title or article citation </li></ul></ul><ul><ul><li>Follow to find in other databases </li></ul></ul><ul><ul><li>Not in a database? Check online catalog for copies in print, and on microfilm or microfiche </li></ul></ul><ul><li>VCU is also a government repository </li></ul><ul><li>VCU library’s resource guides </li></ul>
  18. 18. Tools for the dig <ul><li>Federated search tools </li></ul><ul><ul><li>e.g. Google Scholar, USAsearch.gov </li></ul></ul><ul><ul><li>Finding tools = portals in library’s bookmarks </li></ul></ul><ul><li>Directories </li></ul><ul><ul><li>e.g. Infomine, Intute </li></ul></ul><ul><ul><li>Finding tools = directories in library’s bookmarks </li></ul></ul><ul><li>Metasearch engines </li></ul><ul><ul><li>e.g. Clusty, Dogpile </li></ul></ul><ul><ul><li>Finding tools = metasearch in library’s bookmarks </li></ul></ul>
  19. 19. Tools for the dig <ul><li>Search for search tools </li></ul><ul><ul><li>Finding tools bundle in MW Library’s bookmarks = portals, directories, metasearch </li></ul></ul><ul><ul><li>Search subject + directory, portal , or search </li></ul></ul><ul><li>Social bookmarking sites </li></ul><ul><ul><li>del.icio.us </li></ul></ul><ul><ul><li>digg </li></ul></ul><ul><ul><li>stumbleupon </li></ul></ul>
  20. 20. Choose the right Google <ul><li>Google Book </li></ul><ul><li>Google Scholar </li></ul><ul><li>Google News & News Archive </li></ul><ul><ul><li>Save customizations </li></ul></ul><ul><ul><li>Specify news source location </li></ul></ul><ul><ul><li>Available for other countries </li></ul></ul><ul><ul><li>Set up news alerts </li></ul></ul><ul><li>Google by country </li></ul><ul><li>Google custom search engine </li></ul>By shawnbot
  21. 21. Surveying the landscape <ul><li>Who cares about this topic? </li></ul><ul><li>Where do these people live/gather? </li></ul><ul><ul><li>Research centers </li></ul></ul><ul><ul><li>Regulating/monitoring agencies </li></ul></ul><ul><ul><li>Professional/stakeholder organizations </li></ul></ul><ul><li>Where would their work be collected? </li></ul><ul><ul><li>Key publications, conferences, databases </li></ul></ul><ul><ul><li>Historical societies, museums, galleries </li></ul></ul>
  22. 22. Staying on track <ul><li>Stop periodically </li></ul><ul><ul><li>See where you are </li></ul></ul><ul><ul><li>Reflect on what you’ve found/learned </li></ul></ul><ul><ul><li>Decide where to look next </li></ul></ul><ul><li>Follow leads </li></ul><ul><li>Amend your (concept) map </li></ul><ul><li>Take notes (interacting vs. pasting) </li></ul><ul><li>Document your sources </li></ul>
  23. 23. By ninjaneil902
  24. 24. What questions do you have? <ul><li>MW Library web site </li></ul><ul><ul><li>http://mwlibrary.wordpress.com (or select Media Center on school’s home page) </li></ul></ul><ul><ul><li>Navigation menus at top and on left </li></ul></ul><ul><li>MW Library bookmark account </li></ul><ul><ul><li>http://del.icio.us/dragonlibrary/wilkes_koreanwar </li></ul></ul><ul><li>MW Library research wiki </li></ul><ul><ul><li>http://mwlibrary.wetpaint.com </li></ul></ul><ul><ul><li>See Dig into the Deep Web </li></ul></ul>
  25. 25. By brittanyg
  26. 26. References <ul><li>Dash, R. (2007, April 24). Exposing the invisible web to search engines. Search Engine Journal . Retrieved March 8, 2008, from http://www.searchenginejournal.com/exposing-the-invisible-web-to-search-engines/4771/ </li></ul><ul><li>Deep web. (n.d.). Wikipedia . Retrieved March 8, 2008, from http://en.wikipedia.org/wiki/Deep_web </li></ul><ul><li>He, B., Patel, M., Zhang, Z., & Chang, K. (2007). Accessing the deep web. Communications of the ACM 50 (5), 94-101. Retrieved March 8, 2008, from ACM Digital Library. </li></ul>