Spj110509

517 views

Published on

Mining the Web presentation on effective deep-Web searching for journalists by Alex Johnson of msnbc.com.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
517
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Spj110509

  1. 1. Mining the Web Alex Johnson msnbc.com http://malexjohnson.com @malexjohnson May 9, 2011 Downloadable at: http://malexj.tk/kY
  2. 2. Hidden Google
  3. 3. Hidden Google <ul><li>Advanced Search </li></ul><ul><li>http://www.google.com/advanced_search </li></ul>
  4. 5. Hidden Google <ul><li>Advanced Search </li></ul><ul><li>http://www.google.com/advanced_search </li></ul><ul><li>Realtime Search </li></ul><ul><li>http://www.google.com/realtime </li></ul>
  5. 7. Hidden Google <ul><li>Advanced Search </li></ul><ul><li>http://www.google.com/advanced_search </li></ul><ul><li>Realtime Search </li></ul><ul><li>http://www.google.com/realtime </li></ul><ul><li>Book search </li></ul><ul><li>http://books.google.com </li></ul>
  6. 9. Hidden Google <ul><li>Advanced Search </li></ul><ul><li>http://www.google.com/advanced_search </li></ul><ul><li>Realtime Search </li></ul><ul><li>http://www.google.com/realtime </li></ul><ul><li>Book search </li></ul><ul><li>http://books.google.com </li></ul><ul><li>Academic Search </li></ul><ul><li>http://scholar.google.com </li></ul>
  7. 11. Building Better Searches
  8. 12. Building Better Searches <ul><li>Expand or restrict searches </li></ul>
  9. 13. Building Better Searches <ul><li>Expand or restrict searches </li></ul><ul><li>+example </li></ul><ul><li>-example </li></ul>
  10. 14. Building Better Searches <ul><li>Expand or restrict searches </li></ul><ul><li>+example </li></ul><ul><li>-example </li></ul><ul><li>“ exact phrase” </li></ul>
  11. 15. Building Better Searches <ul><li>Expand or restrict searches </li></ul><ul><li>+example </li></ul><ul><li>-example </li></ul><ul><li>“ exact phrase” </li></ul><ul><li>site: </li></ul>
  12. 16. Building Better Searches <ul><li>Expand or restrict searches </li></ul><ul><li>+example </li></ul><ul><li>-example </li></ul><ul><li>“ exact phrase” </li></ul><ul><li>site: </li></ul><ul><li>combine these switches </li></ul>
  13. 24. Building Better Searches <ul><li>Expand or restrict searches </li></ul><ul><li>+example </li></ul><ul><li>-example </li></ul><ul><li>“ exact phrase” </li></ul><ul><li>site: </li></ul><ul><li>combine these switches </li></ul><ul><li>cache: </li></ul>
  14. 27. Building Better Searches <ul><li>Expand or restrict searches </li></ul><ul><li>+example </li></ul><ul><li>-example </li></ul><ul><li>“ exact phrase” </li></ul><ul><li>site: </li></ul><ul><li>combine these switches </li></ul><ul><li>cache:example.com/example </li></ul><ul><li>Advance operators guide: </li></ul><ul><li>http://www.googleguide.com/advanced_operators.html </li></ul>
  15. 29. The government is your friend
  16. 30. The government is your friend <ul><li>FactFinder </li></ul><ul><li>http://factfinder2.census.gov </li></ul>
  17. 32. The government is your friend <ul><li>FactFinder </li></ul><ul><li>http://factfinder2.census.gov </li></ul><ul><li>State & County QuickFacts </li></ul><ul><li>http://quickfacts.census.gov </li></ul>
  18. 35. The government is your friend <ul><li>FactFinder </li></ul><ul><li>http://factfinder2.census.gov </li></ul><ul><li>State & County QuickFacts </li></ul><ul><li>http://quickfacts.census.gov </li></ul><ul><li>FedStats </li></ul><ul><li>http://www.fedstats.gov </li></ul>
  19. 38. The government is your friend <ul><li>FactFinder </li></ul><ul><li>http://factfinder2.census.gov </li></ul><ul><li>State & County QuickFacts </li></ul><ul><li>http://quickfacts.census.gov/ </li></ul><ul><li>FedStats </li></ul><ul><li>http://www.fedstats.gov </li></ul><ul><li>GovSpot </li></ul><ul><li>http://govspot.com </li></ul>
  20. 40. The government is your friend <ul><li>FactFinder </li></ul><ul><li>http://factfinder.census.gov </li></ul><ul><li>State & County QuickFacts </li></ul><ul><li>http://quickfacts.census.gov </li></ul><ul><li>FedStats </li></ul><ul><li>http://www.fedstats.gov </li></ul><ul><li>GovSpot </li></ul><ul><li>http://govspot.com </li></ul><ul><li>Government Printing Office </li></ul><ul><li>http://www.gpoaccess.gov/multidb.html </li></ul><ul><li>Transactional Records (TRAC) </li></ul><ul><li>http://trac.syr.edu </li></ul><ul><li>FBI, Uniform Crime Reports </li></ul><ul><li>http://www.fbi.gov/ucr/ucr.htm </li></ul><ul><li>Criminal Justice Statistics </li></ul><ul><li>http://www.albany.edu/sourcebook </li></ul>
  21. 41. When the government stiffs you
  22. 42. When the government stiffs you <ul><li>FOIA letter generator </li></ul><ul><li>http://www.splc.org/foiletter.asp </li></ul>
  23. 44. Where to find other stuff
  24. 45. Where to find other stuff <ul><li>Businesses, nonprofits </li></ul><ul><li>GuideStar </li></ul><ul><li>http://www2.guidestar.org/AdvancedSearch.aspx </li></ul><ul><li>SEC Filings </li></ul><ul><li>http://www.sec.gov/edgar.shtml </li></ul>
  25. 46. Where to find other stuff <ul><li>Businesses, nonprofits </li></ul><ul><li>GuideStar </li></ul><ul><li>http://www2.guidestar.org/AdvancedSearch.aspx </li></ul><ul><li>SEC Filings </li></ul><ul><li>http://www.sec.gov/edgar.shtml </li></ul><ul><li>Politics </li></ul><ul><li>OpenSecrets.org </li></ul><ul><li>http://www.opensecrets.org </li></ul><ul><li>PollingReport </li></ul><ul><li>http://www.pollingreport.com </li></ul>
  26. 47. Where to find other stuff <ul><li>Businesses, nonprofits </li></ul><ul><li>GuideStar </li></ul><ul><li>http://www.guidestar.org/ </li></ul><ul><li>SEC Filings </li></ul><ul><li>http://www.sec.gov/edgar.shtml </li></ul><ul><li>Politics </li></ul><ul><li>OpenSecrets.org </li></ul><ul><li>http://www.opensecrets.org/ </li></ul><ul><li>PollingReport </li></ul><ul><li>http://www.pollingreport.com/ </li></ul><ul><li>People </li></ul><ul><li>PeopleSpot </li></ul><ul><li>http://www.peoplespot.com </li></ul><ul><li>Public Records </li></ul><ul><li>http://www.searchsystems.net </li></ul><ul><li>http://www.vitalrec.com </li></ul><ul><li>http://www.pipl.com </li></ul>
  27. 48. Starting points
  28. 49. Starting points <ul><li>Journalism specific </li></ul><ul><li>IRE Beat Resource Guide </li></ul><ul><li>http://www.ire.org/resourcecenter/initial-search-beat.html </li></ul>
  29. 51. Starting points <ul><li>Journalism specific </li></ul><ul><li>IRE Beat Resource Guide </li></ul><ul><li>http://www.ire.org/resourcecenter/initial-search-beat.html </li></ul><ul><li>SPJ Journalist’s Toolbox </li></ul><ul><li>http://www.journaliststoolbox.org </li></ul><ul><li>• Journalists’ Quick Reference Page </li></ul><ul><li>http://malexj.tk/kB </li></ul>
  30. 53. Starting points <ul><li>Journalism specific </li></ul><ul><li>IRE Beat Resource Guide </li></ul><ul><li>http://www.ire.org/resourcecenter/initial-search-beat.html </li></ul><ul><li>Assignment Editor </li></ul><ul><li>http://www.assignmenteditor.com </li></ul><ul><li>SPJ Journalist’s Toolbox </li></ul><ul><li>http://www.journaliststoolbox.org </li></ul><ul><li>• Journalists’ Quick Reference </li></ul><ul><li>http://malexj.tk/kB </li></ul><ul><li>General portals </li></ul><ul><li>• LibrarySpot </li></ul><ul><li>http://www.libraryspot.com </li></ul><ul><li>The Reference Desk </li></ul><ul><li>http://www.martindalecenter.com </li></ul><ul><li>• Scholarly Resource Collections (UC-Riverside) </li></ul><ul><li>http://infomine.ucr.edu </li></ul><ul><li>Profnet </li></ul><ul><li>http://www.profnet.com </li></ul><ul><li>Help A Reporter Out </li></ul><ul><li>http://www.helpareporter.com </li></ul>
  31. 54. The Hidden Web <ul><li>Much of the information on the Web is hidden from search engines. How do you get hold of it? </li></ul>
  32. 55. The Hidden Web <ul><li>Much of the information on the Web is hidden from search engines. How do you get hold of it? </li></ul><ul><li>What we’re talking about is databases. </li></ul>
  33. 56. The Hidden Web <ul><li>Much of the information on the Web is hidden from search engines. How do you get hold of it? </li></ul><ul><li>What we’re talking about is databases. </li></ul><ul><li>National Climatic Data Center (NOAA) </li></ul><ul><li>204 million (380 terabytes as of April 2011, and growing at 80 TB a year) </li></ul>
  34. 57. The Hidden Web <ul><li>Google </li></ul><ul><li>Add database to your query </li></ul>
  35. 60. The Hidden Web <ul><li>Google </li></ul><ul><li>Add database to your query </li></ul><ul><li>Add filetype:xls to your query </li></ul>
  36. 62. The Hidden Web <ul><li>Google </li></ul><ul><li>Add database to your query </li></ul><ul><li>Add xls to your query </li></ul><ul><li>• New York Public Library Databases </li></ul><ul><li>http://www.nypl.org/collections/articles-databases </li></ul>
  37. 64. The Hidden Web <ul><li>• Google </li></ul><ul><li>Add database to your query </li></ul><ul><li>• New York Public Library Databases </li></ul><ul><li>http://www.nypl.org/collections/articles-databases </li></ul><ul><li>• IncyWincy </li></ul><ul><li>http://www.incywincy.com/search-engine/web/web?adv=1 </li></ul>
  38. 66. The Hidden Web <ul><li>• Google </li></ul><ul><li>Add database to your query </li></ul><ul><li>• New York Public Library Databases </li></ul><ul><li>http://www.nypl.org/collections/articles-databases </li></ul><ul><li>• IncyWincy </li></ul><ul><li>http://www.incywincy.com </li></ul><ul><li>• Europe — Europa </li></ul><ul><li>http://europa.eu/documentation/index_en.htm </li></ul>
  39. 68. This presentation can be downloaded at: http://malexj.tk/kY

×