Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
From
DARPA to Shakespeare
and all the data we can handle
Big Data and Digital Humanities
February 2014
http://www.darpa.mi...
1. Big Data
2. Libraries & Librarians
3. University Researchers & Beyond
4. Digital Humanities
1. Big Data
High-Performance Computing (HPC) Act of 1991 (Public
Law 102-194)
as amended by the
Next Generation Internet Research Act ...
Big data...
is a mystery
is a child of the internet
Big Data has grown
from...
CPU's of information
Disks of information
....
Urban computing also aims to deeply understand the nature and sciences behind the phenomenon
occurring in urban spaces, us...
Examples of big data:
• Electronic Health Records
• Text vs tables
• Textual analytics TEI
• Sentiment analysis - FB posts...
Big Data & Science...
• Analyzing output from simulations
• Analyzing instrument output - LHC, Curiosity
• Creating DB's t...
http://www.ibmbigdatahub.com/infographic/four-vs-big-data
Experimental Science
Theoretical Science
Computational Science
Data Science - Big Data
4th Paradigm of Science
From bits to its...
Does the world consist of ...
matter, energy and information?
Newton - matter and motion
Steam engine ...
See presentation:
Philosophy & Big Data: Big Data, the Individual, and Society
by Melanie Swan
January 24, 2013
http://www...
2. Libraries & Librarians
3. University Researchers (YOU)
& Beyond
http://d2c2.lib.purdue.edu/publications
Purdue University
D. Scott Brandt and Jake Carlson
Michael Furlough
Associate Dean for Research and Scholarly Communications
Penn State University Libraries
Libraries roles ...
Librarians - new roles
Instruction - Best Practices
Data Information Literacy
Collaborate - DMP & more
Data Management Pla...
Conversion & Interoperability
Cultures of Practice
Databases & Data Formats
Data Curation & Reuse
Data Management & Organi...
Librarians - new roles
Instruction - Best Practices
Data Information Literacy
Collaborate - DMP & more
Data Management Pla...
Build on successes
MANTRA - Research Management Data Training
http://datalib.edina.ac.uk/mantra/
Data Management Course 20...
See Data Managment Modules
from University of Minnesota
Lisa Johnston
https://sites.google.com/a/umn.edu/data-
management-...
http://www.oucs.ox.ac.uk/oxgarage/
Librarians - new roles
Instruction - Best Practices
Data Information Literacy
Collaborate - DMP & more
Data Management Pla...
What do researchers care about?
Where can I put my stuff?
What is a data management plan?
Data needs to be...
• available
...
DO
DataNet from NSF
http://datafed.org/
Digital Preservation from the LoC
http://www.digitalpreservation.gov/
HathiTrust Digi...
Title:
State of Sustainability Practices among Minnesota Tourism Businesses, 2007-2013
Authors:
Qian, Xinyi (Lisa)
Schneid...
Title:
Public-Use Data from the Obstetrics and Periodontal
Therapy (OPT) Study, a randomized trial of periodontal
therapy ...
Title:
"Laundry Soap" from the Ojibwe Conversational Archives Project
Authors:
Hermes, Mary
Tainter, Rose
Kingbird-Porter,...
https://www.lib.umn.edu/datamanagement/archiving
https://www.lib.umn.edu/datamanagement/archiving
Librarians - new roles
Instruction - Best Practices
Data Information Literacy
Collaborate - DMP & more
Data Management Pla...
Research Data Services
University of Minnesota
https://www.lib.umn.edu/datamanagement/archiving
George Mason University
ht...
For all links please see:
http://guides.lib.cua.edu/hoffman
[tab] BigData
Keeping Research Data Safe
http://www.beagrie.co...
4. Digital Humanities WHY?
4. Digital Humanities
...Using data to tell our story
Data Visualization Catalog
http://blog.visual.ly/the-data-visualization-catalogue/
Visualization
http://www.edwardtufte.com/tufte/posters
http://www.masswerk.at/minard/
http://vannevar.blogspot.com/2009/03...
http://research.google.com/bigpicture/music/?utm_content=buffer662d6&utm_medium=social&utm_so
urce=twitter.com&utm_campaig...
http://www.ucl.ac.uk/infostudies/melissa-terras/DigitalHumanitiesInfographic.pdf
http://www.folgerdigitaltexts.org/
Geography of the London Ballad Trade 1500-
1700
http://ebba.english.ucsb.edu/balladprintersite/L
BP_main.html
World War I ...
Examples and Tools for DH projects
http://miriamposner.com/blog/how-did-they-make-that/#more-1571
ScrollKit
https://www.sc...
JISC media hub
http://jiscmediahub.ac.uk/
Examples of TEI:
American Memory (uses a TEI-conformant DTD)
http://memory.loc.gov/ammem/index.html
Early Canada Online
ht...
Metadata Standards (UNM)
http://libguides.unm.edu/content.php?pid=137795&sid=2556043
Data Formats
Types and formats of dat...
NEVER
DONE
• Data is information
• Libraries can be partners in providing value
- access and analytics
• Deep Collaboration - Federal...
Images:
Images:
http://www.darpa.mil/newsevents/releases/2012/03/29.aspx
http://www.darpa.mil/uploadedImages/Content/NewsE...
References
2012/03/29 DARPA calls for advances in big data to help the warfighter. (2012). Retrieved from
http://www.darpa...
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
From DARPA to Shakespeare: All the Data we Can Handle
Upcoming SlideShare
Loading in …5
×

From DARPA to Shakespeare: All the Data we Can Handle

890 views

Published on

Big Data and Digital Humanities overview presented to CUA LSC874 Digital Humanities Class February 2014.

Published in: Education, Technology
  • Be the first to comment

From DARPA to Shakespeare: All the Data we Can Handle

  1. 1. From DARPA to Shakespeare and all the data we can handle Big Data and Digital Humanities February 2014 http://www.darpa.mil/newsevents/releases/2012/03/29.aspx
  2. 2. 1. Big Data 2. Libraries & Librarians 3. University Researchers & Beyond 4. Digital Humanities
  3. 3. 1. Big Data
  4. 4. High-Performance Computing (HPC) Act of 1991 (Public Law 102-194) as amended by the Next Generation Internet Research Act of 1998 (Public Law 105-305) and America COMPETES Act of 2007 (Public Law 110-69). It’s the law! These laws authorize Federal agencies to set goals, prioritize their investments, and coordinate their activities in networking and information technology research and development. George O. Strawn NITRD Networking and Information Technology Research and Development (NITRD) Program From : Hot Topics in Big Data: What You Need to Know Now! FEDLINK, NFAIS, CENDI; December 11, 2012
  5. 5. Big data... is a mystery is a child of the internet Big Data has grown from... CPU's of information Disks of information ...to Networks of information Sensors everywhere George O. Strawn NITRD
  6. 6. Urban computing also aims to deeply understand the nature and sciences behind the phenomenon occurring in urban spaces, using a variety of heterogeneous data sources, such as traffic flows, human mobility, geographic and map data, environment, energy consumption, populations, and economics, etc. Recently, real-world data reflecting city dynamics becomes widely available, including, e.g., users’ mobile phone signal, GPS traces of vehicles and people, ticketing data in public transportation systems, user-generated content (like tweets, micro-blog, check-ins, photos), data from transportation sensor networks (camera and loop sensors) and environment sensor networks (temperature and air quality), as well as data from the Internet of Things. http://www.meetup.com/UrbanComputing/ Smart Cities
  7. 7. Examples of big data: • Electronic Health Records • Text vs tables • Textual analytics TEI • Sentiment analysis - FB posts, Twitter • Distributed data, distributed computing • Atmospheric sensors, undersea sensors • Hubble telescope • Library ERM
  8. 8. Big Data & Science... • Analyzing output from simulations • Analyzing instrument output - LHC, Curiosity • Creating DB's to support wide collaboration: Human Genome Project • Creating Knowledge Bases from textural information: Semantic Medline • Proteomics will be bigger than genomics How do you move 100TB of information within a University or a research area?
  9. 9. http://www.ibmbigdatahub.com/infographic/four-vs-big-data
  10. 10. Experimental Science Theoretical Science Computational Science Data Science - Big Data 4th Paradigm of Science
  11. 11. From bits to its... Does the world consist of ... matter, energy and information? Newton - matter and motion Steam engine - thermodynamics, matter, energy Computer - science of information, matter, energy and information Data intensive science is revolutionary science Big Data is TOO BIG To KNOW! The dust hasn't settled; dust is swirling all around us; it is FUN dust! George O. Strawn
  12. 12. See presentation: Philosophy & Big Data: Big Data, the Individual, and Society by Melanie Swan January 24, 2013 http://www.slideshare.net/lablogga/philosophy-and-big-data- big-data-the-individual-and-society
  13. 13. 2. Libraries & Librarians 3. University Researchers (YOU) & Beyond
  14. 14. http://d2c2.lib.purdue.edu/publications Purdue University D. Scott Brandt and Jake Carlson
  15. 15. Michael Furlough Associate Dean for Research and Scholarly Communications Penn State University Libraries Libraries roles and challenges: Libraries will have to operate on faith Libraries will need deep collaboration
  16. 16. Librarians - new roles Instruction - Best Practices Data Information Literacy Collaborate - DMP & more Data Management Plans Preserving/curating research DO Manage - RDS Services Keeping up!
  17. 17. Conversion & Interoperability Cultures of Practice Databases & Data Formats Data Curation & Reuse Data Management & Organization Data Processing & Analysis Data Quality & Documentation Discovery & Acquisition Ethics & Attribution Metadata & Data Description Preservation Visualization & Representation See more at: Data Information Literacy Competencies http://wiki.lib.purdue.edu/display/ste/Materials+for+the+DIL+Symposium Data is information
  18. 18. Librarians - new roles Instruction - Best Practices Data Information Literacy Collaborate - DMP & more Data Management Plans Preserving/curating research DO Manage - RDS Services Keeping up!
  19. 19. Build on successes MANTRA - Research Management Data Training http://datalib.edina.ac.uk/mantra/ Data Management Course 2014 - University 0f Minnesota https://sites.google.com/a/umn.edu/data-management-workshop-series/ Data Train http://archaeologydataservice.ac.uk/learning/DataTrain#section- DataTrain-AimsObjectives
  20. 20. See Data Managment Modules from University of Minnesota Lisa Johnston https://sites.google.com/a/umn.edu/data- management-workshop-series/module1
  21. 21. http://www.oucs.ox.ac.uk/oxgarage/
  22. 22. Librarians - new roles Instruction - Best Practices Data Information Literacy Collaborate - DMP & more Data Management Plans Preserving/curating research DO Manage - RDS Services Keeping up!
  23. 23. What do researchers care about? Where can I put my stuff? What is a data management plan? Data needs to be... • available • findable • re-usable • citable
  24. 24. DO
  25. 25. DataNet from NSF http://datafed.org/ Digital Preservation from the LoC http://www.digitalpreservation.gov/ HathiTrust Digital Library http://www.hathitrust.org/ Digital Preservation Network http://www.dpn.org/
  26. 26. Title: State of Sustainability Practices among Minnesota Tourism Businesses, 2007-2013 Authors: Qian, Xinyi (Lisa) Schneider, Ingrid E.
  27. 27. Title: Public-Use Data from the Obstetrics and Periodontal Therapy (OPT) Study, a randomized trial of periodontal therapy to prevent pre-term birth Authors: Hodges, James S. Michalowicz, Bryan S.
  28. 28. Title: "Laundry Soap" from the Ojibwe Conversational Archives Project Authors: Hermes, Mary Tainter, Rose Kingbird-Porter, Margaret
  29. 29. https://www.lib.umn.edu/datamanagement/archiving
  30. 30. https://www.lib.umn.edu/datamanagement/archiving
  31. 31. Librarians - new roles Instruction - Best Practices Data Information Literacy Collaborate - DMP & more Data Management Plans Preserving/curating research DO Manage - RDS Services Keeping up!
  32. 32. Research Data Services University of Minnesota https://www.lib.umn.edu/datamanagement/archiving George Mason University http://dataservices.gmu.edu/resources/data-management University of Maryland http://www.lib.umd.edu/data
  33. 33. For all links please see: http://guides.lib.cua.edu/hoffman [tab] BigData Keeping Research Data Safe http://www.beagrie.com/krds.php
  34. 34. 4. Digital Humanities WHY?
  35. 35. 4. Digital Humanities ...Using data to tell our story
  36. 36. Data Visualization Catalog http://blog.visual.ly/the-data-visualization-catalogue/
  37. 37. Visualization http://www.edwardtufte.com/tufte/posters http://www.masswerk.at/minard/ http://vannevar.blogspot.com/2009/03/minard-napolean-russia-1812-best-chart.html
  38. 38. http://research.google.com/bigpicture/music/?utm_content=buffer662d6&utm_medium=social&utm_so urce=twitter.com&utm_campaign=buffer#
  39. 39. http://www.ucl.ac.uk/infostudies/melissa-terras/DigitalHumanitiesInfographic.pdf
  40. 40. http://www.folgerdigitaltexts.org/
  41. 41. Geography of the London Ballad Trade 1500- 1700 http://ebba.english.ucsb.edu/balladprintersite/L BP_main.html World War I Document Archive http://www.gwpda.org/
  42. 42. Examples and Tools for DH projects http://miriamposner.com/blog/how-did-they-make-that/#more-1571 ScrollKit https://www.scrollkit.com/
  43. 43. JISC media hub http://jiscmediahub.ac.uk/
  44. 44. Examples of TEI: American Memory (uses a TEI-conformant DTD) http://memory.loc.gov/ammem/index.html Early Canada Online http://www.canadiana.org/ Victorian Women Writers Project http://www.indiana.edu/~letrs/vwwp/index.html Oxford Text Archive http://ota.ahds.ac.uk/
  45. 45. Metadata Standards (UNM) http://libguides.unm.edu/content.php?pid=137795&sid=2556043 Data Formats Types and formats of data HDF http://en.wikipedia.org/wiki/Hierarchical_Data_Format Common Data Format http://cdf.gsfc.nasa.gov/ [Also use of protocol buffers]
  46. 46. NEVER DONE
  47. 47. • Data is information • Libraries can be partners in providing value - access and analytics • Deep Collaboration - Federal, University, Business, Researchers/Industry, Future of Research • Data Policies • Renaissance of Archivists • Librarians as information consultants • Librarians as researchers
  48. 48. Images: Images: http://www.darpa.mil/newsevents/releases/2012/03/29.aspx http://www.darpa.mil/uploadedImages/Content/NewsEvents/Releases/2012/cyber_c.jpg http://www.ibm.com/smarterplanet/ie/en/smarter_cities/overview/index.html?re=CS1 http://upload.wikimedia.org/wikipedia/commons/4/4b/OSU_William_Oxley_Thompson_Memorial_Library_Stacks.JPG http://www.lib.ua.edu/wiki/sura/index.php/Data_Life_Cycle_Models
  49. 49. References 2012/03/29 DARPA calls for advances in big data to help the warfighter. (2012). Retrieved from http://www.darpa.mil/newsevents/releases/2012/03/29.aspx Boyle, D. E., Yates, D. C., & Yeatman, E. M. (2013). Urban sensor data streams: London 2013. Internet Computing, IEEE, 17(6), 12-20. doi:10.1109/MIC.2013.85 Domingo, A., Bellalta, B., Palacin, M., Oliver, M., & Almirall, E. (2013). Public open sensor data: Revolutionizing smart cities. Technology and Society Magazine, IEEE, 32(4), 50-56. doi:10.1109/MTS.2013.2286421 Gladney, H. M. (2012). Long-term digital preservation: A digital humanities topic? HISTORICAL SOCIAL RESEARCH-HISTORISCHE SOZIALFORSCHUNG, 37(3), 201-217. IBM smarter cities - overview - ireland. Retrieved from http://www.ibm.com/smarterplanet/ie/en/smarter_cities/overview/index.html?re=CS1 JADH 2013: ODDly pragmatic: Documenting encoding practices in digital humanities projects by james cummings on prezi. Retrieved from http://prezi.com/af2auinap-ug/jadh-2013-oddly-pragmatic-documenting-encoding-practices-in-digital-humanities-projects/ Lisa Johnston, Research Data Management and Curation Lead, & University Libraries University of Minnesota -‐ Twin Cities . (2014). A Workflow Model for Curating Research Data in the University of Minnesota Libraries: Report from the 2013 Data Curation Pilot . ().University Digital of Minnesota Conservancy. Michael Pepi. (2013). The postmodernity of big data – the new inquiry. Retrieved from http://thenewinquiry.com/essays/the-postmodernity-of- big-data/ Van den Eynden, V., Corti, L., Woollard, M., Bishop, L., & Horton, L. (2011). Managing and sharing data: Best practice for researchers

×