Large amounts of scientific data remain uncurated, especially small datasets, which are currently invisible or "dark data". This dark data should be curated locally with the involvement of non-scientists at long-lived institutions like libraries and museums that have experience managing scholarly information over time. New roles and skills are needed for data scientists, digital curators, and biological information specialists to help address this problem by developing the necessary cyberinfrastructure, data standards, and educational programs to make more scientific dark data accessible.

1. P. Bryan Heidorn University of Arizona and JRS Biodiversity Foundation 2011 Scripting Life: the science behind ViBRANT Paris, France 20-21 January 2011 The Path to Enlightened Solutions for Biodiversity's Dark Data

2. University of Arizona Today: 25°C Sunny

3. Thesis Large amounts of data remain uncurated Most of that data is from small data sets and is currently largely invisible – Dark Data This data should be curated locally but not by scientists alone Need for long-lived institutions

4. Cyberinfrastructure Vision “ The anticipated growth in both the production and repurposing of digital data raises complex issues not only of scale and heterogeneity, but also of stewardship, curation and long-term access . ” NSF Cyberinfrastructure Vision for 21st Century Discovery, Chapter 3

5. Recognition of need for data curation “ Recommendation 6 : The NSF, working in partnership with collection managers and the community at large, should act to develop and mature the career path for data scientists and to ensure that the research enterprise includes a sufficient number of high-quality data scientists.” Long-Lived Digital Data Collections: Enabling Research and Education in the 21 st Century, Recommendations

6. Recognition of the importance of Information Recognition of the need for education New work roles within traditional institutions Interagency Working Group on Digital Data

7. Why Libraries and Museums Long history of scholarly data management Skills overlap such a development of metadata standards, ontologies, controlled vocabularies, thesauri Long-lived institutions Existing overlap with museums and archives

8. The problem Recognition of the problem Information is not in accessible format Computer Science, Information Science and Technology has not addressed the problem No training or incentive for data generators

9. Dark data is the data that we know is/was there but we can’t see it. Hubble Space Telescope composite image "ring" of dark matter in the galaxy cluster Cl 0024+17

10. Related Ideas John Porter: Deep verses Wide databases Swanson: Undiscovered Public Knowledge Science Commons: Big Verses Small science

11. GenBank PDB f ( x )= ax k + o ( x k ) Power Law of Science Data f ( x )= ax k + o ( x k )| X<.20 Data Volume Science Projects and Initiatives

12. Does NSF’s Data Follow the Power Law? I do not know but if $1 = X bytes…..

13. 20-80 Rule The small are big! Total Grants 9347 $2,137,636,716 20% 80% Number Grants 1869 7478 Total Dollars $1,199,088,125 $938,548,595 Range $6,892,810-$350,000 $350,000- $831

14. Biology 2009 #Grants: 1886 $Total: $744,168,471 ≈ €550,000,000 Distribution 1266 < $.5 million ≈ €370,000 Mode: $304,691 ≈ €225,000 Myth of the mega-project

15. Because it is high volume Because it is information rich – high entropy While needs of large data are understood small data and integration are not understood Heidorn, P. Bryan (2008). Shedding Light on the Dark Data in the Long Tail of Science. Library Trends 57(2) Fall 2008 . Institutional Repositories: Institutional Repositories: Current State and Future. Edited by Sarah Sheeves and Melissa Cragin. ( http://hdl.handle.net/2142/9127 ). Small data is big science

16. Where to find dark data Scientist’s backpacks and desks Literature/Biodiversity Heritage Library Museum Specimens Field notes Citizen Observations

17. What is dark data good for? Ecological Niche Modeling Climate Change niche change prediction Taxonomic Name Resolution Literature Search Support Taxonomic intelligence Key-like – character searching Phenology and Phenology change Food-web / trophic level

18. Problematic Transition Personal Information Management vs Knowledge Organization Pluralistic vs Unified (Hjørland, 2007)

19. Contrast in Styles (White, in press) Personal Information Management One-Few users Visual/Spatial Project Oriented Knowledge Organization Many users Language based Long-term orientation

20. New Information Disciplines Digital Curator : an expert knowledgeable of and with responsibility for the content of a digital collection(s) Digital Archivist : an expert competent to appraise, acquire, authenticate, preserve, and provide access to records in digital form Data Scientists : the information and computer scientists, database and software engineers and programmers, disciplinary experts, expert annotators, and others, who are crucial to the successful management of a digital data collection (Long Long-Lived Digital Data Collections: Enabling Research and Education in the 21st Century, report of the National Science Board, September, 2005)

21. Roles

22. Skills

23. Library Roles Life Cycle Phases Plan Create Keep Dispose Data Management Function Access Document Organize Protect

24. How to Organize at a higher level? It is difficult to find what is already known Clonal specimens may be stored in different museums around the world DNA analysis may be conducted on one but not the other Micrographs may be in a database Taxonomic treatments or revisions may exist

25. Biological Science Collections (BiSciCol) Tracker Muséum national d'histoire naturelle Nairobi National Museum Living Collection: Missouri Botanical Garden Determination Gene Sequence Parasitism S1: KNM S2: MNHN S3: MBG ? ? GENBANK ? ? ? ? Agave sisalana ?

26. BiSciCol Tracker

27. The Future is all about Data How do we get it? How do we analyze it? How do we disseminate it (Maps, charts tables..)? How do we keep it? Provenance, Storage Weeding How do we make it sustainable?

28. Digital/Data Curation Programs University of Illinois Graduate School of Library and Information Science University of Arizona School of Information Resources and Library Science University of North Carolina School of Information and Library Science

29. Education Needs Biological Information Specialist Concentration in Data Curation (MSLIS) Certificate of Advanced Study in Data Curation for Libraries and Scientist Information and professional education in biodiversity informatics

30. MSLIS Data Curation Concentration Data Curation Educational Program (DCEP) IMLS – Laura Bush 21 st Century Librarian Program, RE-05-06-0036-06 (Heidorn, PI) Students with the DC concentration will be trained to add value to data and promote sharing across labs and disciplinary specializations

31. Biological Information Specialists At present: Biologists at all degree levels self-trained in information technology Information technologists at all degree levels self-trained in biology (both with gaps in knowledge for many months, years) Differing roles of BIS in large and small

32. Master of Science in Biological Informatics Degree Program began September 2007 Part of campus-wide bioinformatics masters program NSF/CISE/IIS, Education Research and Curriculum Development, 0534567 (Palmer, PI) Combines Biology, Bioinformatics, Computer Science core with LIS courses

33. What does a BIS need to know? Biological training and interest in solving biological research problems Information skills Evaluation and implementation of information systems: user based assessment and continual quality improvement for the development of tools that work and are used. Information acquisition, management, and dissemination: development of digital libraries, data archives, institutional repositories, and related tools. Information organization and integration: ontology development, structuring information for optimal use and sharing, and standards development.

34. UIUC bioinformatics core coursework Cross-disciplinary course distribution requirement Bioinformatics: Computing in Molecular Biology Algorithms in Bioinformatics Principles of Systematics Computer Science: Algorithms Database Systems Biology: Human Genetics Introductory Biochemistry Macromolecular Modeling

35. Sample of existing LIS courses Information Organization and Knowledge Representation LIS 551 Interfaces to Information Systems LIS 590DM Document Modeling LIS 590RO Representing and Organizing Information Resources LIS590ON Ontologies in Natural Science Information Resources, Uses and users LIS 503 Use and Users of Information LIS 522 Information Sources in the Sciences LIS 590TR Information Transfer and Collaboration in Science Information Systems LIS 456 Information Storage and Retrieval LIS 509 Building Digital Libraries LIS 566 Architecture of Network Information Systems LIS 590EP Electronic Publishing Disciplinary Focus LIS 530B Health Sciences Information Services and Resources LIS 590HI Healthcare Informatics (Healthcare Infrastructure) LIS 590EI/BDI Ecological Informatics (Biodiversity Informatics)

36. University of Arizona Graduate Certificate in Digital Records Management Six Graduate Courses within MLA program Focus on repositories Cross over with Knowledge Representation and Metadata

37. Workforce Data Curation Workforce Summit Dec 6 th at IDCC Chicago Identify the Skill sets needed to government data curation Department of Energy, US National Science Foundation, Institute of Museum and Library Services, Oak Ridge National Laboratory, USGS National Biological Information Infrastructure, CIESIN

38. The Future is Collaboration and Data Sharing Libraries Museums Government Universities To bring the best data to the major problems and opportunities of our time and the future NGO Private Land Holders Ranches Farms

39. Merci Merci