Richard AkermanNRC-CISTIPresented at Access 2009, Oct. 1, 2009Will We Command Our Data?From the Petascale to the Personal

OverviewDefinitions / AssumptionsHow Big is Data?Four Sources of DataDriversActivities

Definitions / AssumptionsPetabyte = 1000 Terabytesdata = datasets“data is”

How Big is Data? http://www.instructables.com/file/FA9N61CF54HJ6GG/

How Big is Data?http://www.flickr.com/photos/doctorow/2731870631/

How Big is Data?http://en.wikipedia.org/wiki/File:Postduif.jpg

Four Sources of DataResearch dataGovernment dataLibrary dataPersonal data

General DriversSince 2000, a convergence of factors:Value of sharingEase of sharingLevel of sharing (machine level)

Specific Drivers: Research DataOECD Principles and Guidelines for Access to Research Data from Public Funding (April 2007)The Toronto Statement on prepublication data sharing (September 2009)

OECD Principles“Open access to research data from public funding should be easy, timely, user-friendly and preferably Internet-based.”http://www.flickr.com/photos/ben-zvan-photography/468487548/

Specific Drivers: Open Government DataUS Memorandum on Transparency and Open Government (January 2009)US Memorandum on the Freedom of Information Act (January 2009)

Specific Drivers: Open Government DataUK Power of Information Task Force Report (March 2009)Modernise data publishing and reusehttp://poit.cabinetoffice.gov.uk/poit/category/data-final/“public information held by for example the police, health bodies and local authorities is often not available. This is bad for democratic expression, the economy and citizen customers.”Data.gov (May 2009)UK PM Brown meets with Sir Berners-Lee (Sept. 2009)

Specific Drivers: Library DataILS Customer Bill-of-Rights, John Blyberg (November 2005)“Berkeley Accord” (March 2008)

Specific Drivers: Personal DataWired cover feature “Living by numbers” (July 2009)“Know Thyself: Tracking Every Facet of Life, from Sleep to Mood to Pain, 24/7/365”“Numbers are making their way into the smallest crevices of our lives. We have pedometers in the soles of our shoes and phones that can post our location as we move around town. We can tweet what we eat into a database and subscribe to Web services that track our finances. There are sites and programs for monitoring mood, pain, blood sugar, blood pressure, heart rate, … and prayers.”

Why LibrariesAdvocatesExemplarsExperts

Research Data:DataCitehttp://www.datacite.org/“DOIs for data”“The long term vision of the partnership is to support researchers by providing methods for them to locate, identify, and cite research datasets with confidence.”

Research Data: Gateway to Data SetsNRC-CISTI, Gateway to (Canadian) Scientific Data Setshttp://cisti-icist.nrc-cnrc.gc.ca/eng/services/cisti/scientific-data/data-sets/e.g. Canadian Astronomy Data Centre (CADC), Large Synoptic Survey Telescope (LSST)

Government Data: Canada - Federalhttp://geogratis.cgdi.gc.ca/StatsCanData Liberation Initiative (DLI)Ontario Data Documentation, Extraction Service and Infrastructure Initiative (ODESI)“The project will target Statistics Canada datasets... The files will be marked-up using DDI, an international, XML-based metadata tagging system which allows data resource discovery, distributed access, extraction and analysis.”

Government Data: Municipal - Vancouverhttp://data.vancouver.ca/

Government Data:Municipal - SFSan Francisco http://datasf.org/

Library DataA million free covers from LibraryThingOpen Library http://openlibrary.org/dev/docs/dataTalis Connected CommonsMESUR – Serviceshttp://id.loc.gov/ (LCSH)

APIs vs raw dataAPIsAlways serve up latest dataControl over accessTracking/statsAdvanced/complex functionality on top of the dataRaw dataUnconstrained / can do things never imagined by APIHard to track / versionCan lose metadataAllows choice of computing

Personal Data:Daytumhttp://www.daytum.com/

Personal Data:Total Recallhttp://totalrecallbook.com/(Sept. 2009)

Richard Akerman© 2009 Government of CanadaLicensed in the Creative CommonsThank Youhttp://creativecommons.org/licenses/by-nc-sa/2.5/ca/