Trust threads: Provenance for Data Reuse in Long Tail Science
•
1 like•1,545 views
Invited Colloquium talk, Apr 23, 2015, Dept of Information and Library Science, School of Informatics and Computing, Indiana University. Abstract: The world contains a vast amount of digital information which grows vaster ever more rapidly. This makes it possible to do many things on an unprecedented scale: spot social trends, prevent diseases, increase fresh water supplies, accelerate innovation, and so on. As science and technology innovation is essential to improved public health and welfare, the growing sources of data can unlock more secrets. But the rapid growth of data makes accountability and transparency of research increasingly difficult. Data that are not adequately described are not useable except within the research lab that produced it. Data that are intentionally or unintentionally inaccessible or difficult to access and verify are not available to contribute to new forms of research. In this talk I show that data can carry with it thin threads of information that connect it to both its past and its future, forming its lineage particularly as it transitions into a shareable dataset residing in a public repository. In carrying this minimal provenance, the data becomes more trustworthy. This thread of trust is a critical element to the successful sharing, use, and reuse of big data in science and technology research in the future.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Trust threads: Provenance for Data Reuse in Long Tail Science
Similar to Trust threads: Provenance for Data Reuse in Long Tail Science (20)
More from Beth Plale
More from Beth Plale (7)
Recently uploaded
Recently uploaded (20)
Trust threads: Provenance for Data Reuse in Long Tail Science
- 1. Cooperative agreement #OCI0940824 Trust Threads: Provenance for Data Reuse in Long Tail Science Beth Plale Data To Insight Center Indiana University April 23, 2015
- 2. The impact that digital data is having on science is increasing because of phenomenal data growth.
- 3. Data driven agent based modeling for labor sharing in Zambia Enabling new research
- 4. Data growth results in • New and bigger research ques=ons : energy, clean and abundant water, sustainable food, healthy popula=on • Requires interdisciplinary teams : scien=sts, technologists, medical doctors, social scien=sts, policy makers • Requires data understood across discipline boundaries
- 5. Data successfully interpretable across scien=fic discipline boundaries has likely seen some cura=on = curated data equates to reusable data = Data reuse: Data collected for one purpose used for different purpose
- 6. Can technology innova=on (tools, data management, knowledge representa=on) accelerate frequency of data reuse?
- 7. • Data can carry with it thin threads of informa=on “trust threads” that connect data to both its past and its future. • In carrying this minimal provenance, data becomes more trustworthy. • Trustworthiness cri=cal to successful sharing, reuse of data in science and technology research Provenance suitcase
- 8. Data provenance Data Provenance Lineage of data product. Like piece of art, tells what has happened to data product through its life. Shows rela=onships between products Provenance for two polar ice sheet images showing problems with one (on right, missing contribu=ng files) used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used used used wasTriggeredBy wasTriggeredBy used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used used used used used used used used used usedused used usedused used used used wasTriggeredBy wasTriggeredBy usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom used wasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFromusedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom wasTriggeredBy used used used used used usedused used used used used used used used used wasTriggeredBy wasGeneratedBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFromwasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom usedused usedused usedused used used used usedusedused used used used used used usedused used wasTriggeredBy wasTriggeredBy wasGeneratedBy wasDerivedFromwasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFromwasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom usedwasDerivedFrom used wasDerivedFromused wasDerivedFrom usedwasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom usedwasDerivedFrom wasTriggeredBy used wasDerivedFrom used used used used usedused used used used used usedused used used used used used used used used used wasTriggeredBy wasGeneratedBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used usedused used used usedused used used used used used used used usedused wasGeneratedBy wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used used used used used used used used used used usedused used used used used used used wasTriggeredBywasGeneratedBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom wasTriggeredBy used used usedused used used used used used used used used used used used used used used used used usedused used used used wasGeneratedBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromusedwasDerivedFrom usedwasDerivedFrom usedwasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used used used used used used used wasTriggeredBy wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used usedused used used usedused used used used used used used used used used used used usedused used used used used used used wasTriggeredBy wasTriggeredBy wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used used used used wasTriggeredBy usedused used used used used usedused used used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used used used used used used usedused used used used used used used used used used used used used used used used used wasTriggeredBy wasTriggeredBy used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom wasTriggeredBy usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom wasTriggeredBy usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFromusedwasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom usedwasDerivedFrom wasDerivedFrom used usedused used used used used usedused used used used used used used wasTriggeredBy wasGeneratedBy wasGeneratedBy wasTriggeredBy wasTriggeredBy wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromused used wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFromused used used used used used used used used used used used wasTriggeredBy wasTriggeredBy wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom wasTriggeredBy used used used used used used used used used used used used used used used used wasTriggeredBy wasGeneratedBywasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFromwasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used used used used used used used used used wasTriggeredBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasTriggeredBy used wasDerivedFrom wasDerivedFromused wasDerivedFrom wasDerivedFrom used used used used used used used used used used used used used used used used used used used used used used used used used wasGeneratedBy wasGeneratedBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used wasTriggeredBy wasTriggeredBy wasTriggeredBy used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFromused wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom wasTriggeredBy used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFromused wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFromused wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used used used used wasTriggeredBy wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom wasDerivedFrom usedused usedused usedused used used used wasTriggeredBy wasGeneratedBy wasGeneratedBy wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFromused wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used used used used used used used used used used used used used used used used used used used used used used wasTriggeredBy wasGeneratedBy wasTriggeredBy wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom wasGeneratedBywasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used used used used used used used used used used used used used used used used wasTriggeredBy wasTriggeredBy wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFromwasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used used used used used used used used used used used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFromwasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom used wasDerivedFrom used used used used used used used used used used used used used used used usedused wasTriggeredBy used wasDerivedFromused wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used usedused used used used used used used used used wasTriggeredBy wasTriggeredBy wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used used used wasTriggeredBy used wasDerivedFrom used wasDerivedFromused wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom usedused used used used used usedused used used used used used used used used used used used used used used used used used used used wasTriggeredBy wasTriggeredBy wasTriggeredBy used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used usedused used used used usedused used used wasTriggeredBy wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom used used used used used used used usedused used used usedused wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredByused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom used wasDerivedFrom used used used used usedused used used used wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromwasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFromwasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used wasDerivedFrom used used used used used used used used usedused used used used used used used wasTriggeredBy used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom wasTriggeredBy used wasDerivedFrom used wasDerivedFrom wasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used usedused usedused used used used used used used used used used used used used used used used used used used used used usedused used wasTriggeredBywasTriggeredBy used wasDerivedFromused wasDerivedFrom used wasDerivedFromused wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFrom used wasDerivedFromused wasDerivedFromwasGeneratedBy wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom wasDerivedFrom used used used used used used used used used used used used used BT BT BT BT BT BT BT daily BT BT BT BT BT BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT SeaIce12km BT @L3seaiceBT BT BT BT BT BT BT SeaIce12km BT BT BT BT BT BT BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT daily BT BTBT BT BT BTBT BT BT BT BT BT BT BT BT daily @L3seaice BT BT BT BTBT BT BT BT BTBT BTBT BTBT SeaIce12km BT BT BT BT BT BT BT BT BT BT BT @L3seaice SeaIce12km BT BT BT daily BT BT BT BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT BT BT BT BT BT BT daily BT @L3seaice SeaIce12km BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BTBTBT BT daily BTBT BTBT BT BTBT BT BT BT BT @L3seaice SeaIce12km BT BT BT BT BT BT BT BT BT BT BTBT BTBT BT BT BT BT BT BT BT BT BT BT BT BT BT @L3seaice SeaIce12kmBT BT BT BT BT BT daily BT BT BT BT BT BT daily @L3seaice BT BT BT BT BT BT BT BT BT BT BT SeaIce12km BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km @L3seaice BT BT BT BT BT BT BT daily BT BT BT BT @L3seaice SeaIce12km BT BT BT BT BT BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT BT BT BT BT BT daily BT SeaIce12km BT BT BT BT BT BT BT BT BT BT BT daily BT BT BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT SeaIce12km @L3seaice BT BT BT BTBT BT BT BT BT BT BT BT BT BT BT BT BT BT daily BT BT daily BT BT BT BT BT BT BT BT BT BT BT BT BT @L3seaice SeaIce6km SeaIce12km BT BTBTBT BTBT BT BT BT BT BTBT SeaIce12km BT BTBT BTBTSeaIce6km BT @L3seaice BT BT BT BT SeaIceDrift BT BT BT BT BT BT BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT BT BT BT BTdaily @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km BT BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BTBT BT SeaIce12km BT BT BT BT BT daily daily BT BT BT BT BT BT BT BT BT BT BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce6km SeaIce12km BT BT BT BT BT BT BT BT BT BT BT @L3drift BT BT BT BT BT BT BT BT BTBT BT BT dailyBTBT BT BT BT BTBTBT BTBT BT BT SeaIce12km daily BT @L3seaice BT BT BT SeaIce6km BT @L3seaice SeaIce12km SeaIce6km BT BTBT BT BT BT BT BT BT BT daily BT BT BT BT BT BT daily BT BT @L3seaice BT BT BTBT BT BT BT BT BT BT BT BT SeaIce12km BT BT BT BT BT BTBT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT daily BT BT BTBT BT BT BT SeaIce12km BT BT BT BT BT BT BT BTBT SeaIce12km @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km BT BT BT BT BT BT BT BT BT BT BT BT BT BT @L3seaice BT daily BT BT BT BT BT daily BT BT BT BT SeaIce12km @L3seaiceBT BT BT BT BT BT BTBT BT BT BT BTBT BTBTBT BT BT BT BT BT BT BT BT BT BT BTdaily BT BT SeaIce12km @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km @L3seaice BTBT BT BT BT BT daily BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km @L3seaice BT BT BT BT BT BT BT BT BT BT daily BT BT BT SeaIce12km BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km BT BT BT BT BT BT BT BT BT @L3seaice BT BT BT BT BTBT BT daily BTBT BT BT daily BT BT BT BT BT BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT BT BTBTBT daily BT BT BT BT BT BT BT BT BT dailyBT BT BT BT BT BT BT BT BT BT BT BT SeaIce12km BT BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT BTBT BT BT BT BT BT BT BT daily BT BT BT SeaIce12km BT BT BT BT BT BT BT BT @L3seaice BT BT BT BT BT BT BT daily BT BT BT BT BT BT BT BT BT BT SeaIce12km BT BTBT BT BT daily BT BT @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BTBT BT BTBT BT BTBT daily BT BT BT BT BT BT BT BT BT SeaIce12km @L3seaice BT BT BT BT BT BT BT BT BT BT BT BT BT 2012 Month of ice sheet images, linked by template passed from day to day in processing
- 9. Long tail of science DataQuantity Data complexity and dispersion research center project lab team solo inves=gator ci=zen Giga Peta Exa Tera Researchers who produce and analyze heterogeneous data that is unique and at fine resolution and granularity. Researchers who need to collaborate with others to make scientific advances. Researchers who lack access to reliable cyberinfrastructure for managing, sharing, analyzing, publishing and archiving data.
- 10. Publishing Long Tail Science Data Table of Contents • The Flood • The Publishing Exercise
- 11. Terminology • Data publishing : act of making a data set that is associated with a peer reviewed scien=fic publica=on publicly available • Data preserva4on : steps undertaken to ensure a data set is available and useable by subsequent genera=ons of scien=sts • Data sharing : reuse and repurpose of a data set for by a third party • Long tail science: science u=lizing data from mul=ple sources; data and models can be manipulated on worksta=on or small cluster
- 12. Cast of Characters • Hydrologist (PI level) • Hydrologist Postdoc • Data scien=st (PI level) • Research programmer • Data curator (CLIR Fellow) • University library (data management services)
- 13. Extreme Mississippi Flood 2011
- 14. Thanks to SEAD funded by Na4onal Science Founda4on. B. Plale and P. Kumar, co-‐PIs. -‐-‐ Praveen Kumar and his team at UIUC
- 15. Extreme Mississippi Flood 2011* • Birds Point New Madrid (BPNM) Floodway is an agricultural region located west of Mississippi River just south of its confluence with the Ohio River, near city of Cairo, Illinois. • Policy established following disastrous 1927 Flood permits levees surrounding BPNM Floodway to be inten=onally breached during extreme floods. • During historic flood stages of May 2011, Floodway was ac=vated for first =me since 1937. U.S. Army Corps of Engineers used blas=ng agents on May 2, 3 and 5, 2011 to create ar=ficial breaches. * A. E. Goodwell, Z. Zhu, D. Duca, J. A. Greenberg, P. Kumar, M. H. Garcia, B. L. Rhoads, R. R. Holmes, G. Parker, D. P. Berreca, R. B. Jacobson (2014). Assessment of Floodplain Vulnerability during Extreme River Flood 2011, Environmental Science and Technology.
- 16. Why study breach? • Impact was drama=c. Parts of agricultural floodplain were inundated for over a month. • Large floods, such as this, create highly variable spa=al pacerns of erosion and deposi=on. These localized areas, or “hotspots”, of change expose underlying landscape vulnerabili=es. • Study is opportunity to assess causes to guide future ac=ons.
- 17. Sudden eleva=on drop at O’Bryan Ridge(B1, B2) resulted in substan=al erosion there. Oxbow of old river bed (Ten Mile Pond) should have seen substan=al erosion but didn’t because trees had filled into area since 1937. Grey area is floodplain, blue is Mississippi River. Green circles are USGS installed sensors. Levee breach at point A. Credit for images: A. E. Goodwell, et al. (2014). Assessment of Floodplain Vulnerability during Extreme River Flood 2011, Environmental Science and Technology.
- 18. Sudden eleva=on drop at O’Bryan Ridge resulted in substan=al erosion -‐ see B1, B2.
- 19. Part II: Publishing Exercise • Fall 2014 Postdoctoral Fellow asked to publish datasets associated with 2014 EST paper. She used SEAD to ingest into ins=tu=onal repository at Fellow’s university and link dataset and publica=on. • Ques=ons Fellow had to address: – What do I publish? – Why do I publish? – How do I publish?
- 20. Kinds of Data Used • Map of land eleva=on (Digital Eleva=on Model). LIDAR data from 2005 and 2011. US Army Corps of Engineers. Images • Max velocity of water. Hydrosed 2D hydrology model. Modeled data • Woody vegeta=on mapping. AVIRIS – airborne visible/infrared imaging. Images • Water level sensors. USGS Water level sensors. Sensor data • Soil/landscape sensi=vity to erosion. K/T func=on. Computed values
- 21. Published: single Research Object • 5 Hydrosed model max velocity files (whole Birds Point New Madrid floodway (BPNM), Case 1 and 2: O'Bryan, Case 1 and 2 Ten Mile Pond) – Whole BPNM: Umax_BPNM.tif 1.47 MB – Case 1 O’Bryan: Umax_OBR_Case1.tif 3.23 MB – Case 2 O’Bryan: Umax_OBR_Case2.tif 3.23 MB – Case 1 Ten Mile Pond: Umax_TMP_Case1.tif 6.59 MB – Case 2 Ten Mile Pond: Umax_TMP_Case2.tif 6.59 MB • K/T Factor soil map: KT_BPNMShape.tif 7.34 MB • Classified AVIRIS map (woody vegeta4on) AVIRIS_50M_BPNMshape.tif 919.48 KB • Original differen4al DEM (1.5 m resolu=on) Sub_Original_5ft.tif 2.84 GB • Final differen4al DEM (10 m resolu=on) Sub_Corrected_10m.tif 72.75 MB • Final differen4al DEM (3 m resolu=on) Sub_Corrected_3m.tif 798.12 MB
- 22. Publishing Exercise – Why do I publish? • A: publish data necessary to reproduce images in published paper – What do I publish? • A: publish data necessary to reproduce images in published paper
- 23. Publishing Exercise • What do I publish? – Answer: only those files necessary to reproduce images • Implicaons? – All TIF files (geoTIF) – 3.74 GB “Research Object” • Umax_BPNM.tif 1.47 MB • Umax_OBR_Case1.tif 3.23 MB • Umax_OBR_Case2.tif 3.23 MB • Umax_TMP_Case1.tif 6.59 MB • Umax_TMP_Case2.tif 6.59 MB • KT_BPNMShape.tif 7.34 MB • AVIRIS_50M_BPNMshape.tif 919.48 KB • Sub_Original_5ft.tif 2.84 GB • Sub_Corrected_10m.tif 72.75 MB • Sub_Corrected_3m.tif 798.12 MB It is existence of content as TIF files that makes data publishing insufficient for data sharing
- 24. Size Macers Implicaons? • Dspace not well suited to Research Object (RO) that is 3.74 GB in size • Published data is 3.74 GB RO, so shared data will be substanally larger. • Libraries need partnerships with large scale storage providers • Umax_BPNM.tif 1.47 MB • Umax_OBR_Case1.tif 3.23 MB • Umax_OBR_Case2.tif 3.23 MB • Umax_TMP_Case1.tif 6.59 MB • Umax_TMP_Case2.tif 6.59 MB • KT_BPNMShape.tif 7.34 MB • AVIRIS_50M_BPNMshape.tif 919.48 KB • Sub_Original_5ft.tif 2.84 GB • Sub_Corrected_10m.tif 72.75 MB • Sub_Corrected_3m.tif 798.12 MB
- 25. Technology frame of reference: SEAD
- 26. SEAD Leadership Cooperative agreement #OCI0940824 Clockwise from top: Margaret Hedstrom, UMich; Beth Plale, IU; Praveen Kumar, UIUC; Jim Myers, UMich; Sandy Payette, UMich
- 27. SEAD – Researcher’s Perspective Collec=on and analysis of data SEAD Sharing and publica=on of data Management and sharing of data
- 28. SEAD – Researcher’s Perspective Collec=on and analysis of data SEAD Sharing and publica=on of data Management and sharing of data Time Cura=on engagement
- 31. SEAD Technology • Active Project Spaces – Collaboration / file sharing space for research projects – Staging area for data curation prior to publishing • SEADVirtual Archive – Push publishable data (as Research Objects) from an active Project Space to appropriate preservation and discovery environment, and back for reuse • People, Data,Things – Researcher profiles, repository profiles, data profiles – all linked, related, dynamic – JSON LD
- 32. Project Spaces ... organize, describe, visualize geo dashboard dataset collec=ons with metadata
- 33. Virtual Archive Deposit refines for discovery Virtual Archive ... publish and archive data
- 34. Deposit refines for discovery The 3.6 GB Research Object of data for Mississippi River Flood 2011 deposited to IDEALS repository
- 35. Cloud Storage IR Project Space Project Project Space Project Space Match-‐ Maker Cura=on Work Flow Submit Submit Computa=onal Environments Tools Apps Project Space Project Space Project Space Long Tail of Data Producers Domain Repository Scien=fic Workflow VA Registry Services social network
- 36. • At point in =me when researcher ready to publish her data, she must make decision about what data – from amongst all data sources used, consulted, and created, should be include in publishable result. • At this point (at latest) is when data curator gets engaged • Works in-‐situ and alongside researcher to assemble publishable object – Occurs in SEAD Project Spaces Back to Hydrology Researcher
- 37. Tracking life of research object
- 38. Research Object Unique ID Agents StatesRelationships Content Research Object Framework • Data creator • Curator • Data re-use scientist • Live • Curated • Published • Aggregates • Related to • Describes • Derived from • Versioned from • Files • Bitstreams • Pointers • Annotative information Bundles of resources that use common standards and services to transfer and consume them
- 39. Research Object Publish reuse lifecycle • RO begins as Live Object (LO), this is when all future content is s=ll in state of high flux. • Live Object state is collabora=ve, mul=-‐person project team working together in single shared project space in SEAD environment
- 40. Research Object Publish reuse lifecycle • At point in =me when researcher begins assembly for publishing, she will prune and organize material to publish into new directory or set of directories, or tag specific files. • This pruned content becomes the CuraAon Object (CO)
- 41. Research Object Publish reuse lifecycle • Once researcher and digital curator agree that content and descrip=ons of research product are ready, researcher signals intent to publish whereupon research object moves from its state as a Cura=on Object to a new state as a Published Object (PO). • PO exists in “control zone” where changes are tracked in a rigorous manner.
- 42. Publish reuse lifecycle Behavior diagram (aka “state machine”)
- 44. Cloud Storage IR Project Space Project Project Space Project Space Match-‐ Maker Cura=on Work Flow Submit Submit Computa=onal Environments Tools Apps Project Space Project Space Project Space Long Tail of Data Producers Domain Repository Scien=fic Workflow VA Registry Services social network
- 45. Cloud Storage IR Project Space Project Space Project Space Match-‐ Maker Cura=on Work Flow Submit Submit Computa=onal Environments Tools Apps Project Space Project Space Project Space Long Tail of Data Producers Domain Repository Scien=fic Workflow VA Registry Services social network Wild West Boundary Waters Control Zone
- 46. Cloud Storage IR Project Space Project Space Project Space Match-‐ Maker Cura=on Work Flow Submit Submit Computa=onal Environments Tools Apps Project Space Project Space Project Space Long Tail of Data Producers Domain Repository Scien=fic Workflow VA Registry Services social network Wild West Boundary Waters Control Zone Requires two way communica4on with partner repositories
- 47. From point in =me where data is first conceived as ready for publica=on through to its first reuse is cri=cal stage in data’s life. If we are unable to bring technical rigor to handling object cura=on, rela=onships, and deriva=ons in this early stage, we lose cri=cal opportunity for enhancing trustworthiness of data object: an opportunity that is irrecoverable.
- 48. With special thanks • Hydrologist postdoc Allison Goodwell, Professor Praveen Kumar, UIUC • Dr. Inna Kouper, IU • IU SEAD developer team: Scoc McCaulay, Isuru Suriarachchi, Aravindh Varadharaju, Charitha Arachchige, Yuan Luo • Na=onal Science Founda=on