Recommended
More Related Content
Similar to Making Big Data Useful: How NSIDC evolved data products for sea ice
Similar to Making Big Data Useful: How NSIDC evolved data products for sea ice (20)
More from DeVonne Parks, CEM
More from DeVonne Parks, CEM (15)
Recently uploaded
Recently uploaded (20)
Making Big Data Useful: How NSIDC evolved data products for sea ice
- 1. Ronin Institute Ronin Institute Ronin Institute Making Big Data Useful: The NSIDC Experience Ruth Duerr This work is licensed under a Creative Commons Attribution v4.0 License.
- 2. Ronin Institute Ronin Institute Ronin Institute Outline * • Introduction to NSIDC • The story of sea ice • Lessons learned
- 3. Ronin Institute Ronin Institute Ronin Institute Introduction to NSIDC *
- 4. NSIDC: An overview4 Cooperative Institute for Research in Environmental Sciences Main sponsors: NSIDC affiliations and sponsorship National Science Foundation NASA National Oceanographic and Atmospheric Administration World Data System
- 5. The National Snow and Ice Data Center… Provides tools for data access Researches the cryosphere and data science Educates the public about the cryosphere Supports data users Manages and distributes scientific data Supports local and traditional knowledge
- 6. NSIDC: An overview6 ▪ Satellite ▪ In situ (station data and the like) ▪ Model output ▪ Most digital, some analog Products Users More than 600 data and information products, most freely available online
- 7. NSIDC: An overview7 NSIDC 2011 Metrics 2011 Ingest 13 TB 3 million files 2011 Distribution 183 TB 23 million files Total Archive 102 TB 20 million files Metrics are for calendar years.
- 8. ELOKA NSIDC: An overview8 Why • Community-based knowledge of the Arctic informs science, policy, and development. • Arctic communities want their knowledge shared broadly and ethically, and passed on to generations. • There is a need for local knowledge holders to decide how to manage their “data” and how to effectively share it. What • ELOKA provides data management services and user support to facilitate the collection, use, exchange, and preservation, of local observations and knowledge of the Arctic. • ELOKA is mostly funded by NSF. • Data includes interviews, maps, and community measurements of sea ice, observations of narwhal behavior, environmental change, and ecology. • See eloka-arctic.org Exchange for Local Observations and Knowledge in the Arctic (ELOKA)
- 9. Ronin Institute Ronin Institute Ronin Institute The Story of Sea Ice *
- 10. Ronin Institute Ronin Institute Ronin Institute* The Remote Sensing Record • Satellite-based Passive Microwave sensors have been measuring sea ice since 1972 • Consistent collection of data started in 1978 with the SMMR series of instruments • Why passive microwave? ○ Distinguishing sea ice from ocean is straightforward ○ Passive microwave works through clouds and in the dark • Initial user base was cryospheric scientists Arctic sea ice concentration in April 2004, calculated from data measured by the Special Sensor Microwave/Imager (SSM/I) on the Defense Meteorological Satellite Program (DMSP) satellite. The image is centered over the North Pole, with continents shown in green. - Image courtesy of Florence Fetterer and Ken Knowles, National Snow and Ice Data Center, University of Colorado, Boulder, CO.
- 11. Ronin Institute Ronin Institute Ronin Institute Trending and audience – Audience confusion • A search of the NSIDC catalog for “sea ice” data sets returns 132 products! • The need to support these users led to the development of NSIDC’s Sea Ice FAQs which attempts to discuss the pros and cons of each data set, see for example:
- 12. Ronin Institute Ronin Institute Ronin Institute Trending and audience • Audience stayed pretty stable for roughly 20 years • That changed once the science community started reporting statistically significant trends in sea ice extent (in the early 2000’s) • At that point climatologists became interested in the data
- 13. Ronin Institute Ronin Institute Ronin Institute Development of the Sea Ice Index • But did NSIDC really have the data the climatologists needed? • Not really as they need: ○ Monthly averages ○ Climatologies ○ Anomaly maps and trends • Thus the Sea Ice Index (2002) data set was born with support from NOAA NESDIS
- 14. Development of the Sea Ice Index 14 Evolution of the sea ice “designated community”, presented by Ruth Duerr March 15, 2012, Foundations of Data Curation, GSLIS/UIUC7
- 15. Ronin Institute Ronin Institute Ronin Institute • Also starting in 2002 NSIDC started crafting periodic press releases concerning Arctic sea ice conditions at the end of the summer melt season • Media attention rose over the years, initially only from science reporters; eventually the sea ice minimum press release resulted in a barrage of queries from uninformed news reporters • It became painfully obvious that NSIDC needed a site where basic information would be available with answers to questions like “where is the arctic” • Thus was born the Arctic Sea Ice News & Analysis Site (ASINA) • Started as a “skunkworks” project with a bit of funding from a NASA outreach addendum to an existing grant Development of the Arctic Sea Ice News & Analysis Site
- 16. Ronin Institute Ronin Institute Ronin Institute
- 17. Ronin Institute Ronin Institute Ronin Institute ASINA – What is it? • A news blog • Currently updated monthly or more frequently if conditions warrant • Expert analysis and commentary about Arctic sea ice conditions consistently presented and written at a predictable level for returning visitors • High-resolution satellite imagery of current conditions • Additional high-resolution graphics and content as conditions warrant • References and links to scientific work outside NSIDC as appropriate
- 18. Ronin Institute Ronin Institute Ronin Institute ASINA – What is it? • A news blog • Currently updated monthly or more frequently if conditions warrant • Expert analysis and commentary about Arctic sea ice conditions consistently presented and written at a predictable level for returning visitors • High-resolution satellite imagery of current conditions • Additional high-resolution graphics and content as conditions warrant • References and links to scientific work outside NSIDC as appropriate
- 19. Ronin Institute Ronin Institute Ronin Institute ASINA – What happened next? • 2007 sea ice minimum was extreme • Media attention was also extreme • That led to the general public becoming one of NSIDC’s most vocal and consistent audiences • NSIDC’s most popular page • This also led to a rise in requests by the science community for access to NSIDC data • Because of ASINA the audience for our data expanded into a community without a lot of background science knowledge but who are “loyal, tenacious, and perceptive” • They ask a lot of questions and often many of them ask the same or similar questions in a short period of time
- 20. Ronin Institute Ronin Institute Ronin Institute Thus IceLights was born!
- 21. Ronin Institute Ronin Institute Ronin Institute What is Icelights? • A way to respond to user questions in a public forum • Roughly monthly blog-like format with ○ The ability to “Ask Icelights” a question ○ Search for previous topics by tag, free text, etc. ○ Posts that have been reviewed by the science community • “Crash course” content to provide ○ Arctic sea ice 101 ○ Data 101 ○ Reading list
- 22. Ronin Institute Ronin Institute Ronin Institute Distribution for October, 2015 *
- 23. Ronin Institute Ronin Institute Ronin Institute Lessons Learned *
- 24. Data Curation: The Evolution of Data Products for Value and Reach AGU 2015 Karen S. Baker1, Ruth E. Duerr2,1, and Mark Parsons3 1Graduate School of Library and Information Science University of Illinois Champaign-Urbana 2Ronin Institute for Independent Scholarship http://ronin.org 3Research Data Alliance Rensselaer Polytechnic Institute
- 25. Figure 2. A simplified view of the continuing development of scientific data products. Each cycle is initiated by one or more events that create a new audience that leads to generation of a new data product in response to the needs of a recently identified designated user community. Data Products: Multi-cycle Trajectory
- 26. Evolution of Sea Ice Data Products Redrawn from original work by Donna Scott, 2010 who manages the NSIDC Passive Microwave Product Team.
- 27. Evolution of Sea Ice Data Products Redrawn from original work by Donna Scott, 2010 who manages the NSIDC Passive Microwave Product Team.
- 28. Data Product Teams Roles - Skill Sets • Data Managers • Programmers • Technical Writers • Scientists • Science Communications • Systems/Database Managers • User Services Specialists “This active human element of data management is not always recognized by funding agencies, nor is it explicit in the OAIS Reference Model …” - Parsons and Duerr, 2005
- 29. Continuing Development of Data Products
- 30. Sea Ice Data Products: Dependencies & Levels
- 31. Levels of Data
- 32. To the GIS community, the world is: • A collection of features (e.g., roads, lakes, plots of land) with geographic footprints on the Earth's surface • The features are discrete objects described by a set of characteristics such as a shape/geometry (often 2-D) To fluid-earth scientists, the world is: • A set of observations/measurements described by parameters (e.g.,velocity, temperature) that vary as continuous functions in (4-D) space-time • Parameter behaviors are governed by a set of equations To the public, the world is: • The place within which their neighborhood is nested • A place where decision-making is increasing in complexity due to the interdependencies of natural systems, human systems, and human- natural systems * after Mark Parsons, Ben Domenico, and Stefano Nativi Diverse Audiences -> Diverse worldviews
- 33. Greenland Ice Sheet Melt Data Products
- 34. Ronin Institute Ronin Institute Ronin Institute 42 Questions?