Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Open Science and GEOSS: the Cloud Sandbox enablers

691 views

Published on

As part of the European project GEOWOW, Terradue was invited to present views at the GEO-X event on future endeavors to serve data democracy & science literacy in GEOSS (http://www.earthobservations.org/geoss.shtml)

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Open Science and GEOSS: the Cloud Sandbox enablers

  1. 1. Open Science & GEOSS: the Cloud Sandbox enablers GEOSS interoperability for Weather, Ocean and Water THEME[ENV.2011.4.1.3-1]: Inter-operable integration of shared Earth Observation in the Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014 Total EC funding: 6,399,098.00 € Project Web Site: www.geowow.eu EC Grant Agreement no. 282915 GEO-X Plenary Geneva, January 14th, 2014 Hervé Caumont Terradue herve.caumont@terradue.com
  2. 2. the GEOWOW Vision oo ooo Digital Earth Communities A long-term vision for GEOSS - GCI evolution … …considering feedback from all the stakeholders___ to engage with more user categories data providers, data specialists, scientists, decision makers within a more flexible architecture community components, resource enablers, cloud services GEO-X Plenary 2
  3. 3. Digital Earth Communities 14/01/2014 CREATING THE CONDITIONS GEO-X Plenary 3
  4. 4. The concept of Cloud Sandbox enablers Digital Earth Communities Connect web resources into an experiment apparatus http://en.wikipedia.org/wiki/ATLAS_experiment Data assembly APIs in the Cloud Data usage rights globally registered for scientific use Exchange and reuse of a scientist’s workspace 14/01/2014 GEO-X Plenary 4
  5. 5. The present situation for most GEO partners Digital Earth Communities A discovery and download modus operandi Dataset file selections To reach out a multitude of fragmented project environments 14/01/2014 GEO-X Plenary 5
  6. 6. A look at the Future through Cloud Sandboxes Digital Earth Communities Repeatable environments for reuse of scientific work Data as a Service within federated environments, with usage metrics Shared resources across Cloud Computing clusters 14/01/2014 GEO-X Plenary 6
  7. 7. Matching the Open Science goals Digital Earth Communities Open Source Open Data Open Access Open Notebook 29/11/2013 Transparency in experimental approach & collection of observations Public availability & reusability of scientific data Public accessibility of peer-reviewed scientific communication Shared versioning environment to facilitate progress in science GEO-X Plenary 7
  8. 8. Digital Earth Communities LOOKING FOR WHAT’S INSIDE ? 14/01/2014 GEO-X Plenary 8
  9. 9. Cloud Sandbox Enablers Digital Earth Communities •  Cloud Appliances Marketplace: a ‘VM Store’ to manage user Sandboxes •  Platform as a Service (PaaS): an algorithms integration environment •  Data staging tools: access and manage dataset slices required by applications Sandboxes Sandbox instance Tools (Python, Libraries, ...)
  10. 10. The PaaS environment Digital Earth Communities Apache Hadoop Streaming programming model Scale up MapReduce Jobs from single servers to thousands of computing nodes Automate failure handling at the application layer
  11. 11. Cloud Sandbox enablers Digital Earth Communities Cloud Sandbox UI dashboard -  VM Information, App Descriptor, App runs workflow status, VM monitoring, Run invocation, Support tools Cloud Applications linked to GitHub repositories –  Automated Code Versioning & Collaborative developments –  A new paradigm: a distributed service for dissemination Data Casting enablers for data preparation -  Get the atomic, scalable, data slice units -  Validate a processing job in Sandbox simulation mode -  Scale out on a cluster (e.g. by time slices) 14/01/2014 GEO-X Plenary 11
  12. 12. Tutorials & Training material Digital Earth Communities
  13. 13. THEY DARED TO TEST IT Digital Earth Communities 14/01/2014 GEO-X Plenary 13
  14. 14. Developers on Cloud Sandboxes Digital Earth Communities On-boarding GEOSS partners: Dec’12: Apr’13: May’13: Sep’13: 14/01/2014 UNESCO INPE ECMWF ESA GEO-X Plenary 14
  15. 15. Catalyzing resources Digital Earth Communities Support researchers to compute indicators for policy makers Take-up for future commercial applications (under ad-hoc conditions) Feed resources to Computing Clusters to run global & regional marine ecosystems assessments Go through research oriented, not for profit, uses of TIGGE-LAM data in order to spread innovation Expand uses of ESA satellites data Improve natural resources management and evolve policies Development of new applications leveraging innovative uses of earth observations 14/01/2014 Handle large Earth Observation Temporal Series for Land Change Events Detection GEO-X Plenary 15
  16. 16. Capacity Building Digital Earth Communities Ability to leverage ESGF’s CMIP5 Climate projections data, slice it and process it Explore and visualize ECMWF data from integrated Cloud appliances Support reproducibility of scientific experiments & open science Improve accessibility of key TIGGE data for a wide user community and better support ECMWF partner users Experiment with ENVISAT SCIAMACHY, global measuring of trace gases in the troposphere and in the stratosphere Streamline the processing of large EO data Temporal Series Towards integration of Earth Explorers missions: catalogs & processors on Cloud Sandboxes 14/01/2014 Experiment flexible access to compute-intensive resources GEO-X Plenary 16
  17. 17. Data Sharing Digital Earth Communities Ocean Biogeographical Information System (OBIS) Transboundary Waters Assessment Programme (TWAP) indices and indicators ESA Missions data (archived and Earth Explorer) 14/01/2014 TIGGE and TIGGE LAM, THORPEX Interactive Grand Global Ensemble Limited Area Model ERA Re-Analysis archives Earth System Grid Federation – Model Intercomparison Project Phase 5 (CMIP5) data GEO-X Plenary 17
  18. 18. Digital Earth Communities Developer Cloud Sandboxes – What’s next PERSPECTIVES IN THE NEWS 18
  19. 19. What we can have for GEOSS Digital Earth Communities Smooth deployments of Cloud environments for users. Lean on-boarding process for custom user needs. Data casting services for data intensive computing needs. EO data processing applications shared and reused. 14/01/2014 GEO-X Plenary 19
  20. 20. Bringing in the lean approach Digital Earth Communities Build Monitor Learn
  21. 21. Coming next from GEOWOW Digital Earth Communities Open Science ready Success stories Best practices Hands-on tutorials Online demos Getting users from all GEOSS communities on Cloud Sandboxes 14/01/2014 GEO-X Plenary 21
  22. 22. Open Science & GEOSS: the Cloud Sandbox enablers GEOSS interoperability for Weather, Ocean and Water THEME[ENV.2011.4.1.3-1]: Inter-operable integration of shared Earth Observation in the Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014 Total EC funding: 6,399,098.00 € Project Web Site: www.geowow.eu EC Grant Agreement no. 282915 GEO-X Plenary Geneva, January 14th, 2014

×