Download as PDF, PPTX
Uploaded byGlobus
Accelerating Science with Cloud Technologies in the ABoVE Science Cloud
This document summarizes the use of the ABoVE Science Cloud (ASC) to support research for the Arctic-Boreal Vulnerability Experiment (ABoVE). The ASC provides researchers with large datasets, computing resources, and tools to process and analyze remote sensing and model data related to Alaska and northern Canada. Several examples are given of projects using the ASC, including analyzing satellite imagery to map forest structure, tracking surface water changes over time, characterizing fire history, and modeling future forest composition under climate change. The ASC aims to facilitate collaboration by allowing scientists to access common datasets and run computationally-intensive processes in the cloud without having to directly transfer large amounts of data.
More Related Content
Similar to Accelerating Science with Cloud Technologies in the ABoVE Science Cloud
Accelerating Science with Cloud Technologies in the ABoVE Science Cloud
- 1. Above.nasa.gov @NASA_ABoVE Accelerating Sciencewith Cloud Technologies in the ABoVE Science Cloud Elizabeth E. Hoy1,3, Peter C Griffith1,4, Dan Duffy2, Ellen Salmon2, Laura Carriere2, Jim Shute2, Julien Peters2, Hoot Thompson2 1NASA Carbon Cycle and Ecosystems Office; 2NASA Center for Climate Simulation; 3GST, Inc.; 4SSAI
- 2.
- 3. Where are weworking? 3 • 88 total projects (including NASA funded, Partner, Affiliated) • >550 participants from universities, national agencies/labs, state/provincial/territorial groups, private, and native/aboriginal organizations • Summer 2017 airborne campaign: • 10 aircraft, 20 deployments, and 200 science flights • April to October 2017 • 4 million km2 in Alaska and Canada • 2018 included additional sampling
- 4. Above.nasa.gov @NASA_ABoVE Why dowe need a new approach? • Science datasets are becoming larger, with intensive computation needed for data processing • And collaboration across diverse research groups is essential, • But it is often time consuming and expensive to transfer, download, process and share data with others • Therefore the ABoVE Science Cloud (ASC) was created to meet the needs of ABoVE investigators and encourage collaboration within the field campaign.
- 5. Above.nasa.gov @NASA_ABoVE Augmented fromRüegg et al 2014 in Front Ecol Environ The ABoVE Science Cloud is the center of the data lifecycle.
- 6. Above.nasa.gov @NASA_ABoVE Augmented fromRüegg et al 2014 in Front Ecol Environ The ABoVE Science Cloud is the center of the data lifecycle.
- 7. Above.nasa.gov @NASA_ABoVE Analysis isDifferent than HPC High Performance Computing Data Analysis Takes in small amounts of input and creates large amounts of output… Takes in large amounts of input and creates a small amount of output… Relatively small amount of observation data, models generate forecasts Large amounts of observational/model data generate science Tightly coupled processes require synchronization within the simulation Loosely coupled processes requiring little to no synchronization Simulation applications are typically 100,000’s of lines of code Analysis applications are typically 100’s of lines of code Fortran, Message Passing Interface (MPI), large shared parallel file systems Python, IDL, Matlab, custom Rigid environment – users adhere to the HPC systems Agile environment – users run in their own environments
- 8. Above.nasa.gov @NASA_ABoVE Advanced DataAnalytics Platform (ADAPT) “High Performance Science Cloud” Conceptual Architecture Adjunct to the NCCS HPC environment • Lower barrier to entry for scientists • Customized run-time environments • Reusable HPC/Discover hardware Expanded customer base • Scientist brings their analysis to the data • Extensible storage; build and expand as needed • Persistent data services build in virtual machines • Create purpose built VMs for specific science projects Difference between a commodity cloud • Platform-as-a-Service • Critical Node-to-node communication – high speed, low latency • Shared, high performance file system • Management and rapid provisioning of resources High Performance Science Cloud is uniquely positioned to provide data processing and analytic services for NASA Science projects. A portion of ADAPT is dedicated to ABoVE (the ABoVE Science Cloud).
- 9. Above.nasa.gov @NASA_ABoVE ASC SoftwareStack Operating Systems Linux (Debian, CentOS) and Windows Commercial Tools Intel Compiler (C, C++, Fortran), IDL (4 seats) Open Source Tools Python, NetCDF, etc. External License Servers Virtual machines can be set up to reach out to external license servers.
- 10.
- 11. Above.nasa.gov @NASA_ABoVE ABoVE ScienceCloud Data Holdings Large Collections Amount Landsat 186 TB MODIS MODAPS collection remotely mounted MERRA & MERRA2 406 TB DigitalGlobe Imagery 3 PB Total > 3.5 PB Find a list of all common datasets available on the ASC here>> Other Data Sets • Elevation datasets • Vegetation products • Land cover products • Products generated by the science team *Others as the team requests…
- 12. Above.nasa.gov @NASA_ABoVE . ABoVE ScienceCloud DigitalGlobe Imagery: Circumpolar
- 13. Above.nasa.gov @NASA_ABoVE Examples ofthe ASC In Action 13
- 14. Above.nasa.gov @NASA_ABoVE Forest CanopySurface Elevations – C. Neigh & P. Montesano © 2016 Digital Globe Nextview License © 2016 Digital Globe Nextview License • Understanding forest patterns using DigitalGlobe high-resolution satellite imagery • Using multiple VMs and Ames Stereo Pipeline (ASP) on the ASC to process Digital Elevation Models
- 15. Above.nasa.gov @NASA_ABoVE • Decisiontree classification on each Landsat scene • Extract theme (water) from each date, build data stack • Sum water observations for entire epoch • Mosaic each themed scene into ABoVE tile (no overlap) • Sum mosaicked tiles to create a total per theme for each ABoVE tile (utilizes all available observations including overlap) Extent and Dynamics of Surface Water – M. Carroll
- 16. Above.nasa.gov @NASA_ABoVE Burned area unburnedburnedvery low low-moderate moderate-high very high 1985 2005 2002 2010 fill value Landsat data record Burn severity Fire History for ABoVE – T. Loboda et al. • Fire history across the ABoVE study region is compiled from available and new data products and enhanced • Multiple VMs on the ASC are used to process Landsat and MODIS data to develop the burn severity characterization Loboda, T.V., D. Chen, J.V. Hall, and J. He. 2018. ABoVE: Landsat-derived Burn Scar dNBR across Alaska and Canada, 1985-2015. ORNL DAAC, Oak Ridge, Tennessee, USA. https://doi.org/10.3334/ORNLDAAC/1564
- 17. Above.nasa.gov @NASA_ABoVE Landscape-Scale Historiesof Disturbance, Seasonality and Greenness Trends - C. Woodcock et al. • 30+ year historical record and ongoing characterization of disturbance events and phenology across the ABoVE study domain • Using multiple VMs to move Landsat data into the ABoVE grid and then develop the landscape histories
- 18. Above.nasa.gov @NASA_ABoVE Land surfacealbedo – B. Rogers Year since fireYear since fire Machine learning modeled post-fire albedo • Modeled for every pixel, month, and year since fire • Accounts for heterogeneity in soils, vegetation, climate, and fire weather MODIS ‘blue sky’ albedo product • Accounts for multiple scattering based on Roman et al. (2010) • More accurate at high solar zenith angles and high albedos (e.g., non-summer months at high latitudes)
- 19. Above.nasa.gov @NASA_ABoVE Foster etal., in prep for Environmental Research Letters Climate change: RCP 8.5 Scenario 0 20 40 60 80 2000 2050 2100 2150 2200 Year AbovegroundBiomass(tonnesCha-1 ) Species Kenai birch AK birch Larch White spruce Black spruce Balsam poplar Quaking aspen Site 223082 Climate change white spruce aspen birch poplar Forest Change Modeling – A. Foster
- 20.