Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AusCover Earth Observation Services and Data Cubes

448 views

Published on

The presentation provides an overview of earth observation services offered by AusCover Facility of TERN. The presentation was part of the Workshop on Approaches to Terrestrial Ecosystem Data Management : from collection to synthesis and beyond which was held on 9th of March 2016 in University of Queensland.

Published in: Science
  • Be the first to comment

  • Be the first to like this

AusCover Earth Observation Services and Data Cubes

  1. 1. AusCover Earth Observation Services & Data Cubes Matt Paget – AusCover data and systems lead coordinator & AGDC team leader CSIRO Land and Water, Canberra
  2. 2. Introduction TERN / AusCover • TERN’s Remote Sensing Data Facility • Coordinated infrastructure for management and access to EO products from Universities, CSIRO, Geoscience Australia (GA) and Bureau or Meteorology (BoM) Australian Geoscience Data Cube (AGDC) • Partnership between GA, CSIRO and National Computation Infrastructure (NCI) • Version 2 is presently in development, due June 2016 Committee on Earth Observation Satellites (CEOS) • AGDC forked as a collaboration between CEOS Systems Engineering Office (NASA-SEO), CSIRO and GA • Implementations for Kenya and Colombia Copernicus Data Hub • Australia and south Asia mirror of Europe’s Sentinel satellite series and services • Partnership between Federal and State Govt agencies and departments and AARNET/RDS/NCI
  3. 3. Two Pathways Present Data producers / custodians • Internal infrastructure, downloads etc • Storage and services close to custodians Open data access • Low level programmable data services • Interoperability with related and cross-discipline activities Aggregation • Promote open data formats and consistent metadata • Metadata services and data portals Emerging Data producers / custodians • Large central data stores • Coordinated data and resource management between custodians Open data access • Analytics facility close to data • Optimised storage units with API access Aggregation • Mature algorithms and exploratory data analysis • Community engagement in validation and methods
  4. 4. AusCover www.auscover.org.au (in transition) Remote sensing data delivery backbone • Interoperability formats and services • Storage and infrastructure Data production network • Central coordination and regional nodes • National biophysical data products Ground validation and characterisation • Data collection and processing protocols • Field campaigns with airborne hyperspectral and lidar
  5. 5. AusCover data system Distributed network of storage and data access services • Extensible nationwide system • Data providers upload to their own storage Open standards, systems and services • Non-proprietary formats, OGC Direct data access • Programmable and Human • No license click-throughs Metadata and data licensing • Aligned with TERN, ANZLIC and ANDS/RDA Central coordination • Website, Metadata, Services • TERN coordination
  6. 6. Emerging EO informatics Estimated storage and processing needs and opportunities for the next decade • Increasing volume: spatial, temporal and spectral resolution and number of sensors • Increasing velocity: near real time applications and automation • Increasing variety: best practice processing and integration of disparate sources
  7. 7. AGDC and CEOS Data Cubes Dice… …and Stack Version 1 • GA’s Unlocking Landsat Archive • Tiling, Indexing, Geotiff, HPC workflow • Water Observations from Space • Collaboration between GA, CSIRO and NCI • Uptake by CEOS and Kenya Cube development Version 2 • Analysis Ready Data • Relax requirement for a common grid • Modular design • Multiple platforms & environments • Partnership and Governance • Transition Plan (archives, products & users) • Greater open collaboration with the community • Functional requirements due June 2016 www.datacube.org.au github.com/data-cube/agdc-v2 github.com/ceos-seo
  8. 8. Storage Query and Access Module Task Execution N-Dimensional Array Interface Data Index Store Data Multi- Dimensional Storage Unit Business Processes (Data Management, Access Controls, Job Management, Etc.) Integrating Tool Forest Economics App N Reporting Visualization Application / GIS Mobile App Data Cube Infrastructure Data & Application Platform UI & Application Layer Virtual Lab Platform Analytical Task Definition Data Acquisition & Inflow Analysis Ready DataObservations Data Cube Notional Architecture Data Collection 1 Data Collection 2 Data Index Store Data Multi- Dimensional Storage Unit Authentication, Authorisation, Accounting Discovery Web Services Ingester (ARD and Result)
  9. 9. Summary NCRIS and TERN • Investment has enabled open data access across Australian institutions • Major contribution to, and application of, open source standards and services International links • CEOS WGISS Future Data Architectures (NASA, USGS, ESA, JAXA) • Google Earth Engine and Amazon Web Services • Unprecedented Australian Govt and research collaboration through Copernicus Data Hub Implications and opportunities • Reducing the need for expert EO know-how • Enabling best available data and analytics from the research sector for applications and operations • Building innovative and common infrastructures that adapt to massive data volumes and emerging storage and compute environments
  10. 10. Contact details AusCover www.auscover.org.au (in transition) Alex Held, CSIRO Canberra alex.held@csiro.au Stuart Phinn, UQ & JRSRP stuart.phinn@uq.edu.au Matt Paget, CSIRO Canberra matt.paget@csiro.au Peter Scarth, UQ & DSITI peter.scarth@qld.gov.au AGDC www.datacube.org.au (in transition) Trevor Dhu, GA trevor.dhu@ga.gov.au Rob Woodcock, CSIRO Canberra robert.woodcock@csiro.au Lesley Wyborn, ANU & NCI lesley.wyborn@anu.edu.au Leo Lymburner, GA leo.lymburner@ga.gov.au CEOS Data Cube www.ceos.org Alex Held, CSIRO Canberra alex.held@csiro.au Brian Killough, NASA-SEO brian.d.killough@nasa.gov
  11. 11. Australia’s Regional Copernicus Data Access/Analysis Hub An integrated ‘Team Australia’ approach to Support government information requirements. Support the broader objective of enhancing access to satellite Earth observation data by research, industry and civil society. Facilitate collaboration between Australians, Europeans and inhabitants of the South-East Asia-South Pacific region in exploitation of Earth observation data. Benefits for: Australia, the region, EC/ESA/EUMETSAT, and the global satellite EO community. PartnersConsortium CollaboratorsSupporting All Sentinel Products For
  12. 12. Partnerships (pay-per-use) Australia’s Regional Copernicus Data Access/Analysis Hub Master Repository (NCI) ESA IntHub EUMETCAST Terrestrial Operational Products Deep National Archives (Government Systems) GEANT/AARNET Original Format Data Files (HSM Storage) Analysis-Ready Data (Lustre/spinning disk) Cloud and HPC (Petascale) Agreed Standards Interoperable Services (e.g.Thredds) Regional Portal (ala‘SciHub’) Industry Exploitation R&D for: Research/Academia Civil Society Government Industry Australia and Region Delivery for: Australia Region

×