The document outlines the collaboration between various Australian organizations like CSIRO, Geoscience Australia, and TERN to enhance access to Earth observation data via the Australian Geoscience Data Cube (AGDC). It discusses the infrastructure for data management, open data access, and the development of analysis-ready data, aiming to support both national and regional users in exploiting satellite data. The initiative promotes interoperability, community engagement, and the integration of diverse data sources to improve analytics and scientific research.

1. AusCover Earth Observation Services
& Data Cubes
Matt Paget – AusCover data and systems lead coordinator & AGDC team leader
CSIRO Land and Water, Canberra

2. Introduction
TERN / AusCover
• TERN’s Remote Sensing Data Facility
• Coordinated infrastructure for management and access to EO products from
Universities, CSIRO, Geoscience Australia (GA) and Bureau or Meteorology (BoM)
Australian Geoscience Data Cube (AGDC)
• Partnership between GA, CSIRO and National Computation Infrastructure (NCI)
• Version 2 is presently in development, due June 2016
Committee on Earth Observation Satellites (CEOS)
• AGDC forked as a collaboration between CEOS Systems Engineering Office
(NASA-SEO), CSIRO and GA
• Implementations for Kenya and Colombia
Copernicus Data Hub
• Australia and south Asia mirror of Europe’s Sentinel satellite series and services
• Partnership between Federal and State Govt agencies and departments and
AARNET/RDS/NCI

3. Two Pathways
Present
Data producers / custodians
• Internal infrastructure, downloads etc
• Storage and services close to
custodians
Open data access
• Low level programmable data services
• Interoperability with related and
cross-discipline activities
Aggregation
• Promote open data formats and
consistent metadata
• Metadata services and data portals
Emerging
Data producers / custodians
• Large central data stores
• Coordinated data and resource
management between custodians
Open data access
• Analytics facility close to data
• Optimised storage units with API
access
Aggregation
• Mature algorithms and exploratory
data analysis
• Community engagement in
validation and methods

4. AusCover
www.auscover.org.au
(in transition)
Remote sensing data delivery
backbone
• Interoperability formats and
services
• Storage and infrastructure
Data production network
• Central coordination and
regional nodes
• National biophysical data
products
Ground validation and
characterisation
• Data collection and processing
protocols
• Field campaigns with airborne
hyperspectral and lidar

5. AusCover data system
Distributed network of storage and data
access services
• Extensible nationwide system
• Data providers upload to their own
storage
Open standards, systems and services
• Non-proprietary formats, OGC
Direct data access
• Programmable and Human
• No license click-throughs
Metadata and data licensing
• Aligned with TERN, ANZLIC and
ANDS/RDA
Central coordination
• Website, Metadata, Services
• TERN coordination

6. Emerging EO informatics
Estimated storage and processing needs and opportunities for the next decade
• Increasing volume: spatial, temporal and spectral resolution and number of sensors
• Increasing velocity: near real time applications and automation
• Increasing variety: best practice processing and integration of disparate sources

7. AGDC and CEOS Data Cubes
Dice… …and Stack
Version 1
• GA’s Unlocking Landsat Archive
• Tiling, Indexing, Geotiff, HPC workflow
• Water Observations from Space
• Collaboration between GA, CSIRO and NCI
• Uptake by CEOS and Kenya Cube
development
Version 2
• Analysis Ready Data
• Relax requirement for a common grid
• Modular design
• Multiple platforms & environments
• Partnership and Governance
• Transition Plan (archives, products & users)
• Greater open collaboration with the
community
• Functional requirements due June 2016
www.datacube.org.au
github.com/data-cube/agdc-v2
github.com/ceos-seo

8. Storage Query and Access Module
Task
Execution
N-Dimensional Array Interface
Data
Index Store Data
Multi-
Dimensional
Storage Unit
Business Processes
(Data Management, Access Controls, Job Management, Etc.)
Integrating Tool
Forest Economics App N
Reporting Visualization
Application / GIS Mobile App
Data Cube
Infrastructure
Data & Application Platform
UI & Application Layer
Virtual Lab
Platform
Analytical
Task
Definition
Data Acquisition
& Inflow Analysis Ready DataObservations
Data Cube Notional Architecture
Data Collection 1 Data Collection 2
Data
Index Store Data
Multi-
Dimensional
Storage Unit
Authentication,
Authorisation,
Accounting
Discovery
Web
Services
Ingester (ARD and Result)

9. Summary
NCRIS and TERN
• Investment has enabled open data access across Australian institutions
• Major contribution to, and application of, open source standards and
services
International links
• CEOS WGISS Future Data Architectures (NASA, USGS, ESA, JAXA)
• Google Earth Engine and Amazon Web Services
• Unprecedented Australian Govt and research collaboration through
Copernicus Data Hub
Implications and opportunities
• Reducing the need for expert EO know-how
• Enabling best available data and analytics from the research sector for
applications and operations
• Building innovative and common infrastructures that adapt to massive
data volumes and emerging storage and compute environments

10. Contact details
AusCover www.auscover.org.au (in transition)
Alex Held, CSIRO Canberra alex.held@csiro.au
Stuart Phinn, UQ & JRSRP stuart.phinn@uq.edu.au
Matt Paget, CSIRO Canberra matt.paget@csiro.au
Peter Scarth, UQ & DSITI peter.scarth@qld.gov.au
AGDC www.datacube.org.au (in transition)
Trevor Dhu, GA trevor.dhu@ga.gov.au
Rob Woodcock, CSIRO Canberra robert.woodcock@csiro.au
Lesley Wyborn, ANU & NCI lesley.wyborn@anu.edu.au
Leo Lymburner, GA leo.lymburner@ga.gov.au
CEOS Data Cube www.ceos.org
Alex Held, CSIRO Canberra alex.held@csiro.au
Brian Killough, NASA-SEO brian.d.killough@nasa.gov

11. Australia’s Regional Copernicus Data Access/Analysis Hub
An integrated ‘Team Australia’ approach to
Support government information requirements.
Support the broader objective of enhancing
access to satellite Earth observation data by
research, industry and civil society.
Facilitate collaboration between Australians,
Europeans and inhabitants of the South-East
Asia-South Pacific region in exploitation of
Earth observation data.
Benefits for: Australia, the region,
EC/ESA/EUMETSAT, and the global
satellite EO community.
PartnersConsortium CollaboratorsSupporting
All Sentinel Products For

12. Partnerships
(pay-per-use)
Australia’s Regional Copernicus Data Access/Analysis Hub
Master Repository (NCI)
ESA
IntHub
EUMETCAST
Terrestrial
Operational Products
Deep National Archives
(Government Systems)
GEANT/AARNET
Original Format Data Files
(HSM Storage)
Analysis-Ready Data
(Lustre/spinning disk)
Cloud and HPC
(Petascale)
Agreed Standards
Interoperable
Services
(e.g.Thredds)
Regional
Portal
(ala‘SciHub’)
Industry
Exploitation
R&D for:
Research/Academia
Civil Society
Government
Industry
Australia and Region
Delivery for:
Australia
Region