AusCover portal presentation
Upcoming SlideShare
Loading in...5
×
 

AusCover portal presentation

on

  • 739 views

 

Statistics

Views

Total Views
739
Views on SlideShare
658
Embed Views
81

Actions

Likes
0
Downloads
4
Comments
0

2 Embeds 81

http://aceas.org.au 79
http://translate.googleusercontent.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    AusCover portal presentation AusCover portal presentation Presentation Transcript

      • AusCover - The Terrestrial Remote Sensing and Spatial Data Facility for TERN 21 March 2011
      Matt Paget and Edward King CMAR Canberra TERRESTRIAL ECOSYSTEM RESEARCH NETWORK
    • Discovering AusCover datasets and products
      • For all things dataset and product related:
        • www.auscover.org.au -> AusCover Portal
      • Visualisation portal is up to this point
        • http://portal.auscover.org.au
      • Metadata for datasets and products can be discovered here:
        • http://mest.auscover.org.au
      • For remote sensing experts, direct access to datasets is possible
        • OPeNDAP, THREDDS, Geoserver, HTTP
      • All currently available datasets and products
        • http://data.auscover.org.au
      • Servers hosting AusCover data:
        • http://data.auscover.org.au/servers
    • Accessing AusCover datasets and products Online Coming soon Source: http://www.aarnet.edu.au BoM NCI CSIRO UQ iVEC VPAC Data Fabric
    • AusCover strategic vision
      • TERN strategic priority 5: Sustainable production of an accessible and nationally consistent long-time series set of satellite images , satellite-based biophysical map products, next generation remote sensing research data and product validation for Australian conditions.
      Web-enabled data delivery for gridded remote sensing products and associated validation data
    • Data Needs and Inputs to Ecosystem & Climate Modelling for Land-Surface Condition & Change Detection
      • Time-series of Vegetation Biophysical Characteristics
      • Cover type & extent
      • Phenology
      • Fraction of absorbed photosynthetically active radiation (FAPAR)
      • Leaf area index (LAI)
      • Disturbance – Degradation: type/extent/intensity
      • Biomass
      • Net primary productivity
      • Future Climate Predictions ?
      • Other Land-Surface Characteristics
      • Evapotranspiration
      • Land surface temperature
      • Near-surface meteorology
      • Precipitation
      • Albedo
      • Radiation fluxes
      • Energy balance
      • Surface roughness
      • Soil physics and chemistry
      • Topography
      • Runoff
      • Snow and ice
      • Fires and biomass burning
      Source: ISLSCP – GCOS ECVs
    • Principles
      • Best available remote sensing and validation datasets and products
      • Multiple access pathways for users
      • Evolution away from proprietary data formats to open standards for formats and access
      • Discoverability via metadata standards and catalogue federation
      • Scalable and distributed data management system
      • Building sustainable relationships and collaborative links with TERN facilities, other NCRIS facilities, Aust Govt. and national computing facilities.
    • System design
      • Storage
      • Metadata
      • Web-enabled data servers
      • Whole of system tools
    • Storage
      • We are not resourced to build a monolithic “peak facility” to store all AusCover data
      • The maintenance task would be overwhelming
      • Data is almost always best curated and cared for locally by its creators
      • This suggests that some sort of distributed storage model will be both necessary and desirable
      Collection Catalogue (portal) Data Server Data Server Data Server
    • Metadata…
      • 10 100 1000 10000 100000
      • Granules
      Collections
    • Metadata standards
      • There are LOTS to choose from
      • Adopt simple BUT sufficient to:
        • Maximise inclusiveness
        • Guarantee minimum level of service
      • Mandatory minimum standard for collection metadata
        • ANZLIC profile of ISO19115 (geospatial data)
        • Fit for purpose + widely supported
        • No need to distinguish ourselves from other communities
      • Support a range of levels for granule metadata
        • Strive for CF (climate and forecast) Conventions
        • Optionally ACDD (attribute convention for data discovery)
      translation translation
    • Whole of system tools Whole of system tools
      • Discovery – http://mest.auscover.org.au
      • Visualisation – http://portal.auscover.org.au
      • Data subsetting and aggregation –proof of concept prototype ( http://www.marine.csiro.au/remotesensing/imos/ )
      Data access
      • Currently available products – http://data.auscover.org.au
      • Browse all AusCover servers - http://data.auscover.org.au/servers
      Collaborations
      • Australian institutions – UQ, BoM, GA, iVEC, CSIRO;
      • TERN facilities – Ecoinformatics, Supersites, TERN Central;
      • NCRIS platforms – IMOS, ALA, AuScope;
      • eResearch strategies – ANZLIC, ANDS, NeAT
    • Present systems
      • BoM
        • Technical lead is Tim Pugh
        • Existing Opendap/Thredds server
      • UQ
        • Technical lead is Leslie Elliot
        • Server powered up and install/config of software is in progress
      • CSIRO
        • Technical lead is Ron Jones
        • Beta servers built for Portal and MEST
        • Development Opendap, Thredds, Geoserver, Ncwms and Erddap servers
      • Perth
        • Opendap/Thredds server supporting both TERN and IMOS
      • GA
        • Pathway to push data to CSIRO
        • MODIS swath processing to be initiated at the ANU NCI
      • Darwin
        • Pathway to push data to CSIRO
        • MODIS swath processing to be initiated at the ANU NCI
    • Next steps Q1 2011 Publish data formats and metadata cookbooks for data providers. With appropriate guidance, the expectation is for data providers to do the majority of data formatting and metadata creation. Q2 2011 Refine implementation of data services at participating organisations. We aim to have reasonably consistent yet adaptable systems. The expectation is that the addition of new nodes will be relatively easy. Identified tasks but no timeline yet
      • Enhancement of the whole-of-system tools:
      • (semi-)Automated harvesting of metadata from datasets;
      • Implementation of usage and data download statistics;
      • Improved integration of the metadata and visualisation portals;
      • Continued co-development of the granule aggregation and subsetting tool with IMOS/eMII.
      Ongoing Stay abreast of TERN, NCRIS and eResearch capabilities. Web-enabled data delivery and system interoperability is a dynamic field and we’re at the leading edge.
    • Threats/Risks
      • Failure to secure viable storage
        • Multi-provider strategy
      • Inability to straddle the GIS/Global divide
        • Adopting loose strategy in terms of mandated formats
      • Failure to establish/support management/ingest workflows & transparency
        • Work with data sets team and community
      • Nodes fail to adopt
        • Outreach, feedback, documentation, roadshows, help
      • Granule manager technology superseded
        • Currently no other solution exists (but it might), we’re watching
    • AusCover, TERN and NCRIS
    • Generic and Facility architecture
    • Data and Metadata Team role
      • Provision of remote sensing data sets.
      • Providing advice and a process for creation of metadata for data sets
      • Guiding the reformatting of data files for improved web-enabled access and inclusion of metadata for “future-proofing”
      • Building relationships with Australia’s large computing facilities to both allow a platform for serving AusCover datasets and to promote coordination and consistency with other NCRIS Capabilities
      • Developing new server-side subsetting and aggregating tools in a partnership with IMOS to enhance the usability of large remote sensing datasets.
    • Servers and services Selected data servers (standard layer services)
      • Tier 1 (core): OPeNDAP, THREDDS, Geoserver.
      • Tier 2 (value-adding): ncWMS, ERDDAP.
      Reasons for their selection
      • Use of, and alignment to, open standards for data access (eg, DAP, OGC, KML).
      • Scalability with respect to a distributed network of data servers around Australia.
      • Common use within other NCRIS platforms.
      Key requirements for effective discovery, management and use of data
      • Searchable collection metadata records (for each distinct dataset served by AusCover).
      • Evolution away from proprietary file formats towards open-standard file formats for datasets.
      • Early and accurate recording of metadata including provenance, algorithm and theoretical basis, maturity, validation, contacts and online resources.