Source: http://hdfeos.org/workshops/ws01/presentations/UAH/matt.ppt

1. Dataset Independent Subsetting
A Dataset Independent Subsetting Prototype
http://minnie.cs.uah.edu/
Matthew R. Smith - matt.smith@msfc.nasa.gov
Bruce Beaumont
Dr. Sara J. Graves
The University of Alabama in Huntsville
Information Technology & Systems Laboratory
UAH
The University of Alabama in Huntsville
8-10 September 1997

2. Outline
Context
Purpose
Design
Functionality
Web pages
Future
Summary
UAH
The University of Alabama in Huntsville
8-10 September 1997

3. Context
NASA’s Mission to Planet Earth (MTPE)
Earth Observing System (EOS)
Data and Information System (DIS)
EOSDIS Core System (ECS) Contractor:
Hughes Information Technology Systems
Design and Implement a prototype datasetindependent subsetter
UAH
The University of Alabama in Huntsville
8-10 September 1997

4. Subsetting?
l Goal:
to provide a science data user with only
the data they request as quickly as possible.
l
Benefits science data users and data centers:
- reduces analysis time by reducing amount of data
- reduces time for data delivery
- reduces resources (network, personnel, media, etc.)
l
Steps:
- locate spatial, temporal, and spectral area of interest
- extract data
- re-assemble for distribution
UAH
The University of Alabama in Huntsville
8-10 September 1997

5. Design
Web-based
Dataset - independent
HDF-EOS formatted data
HDF-EOS software library
Data types
Swath
Grid
UAH
The University of Alabama in Huntsville
8-10 September 1997

6. Functionality
Front-end ( user interface )
Forms-based Web application - obtains subsetting
selection criteria
criteria file (ODL)
Back-end ( subsetter )
C software using HDF-EOS and HDF libraries
executed in batch mode
UAH
The University of Alabama in Huntsville
8-10 September 1997

7. User Interface
File selection
Parameters/channels
Geographic bounding box
Time range
Subsampling stride
Non-geolocated objects
UAH
The University of Alabama in Huntsville
8-10 September 1997

13. Summary of Current
Functionality
Subsetter Functionality
Can subset grid and swath data
Files may contain multiple grids and/or swaths; user may select
any or all for subsetting
Subset swath data on latitude/longitude and/or time
Subset grid data on latitude/longitude
Non-geolocated data may be included or excluded
Output is HDF-EOS file using same data types
“Back-end” runs as a batch job at archive center
User may check status of job and/or cancel it
E-mail sent to user when complete
Data retrieved via FTP
UAH
The University of Alabama in Huntsville
8-10 September 1997

14. Restrictions
Number of subsettable datasets limited by HDF-EOS library
subsetting functions:
Latitude must be “Latitude” or “Colatitude”
Longitude must be “Longitude”
Latitude and longitude must be FLOAT32 or FLOAT64
Latitude and longitude must be 1- or 2-dimensional
Latitude and longitude must have identical dimensions
Time must be “Time”
Time must be FLOAT64 in TAI93 format
Time must be 1- or 2-dimensional
“Track” must be slowest varying dimension in geo fields
Grid data must be in one of six supported projections
UAH
The University of Alabama in Huntsville
8-10 September 1997

15. Future Plans
Relax requirements for latitude/longitude and time in
swath datasets
Provide Java-based GUI for area-of-interest selection
Allow user to apply one subset specification to multiple
input files
Study integrating subsetter with a data visualization tool
Study separating structural metadata from data
UAH
The University of Alabama in Huntsville
8-10 September 1997

16. What is Needed
More test datasets in HDF-EOS format
Additional support for modifications to HDF-EOS calls
Accurate HDF-EOS documentation (internal and external)
Functional Java map applet
Resolution of metadata issues
Publication of official metadata standards
Name, content, and format of granule metadata
UAH
The University of Alabama in Huntsville
8-10 September 1997

17. Risks
HDF-EOS not currently in widespread use
HDF-EOS requirements for dataset-independent
subsetting not widely known to data producers
Legacy datasets are not in HDF-EOS format
Converting to HDF-EOS may increase storage
requirements
Many datasets are on non-volatile media
UAH
The University of Alabama in Huntsville
8-10 September 1997

18. Summary
A prototype Web-based dataset-independent subsetter has
been developed by UAH.
Allows spatial, temporal, and spectral subsetting and
subsampling of HDF-EOS datasets
Benefits science data users and data centers
Great potential. but limited current use
UAH
The University of Alabama in Huntsville
8-10 September 1997