Dataset Independent Subsetting

345 views

Published on

Source: http://hdfeos.org/workshops/ws01/presentations/UAH/matt.ppt

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
345
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dataset Independent Subsetting

  1. 1. Dataset Independent Subsetting A Dataset Independent Subsetting Prototype http://minnie.cs.uah.edu/ Matthew R. Smith - matt.smith@msfc.nasa.gov Bruce Beaumont Dr. Sara J. Graves The University of Alabama in Huntsville Information Technology & Systems Laboratory UAH The University of Alabama in Huntsville 8-10 September 1997
  2. 2. Outline Context Purpose Design Functionality Web pages Future Summary UAH The University of Alabama in Huntsville 8-10 September 1997
  3. 3. Context NASA’s Mission to Planet Earth (MTPE) Earth Observing System (EOS) Data and Information System (DIS) EOSDIS Core System (ECS) Contractor: Hughes Information Technology Systems Design and Implement a prototype datasetindependent subsetter UAH The University of Alabama in Huntsville 8-10 September 1997
  4. 4. Subsetting? l Goal: to provide a science data user with only the data they request as quickly as possible. l Benefits science data users and data centers: - reduces analysis time by reducing amount of data - reduces time for data delivery - reduces resources (network, personnel, media, etc.) l Steps: - locate spatial, temporal, and spectral area of interest - extract data - re-assemble for distribution UAH The University of Alabama in Huntsville 8-10 September 1997
  5. 5. Design Web-based Dataset - independent HDF-EOS formatted data HDF-EOS software library Data types Swath Grid UAH The University of Alabama in Huntsville 8-10 September 1997
  6. 6. Functionality Front-end ( user interface ) Forms-based Web application - obtains subsetting selection criteria criteria file (ODL) Back-end ( subsetter ) C software using HDF-EOS and HDF libraries executed in batch mode UAH The University of Alabama in Huntsville 8-10 September 1997
  7. 7. User Interface File selection Parameters/channels Geographic bounding box Time range Subsampling stride Non-geolocated objects UAH The University of Alabama in Huntsville 8-10 September 1997
  8. 8. Summary of Current Functionality Subsetter Functionality Can subset grid and swath data Files may contain multiple grids and/or swaths; user may select any or all for subsetting Subset swath data on latitude/longitude and/or time Subset grid data on latitude/longitude Non-geolocated data may be included or excluded Output is HDF-EOS file using same data types “Back-end” runs as a batch job at archive center User may check status of job and/or cancel it E-mail sent to user when complete Data retrieved via FTP UAH The University of Alabama in Huntsville 8-10 September 1997
  9. 9. Restrictions Number of subsettable datasets limited by HDF-EOS library subsetting functions: Latitude must be “Latitude” or “Colatitude” Longitude must be “Longitude” Latitude and longitude must be FLOAT32 or FLOAT64 Latitude and longitude must be 1- or 2-dimensional Latitude and longitude must have identical dimensions Time must be “Time” Time must be FLOAT64 in TAI93 format Time must be 1- or 2-dimensional “Track” must be slowest varying dimension in geo fields Grid data must be in one of six supported projections UAH The University of Alabama in Huntsville 8-10 September 1997
  10. 10. Future Plans Relax requirements for latitude/longitude and time in swath datasets Provide Java-based GUI for area-of-interest selection Allow user to apply one subset specification to multiple input files Study integrating subsetter with a data visualization tool Study separating structural metadata from data UAH The University of Alabama in Huntsville 8-10 September 1997
  11. 11. What is Needed More test datasets in HDF-EOS format Additional support for modifications to HDF-EOS calls Accurate HDF-EOS documentation (internal and external) Functional Java map applet Resolution of metadata issues Publication of official metadata standards Name, content, and format of granule metadata UAH The University of Alabama in Huntsville 8-10 September 1997
  12. 12. Risks HDF-EOS not currently in widespread use HDF-EOS requirements for dataset-independent subsetting not widely known to data producers Legacy datasets are not in HDF-EOS format Converting to HDF-EOS may increase storage requirements Many datasets are on non-volatile media UAH The University of Alabama in Huntsville 8-10 September 1997
  13. 13. Summary A prototype Web-based dataset-independent subsetter has been developed by UAH. Allows spatial, temporal, and spectral subsetting and subsampling of HDF-EOS datasets Benefits science data users and data centers Great potential. but limited current use UAH The University of Alabama in Huntsville 8-10 September 1997

×