HDF OPeNDAP update

712 views

Published on

HDF and HDF-EOS Workshop XVII (2014)
Kent Yang

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
712
On SlideShare
0
From Embeds
0
Number of Embeds
23
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • MOD08_M3, AIRS version 6 grid
  • Ncdump to dump the file
  • The Earth Science Group is a subgroup within The HDF Group formed to concentrate on issues like these. We’re excited by the opportunity to join in the conversation and help form the emerging landscape in Earth Sciences software, data and metadata conventions and their uses in current and upcoming missions.
  • HDF OPeNDAP update

    1. 1. www.hdfgroup.org The HDF Group ESIP Summer Meeting HDF OPeNDAP update Kent Yang The HDF Group 1July 8 – 11, 2014
    2. 2. www.hdfgroup.orgESIP Summer MeetingJuly 8 – 11, 2014 2 Why OPeNDAP? • Check metadata remotely (in various forms) • Obtain the subset of data easily and efficiently • Hide the original data sources • netCDF, HDF4, HDF5 or GeoTiFF, GRIB • Many popular earth science tools can visualize and analyze the data via OPeNDAP • OPeNDAP output(including subsets) can be downloaded as other formats
    3. 3. www.hdfgroup.orgESIP Summer MeetingJuly 8 – 11, 2014 3 • The visualization of HDF(5) data by OPeNDAP • What users request the most What makes HDF OPeNDAP support special?
    4. 4. www.hdfgroup.orgESIP Summer Meeting HDF Handlers July 8 – 11, 2014 4 Access HDF via OPeNDAP Users Clients Servers HDF Files AIRS/OMI DAP HDF Handlers(CF) Aqua/Aura
    5. 5. www.hdfgroup.orgESIP Summer Meeting HDF4 handler update • Add TRMM 7 support • level 1 and level 2 swath • Level 3 grid • Improve the performance to handle AIRS version 6 grid and MOD08_M3 products • Improve the performance to handle MODIS products with no-CF scale and offset rules • Reduce the number of file open/close calls when using file netCDF module to access HDF data • Map HDF4 char array to DAP String July 8 – 11, 2014 5
    6. 6. www.hdfgroup.orgESIP Summer Meeting MOD08_M3 and AIRS version 6 Performance • Disable the generation of StructMetadata as a DAP attribute for the CF option • Use the special features in these products to efficiently build DDS and DAS July 8 – 11, 2014 6
    7. 7. www.hdfgroup.orgESIP Summer Meeting TRMM 7 Demo – swath(level 1B) July 8 – 11, 2014 7
    8. 8. www.hdfgroup.orgESIP Summer Meeting TRMM 7 Demo – swath(level 2A25) July 8 – 11, 2014 8
    9. 9. www.hdfgroup.orgESIP Summer Meeting TRMM 7 Demo multi-grid(level 3A25) July 8 – 11, 2014 9
    10. 10. www.hdfgroup.orgESIP Summer Meeting TRMM 7 Demo grid(level 3B43) July 8 – 11, 2014 10
    11. 11. www.hdfgroup.orgESIP Summer Meeting HDF5 handler update • Support GPM level-1 products • Make the products follow CF • Reduce the number of file open/close calls when using file netCDF module to access HDF data July 8 – 11, 2014 11
    12. 12. www.hdfgroup.orgESIP Summer Meeting GPM HDF5 structure July 8 – 11, 2014 12
    13. 13. www.hdfgroup.orgESIP Summer Meeting HDF5 handler demo: GPM level 1A July 8 – 11, 2014 13
    14. 14. www.hdfgroup.orgESIP Summer Meeting HDF5 handler demo: GPM level 1B July 8 – 11, 2014 14
    15. 15. www.hdfgroup.orgESIP Summer Meeting Hyrax file-netCDF module • Can work with HDF handlers to convert HDF files to netCDF-3 or netCDF-4 classic files that follow the CF conventions • We also help debug and provide fixes for this module to help NASA July 8 – 11, 2014 15
    16. 16. www.hdfgroup.orgESIP Summer Meeting File netCDF module demo • Can use besstandalone program • End users can install Hyrax and use this program to convert HDF4 and HDF5 files to netCDF3 or netCDF4 16
    17. 17. www.hdfgroup.orgESIP Summer MeetingJuly 8 – 11, 2014 17 More challenges • CF conventions evolve and tools evolve • New versions of existing HDF products and new HDF products may require significant updates • The data aggregation service requires decent performance of the data service per file
    18. 18. www.hdfgroup.orgESIP Summer Meeting Future - Performance • Reduce the DDS and DAS access time • Provide a caching mechanism in the handler level July 8 – 11, 2014 18
    19. 19. www.hdfgroup.org The HDF Group ESIP Summer Meeting Earth Science Group Ted Habermann Aleksandar Jelenak H. Joe Lee Joel Plutchak John Readey Kent Yang 19July 8 – 11, 2014

    ×