Status of HDF-EOS,

Related Software, and Tools
Abe Taaheri, Raytheon IIS
HDF & HDF-EOS Workshop XII
October 2008

Page 1
TOOLKIT / HDF-EOS Support
•
•

Maintenance support was funded on EMD Evolution
through May 2008
One year extension until M...
ECS Support of HDF-EOS
• Both HDF4 and HDF5 based flavors of HDF-EOS are part
of ECS baseline.

• OS’s: Solaris (9, 10), I...
Downloadable Software

• SDP/MTD Toolkit (Science Data Processing Tools)
• HDF-EOS2 (HDF4 based)
• HDF-EOS5 (HDF5 based)
•...
Downloadable Applications
• HEG
- GeoTIFF conversion
- Spatial subsetting, Field Subsetting
- reprojection
- stitching, su...
Software Releases
Toolkit/HDF-EOS:
• Last release – March 2008
– TOOLKIT 5.2.15
– HDF-EOS 2.15, HDF- EOS5.1.11
with HDF4.2...
Software Releases (Cont.)
HEG (HDF-EOS2 to GeoTIFF Converter):
• Last release – August 2008
– Version 2.9
• Next release –...
Software Releases (Cont.)
HDF-EOS Plug-in for HDFView:
• Last release – September 2008
– Version 2.4
• Next release –
– Fo...
Software Releases (Cont.)
HECONVERT (HDF-EOS2 to HDF-EOS5
Converter):
• Last release – 2001, 2004
– Version ?
• Next relea...
SDP Toolkit
•

Science Data Production (SDP) Toolkit :
collection of tools used by
- Science software developers who produ...
SDP Toolkit (cont.)
•

Tools provided in (SDP, MTD) Toolkit:
-

AA
CBP
CSC
CUC
DEM
EPH
GCT
IO
MEM
MET
PC
SMF
TD

(Ancillar...
What is HDF-EOS ?
• HDF-EOS is an extension to HDF which

standardizes storage of and access to common
Earth Sciences data...
HDF-EOS 2
• HDF4 - based, storage format for EOS standard
products.
• Used operationally by MODIS, MISR, ASTER,
Landsat, A...
HDF-EOS 5
• Based on HDF5, a complete rewrite of HDF-EOS2
with a different user interface.
– First released in 2000.
– For...
HDFView
•
•
•

•
•
•

HDFView: A Java-based browser from THG for HDF4
and HDF5 – based data.
HDF-EOS plug-in extends HDFVi...
Swath, Grid, and Point Objects

Page 16
AMSR-E and OMI Point objects

Page 17
AMSR-E Swath

New Features in Release Version 2.4
Page 18
HE2 To HE5 Conversion

New Features in Release Version 2.4
Page 19
HDF-EOS to GeoTIFF Converter (HEG)
HEG: Utility that converts EOSDIS data from
HDF-EOS format to common Geographical
Infor...
HEG Functionality
• Conversion:
– HDF-EOS (Swath or Grid ) to a single-band or multi-

•
•
•
•

band GeoTIFF’s
– HDF-EOS S...
HEG Functionality
• Metadata preservation/creation.
• Currently supports MODIS, MISR, ASTER, AIRS and AMSR-E
products on T...
Mt. Etna Eruption: MISR L1B
Stitched Blocks

Page 23
2004 Hurricane Francis, MODIS

MOD021KM, Multi-Band GeoTIFF, 3 bands:
EV_1KM_RefSB, Bands 1,2,5
Page 24
AST_L1B, 2003 California Fires

Geo. projection, Multi-Band GeoTIFF,
(3 bands: Visible Near Infrared (VNIR))

….. and Subs...
Major Development/Fixes
Last Releases
TOOLKIT/HDF-EOS:
 SZIP Insatallation: Default decoder,
Choice for encoder
 Choice ...
Major Development/Fixes (cont.)
 Support for new releases of HDF4/HDF5/SZIP
(HDF4.2r3, HDF5-1.6.7, SZIP-2.1)

 Installat...
Major Development/Fixes (cont.)
 Support for Mac Intel platform. Drop of support
for Sun5.8, Dec, HP, HP 11, IBM

 Fixes...
Major Development/Fixes (cont.)

 HDFEOSVersion string clean up in HDF-EOS5
 Added new utilities to dump lat/lon arrays ...
Major Development/Fixes (cont.)
HDF-EOS Plug-in for HDFView:
Current Support:

–
–
–

SUN, LINUX, WINDOWS,
and MAC (Power ...
Major Development/Fixes (cont.)
–
–
–
–
–

Display Lat/Lon for a selected pixel in the
data field tables
Full Mac OSX supp...
Major Development/Fixes (cont.)
HEG: Release
Spatial subsetting:
- AE-DyOcn, AE_DySno
• New Product: AMSR-E and ASTER (On-...
Major Development/Fixes (cont.)
• Stitching:
− MOD14A1 3-D fields (band selection)
− TM grids
• Multi-band geotif/HDF-EOS ...
Major Development/Fixes (cont.)
− Correction for droping a column when
stitching MODIS SIN tiles.
− Correction to ignoring...
Future Plans

TOOLKIT/HDF-EOS:
• Auto Configuration for TOOLKIT
• Full Support for Cygwin platform
• Support for HDF5-1.8
...
Future Plans (cont.)

HDF-EOS Plug-in for HDFView:
• More functionalities :
- Ability to modify input HDF-EOS2 granules.
-...
Availability
• Access to libraries, applications, Documentation:
TOOLKIT:
http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit...
Upcoming SlideShare
Loading in …5
×

Status of HDF-EOS, Related Software, and Tools

238 views

Published on

Current status of HDF-EOS and access tools will be summarized. Update on HDF-EOS, HDFView plug-in and The HDF-EOS to GeoTIFF (HEG) conversion tool, including recent changes to the software, ongoing maintenance, upcoming releases, future plans, and issues will be discussed.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
238
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Status of HDF-EOS, Related Software, and Tools

  1. 1. Status of HDF-EOS, Related Software, and Tools Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshop XII October 2008 Page 1
  2. 2. TOOLKIT / HDF-EOS Support • • Maintenance support was funded on EMD Evolution through May 2008 One year extension until May 2009 • Current support       HDF-EOS2 & 5 SDP/MTD Toolkit HDF-EOS plug-in for HDFView HDF-EOS2 to GeoTIFF Converter (HEG) Other Toolkit/HDF-EOS related software Toolkit/HDF-EOS/HEG helpdesk Page 2
  3. 3. ECS Support of HDF-EOS • Both HDF4 and HDF5 based flavors of HDF-EOS are part of ECS baseline. • OS’s: Solaris (9, 10), Irix6.5 (64, n32), Linux (including 64-bit Opteron and Itanium), Mac OS X (Power PC and Mac Intel), Windows XP(MTD Toolkit, HDF-EOS2 & 5, HEG, HDFView) (Dropped support for Solaris 8, HP, HP11, Dec, IBM in 2.15 release) • Planned full Cygwin support for late 2008 • Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++, Intel Fortran (SGI Altix), gfortran (Mac Intel) Page 3
  4. 4. Downloadable Software • SDP/MTD Toolkit (Science Data Processing Tools) • HDF-EOS2 (HDF4 based) • HDF-EOS5 (HDF5 based) • heconvert (converts HDF4 - based Grid/Point/Swath structures to HDF-EOS 5 equivalents) Page 4
  5. 5. Downloadable Applications • HEG - GeoTIFF conversion - Spatial subsetting, Field Subsetting - reprojection - stitching, sub-sampling and more …. • HDFView Plug-in, Java-based browser (HDF4, HDF5, HDF-EOS 2 and 5 access) • HE5View* (HDF5 - based browser) • EOSView* (HDF4 - based browser) * Dropped support for OS updates on July 2006 Page 5
  6. 6. Software Releases Toolkit/HDF-EOS: • Last release – March 2008 – TOOLKIT 5.2.15 – HDF-EOS 2.15, HDF- EOS5.1.11 with HDF4.2r3, HDF5-1.6.7, Szip2.1 • Next release – March 2009 – TOOLKIT 5.2.16 – HDF-EOS 2.16, HDF-EOS5.1.12 with new releases of HDF4, 5 (possibly 1.8.X) Page 6
  7. 7. Software Releases (Cont.) HEG (HDF-EOS2 to GeoTIFF Converter): • Last release – August 2008 – Version 2.9 • Next release – May 2009 – Version 2.10 Page 7
  8. 8. Software Releases (Cont.) HDF-EOS Plug-in for HDFView: • Last release – September 2008 – Version 2.4 • Next release – – For HDFView Version 2.5 Page 8
  9. 9. Software Releases (Cont.) HECONVERT (HDF-EOS2 to HDF-EOS5 Converter): • Last release – 2001, 2004 – Version ? • Next release – Dec. 2008 – Version ? Page 9
  10. 10. SDP Toolkit • Science Data Production (SDP) Toolkit : collection of tools used by - Science software developers who produce code to process instrument data. - HDF-EOS users who need to • Access metadata in HDF-EOS files • Perform Time/Date conversion • Access Digital Elevation Model Data • Access Ancillary Data, such as Digital Chart of the World database (DCW), Olson World Ecosystem files, DEM data files, etc. Page 10
  11. 11. SDP Toolkit (cont.) • Tools provided in (SDP, MTD) Toolkit: - AA CBP CSC CUC DEM EPH GCT IO MEM MET PC SMF TD (Ancillary Data Access) (CELESTIAL Body Position) (Coordinate System Conversion) (Constant and Unit Conversions) (Digital Elevation Model Access) (Ephemeris Data Access) (Geo Coordinate Transformation) (Input Output (File I/O)) (Memory Management) (Metadata Access) (Process Control) (Status Message File (Error/Status)) (Time and Date Conversion) Page 11
  12. 12. What is HDF-EOS ? • HDF-EOS is an extension to HDF which standardizes storage of and access to common Earth Sciences data structures – By providing a single interface to data structures common to earth science – By providing a container for EOS inventory, archive and product specific metadata – By establishing a consistent relationship between geolocation and science data. Page 12
  13. 13. HDF-EOS 2 • HDF4 - based, storage format for EOS standard products. • Used operationally by MODIS, MISR, ASTER, Landsat, AIRS and other EOS instruments • Support for Grid/Point/Swath structures • HDF4 and HDF-EOS 2 will be supported by THG/NASA for the fore-seeable future. Page 13
  14. 14. HDF-EOS 5 • Based on HDF5, a complete rewrite of HDF-EOS2 with a different user interface. – First released in 2000. – Format used by EOS Aura instruments • Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible. – Supports same data structures – Added prefix ‘HE5_’ to HDF-EOS 2 functions. Page 14
  15. 15. HDFView • • • • • • HDFView: A Java-based browser from THG for HDF4 and HDF5 – based data. HDF-EOS plug-in extends HDFView to browse HDFEOS data in both versions. The extension is a plug-in module and does not change THG HDFView functionality. All copy, save, editing, image display, etc. for HDF4 and 5 objects remains. Identical look and feel when displaying HDF and HDF-EOS objects. Users can access a complete breakdown of HDFEOS objects. e.g. grid information, projection info., dimensions, datafields, and attributes are displayed. Current Support: SUN, SGI, LINUX, Mac, WINDOWS platforms Page 15
  16. 16. Swath, Grid, and Point Objects Page 16
  17. 17. AMSR-E and OMI Point objects Page 17
  18. 18. AMSR-E Swath New Features in Release Version 2.4 Page 18
  19. 19. HE2 To HE5 Conversion New Features in Release Version 2.4 Page 19
  20. 20. HDF-EOS to GeoTIFF Converter (HEG) HEG: Utility that converts EOSDIS data from HDF-EOS format to common Geographical Information System (GIS)-compatible formats. • Versions Available: – Stand-alone: A downloadable desktop version – Data Pool: Access through NASA archive online storage – Implemented also in prototypes for Stand-alone Data Pool, OGC (Open Geospatial Consortium) for Web Services Chaining, and DOWS (Deploy OGC Web Services) on the DAACs Page 20
  21. 21. HEG Functionality • Conversion: – HDF-EOS (Swath or Grid ) to a single-band or multi- • • • • band GeoTIFF’s – HDF-EOS Swath to HDF-EOS Grid or Swath – HDF-EOS Swath or Grid to generic Binary (with metadata file) Subsetting (spatial, field, band) Stitching (mosaicing) + subsetting + Reprojecting Reprojection ( to UTM, PS, TM, STP, LCC, LAMAZ, GEO, SIN, Albers) Subsampling (Subsample stacks) • Running from Command Line (batch jobs) or with Java-based GUI Page 21
  22. 22. HEG Functionality • Metadata preservation/creation. • Currently supports MODIS, MISR, ASTER, AIRS and AMSR-E products on TERRA and AQUA (>115 products), • Allows file selection from users local storage • Operable on Sun, SGI, Win, Linux, MAC • Integrated into ECS Data Pool – Reduces the transfer time of HDF-EOS data sets (if subset is requested) – Provides the end-user with the exact file required by their application • User Interface – Portable, written in Java. – Not dependent on COTS (eg. IDL). Page 22
  23. 23. Mt. Etna Eruption: MISR L1B Stitched Blocks Page 23
  24. 24. 2004 Hurricane Francis, MODIS MOD021KM, Multi-Band GeoTIFF, 3 bands: EV_1KM_RefSB, Bands 1,2,5 Page 24
  25. 25. AST_L1B, 2003 California Fires Geo. projection, Multi-Band GeoTIFF, (3 bands: Visible Near Infrared (VNIR)) ….. and Subsetted Two Swaths Stitched ….. Page 25
  26. 26. Major Development/Fixes Last Releases TOOLKIT/HDF-EOS:  SZIP Insatallation: Default decoder, Choice for encoder  Choice for configuring HDF4 with/without SZIP  Choice for disabling HDF4’s internal netCDF  Choice for installing Toolkit/HDF-EOS in 32 or 64-bit modes in 64-bit Linux platforms (except for Itanium; no –m32 flag) Page 26
  27. 27. Major Development/Fixes (cont.)  Support for new releases of HDF4/HDF5/SZIP (HDF4.2r3, HDF5-1.6.7, SZIP-2.1)  Installation fix for a problem caused by limits.h in some Linux platforms  Auto Configuration capability (like HDF) added for both HDF-EOS2 and HDF-EOS5 Page 27
  28. 28. Major Development/Fixes (cont.)  Support for Mac Intel platform. Drop of support for Sun5.8, Dec, HP, HP 11, IBM  Fixes for Lat/Lon to pixel conversions in DEM tools for Linux to get results similar to those in Unix platforms Add GDij2ll and GDll2ij APIs to HDF-EOS and HDF-EOS5  Modified accuracy check for quaternion in CSC tools Page 28
  29. 29. Major Development/Fixes (cont.)  HDFEOSVersion string clean up in HDF-EOS5  Added new utilities to dump lat/lon arrays for grids in HDF-EOS and HDF-EOS5  Fixed several other bugs for Grid. Swath and Point objects (complete list in release notes) Page 29
  30. 30. Major Development/Fixes (cont.) HDF-EOS Plug-in for HDFView: Current Support: – – – SUN, LINUX, WINDOWS, and MAC (Power and Intel) Auto installation of plug-in into HDFView running self-extracting archives. Adding to plug-in the fixes and enhancement that went into HDFView2.4 Display image for selected pixels in the viewed tables Page 30
  31. 31. Major Development/Fixes (cont.) – – – – – Display Lat/Lon for a selected pixel in the data field tables Full Mac OSX support Improved Help Window New icons for activating Lat/Lon display for Swath & Grid Format conversion: HDF-EOS2 -> HDF-EOS5 Page 31
  32. 32. Major Development/Fixes (cont.) HEG: Release Spatial subsetting: - AE-DyOcn, AE_DySno • New Product: AMSR-E and ASTER (On-Demand L2) - AMSR-E 5 day Snow - AMSR-E Monthly Snow - AMSR-E Weekly Ocean - AMSR-E Monthly Ocean - AMSR-E Daily 12 Km Sea Ice - AMSR-E Daily 6 Km Sea Ice - AMSR-E L2 Rain, AMSR-E Rain Grid - ASTR_07XT, ASTR_09XT Page 32
  33. 33. Major Development/Fixes (cont.) • Stitching: − MOD14A1 3-D fields (band selection) − TM grids • Multi-band geotif/HDF-EOS in Datapool • Other Fixes: − Interpolation correction with AIRS products containing fillvalue in the data fields − Pixel size unit consistency in conversion and stitch GUIs. Page 33
  34. 34. Major Development/Fixes (cont.) − Correction for droping a column when stitching MODIS SIN tiles. − Correction to ignoring of Geographic pixel size and processing with default pixel size − Spatial subsetting fix for global AMSR-E garanules − Correction for ignoring re-projection interpolation type NN after stitching Page 34
  35. 35. Future Plans TOOLKIT/HDF-EOS: • Auto Configuration for TOOLKIT • Full Support for Cygwin platform • Support for HDF5-1.8 • Validation tool for HDF-EOS5 files • Performance improvement – reading HDF format eph/att files in Toolkit – HDF-EOS5 by redesigning some functions Page 35
  36. 36. Future Plans (cont.) HDF-EOS Plug-in for HDFView: • More functionalities : - Ability to modify input HDF-EOS2 granules. - Ability to cut/paste objects, modify/delete attributes. - Ability to create new grid/swath HEG: - Support for AMSR-E Level 3 Land (CEA projection) - New MRT (MODIS Reprojection Tool) release updates Page 36
  37. 37. Availability • Access to libraries, applications, Documentation: TOOLKIT: http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html HDFView: http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/ HDFView_hdfeos_plugin.html HEG: http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html • Email – Abe_Taaheri@raytheon.com – Landover_PGSTLKIT@raytheon.com Page 37

×