The document outlines the maintenance and support of HDF-EOS tools and software, detailing staff roles, current and upcoming releases, and software features. It highlights the functionality of various applications like the science data production toolkit, HDF-EOS, and HEG for converting HDF-EOS data into GIS-compatible formats. Future development plans include enhancements for compatibility and performance improvements, particularly in support of new HDF versions and systems.

1. HDF-EOS Maintenance,
Current Development
and Tools
Abe Taaheri, Raytheon Information Systems
Larry Klein, RS Information Systems
HDF-EOS Workshop X, November 2006

2. HDF-EOS Support
• Maintenance support is funded on EMD Evolution
through May 2008
• Current staff
- One full time developer for maintaining:
SDP/MTD Toolkit,
HDF-EOS2 & 5,
HDF-EOS plug-in for HDFView,
HEG,
other Toolkit/HDF-EOS related software/maintenance:
- Users Guides
- Web sites
- User help
- Training another developer for HEG maintenance
Page 2

3. ECS Support of HDF-EOS
• Both HDF4 and HDF5 based flavors of HDF-EOS are part
of ECS baseline.
• OS's: Solaris (8, 9, 10), Irix6.5, HP 11, AIX, DEC,
Windows NT/2000/XP, Linux (including 64-bit Opteron
and Itanium), Mac OS X, Cygwin (MTD Toolkit, HDF-EOS2 & 5)
• Adding Cygwin support for SDP Toolkit
• Possibly supporting Intel based Mac for next release
• Compilers: Fortran 77/90 & g77/pgf90 , C, C++, gcc, g++
• Support for SGI 7.4.2 besides SGI 7.3.1
Page 3

4. Software Releases
SDP Toolkit/HDF-EOS:
• Last release – March 2006
TOOLKIT 5.2.14, HDF-EOS 2.14, HDF-EOS5.1.10
with HDF4.2r1, HDF5-1.6.5-post5, Szip2.0
• Next release – Early 2007
TOOLKIT 5.2.15, HDF-EOS 2.15, HDF-EOS5.1.11
with new releases of HDF4, and HDF5
MTD Toolkit: Last Release – Nov. 2006
HDF-EOS Plug-in for HDFView:
• Last release – May 2006
– Version 2.3
• Next release – ?
– After HDFView2.4 release by The HDF Group possibly in early 2007
HEG:
• Last release – March 2006
– Version 2.7
• Next release – Early 2007
Page 4

5. Downloadable Applications
• HE5View* (HDF5 - based browser)
• EOSView* (HDF4 - based browser)
• HDFView Plug-in, Java-based browser (HDF4,
HDF5, HDF-EOS 2 and 5 access)
• heconvert (converts HDF4 - based Grid/Point/Swath
structures to HDF-EOS 5 equivalents)
– Compliments HDF4 -> HDF5 conversion tool
– Not tested on all products
• HEG (HDF-EOS2 to GeoTIFF converter, subsetting,
reprojection, stitching, sub-sampling, etc.)
* Dropped support for OS updates since July 2006
Page 5

6. SDP/MTD Toolkit
Science Data Production (SDP) Toolkit is a collection of
tools used by
- Science software developers who produce
code to process instrument data.
- HDF-EOS users who need to
• Access metadata in HDF-EOS files
• Perform Time/Date conversion
• Access Digital Elevation Model Data
• Access Ancillary Data, such as Digital Chart of
the World database (DCW), Olson World
Ecosystem files, DEM data files, etc.
Page 6

7. SDP Toolkit (cont.)
•
Tools provided in (SDP, MTD) Toolkit:
- AA
(Ancillary Data Access)
- CBP
(CELESTIAL Body Position)
- CSC
(Coordinate System Conversion)
- CUC
(Constant and Unit Conversions)
- DEM
(Digital Elevation Model Access)
- EPH
(Ephemeris Data Access)
- GCT
(Geo Coordinate Transformation)
- IO
(Input Output (File I/O))
- MEM
(Memory Management)
- MET
(Metadata Access)
- PC
(Process Control)
- SMF
(Status Message File (Error/Status))
- TD
(Time Date Conversion)
Page 7

8. What is HDF-EOS ?
• HDF-EOS is an extension to HDF which
standardizes storage of and access to common
Earth Sciences data structures
–By providing a single interface to data structures
common to earth science
–By providing a container for EOS inventory,
archive and product specific metadata
–By establishing a consistent relationship
between geolocation and science data.
Page 8

9. HDF-EOS 2
• HDF4 - based, storage format for EOS standard
products.
• Used operationally by MODIS, MISR, ASTER,
Landsat, AIRS and other EOS instruments
• Support for Grid/Point/Swath structures
• HDF4 and HDF-EOS 2 will be supported by
THG/NASA for the fore-seeable future.
Page 9

10. HDF-EOS 5
• Based on HDF5, a complete rewrite of HDF-EOS2
with a different user interface.
–First released in 2000.
–Format used by EOS Aura instruments
• Designed to ‘resemble’ HDF-EOS 2 to the
maximum extent possible.
–Supports same data structures
–Added prefix ‘HE5_’ to HDF-EOS 2 functions.
Page 10

11. HDF-EOS Grid Structure
• Usage - Data which is organized
by regular geographic spacing,
specified by projection parameters.
• Structure
– Any number of 2-D to 8-D data arrays per structure
– Geolocation information contained in projection formula,
coupled by structural metadata.
– Any number of Grid structures per file allowed.
Page 11

12. HDF-EOS Swath Structure
• Data which is organized by time, or
other track parameter.
• Spacing can be irregular.
• Structure
– Geolocation information stored
explicitly in Geolocation
Field (2-D array)
– Data stored in 2-D or 3-D arrays
– Time stored in 1-D or 2-D array,
TAI is standard.
– Geolocation/science data
connected by structural metadata
Page 12

13. HDF-EOS Point Structure
• Data is specified temporally and/or spatially, but
with no particular organization
• Structure
–Tables used to store science
data at a particular
Lat/Long/Height
– Up to eight levels of
Station
Chicago
Los Angeles
Washington
Miami
data allowed. Structural
metadata specifies
relationship between levels.
Lat
Lon
41.49 -87.37
34.03 -118.14
38.50 -77.00
25.45 -80.11
Time
0800
0900
1000
0800
0900
1000
1100
1000
1100
1200
1300
1400
0600
0700
Temp(C)
-3
-2
-1
20
21
22
24
6
8
9
11
12
15
16
Page 13

14. Zonal Average Structure
• Generalized array structure with no geolocation
linkage
• Data can be organized by time or track
parameter
• Data spacing can be irregular
• Structure
– Data stored in multidimensional arrays
– Time stored in 1-D or 2-D array
Page 14

15. Major Development/Fixes
Last Release:
HDF-EOS5:
 POINT object fixes of:
-
-
linkage between levels
updating levels
writing Forward and Backward Linkage data
char field inquiry
 Added a routine to return list of aliases for a given field
 Used HDF5’s H5T_C_S1 datatype for string type
 Added a few more Fortran wrappers
 Modified symbol “H5E_NONE_MAJOR” to other symbols
for better description of errors
 Ported to Solaris 9, Solaris 10, and Cygwin
 C++ fixes for SGI 7.4.2 compiler support
Page 15

16. Major Development/Fixes (cont.)
 Fixed subsetting problem in HDF-EOS5 Grid, resulting
from faulty conversion of DMS degree to Decimal degree.
 Setting LD_LIBRARY_PATH for SZIP/HDF5 shared
libraries in installation and environment setting scripts
TOOLKIT/HDF-EOS2:
 Ported to Solaris 9 and Solaris 10
 Keeping more significant digits after the decimal point in
projection parameters if needed.
 chkeph utility fixes for reading binary ephemeris files
(Problem in reading EPH files with different endiannes than the machine )
 SGI 7.4.2 compiler support
 orbsim fixes for Linux to create files similar to those in
UNIX platforms
Page 16

17. Future Development/Fixes
Next Release:
 Default installation of SZIP
 Choice for installing Toolkit/HDF-EOS in 32 or 64 bit
modes in 64-bit Linux platforms
 New HDF4/HDF5 Release support
 Lat/Lon conversions to pixel number in DEM tools in
Linux for getting results similar to those in Unix platforms
Future Plans:
− Support for Cygwin
− Support for HDF4.2r2 and HDF5-1.8
− Performance improvement, if possible, for reading HDF
format ephemeris/attitude files in Toolkit
− Performance improvement in HDF-EOS5 by redesigning
some functions
Page 17

18. HDFView
•
•
•
•
•
•
HDFView: A Java-based browser from THG for HDF4
and HDF5 – based data.
HDF-EOS plug-in extends HDFView to browse HDFEOS data in both versions.
The extension is a plug-in module and does not
change THG HDFView functionality. All copy, save,
editing, image display, etc. for HDF4 and 5 objects
remains.
Identical look and feel when displaying HDF and
HDF-EOS objects.
Users can access a complete breakdown of HDFEOS objects. e.g. grid information, projection info.,
dimensions, datafields, and attributes are displayed.
Current Support: SUN, SGI, LINUX, WINDOWS
platforms
Page 18

19. HDFView (Cont.)
• Release 2.2 October 2005
- A separate HDF-EOS plug-in: Users install HDFView 2.2 and
integrates HDF-EOS plug-in into it.
- Ability to do "Save as" for HDF-EOS5:
modify individual values in grid or swath data.
Delete entire grids or swaths.
Save the result.
- Separate help window for HDF-EOS objects
• Release 2.3 January 2006
− Display Point Objects
− Scientific notation in Table data
− Geolocating pixels for swath and grid images
Page 19

20. HDFView (Cont.)
• Next Release: Version 2.4
– Automatic installation of plug-in into HDFView running selfextracting archives.
– Display image for selected pixels in the viewed tables
– Geolocating cells in the data field tables for swath and grid
– Mac OSX porting
– Better display of Plug-in User’s Guide
– New icons for activating Lat/Lon display for Swath & Grid
- Future Plans:
- Ability to modify input HDF-EOS2 granules.
- Ability to cut/paste objects, modify/delete attributes.
- Ability to convert HDF-EOS2 to HDF-EOS5
- Ability to create new grids/swaths
Page 20

21. Swath, Grid, and Point Objects
Page 21

22. AMSR-E Grid, Northern Hemisphere,
with selected palette
Page 22

23. AMSR-E and OMI Point objects
Page 23

24. AMSR-E Swath
New Features in Next Release Version 2.4
Page 24

25. Aster Swath, with Image menu
Page 25

26. HDF-EOS to GeoTIFF Converter (HEG)
HEG: Utility that converts EOSDIS data from
HDF-EOS format to common Geographical
Information System (GIS)-compatible formats.
• Two versions available:
– A downloadable desktop version
– Access through NASA archive online storage (Data Pools)
– Implemented also in prototypes for Stand-alone Data Pool,
OGC (Open Geospatial Consortium) for Web Services
Chaining, and DOWS (Deploy OGC Web Services) on the
DAACs
Page 26

27. HEG Functionality
• Conversion:
– HDF-EOS to a single-band or multi-band GeoTIFF’s
– HDF-EOS Swath to HDF-EOS Grid and Swath
– HDF-EOS Swath or Grid to GeoTIFF
– HDF-EOS Swath or Grid to generic Binary (with
•
•
•
•
metadata file)
Subsetting (spatial, field, band)
Stitching (mosaicing) + subsetting + Reprojecting
Reprojection ( to UTM, PS, TM, STP, LCC, LAMAZ, GEO, SIN)
Subsampling (Subsample stacks)
• Running from Command Line (batch jobs)
or with Java-based GUI
Page 27

28. HEG Functionality
• Metadata preservation/creation.
• Currently supports MODIS, MISR, ASTER, AIRS and AMSRE products on TERRA and AQUA (>110 products),
• Allows file selection from users local storage
• Operable on Sun, SGI, Win, Linux, MAC
• Integrated into ECS Data Pool
– Reduces the transfer time of HDF-EOS data sets (if subset is
requested)
– Provides the end-user with the exact file required by their application
• User Interface
– Portable, written in Java.
– Not dependent on COTS (eg. IDL).
Page 28

29. HEG Releases
– Support for:
AMSR-E L3 GRID products (in Polar Stereographic and EASE-Grid)
AMSR-E L3 GRID products (in Geographic projection)
MISR L3 products (in Geographic projection).
AMSR-E L2 SWATH products
AIRS L3 products (in Geographic projection)
ASTER DEM products: Correction for missing data in the internal
geolocation fields of the ASTER DEMs.
MOD43C1, MOD43C2, MOD43C3, MCD43C1, MCD43C2, MCD43C3,
MOD12C1, MYD12C1: Corner points of GRID in DD instead of DMS units.
MOD11_L2: fillvalues (-999.0) in the Lat/Lon geolocations while (-999.9) in
internal metadata.
– Added Albers Projection
– New "Subsetter" functionality for Parameters (aka Fields). This results in:
• Output to HDF-EOS SWATH from input HDF-EOS SWATH
• No alteration of input data because of projection operation.
Page 29

30. Mt. Etna Eruption: MISR L1B
Swaths: Stitched
Page 30

31. MODIS L1B: La Plata Tornado
Reprojected, Co-Registered with ASTER
Page 31

32. 2004 Hurricane Francis, MODIS
MOD021KM, Multi-Band GeoTIFF, 3 bands:
EV_1KM_RefSB, Bands 1,2,5
Page 32

33. AST_L1B, 2003 California Fires
Geo. projection, Multi-Band GeoTIFF,
(3 bands: Visibile Near Infrared (VNIR))
….. and Subsetted
Two Swaths Stitched …..
Page 33

34. MODIS Land Products: Reprojected,
Co-registered, Processed in ArcInfo/ArcMap
Vegetation Index
Leaf Area Index
Surface Reflectance
Page 34

35. Availability
• Access to libraries and applications:
TOOLKIT:
http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html
HDFView:
http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/HDFView_hdfeos_plugin.html
HEG:
http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html
• Email
– Abe_Taaheri@raytheon.com
– Landover_PGSTLKIT@raytheon.com
Page 35