Accessibility and usability of NPP/NPOESS data in HDF5 can be enhanced by providing tools that simplify and standardize how data is accessed and presented. In this project, The HDF Group is creating such tools in the form of software to read and write certain key data types and data aggregates used in NPP/NPOESS data products, and extending HDFView to extract, present and export these data effectively. In particular, the work will focus on NPP/NPOESS use of HDF5 region references and quality flags. The HDF Group will also provide high quality user support for the project.

1. Support for NPP/NPOESS
by The HDF Group
Mike Folk
The HDF Group
HDF and HDF-EOS Workshop XII
October 17, 2008
Oct. 17 2008
HDF and HDF-EOS Workshop XII
1

2. Goal
To develop and adapt HDF5
software and provide HDF5
support for NPP/NPOESS.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
2

3. NPOESS Priorities and HDF5
•
•
•
•
•
Data calibration/validation
Data accessibility and usability
Tool development
System maintenance
User support
Oct. 17 2008
HDF and HDF-EOS Workshop XII
3

4. Areas of immediate need
1. Ensure easy and intuitive access and use by
diverse communities
2. Enable producers and consumers to view
content, manage metadata, and convert data to
other formats.
3. Provide high-quality, rapid-response support of
HDF5 for NPOESS users at all levels.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
4

5. 1. Ensure easy and intuitive
data access and use by
diverse communities
Oct. 17 2008
HDF and HDF-EOS Workshop XII
5

6. Region references
Root - /
Product Group
Agg
Reference
Object
Data Array
Gran N
Root -/
Reference
Region
Reference
Region
Oct. 17 2008
HDF and HDF-EOS Workshop XII
6

7. New routines to facilitate data access
• Access data pointed to by region references
Retrieving data pointed to by region references
requires a series of complex function calls.
New API would encapsulate these into a few routines.
• Rewrite region references in datasets
When a dataset with region references is moved from
one file to another, the references of the datasets are
invalid in the new file.
The process of updating reference values is complex
and error prone.
New functions can address both of these concerns.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
7

8. • Access individual quality flags.
NPOESS quality flags are packed into an integer.
In HDF5, integer types are “atomic” – any I/O
operation access an entire integer.
To access individual NPOESS quality flags in a
convenient and consistent manner, routines are
needed that can unpack and return individual
quality flags represented by bits packed in an
HDF5 integer.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
8

9. 2. Enable producers and
consumers to view content,
manage metadata, and convert
data to other formats.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
9

10. Tools enhancements
• There is interest in tools to view and export
packed quality flag values, as well as data
pointed to by region references.
• Tools of particular interest:
h5dump – a command line utility for dumping
information from an HDF5 file.
HDFView – a graphical tool for displaying and
editing the contents of HDF5 files.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
10

11. Tool enhancements
• H5dump modifications
Extract, display, and export a subset of packed
bits corresponding to NPOESS quality flags.
Extract, display, and export data pointed to by
region references.
• HDFView modifications
Display a rectangular region of data in a dataset
corresponding to set of region references.
Display packed bits corresponding to the
NPOESS quality flags.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
11

12. 3. Provide high-quality, rapidresponse support of HDF5 for
NPOESS users at all levels.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
12

13. Support enhancements
• Assign high priority to helpdesk requests from
the NPOESS user community, and ensure a one
business-day response to all queries known to
be NPOESS-related.
• Support HDF5 on AIX 5.2.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
13

14. Thank You!
This report is based upon work to be supported by a
Cooperative Agreement with NASA under NASA contract
NNG08HZ14C. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of the
National Aeronautics and Space Administration.
Oct. 17 2008
HDF and HDF-EOS Workshop XII
14

15. Questions/comments?
Oct. 17 2008
HDF and HDF-EOS Workshop XII
15