1. nub - npoess user block tool
Richard Ullman
NASA GSFC

2. nub
• nub (nŭb) n.
– 1. A protuberance or knob.
– 2. A small lump.
– 3. The essence; the core: the nub of a story.
– [Variant of knub, probably from Low German knubbe, from
Middle Low German, variant of knobbe.]
• (selected from http://www.answers.com/topic/nub)
2

3. NPOESS user block
• For NPOESS files, the user block is ASCII text written in XML
that provides a quick-look into the metadata attributes contained in
the HDF5 file.
– The content of the user block includes metadata that is attached
to the NPOESS:
• / (root) group
• <collection_shortname> group, and
• <collection_shortname>_Aggr group.
3

4. nub_Motivation
• When evaluating NPOESS sample data last year, I wished to
examine the user block, as a stand-alone item, but I found that the
h5unjam tool to be expensive
– h5unjam reads an HDF5 file and writes two output files
• 1. a file containing the content of the user block
• 2. a file containing the H5 data without the user block.
– The second part was unnecessary for my purposes, but because
the NPOESS files are large, it is expensive.
• So .. I wrote nub …
4

5. nub_Design
• Design criteria:
– “c” program (that’s all I know how to write)
– Efficient - do no ‘EXTRA’ work.
– Command line operation, with Unix-like options.
– Pipe-able. (stdin/stdout)
– Assume user block is text.
– Work only on the front user block.
– Complete set of functions to manipulate the user block:
• List, Create, Overwrite, Append, Delete, Get size
• Requirements creep …
– Complete replacement for h5jam/h5unjam/tellub
– No reliance on h5 library (no need).
• Availability:
– Applied for NASA Open Source release. All paperwork completed.
Waiting for final approval.
– Evaluation copy available to anyone interested in NPOESS.
5

6. nub operates on the front user block
ONLY
signature: “211HDFrn32n”
user block
user block
HDF5
HDF5
sizeof( user_block) = 2^n, where n>8, (or n=0)
6

7. nub file.h5
signature: “211HDFrn32n”
content,content/0/0/0/0/0/0/0/0
content,content/0/0/0/0/0/0/0/0
--tell -b
HDF5
HDF5
--tell -s
--tell -a
• Tell: name comes from HDF5 internal tool “tellub”
• This is the nub default operation.
– Count the size of the user block content [-b], the size of the user block
[-s], and the size of the complete H5 file [-a] in bytes.
– No options given is equivalent to: nub --tell -bsa file.h5
– Output is similar to wc
7

8. nub -l file.h5
signature: “211HDFrn32n”
file.txt
file.txt
content,content/0/0/0/0/0/0/0/0
content,content/0/0/0/0/0/0/0/0
HDF5
HDF5
• List: list the content of the user block
write to stdout, or if -f write to the given file
8

9. nub -w file.h5
content,content/0/0/0/0/0/0/0/0
content,content/0/0/0/0/0/0/0/0
signature: “211HDFrn32n”
user_file.txt
user_file.txt
HDF5
HDF5
• Write: write text content to the user block
– append by default, or clobber (replace)
– fit into existing space (by overwriting only null chars) by default, or
“jam” (extend the block)
– “kram” option shrinks user block into smallest that will fit the content
– read from stdin, or use the file specified on the -u switch
9

10. nub_ compatibility
• nub will respond to the same command line options that are used
with h5jam and h5unjam.
• “nub -J” is an explicit substitute for h5jam
• “nub -U” is an explicit substitute for h5unjam
• “nub -T” is an explicit substitute for tellub
• Or, use just “nub” as a substitute for either and the code will decide
which of the two is most appropriate based on the flags provided.
10

11. nub_test
• Results of nub as a substitute for h5jam, h5unjam in the standard
hdf5 tools regression tests …
• 68 PASSED
• 6 FAILED
– fails are due to difference in concept of concatenation.
– nub appends to the end of the content of an existing user block.
– h5jam appends to the end of the existing user block.
– User blocks are of size=2^n, where size > 256 and is big enough
to fit the content. So, if user block content is 513 bytes, the size
of the user block is 1024 bytes, including 511 bytes of NULL
padding.
– nub will overwrite the NULL padding, h5jam leaves the padding
in place.
11