HDF5 Tools Updates

290 views
205 views

Published on

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
290
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

HDF5 Tools Updates

  1. 1. The HDF Group HDF5 Tools Updates Peter Cao, The HDF Group September 28-30, 2010 HDF and HDF-EOS Workshop XIV 1
  2. 2. Tools Updates • HDF5 Command-line tools • HDF-Java Products September 28-30, 2010 HDF and HDF-EOS Workshop XIV 2 www.hdfgroup.org
  3. 3. HDF5 Command-line tools • Look or check file content h5dump, h5diff, h5ls, h5check, etc. • Change file content or create a new file h5repack, h5import, h5jam, h5copy, etc. • Convert an HDF5 file to a different file h4toh5, h5toh4, gif2h5, h52gif, etc. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 3 www.hdfgroup.org
  4. 4. h5ls Prints information about one or more HDF5 files or objects September 28-30, 2010 HDF and HDF-EOS Workshop XIV 4 www.hdfgroup.org
  5. 5. h5ls • New features • ‘--follow-symlinks’ option to follow soft and external link. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 5 www.hdfgroup.org
  6. 6. h5ls • New features • ‘--no-dangling-links’ option to check dangling links September 28-30, 2010 HDF and HDF-EOS Workshop XIV 6 www.hdfgroup.org
  7. 7. h5dump Dumps contents of HDF5 file to stdout • • • September 28-30, 2010 ASCII text XML Binary HDF and HDF-EOS Workshop XIV 7 www.hdfgroup.org
  8. 8. h5dump • New features • Show data pointed by region references September 28-30, 2010 HDF and HDF-EOS Workshop XIV 8 www.hdfgroup.org
  9. 9. h5dump • New features • Bit(s) values packed in a byte/short integer E.g. Quality flags Land/Water Background SST Skin Quality SST Bulk Quality Aerosol Correction September 28-30, 2010 HDF and HDF-EOS Workshop XIV -- 1-bit -- 2-bits -- 2-bits -- 3-bits 9 www.hdfgroup.org
  10. 10. h5dump • Bug fixes • • • • Datatype of object references was displayed as the datatype of region references. Does not export data pointed by a region reference as a binary file. Ignore –m <set the floating point output format> option when generating XML with –u. Subsetting does not check dimension bounds. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 10 www.hdfgroup.org
  11. 11. h5diff Show differences between… two objects or two files September 28-30, 2010 HDF and HDF-EOS Workshop XIV 11 www.hdfgroup.org
  12. 12. h5diff • New features • ‘--follow-symlinks’ option to follow soft and external link. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 12 www.hdfgroup.org
  13. 13. h5diff • New features • ‘--no-dangling-links’ option to check dangling links September 28-30, 2010 HDF and HDF-EOS Workshop XIV 13 www.hdfgroup.org
  14. 14. h5diff • New features • ‘--exclude-path’ option to exclude specified object pointed by a given path for comparison. • Will apply to attributes in the future. E.g. h5diff --exclude-path /g1/g2/f tools_diff1.h5 tools_diff2.h5 September 28-30, 2010 HDF and HDF-EOS Workshop XIV 14 www.hdfgroup.org
  15. 15. h5diff • Bug fixes • Improved to traverse groups recursively for comparing child objects when the groups are specified to be compared via command line. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 15 www.hdfgroup.org
  16. 16. h5repack Copies an entire file to a new file… September 28-30, 2010 HDF and HDF-EOS Workshop XIV 16 www.hdfgroup.org
  17. 17. h5repack • Bug fixes • • • Attributes on the datasets with H5T_REFERENCE type were not present in the repacked file A data element which was a reference to a group became NULL in the repacked file Display compression information incorrectly. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 17 www.hdfgroup.org
  18. 18. h5copy Copies objects to a different file / / floats integers i1 i2 64-bit f32 f1 September 28-30, 2010 HDF and HDF-EOS Workshop XIV f2 18 www.hdfgroup.org
  19. 19. h5copy • Bug fixes • • • Failed to copy dangling links when it’s specified directly as the source object. Couldn’t deal with NULL elements of object references Copying references was not tested and properly documented September 28-30, 2010 HDF and HDF-EOS Workshop XIV 19 www.hdfgroup.org
  20. 20. New Tools in development h5watch September 28-30, 2010 h5edit HDF and HDF-EOS Workshop XIVXIII HDF/HDF-EOS Workshop 20 www.hdfgroup.org
  21. 21. h5watch • • • Allows the user to monitor the growth of a dataset Prints out the new elements appended to a dataset Includes options on what to be monitored. September 28-30, 2010 HDF and HDF-EOS Workshop XIV 21 www.hdfgroup.org
  22. 22. h5edit • Motivation • Create an attribute • Change the value of an attribute in a dataset • Edit other objects in an HDF5 file • Requirements • Familiar syntax for the input data (DDL or h5import) • Scriptable (can be used in a shell script ) • Extendible to support other HDF5 object September 28-30, 2010 HDF and HDF-EOS Workshop XIV 22 www.hdfgroup.org
  23. 23. h5edit • New tool for editing an HDF5 file RFC: http://www.hdfgroup.org/pubs/rfcs/ • Initial implementation will include creating and deleting attributes • Prototype will be released soon • Please send us your comments! September 28-30, 2010 HDF and HDF-EOS Workshop XIV 23 www.hdfgroup.org
  24. 24. Example: h5edit  H5edit --command “delete ATTRIBUTE Percentage_per_Volume /m1;” file1.h5  h5edit --command “create September 28-30, 2010 ATTRIBUTE Percentage_per_Volume HDF and HDF-EOS Workshop XIV 40 24 /m1;” file1.h5 www.hdfgroup.org
  25. 25. The HDF Group HDF-Java Products Support for HDF5 1.8 HDF-Java 2.7 release September 28-30, 2010 HDF and HDF-EOS Workshop XIV 25
  26. 26. HDF-Java Products HDF-Java HDF Java Wrapper September 28-30, 2010 Object Package HDF and HDF-EOS Workshop XIV Viewer Editor 26 www.hdfgroup.org
  27. 27. Java Wrapper HDF JNI A programming framework that allows Java code to call and be called by applications and libraries written in other languages HDF September 28-30, 2010 HDF and HDF-EOS Workshop XIV 27 www.hdfgroup.org
  28. 28. Object Package: basic objects File Group Dataset Datatype Attribute http://www.hdfgroup.org/hdf-java-html/hdf-object/ September 28-30, 2010 HDF and HDF-EOS Workshop XIV 28 www.hdfgroup.org
  29. 29. HDFView September 28-30, 2010 HDF and HDF-EOS Workshop XIV 29 www.hdfgroup.org
  30. 30. Support for HDF 1.8 in HDF-Java • HDF5 JNI (funded by Sandia National Lab) • Added about 118 (out of 169) new functions • Added unit tests for theJNI functions Package H5S H5T H5D H5O H5L H5A H5R H5I H5P 1.6 compatable APIs Total September 28-30, 2010 No. of new APIs 3 7 5 17 18 14 1 2 34 17 118 HDF and HDF-EOS Workshop XIV 30 www.hdfgroup.org
  31. 31. Support for HDF 1.8 in HDF-Java • Added HDF5 1.8 related features to the Object Layer and HDFView • • • • • • Set the bounds of library versions Set link creation order Set link storage type Show groups and attributes in creation order (not in HDFView) Create soft and external links Retrieve link information September 28-30, 2010 HDF and HDF-EOS Workshop XIV 31 www.hdfgroup.org
  32. 32. HDF-Java 2.7 • New features (17), e.g. • • • • • • Show bit(s) values in byte or short integers Move objects within the same file Import/export binary data Use H5Ocopy() API (important for copying large datasets) Rename Attributes Support for 64-bit Windows September 28-30, 2010 HDF and HDF-EOS Workshop XIV 32 www.hdfgroup.org
  33. 33. HDF-Java 2.7 • Bug fixes (29), e.g. • • • • Java binary jar files are not the same for 32-bit and 64-bit Datasets displayed incorrectly if you swap dimensions H5Sset_extent_simple in Java does not seem to take effect Possible useful information for HDF-EOS2 files are not displayed by HDFView September 28-30, 2010 HDF and HDF-EOS Workshop XIV 33 www.hdfgroup.org
  34. 34. HDF-Java 2.7 • • • • Support for HDF5 1.8 Bug fixes/other new features Beta is out for you to try Official release – Nov. 2010 September 28-30, 2010 HDF and HDF-EOS Workshop XIV 34 www.hdfgroup.org
  35. 35. HDF-Java Features/Roadmap Nov. 2010 • Support for HDF5 1.8 • N-dimension Array • Buffered Array • Function pointers • Unit tests for JNI • Creation Order * • Buffered Array • Edit complex data • Bug fixes • Import/Export GeoTiff * • Import/export XML * • 3D Graphic view * • Other new features • GUI Widgets * • Drag/drop (in & ex) * • Automatic GUI tests * * For HDFView only Please give us your feedback September 28-30, 2010 HDF and HDF-EOS Workshop XIV 35 www.hdfgroup.org
  36. 36. Thank You! Questions/comments? September 28-30, 2010 HDF and HDF-EOS Workshop XIV 36 www.hdfgroup.org

×