The HDF Group

HDF Tools Tutorial
Peter Cao
The HDF Group
The 15th HDF and HDF-EOS Workshop
April 17-19, 2012

Apr. 17-19,...
Tools Tutorial
HDF5
Command-line
tools

HDFView

GUI
Built in Java

Text
Built in C

Separate release

Part of the lib rel...
HDFView

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

3

www.hdfgroup.org
HDFView

The best way to get help
a)
b)
c)
d)
e)

Apr. 17-19, 2012

Call Peter
Email to Barbara
Stop by Mike’s office
All ...
HDFView

HDFView

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

5

www.hdfgroup.org
View File Content

File structure

File Content

Metadata and log info.
Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

6

www....
Add/delete, Copy/paste Objects

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

7

www.hdfgroup.org
Add/delete, Copy/paste Objects

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

8

www.hdfgroup.org
Add/Delete, Copy/Paste Objects

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

9

www.hdfgroup.org
Modify, Copy/Paste Data Values

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

10

www.hdfgroup.org
Modify, Add, Delete Attributes

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

11

www.hdfgroup.org
Modify, Add, Delete Attributes

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

12

www.hdfgroup.org
Modify, Add, Delete Attributes

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

13

www.hdfgroup.org
Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

14

www.hdfgroup.org
HDFView
• View file content
• File structure/metadata
• Data values or Images

• Change file content
• Add/delete objects/...
The HDF Group

HDF5 Command-line tools

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

16

www.hdfgroup.org
HDF5 Command-line tools
•

View or check file content
h5dump, h5diff, h5ls, h5check, etc.

•

Change file content or creat...
HDF5 Command-line tools
• H5dump
• H5repack

• H5diff
• h5copy

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

18

www.hdfgrou...
HDF5 Command-line tools

“-h” or “-help”
h5dump -h

HDF5 tools

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

19

www.hdfgrou...
h5dump
h5dump -H ami.h5
HDF5 "ami.h5" {
GROUP "/" {
GROUP "EnSight_model" {
ATTRIBUTE "description_1" {
DATATYPE H5T_STRIN...
h5dump: Dataset Information
H5dump –H

-d

/EnSight_model/geometry/parts/part_2/quad4

ami.h5

HDF5 "ami.h5" {
DATASET "/E...
h5dump: Dataset Content
h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5
HDF5 "ami.h5" {
DATASET "/EnSight_mode...
h5dump: Data Subset
h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -s 10,0 -c 10,2 ami.h5
HDF5 "ami.h5" {
DATASET "/...
h5repack

Copies a whole file to a new file

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

24

www.hdfgroup.org
H5repack: Remove Junk Space

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

25

www.hdfgroup.org
H5repack: change the storage layout
2x2 chunks

h5repack -l CHUNK=16x16 tools_bad_layout.h5 tmp.h5
162x16 chunks

Apr. 17-...
H5repack:Apply compression filter

10 GB

h5repack -f GZIP=6 tmp.h5 tmp2.h5
1 GB

Apr. 17-19, 2012

HDF/HDF-EOS Workshop X...
h5diff
Show differences
between
two files or two objects

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

28

www.hdfgroup.org
H5diff: Two files

h5diff tools_diff1.h5 tools_diff2.h5

Apr. 17-19, 2012

HDF/HDF-EOS Workshop XV

29

www.hdfgroup.org
H5diff: Two datasets

h5diff tools_diff1.h5 tools_diff2.h5 /dset

h5diff tools_diff1.h5 tools_diff2.h5 /g1/uint8 /dset

Ap...
h5copy

Copy one or more objects to another file
h5copy -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /float

-h
Apr. 17-19, ...
Acknowledgements
This work was supported by Subcontract number
114820 under Raytheon Contract number
NNG10HP02C, funded by...
Upcoming SlideShare
Loading in …5
×

HDF Tools Tutorial

384 views
257 views

Published on

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
384
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Hdf4, netcdf, etc.
  • Copy and paste
  • Add general view
  • Add general view
  • Add general view
  • Introduce 4 tools, mention Gerd’spowershell, Dan’s python, and other tools
  • Find these are the most useful/used tools
  • Does not show content of a dataset
  • A little too many details.
  • One or more objects to another file
  • Mention the updates and other things
  • HDF Tools Tutorial

    1. 1. The HDF Group HDF Tools Tutorial Peter Cao The HDF Group The 15th HDF and HDF-EOS Workshop April 17-19, 2012 Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 1 www.hdfgroup.org
    2. 2. Tools Tutorial HDF5 Command-line tools HDFView GUI Built in Java Text Built in C Separate release Part of the lib release HDF4, HDF5, NetCDF, etc. HDF5 only Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 2 www.hdfgroup.org
    3. 3. HDFView Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 3 www.hdfgroup.org
    4. 4. HDFView The best way to get help a) b) c) d) e) Apr. 17-19, 2012 Call Peter Email to Barbara Stop by Mike’s office All of above None of above HDF/HDF-EOS Workshop XV 4 www.hdfgroup.org
    5. 5. HDFView HDFView Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 5 www.hdfgroup.org
    6. 6. View File Content File structure File Content Metadata and log info. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 6 www.hdfgroup.org
    7. 7. Add/delete, Copy/paste Objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 7 www.hdfgroup.org
    8. 8. Add/delete, Copy/paste Objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 8 www.hdfgroup.org
    9. 9. Add/Delete, Copy/Paste Objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 9 www.hdfgroup.org
    10. 10. Modify, Copy/Paste Data Values Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 10 www.hdfgroup.org
    11. 11. Modify, Add, Delete Attributes Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 11 www.hdfgroup.org
    12. 12. Modify, Add, Delete Attributes Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 12 www.hdfgroup.org
    13. 13. Modify, Add, Delete Attributes Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 13 www.hdfgroup.org
    14. 14. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 14 www.hdfgroup.org
    15. 15. HDFView • View file content • File structure/metadata • Data values or Images • Change file content • Add/delete objects/attributes • Modify values of datasets or attribute • Import/export • Image files • Text files Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 15 www.hdfgroup.org
    16. 16. The HDF Group HDF5 Command-line tools Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 16 www.hdfgroup.org
    17. 17. HDF5 Command-line tools • View or check file content h5dump, h5diff, h5ls, h5check, etc. • Change file content or create a new file h5repack, h5import, h5jam, h5copy, etc. • Convert between HDF5 and another format h4toh5, h5toh4, gif2h5, h52gif, etc. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 17 www.hdfgroup.org
    18. 18. HDF5 Command-line tools • H5dump • H5repack • H5diff • h5copy Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 18 www.hdfgroup.org
    19. 19. HDF5 Command-line tools “-h” or “-help” h5dump -h HDF5 tools Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 19 www.hdfgroup.org
    20. 20. h5dump h5dump -H ami.h5 HDF5 "ami.h5" { GROUP "/" { GROUP "EnSight_model" { ATTRIBUTE "description_1" { DATATYPE H5T_STRING { STRSIZE 256; STRPAD H5T_STR_NULLPAD; CSET H5T_CSET_ASCII; CTYPE H5T_C_S1; } DATASPACE SIMPLE { ( 1 ) / ( 1 ) } } …. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 20 www.hdfgroup.org
    21. 21. h5dump: Dataset Information H5dump –H -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 HDF5 "ami.h5" { DATASET "/EnSight_model/geometry/parts/part_2/quad4" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 768, 4 ) / ( 768, 4 ) } } } Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 21 www.hdfgroup.org
    22. 22. h5dump: Dataset Content h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 HDF5 "ami.h5" { DATASET "/EnSight_model/geometry/parts/part_2/quad4" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 768, 4 ) / ( 768, 4 ) } DATA { (0,0): 5, 6, 431, 430, (1,0): 6, 7, 432, 431, (2,0): 7, 8, 433, 432, (3,0): 8, 9, 434, 433, (4,0): 9, 10, 435, 434, (5,0): 10, 11, 436, 435, … Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 22 www.hdfgroup.org
    23. 23. h5dump: Data Subset h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -s 10,0 -c 10,2 ami.h5 HDF5 "ami.h5" { DATASET "/EnSight_model/geometry/parts/part_2/quad4" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 768, 4 ) / ( 768, 4 ) } SUBSET { START ( 10, 0 ); STRIDE ( 1, 1 ); COUNT ( 10, 2 ); BLOCK ( 1, 1 ); DATA { (10,0): 15, 16, (11,0): 16, 17, (12,0): 17, 18, (13,0): 18, 19, (14,0): 19, 20, } Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 23 www.hdfgroup.org
    24. 24. h5repack Copies a whole file to a new file Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 24 www.hdfgroup.org
    25. 25. H5repack: Remove Junk Space Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 25 www.hdfgroup.org
    26. 26. H5repack: change the storage layout 2x2 chunks h5repack -l CHUNK=16x16 tools_bad_layout.h5 tmp.h5 162x16 chunks Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 26 www.hdfgroup.org
    27. 27. H5repack:Apply compression filter 10 GB h5repack -f GZIP=6 tmp.h5 tmp2.h5 1 GB Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 27 www.hdfgroup.org
    28. 28. h5diff Show differences between two files or two objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 28 www.hdfgroup.org
    29. 29. H5diff: Two files h5diff tools_diff1.h5 tools_diff2.h5 Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 29 www.hdfgroup.org
    30. 30. H5diff: Two datasets h5diff tools_diff1.h5 tools_diff2.h5 /dset h5diff tools_diff1.h5 tools_diff2.h5 /g1/uint8 /dset Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 30 www.hdfgroup.org
    31. 31. h5copy Copy one or more objects to another file h5copy -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /float -h Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 31 www.hdfgroup.org
    32. 32. Acknowledgements This work was supported by Subcontract number 114820 under Raytheon Contract number NNG10HP02C, funded by the National Aeronautics and Space Administration (NASA) and by cooperative agreement number NNX08AO77A from the NASA. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of Raytheon or the National Aeronautics and Space Administration. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 32 www.hdfgroup.org

    ×