The HDF Group

HDF Tools Tutorial

Peter Cao, The HDF Group
Jonathan Kim, The HDF Group

September 28-30, 2010

HDF and HD...
Tools Tutorial

• HDFView
• HDF5 Command-line tools

September 28-30, 2010

HDF and HDF-EOS Workshop XIV

2

www.hdfgroup....
HDFView

• View file content
• Edit file content
• Show NPOESS/JPSS related features

September 28-30, 2010

HDF and HDF-E...
HDFView

• Download HDF-Java 2.7 beta from
http://www.hdfgroup.org/ftp/HDF5/hdf-java-2.7
• Download example files from
ftp...
HDFView
• View file content
•
•
•
•
•

Open OMI-Aura_L3.he5
Open groups (try expand all)
Stretch the metadata view for sho...
HDFView
• Copy/paste a dataset and a group
• Create a new file: tmp.h5
• Copy/paste “ColumnAmountO3” and show the image fr...
HDFView
• Create a dataset and add an attribute
•
•
•
•
•

Create /d1 (32-bit float, 720x1440)
Copy “ColumnAmountO3” to /d...
HDFView
• NPOESS/JPSS related features
• Object reference
• Region references
• Quality flags

September 28-30, 2010

HDF ...
HDFView
• Object reference
1)
2)
3)
4)
5)
6)

Open file: SVI-NPP.h5
Open “VIIRS-I1-SDR_Aggr”
Click on object ID and shows ...
HDFView
• Region references
1)
2)
3)
4)

Open “VIIRS-I1-SDR_Gran”
Right mouse to click to open data as image
Open data in ...
HDFView
• Quality flags
1) Quality flags
•
•
•
•

Land/Water Background
SST Skin Quality
SST Bulk Quality
Aerosol Correcti...
HDFView
• Advanced features
1) Import/export images
2) Use HDF-EOS plugin

September 28-30, 2010

HDF and HDF-EOS Workshop...
HDF5 Command-line tools

•
•
•
•
•

September 28-30, 2010

h5ls
h5dump
h5repack
h5diff
h5copy

HDF and HDF-EOS Workshop XI...
h5ls

• Show file content list
• h5ls ami.h5
• h5ls –r ami.h5

September 28-30, 2010

HDF and HDF-EOS Workshop XIV

14

ww...
h5dump

•
•
•
•
•

September 28-30, 2010

Structure
Dataset
Binary
XML
Region references

HDF and HDF-EOS Workshop XIV

15...
h5dump
• Structure
• h5dump -H tools_diff1.h5
• h5dump -H ami.h5

September 28-30, 2010

HDF and HDF-EOS Workshop XIV

16
...
h5dump
• Dataset
• Content
• h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5

• Information
• h5dump –p –H -d ...
h5dump
• Binary
•

h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -b LE –o out.bin ami.h5

• XML
• h5dump –x ami.h5
...
h5repack
• Remove inaccessible objects / junk spaces
• Change storage layout
• Apply compression filter
Copies a file to a...
h5repack
• Remove inaccessible objects
• h5dump tools_junk.h5
• h5repack tools_junk.h5 tmp.h5

• Change layout
• h5dump -H...
h5diff

Show differences between
two files or two objects

• Dataset

• Whole file

September 28-30, 2010

HDF and HDF-EOS...
h5diff
• Dataset
• h5diff -r tools_diff1.h5 tools_diff2.h5 /g1/uint8 /x

• Whole file
• Run “h5ls –r tools_diff1.h5” and “...
h5copy

• Dataset
• Group

Copies
objects
to files…

September 28-30, 2010

HDF and HDF-EOS Workshop XIV

23

www.hdfgroup...
h5copy
• Dataset
• h5copy -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /float

• Group
• Default: h5copy -i tools_input.h5 -...
Thank You!
Questions/comments?

September 28-30, 2010

HDF and HDF-EOS Workshop XIV

25

www.hdfgroup.org
Upcoming SlideShare
Loading in …5
×

HDF Tools Tutorial

344 views
224 views

Published on

This tutorial is designed for anyone who needs to work with data stored in HDF and HDF5 files.

The first part of the tutorial will focus on the HDF5 utilities to display the contents of HDF5 files, to extract and to import data from and to HDF5 files, to compare two HDF5 files, and more. Participants will be guided through the hand-on examples and will learn about different tools options. New changes and advanced features will be covered in a separate session (Updates on HDF tools) on Wednesday.

The second part of tutorial includes a hands-on session to learn the HDF (4 & 5) Java browsing tool, HDFView. The tool and special plug-ins will be used to work with the existing HDF, HDF-EOS, and netCDF-4 files, and to create a new HDF5 file. The tutorial will cover basic features of HDFView.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
344
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

HDF Tools Tutorial

  1. 1. The HDF Group HDF Tools Tutorial Peter Cao, The HDF Group Jonathan Kim, The HDF Group September 28-30, 2010 HDF and HDF-EOS Workshop XIV 1 www.hdfgroup.org
  2. 2. Tools Tutorial • HDFView • HDF5 Command-line tools September 28-30, 2010 HDF and HDF-EOS Workshop XIV 2 www.hdfgroup.org
  3. 3. HDFView • View file content • Edit file content • Show NPOESS/JPSS related features September 28-30, 2010 HDF and HDF-EOS Workshop XIV 3 www.hdfgroup.org
  4. 4. HDFView • Download HDF-Java 2.7 beta from http://www.hdfgroup.org/ftp/HDF5/hdf-java-2.7 • Download example files from ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/WorkshopXIV/HDF5-Files/ September 28-30, 2010 HDF and HDF-EOS Workshop XIV 4 www.hdfgroup.org
  5. 5. HDFView • View file content • • • • • Open OMI-Aura_L3.he5 Open groups (try expand all) Stretch the metadata view for showing metadata Double click on “ColumnAmountO3” to show data in table Use “Open As” to show “ColumnAmountO3” in image • View a subset • Open weather.h5 • Select from preview image or selection fields September 28-30, 2010 HDF and HDF-EOS Workshop XIV 5 www.hdfgroup.org
  6. 6. HDFView • Copy/paste a dataset and a group • Create a new file: tmp.h5 • Copy/paste “ColumnAmountO3” and show the image from the new file • Copy the group “GRIDS” to show the whole group is copied September 28-30, 2010 HDF and HDF-EOS Workshop XIV 6 www.hdfgroup.org
  7. 7. HDFView • Create a dataset and add an attribute • • • • • Create /d1 (32-bit float, 720x1440) Copy “ColumnAmountO3” to /d1 Add attribute “IMAGE_MINMAXRANGE = 123.0,447.0” Export /d1 to text file Create /d2 and import d1.txt September 28-30, 2010 HDF and HDF-EOS Workshop XIV 7 www.hdfgroup.org
  8. 8. HDFView • NPOESS/JPSS related features • Object reference • Region references • Quality flags September 28-30, 2010 HDF and HDF-EOS Workshop XIV 8 www.hdfgroup.org
  9. 9. HDFView • Object reference 1) 2) 3) 4) 5) 6) Open file: SVI-NPP.h5 Open “VIIRS-I1-SDR_Aggr” Click on object ID and shows object name Right mouse to click to open data as image Open data in table Select two cells and open them at once September 28-30, 2010 HDF and HDF-EOS Workshop XIV 9 www.hdfgroup.org
  10. 10. HDFView • Region references 1) 2) 3) 4) Open “VIIRS-I1-SDR_Gran” Right mouse to click to open data as image Open data in table Select two cells and open at once September 28-30, 2010 HDF and HDF-EOS Workshop XIV 10 www.hdfgroup.org
  11. 11. HDFView • Quality flags 1) Quality flags • • • • Land/Water Background SST Skin Quality SST Bulk Quality Aerosol Correction -- 1-bit -- 2-bits -- 2-bits -- 3-bits 2) Open QF_VIIRSI1SDR_Array 3) Use "Open As" on QF_VIIRSI1SDR_Array 4) Select first bit September 28-30, 2010 HDF and HDF-EOS Workshop XIV 11 www.hdfgroup.org
  12. 12. HDFView • Advanced features 1) Import/export images 2) Use HDF-EOS plugin September 28-30, 2010 HDF and HDF-EOS Workshop XIV 12 www.hdfgroup.org
  13. 13. HDF5 Command-line tools • • • • • September 28-30, 2010 h5ls h5dump h5repack h5diff h5copy HDF and HDF-EOS Workshop XIV 13 www.hdfgroup.org
  14. 14. h5ls • Show file content list • h5ls ami.h5 • h5ls –r ami.h5 September 28-30, 2010 HDF and HDF-EOS Workshop XIV 14 www.hdfgroup.org
  15. 15. h5dump • • • • • September 28-30, 2010 Structure Dataset Binary XML Region references HDF and HDF-EOS Workshop XIV 15 www.hdfgroup.org
  16. 16. h5dump • Structure • h5dump -H tools_diff1.h5 • h5dump -H ami.h5 September 28-30, 2010 HDF and HDF-EOS Workshop XIV 16 www.hdfgroup.org
  17. 17. h5dump • Dataset • Content • h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 • Information • h5dump –p –H -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 • Subset • H5dump -d /EnSight_model/geometry/parts/part_2/quad4 -s 10,0 -c 20,2 ami.h5 • <-s and –c must be used after –d> September 28-30, 2010 HDF and HDF-EOS Workshop XIV 17 www.hdfgroup.org
  18. 18. h5dump • Binary • h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -b LE –o out.bin ami.h5 • XML • h5dump –x ami.h5 • Region • • h5dump -d /Data_Products/VIIRS-I1-SDR/VIIRS-I1-SDR_Gran_0 -s 3 -c 1 SVI-NPP.h5 h5dump –R -d /Data_Products/VIIRS-I1-SDR/VIIRS-I1-SDR_Gran_0 -s 3 -c 1 SVI-NPP.h5 • Packed fields <only 1.8.5+> September 28-30, 2010 HDF and HDF-EOS Workshop XIV 18 www.hdfgroup.org
  19. 19. h5repack • Remove inaccessible objects / junk spaces • Change storage layout • Apply compression filter Copies a file to a new file with different storage layouts and compression filters September 28-30, 2010 HDF and HDF-EOS Workshop XIV 19 www.hdfgroup.org
  20. 20. h5repack • Remove inaccessible objects • h5dump tools_junk.h5 • h5repack tools_junk.h5 tmp.h5 • Change layout • h5dump -H tools_bad_layout.h5 • h5repack tools_bad_layout.h5 tmp.h5 • h5repack -l CHUNK=16x16 tools_bad_layout.h5 tmp.h5 • Change compression • h5repack -f GZIP=6 tmp.h5 tmp2.h5 • More: use latest version, link storage, shared message, and etc September 28-30, 2010 HDF and HDF-EOS Workshop XIV 20 www.hdfgroup.org
  21. 21. h5diff Show differences between two files or two objects • Dataset • Whole file September 28-30, 2010 HDF and HDF-EOS Workshop XIV 21 www.hdfgroup.org
  22. 22. h5diff • Dataset • h5diff -r tools_diff1.h5 tools_diff2.h5 /g1/uint8 /x • Whole file • Run “h5ls –r tools_diff1.h5” and “h5ls -r tools_diff2.h5” • h5diff tools_diff1.h5 tools_diff2.h5 • h5diff –v tools_diff1.h5 tools_diff2.h5 • Follow symlinks • h5diff -r --follow-symlinks tools_diff1.h5 tools_diff2.h5 /sl1 September 28-30, 2010 HDF and HDF-EOS Workshop XIV 22 www.hdfgroup.org
  23. 23. h5copy • Dataset • Group Copies objects to files… September 28-30, 2010 HDF and HDF-EOS Workshop XIV 23 www.hdfgroup.org
  24. 24. h5copy • Dataset • h5copy -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /float • Group • Default: h5copy -i tools_input.h5 -o tmp.h5 -s /g1 -d /g1 • Parent groups: h5copy -p -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /g1/g2/f September 28-30, 2010 HDF and HDF-EOS Workshop XIV 24 www.hdfgroup.org
  25. 25. Thank You! Questions/comments? September 28-30, 2010 HDF and HDF-EOS Workshop XIV 25 www.hdfgroup.org

×