Transition from HDF4 to
HDF5: Issues
Robert E. McGrath
NCSA
University of Illinois at Urbana-Champaign

University of Illi...
Overall Goal
A discussion of compatibility, conversion,
and interoperability issues for HDF4 and
HDF5, and HDFEOS.
For mor...
Conclusions (Preview)
Transition from HDF4 to HDF5 is not
necessarily transparent or effortless.

University of Illinois a...
Conclusions: HDF
• Transition from HDF4 to HDF5 is not
transparent: in most cases source code will
have to be written or r...
Conclusions: HDF-EOS
• Transition from HDFEOS2 to HDFEOS3 is
not transparent: in most cases source code
will have to be wr...
What Does Compatibility Mean?

University of Illinois at Urbana-Champaign

9/19/2000

McGrath 9/19/2000

6

HDF
What does “compatibility” mean?
• Ability to “do the same things” with new
software
– I.e., “compatibility” is relative to...
Some Types of Compatibility
Compatibility
Issue
Conceptual

Description

Format
Software
Programming
model
Features
Interf...
Compatibility of HDF and HDF-EOS
Compatibility Issue HDF4/HDF5 HDFEOS2/HDFEOS3
Conceptual
Yes, H5 is a Yes
superset
Format...
Interoperation

• My definition of “interoperation”:
one program or tool may use HDF4 or
HDF5 or both

University of Illin...
NCSA HDF4 and HDF5

University of Illinois at Urbana-Champaign

9/19/2000

McGrath 9/19/2000

11

HDF
Interoperability: Any program may use either HDF4 or 5
or both:
Not Supported

dataset1

dataset2

image1

image2

Single ...
Interoperability: Specific Application or tool may use
either HDF4 or 5 or both:
Up to Application and Tool Developers

da...
HDF-EOS Version 2 and Version 3

University of Illinois at Urbana-Champaign

9/19/2000

McGrath 9/19/2000

14

HDF
Interoperability:
Any program may using HDF-EOS Objects with either
HDF4 or HDF5 or both:
Not supported
HDFEOS Library

Gr...
Interoperability: One application may use HDF-EOS
Objects with either HDF4 or HDF5 or both:
Not supported
Grid1

Grid2

HD...
Translation

• Convert HDF4 objects (file) to HDF5
objects (file)

University of Illinois at Urbana-Champaign

9/19/2000

...
Translation of HDF4 to HDF5
Supported: NCSA mapping and h4toh5 tool and library

HDF4 Library

HDF5 Library
/

Stored as
H...
Translation of HDF-EOS V2 to V3
Supported
HDF EOS V. 2.x Library
Grid1

Grid2

HDF EOS V. 3 Library

Grid2

Grid1

SAME
HD...
“Hybrid” HDF-EOS files

• HDF-EOS files may contain native HDF
objects (images, arrays, etc.) as well as
HDF-EOS objects (...
“Hybrid” HDF-EOS File
Science Application Code
HDFEOS

Other

NCSA HDF
Components of
HDFEOS Objects
stored in HDF
Objects
...
Conclusions
Transition from HDF4 to HDF5 is not
necessarily transparent or effortless.

University of Illinois at Urbana-C...
Conclusions: HDF
• Transition from HDF4 to HDF5 is not
transparent: in most cases source code will
have to be rewritten.
•...
Conclusions: HDF-EOS
• Transition from HDFEOS2 to HDFEOS3 is
not transparent: in most cases source code
will have to be re...
For More Information
• NCSA White paper: “Transition From HDF4 to HDF5”:
http://hdf.ncsa.uiuc.edu/HDF5/doc/TechNotes/Trans...
Upcoming SlideShare
Loading in …5
×

Transitions from HDF4 to HDF5: Issues

238 views

Published on

Source: http://hdfeos.org/workshops/ws04/presentations/McGrath/McGrath.ppt

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
238
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transitions from HDF4 to HDF5: Issues

  1. 1. Transition from HDF4 to HDF5: Issues Robert E. McGrath NCSA University of Illinois at Urbana-Champaign University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 1 HDF
  2. 2. Overall Goal A discussion of compatibility, conversion, and interoperability issues for HDF4 and HDF5, and HDFEOS. For more details see the white paper at: http://hdf.ncsa.uiuc.edu/HDF5/doc/TechNotes/TransitionIssues.pdf University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 2 HDF
  3. 3. Conclusions (Preview) Transition from HDF4 to HDF5 is not necessarily transparent or effortless. University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 3 HDF
  4. 4. Conclusions: HDF • Transition from HDF4 to HDF5 is not transparent: in most cases source code will have to be written or rewritten. • Many tools will support both HDF4 and HDF5. • Default translation from HDF4 to HDF5 will be supported as a utility and library. University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 4 HDF
  5. 5. Conclusions: HDF-EOS • Transition from HDFEOS2 to HDFEOS3 is not transparent: in most cases source code will have to be written or rewritten • Translation of ‘pure’ HDF-EOS will be supported • Translation of ‘hybrid’ HDF-EOS files is very difficult. • Not possible to support V.2 and V.3 in a single program University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 5 HDF
  6. 6. What Does Compatibility Mean? University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 6 HDF
  7. 7. What does “compatibility” mean? • Ability to “do the same things” with new software – I.e., “compatibility” is relative to some purpose – Many dimensions and shades of compatibility – No golden solution sufficient for every use • Assume HDF4 users want to continue doing what they are already doing, but move to HDF5 University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 7 HDF
  8. 8. Some Types of Compatibility Compatibility Issue Conceptual Description Format Software Programming model Features Interfaces The overall conceptual approach: the formal and informal models. Storage layout and other format details. Compatibility of the software The assumptions and approach to programming. The set of features supported. The programming interfaces. University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 8 HDF
  9. 9. Compatibility of HDF and HDF-EOS Compatibility Issue HDF4/HDF5 HDFEOS2/HDFEOS3 Conceptual Yes, H5 is a Yes superset Format No No Software Programming model No? Yes Features Some Yes, V3 is a superset Interfaces No No University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 9 HDF
  10. 10. Interoperation • My definition of “interoperation”: one program or tool may use HDF4 or HDF5 or both University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 10 HDF
  11. 11. NCSA HDF4 and HDF5 University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 11 HDF
  12. 12. Interoperability: Any program may use either HDF4 or 5 or both: Not Supported dataset1 dataset2 image1 image2 Single Combined HDF Library HDF 4 Library HDF 5 Library Stored as HDF4 Objects Stored as HDF5 Objects HDF4 file HDF5 file University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 12 HDF
  13. 13. Interoperability: Specific Application or tool may use either HDF4 or 5 or both: Up to Application and Tool Developers dataset1 dataset2 image1 image2 Application Specific Library specific HDF 4 Library HDF 5 Library Stored as HDF4 Objects Stored as HDF5 Objects HDF4 file HDF5 file University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 13 HDF
  14. 14. HDF-EOS Version 2 and Version 3 University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 14 HDF
  15. 15. Interoperability: Any program may using HDF-EOS Objects with either HDF4 or HDF5 or both: Not supported HDFEOS Library Grid1 Grid2 HDF 4 Library Grid3 Grid4 HDF 5 Library Stored as HDF4 Objects Stored as HDF5 Objects HDF4 file HDF5 file University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 15 HDF
  16. 16. Interoperability: One application may use HDF-EOS Objects with either HDF4 or HDF5 or both: Not supported Grid1 Grid2 HDFEOS V2 Library HDF 4 Library Grid3 Grid4 HDFEOS V3 Library HDF 5 Library Stored as HDF4 Objects Stored as HDF5 Objects HDF4 file HDF5 file University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 16 HDF
  17. 17. Translation • Convert HDF4 objects (file) to HDF5 objects (file) University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 17 HDF
  18. 18. Translation of HDF4 to HDF5 Supported: NCSA mapping and h4toh5 tool and library HDF4 Library HDF5 Library / Stored as HDF4 Objects Stored as Analogous HDF5 Objects HDF5 File HDF4 File University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 18 HDF
  19. 19. Translation of HDF-EOS V2 to V3 Supported HDF EOS V. 2.x Library Grid1 Grid2 HDF EOS V. 3 Library Grid2 Grid1 SAME HDFEOS Objects HDFEOS Objects HDF4 Library HDF5 Library Stored as HDF4 Objects Stored as HDF5 Objects / HDF5 file HDF4 file University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 19 HDF
  20. 20. “Hybrid” HDF-EOS files • HDF-EOS files may contain native HDF objects (images, arrays, etc.) as well as HDF-EOS objects (Grid, etc.) • Almost all HDF-EOS files are ‘hybrids’. • Very difficult to automatically translate hybrid files. University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 20 HDF
  21. 21. “Hybrid” HDF-EOS File Science Application Code HDFEOS Other NCSA HDF Components of HDFEOS Objects stored in HDF Objects Other HDF Objects Stored Directly Through HDF One HDF file, objects mixed together. Very difficult to tell which objects are part of HDF-EOS University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 21 HDF
  22. 22. Conclusions Transition from HDF4 to HDF5 is not necessarily transparent or effortless. University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 22 HDF
  23. 23. Conclusions: HDF • Transition from HDF4 to HDF5 is not transparent: in most cases source code will have to be rewritten. • Many tools will support both HDF4 and HDF5. • Default translation from HDF4 to HDF5 will be supported as a utility and library. University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 23 HDF
  24. 24. Conclusions: HDF-EOS • Transition from HDFEOS2 to HDFEOS3 is not transparent: in most cases source code will have to be rewritten • Translation of ‘pure’ HDF-EOS will be supported • Translation of ‘hybrid’ HDF-EOS files is very difficult. • Not possible to support V.2 and V.3 in a single program University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 24 HDF
  25. 25. For More Information • NCSA White paper: “Transition From HDF4 to HDF5”: http://hdf.ncsa.uiuc.edu/HDF5/doc/TechNotes/TransitionIssues.pdf • “Guidelines for Mapping HDF4 to HDF5 Objects”, http://hdf.ncsa.uiuc.edu/HDF5/papers/H4-to-H5MappingGuidelines.pdf • h4toh5 tool: Soon University of Illinois at Urbana-Champaign 9/19/2000 McGrath 9/19/2000 25 HDF

×