Your SlideShare is downloading. ×
0
HDF Update
Mike Folk
HDF-EOS Workshop IV
Sept. 19-21, 2000
University of Illinois at Urbana-Champaign

-1-

HDF
Topics
• What is HDF?
• HDF community
• HDF4
– past and future work

• HDF5
–
–
–
–

HDF5-HDF5 interoperability
Activities...
NCSA HDF Mission

To develop, promote, deploy, and support
open and free technologies that facilitate
scientific data stor...
What is HDF?
•
•
•
•
•
•

Format and software for scientific data
Stores images, arrays, tables, etc.
Emphasis on storage ...
HDF People
• NCSA
– QA, maintanance & support
– Library development & performance
– Tools development

• EOS & ASCI
– User...
Who is supporting HDF?
• NASA/ESDIS
– Earth science applications, instrument data
– All aspects of data management
• DOE/A...
HDF4
•
•
•
•

User support, tools, maintenance, QA
JHV (Java viewer) - two update releases
HDF4 dumper (hdp) - many improv...
HDF4 Archiving
• New File Format and Specification Manual
–
–
–
–

Facilitate long term maintenance of HDF4
Necessary for ...
HDF5
University of Illinois at Urbana-Champaign

-9-

HDF
The growth of HDF5
• Users in 27 countries
• Users include
–
–
–
–

Government agencies
National labs
Companies
Universiti...
The growth of HDF5
• Scientific fields
– Astronomy, astrophysics, aerospace engineering
– Geophysics, remote sensing, mete...
Facilitating interoperability
between HDF4 and HDF5
• HDF4 to HDF5 mapping specification
– “Mapping HDF4 Objects to HDF5 O...
HDF5 Activities in 2000
•
•
•
•
•
•

HDF5 1.2.2 library release
Fortran 90 & C++ API
HDF5 Abstract Data Model
XML Document...
Tools & Utilities
• NCSA
–
–
–
–

Java wrapper for HDF5
H5View: Java browser/Editor for HDF5
H5gen: XML-to-HDF5 file gener...
H5View

University of Illinois at Urbana-Champaign

- 16 -

HDF
H5View
• Java-based tool for browsing and editing
• Display structure of file
• Display content of objects
• Create and de...
HDF5 XML DTD
• A flexible standard language that can
describe an HDF5 file in precise detail
– Datasets, dataspaces, datat...
H5Gen: XML HDF5
• Reads an XML description of an HDF5 file
• Generates the corresponding HDF5 file
• Validates XML descrip...
Some uses of XML
Archive

XML
documentation
of metadata
& structure

XML to HDF5 (H5gen)
HDF5 to XML
Generate, validate,
r...
Focus areas for 2001
•
•
•
•
•
•

Support Terra & Aqua
Get ready for Aura
Support HDF-EOS 3
Enhance HDF5View
Refine XML DT...
Focus areas for 2001
•
•
•
•
•
•
•
•

Expand list of applications and users
Facilitate access to other tools & software
Ge...
Thank you!
HDF • HDF website
– http://hdf.ncsa.uiuc.edu/

5 • HDF5 Information Center

– http://hdf.ncsa.uiuc.edu/HDF5/

•...
Upcoming SlideShare
Loading in...5
×

HDF Update

151

Published on

Source: http://hdfeos.org/workshops/ws04/presentations/Folk/Folk.ppt

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
151
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • <number>
  • Format and software for scientific data. HDF5 is a different format from earlier versions of HDF, as is the library.
    Stores images, multidimensional arrays, tables, etc. That is, you can construct all of these different kinds structures and store them in HDF5. You can also mix and match them in HDF5 files according to your needs.
    Emphasis on storage and I/O efficiency Both the library and the format are designed to address this.
    Free and commercial software support As far as HDF5 goes, this is just a goal now. There is commercial support for HDF4, but little if any for HDF5 at this time. We are working with vendors to change this.
    Emphasis on standards You can store data in HDF5 in a variety of ways, so we try to work with users to encourage them to organize HDF5 files in standard ways.
    Users from many engineering and scientific fields
  • Transcript of "HDF Update"

    1. 1. HDF Update Mike Folk HDF-EOS Workshop IV Sept. 19-21, 2000 University of Illinois at Urbana-Champaign -1- HDF
    2. 2. Topics • What is HDF? • HDF community • HDF4 – past and future work • HDF5 – – – – HDF5-HDF5 interoperability Activities and growth Tools and utilities Focus for 2001 University of Illinois at Urbana-Champaign -2- HDF
    3. 3. NCSA HDF Mission To develop, promote, deploy, and support open and free technologies that facilitate scientific data storage, exchange, access, analysis and discovery. University of Illinois at Urbana-Champaign -3- HDF
    4. 4. What is HDF? • • • • • • Format and software for scientific data Stores images, arrays, tables, etc. Emphasis on storage and I/O efficiency Free and commercial software support Emphasis on standards Users from many engineering & scientific fields University of Illinois at Urbana-Champaign -4- HDF
    5. 5. HDF People • NCSA – QA, maintanance & support – Library development & performance – Tools development • EOS & ASCI – Users, applications and tools – Performance • Volunteers from all over – Users, applications & tools University of Illinois at Urbana-Champaign -5- HDF
    6. 6. Who is supporting HDF? • NASA/ESDIS – Earth science applications, instrument data – All aspects of data management • DOE/ASCI (Accelerated Strategic Computing Init.) – Simulations on massively parallel machines – Emphasis on parallel I/O performance, functionality • NCSA – Grid, Vis, other R&D, user support • Others – Applications, support, some R&D University of Illinois at Urbana-Champaign -6- HDF
    7. 7. HDF4 • • • • User support, tools, maintenance, QA JHV (Java viewer) - two update releases HDF4 dumper (hdp) - many improvements Next HDF4 Release (Oct. 2000) – – – – Bug fixes Chunking for General Raster (GR) images Unlimited dimensions for SDS GIF <--> HDF4 conversion tools University of Illinois at Urbana-Champaign -7- HDF
    8. 8. HDF4 Archiving • New File Format and Specification Manual – – – – Facilitate long term maintenance of HDF4 Necessary for bug fixing, adding new features Decreases learning curve for new developers Available soon in • PDF, HTML and PS formats University of Illinois at Urbana-Champaign -8- HDF
    9. 9. HDF5 University of Illinois at Urbana-Champaign -9- HDF
    10. 10. The growth of HDF5 • Users in 27 countries • Users include – – – – Government agencies National labs Companies Universities University of Illinois at Urbana-Champaign - 10 - HDF
    11. 11. The growth of HDF5 • Scientific fields – Astronomy, astrophysics, aerospace engineering – Geophysics, remote sensing, meteorology, – Oceanography, environmental Science – Informational Science – Medical Research (brain, cancer, biotech) – Product model data University of Illinois at Urbana-Champaign - 11 - HDF
    12. 12. Facilitating interoperability between HDF4 and HDF5 • HDF4 to HDF5 mapping specification – “Mapping HDF4 Objects to HDF5 Objects” • http://hdf.ncsa.uiuc.edu/HDF5/papers – Rules for mapping high level HDF4 objects to HDF5 • How to describe HDF4 objects in HDF5 • How to interpret HDF5 objects as HDF4 objects • HDF4-to-HDF5 conversion software (later) • Space Research, Inc. Explorer 1.1 – reads both HDF4 and HDF5 University of Illinois at Urbana-Champaign - 12 - HDF
    13. 13. HDF5 Activities in 2000 • • • • • • HDF5 1.2.2 library release Fortran 90 & C++ API HDF5 Abstract Data Model XML Document Type Definition (DTD) HDF5 tools Support for users and application developers • HDF5 Tutorial • Searchable, printable documentation University of Illinois at Urbana-Champaign - 13 - HDF
    14. 14. Tools & Utilities • NCSA – – – – Java wrapper for HDF5 H5View: Java browser/Editor for HDF5 H5gen: XML-to-HDF5 file generator (Java) H5dump & H5ls • Others – – – – VisAD data adapter for HDF5 (Java toolkit) HDF Inspector/Explorer Open Data Explorer (IBM) Ensight University of Illinois at Urbana-Champaign - 15 - HDF
    15. 15. H5View University of Illinois at Urbana-Champaign - 16 - HDF
    16. 16. H5View • Java-based tool for browsing and editing • Display structure of file • Display content of objects • Create and delete objects • Modify values of objects and attributes University of Illinois at Urbana-Champaign - 17 - HDF
    17. 17. HDF5 XML DTD • A flexible standard language that can describe an HDF5 file in precise detail – Datasets, dataspaces, datatypes – Groups and links, structure of the file – Values of the attributes and data • http://hdf.ncsa.uiuc.edu/HDF5/XML University of Illinois at Urbana-Champaign - 18 - HDF
    18. 18. H5Gen: XML HDF5 • Reads an XML description of an HDF5 file • Generates the corresponding HDF5 file • Validates XML description vs. HDF5 DTD University of Illinois at Urbana-Champaign - 19 - HDF
    19. 19. Some uses of XML Archive XML documentation of metadata & structure XML to HDF5 (H5gen) HDF5 to XML Generate, validate, reconstruct HDF5 files Description in XML using HDF5 DTD XML to HTML Catalog records in XML View HDF5 files using a web browser Data location services XML to Java Java to XML HDF5 file - 20 - Java viewers,editors, University of Illinois at Urbana-Champaign other tools HDF
    20. 20. Focus areas for 2001 • • • • • • Support Terra & Aqua Get ready for Aura Support HDF-EOS 3 Enhance HDF5View Refine XML DTD & design tools around it HDF5 converters – HDF5 XML – HDF4-to-HDF5 – Others (e.g. GIF HDF5) University of Illinois at Urbana-Champaign - 21 - HDF
    21. 21. Focus areas for 2001 • • • • • • • • Expand list of applications and users Facilitate access to other tools & software Get vendors on board Performance testing and tuning Extend API with ease-of-use functions Clusters and other new environments Implement a thread-safe version HDF5 advanced tutorial University of Illinois at Urbana-Champaign - 22 - HDF
    22. 22. Thank you! HDF • HDF website – http://hdf.ncsa.uiuc.edu/ 5 • HDF5 Information Center – http://hdf.ncsa.uiuc.edu/HDF5/ • HDF Helpdesk – hdfhelp@ncsa.uiuc.edu • HDF users mailing list – hdfnews@ncsa.uiuc.edu University of Illinois at Urbana-Champaign - 23 - HDF
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×