The EOSDIS Metadata Archive
Ted Habermann
Director of Earth Science
The HDF Group
thabermann@hdfgroup.org
January 8-10, 2014
ESIP Winter 2014
1

The EOSDIS Metadata Archive
Ted Habermann
Director of Earth Science
The HDF Group
thabermann@hdfgroup.org
January 8-10, 2014
ESIP Winter 2014
2

Metadata Types and Sharing
HDF-EOS
Discovery
Use / Mashup
Understanding
January 8-10, 2014
ESIP Winter 2014
netCDF
Core
ACDD
Structural
NUG,
CF
Archive
3

Data Tools Need Use Metadata
78 Tools for EOSDIS Data
ESIP Winter 2014
January 8-10, 2014

Data Tools Need Use Metadata
78 Tools for EOSDIS Data
5
ESIP Winter 2014
January 8-10, 2014

Data Tools Need Use Metadata
78 Tools for EOSDIS Data
6
ESIP Winter 2014
January 8-10, 2014

HDF-EOS Grid Metadata
StructMetadata
+ group
GridStructure
+ group 0..*
Grid
+ group 0..1
Dimension
+ GridName : CharacterString
+ Xdim : Integer
+ Ydim ; Integer
+ UpperLeftPointMtrs : CharacterString
+ LowerRightMtrs : CharacterString
+ Projection : CharacterString
+ ProjParams : Real
+ GridOrigin : Code
+ SphereCode : Integer
+ PixelRegistration : Code
+ ZoneCode : Code
+ group 0..1
MergedFields
+ group 0..1
DataField
+ Dimension
0..*
Dimension
+ DimensionName: CharacterString
+ Size : Integer
DataField
+ DataField 0..*
+ DataFieldName: CharacterString
+ DataType: CharacterString
+ DimList: CharacterString
+ MaxDimList : CharacterString
+ CompressionType : Code
+ DeflateLevel : Integer
+ BlockSize : Integer
+ CompressionParameters : Integer

HDF-EOS Grid Metadata
StructMetadata
+ group
GridStructure
Grid
+ group 0..*
Grid
+ GridName : CharacterString
+ GridName : CharacterString
+ Xdim Integer
+ XDim : Integer ;: Integer
+ Ydim
+ UpperLeftPointMtrs : CharacterString
+ YDim : Integer
+ LowerRightMtrs : CharacterString
+ Projection : CharacterString
+
+ group 0..1 UpperLeftPointMtrs : CharacterString
+ group 0..1
+ ProjParams : Real
+ GridOrigin : Code
+ LowerRightMtrs : CharacterString
+ SphereCode : Integer
+ PixelRegistration : Code
+ Projection : CharacterString
+ ZoneCode : Code
Dimension
MergedFields
+ ProjParams : CharacterString
+ group
+ GridOrigin : Code 0..1
DataField
+
+ Dimension SphereCode : Integer
DataField
0..*
+ PixelRegistration : CodeDataField 0..*
+ DataFieldName: CharacterString
Dimension
+
+ DataType: CharacterString
+ ZoneCode : Code
+ DimensionName: CharacterString
+ DimList: CharacterString
+ Size : Integer
+ MaxDimList : CharacterString
+ CompressionType : Code
+ DeflateLevel : Integer
+ BlockSize : Integer
+ CompressionParameters : Integer

HDF-EOS Swath Metadata
StructMetadata.N
SwathStructure
+ group
+ group 0..*
Swath
+ SwathName : CharacterString
+ group 0..1
Dimension
DimensionMap
IndexDimensionMap
+ group 0..1
+ group 0..1
+ group
0..1
+ group 0..1
+ group 0..1
DataField
GeoField
+ GeoField 0..*
+ Dimension
0..*
+ DimensionMap 0..*
MergedFields
+ DataField 0..*
+ IndexDimensionMap 0..*
GeoField
Dimension
DimensionMap
+ DimensionName: CharacterString
+ Size : Integer
+ GeoDimension: CharacterString
+ DataDimension: CharacterString
+ Offset: Integer
+ Increment: Integer
IndexDimensionMap
+ GeoDimension: CharacterString
+ DataDimension: CharacterString
+ GeoFieldName: CharacterString
+ DataType: Code
+ DimList: CharacterString
+ MaxDimList : CharacterString
+ CompressionType : Code
+ DeflateLevel : Integer
+ BlockSize : Integer
+ CompressionParameters : Integer
DataField
+ DataFieldName: CharacterString
+ DataType: CharacterString
+ DimList: CharacterString
+ MaxDimList : CharacterString
+ CompressionType : Code
+ DeflateLevel : Integer
+ BlockSize : Integer
+ CompressionParameters : Integer

The NASA Metadata Archive
HDF4MapWriter
was used to
extract
metadata from
600+ HDF4
Products for this
analysis.
HDF4Mapper
ODL2XML.xsl

Results
Metadata Groups
50%
Granules
HDF-EOS
76%
Products
HDF-EOS
January 8-10, 2014
ESIP Winter 2014
11

Conclusion
Metadata from the entire HDF4 archive could be read
using a single tool (HDF4Mapper) because the entire
archive is in a single format (HDF4).
Roughly half of the
archive includes
conventional use
metadata
(StructMetadata.N) that
allows it to be plotted /
mapped with
conventional tools.
January 8-10, 2014
ESIP Winter 2014
12

Conclusion
Metadata from the entire HDF4 archive could be read
using a single tool (HDF4Mapper) because the entire
archive is in a single format (HDF4).
Roughly half of the
archive includes
conventional use
metadata
(StructMetadata.N) that
allows it to be plotted /
mapped with
conventional tools.
January 8-10, 2014
ESIP Winter 2014
13

The Earth Science Platform
Conventions
The Earth Science
Platform depends on
consistent formats and
conventions to support
efficient tool/service
development
Formats are working well
but need to be updated
Conventions need work
Tools
Formats
Services
January 8-10, 2014
ESIP Winter 2014
14

The Earth Science Platform
Conventions
The Earth Science
Platform depends on
consistent formats and
conventions to support
efficient tool/service
development
Formats are working well
but need to be updated
Conventions need work
Tools
Formats
Services
January 8-10, 2014
ESIP Winter 2014
15

The Earth Science Platform
Conventions
The Earth Science
Platform depends on
consistent formats and
conventions to support
efficient tool/service
development
Formats are working well
but need to be updated
Conventions need work
Tools
Formats
Services
January 8-10, 2014
ESIP Winter 2014
16

Questions?
tedhabermann@hdfgroup.org
17

Acknowledgements
This work was partially supported by contract number NNG10HP02C from NASA.
Any opinions, findings, conclusions, or recommendations expressed in this material are
those of the author and do not necessarily reflect the views of NASA or The HDF Group.
July 24, 2013
BESSIG
18