The HDF format is the foundation for sharing data in many communities that have created domain-specific conventions on top of HDF. This presentation was given at the Winter meeting of the Earth Science Information Partnership (ESIP).
Python and HDF
From the Introduction:
As a graduate student I didn’t care
that much about the details of how
my data were stored… As a
scientist, I eventually came to
recognize that the choices we make
for organizing and storing our data
are also choices about
communication. Not only do
standard, well-designed formats
make life easier for individuals, but
they make it possible to share data
with a global audience.
2
New book on Pythin and HDF by Andrew Collette. The quote from the Introduction indicates that Andrew understands the important role of standard formats in data sharing.
Andrew works at the Colorado Center for Lunar Dust and Atmospheric Studies (http://lasp.colorado.edu/home/ccldas/). This large NASA facility is used by scientists in NASA and around the world to study very high speed dust impacts on instrument windows and other objects. Scientists take their observations home in HDF5 files.
The Colorado Center for Lunar Dust and Atmospheric Studies (http://lasp.colorado.edu/home/ccldas/) has HDFIndise.
The NASA Earth Observing System Data and Information System (https://earthdata.nasa.gov) has #HDFInside.
netCDF4 (http://www.unidata.ucar.edu/software/netcdf/) has #HDFInside
The Bathymetry Attributed Grid (BAG) developed by the Open Navigational Surface (http://www.opennavsurf.org) and used by the National Ocean Service (http://www.ngdc.noaa.gov/mgg/bathymetry/hydro.html) has #HDFInside
The NeXus Scientific Format has #HDFInside http://www.nexusformat.org
HDF5-FastQuery developed by the Visualization Groupat Lawrence Berkely National Lab (http://www-vis.lbl.gov/Events/SC05/HDF5FastQuery/) has #HDFInside
h5py (http://www.h5py.org) developed at the Colorado Center for Lunar Dust and Atmospheric Studies (http://lasp.colorado.edu/home/ccldas/) and PyTables (http://www.pytables.org/moin) have #HDFInside.
GLOBE Claritas (http://www.globeclaritas.com) seismic data processing software and h5vc software for scalable nucleotide tallies (http://www.bioconductor.org/packages/2.14/bioc/html/h5vc.html) have #HDFInside.
The KAIRA (Kilpisjärvi Atmospheric Imaging Receiver Array) dual array of omni-directional VHF radio antennas,a project of the Sodankylä Geophysical Observatory and the Low Frequency Array (LOFAR http://www.lofar.org) have #HDFInside.
MathWorks (MATLAB, http://www.mathworks.com ) and Exelis (IDL, ENVI, http://www.exelisinc.com/Pages/default.aspx) have #HDFInside.
The NASA Earth Observing System Data and Information System (https://earthdata.nasa.gov) has #HDFInside.netCDF4 (http://www.unidata.ucar.edu/software/netcdf/) has #HDFInsideThe Bathymetry Attributed Grid (BAG) developed by the Open Navigational Surface (http://www.opennavsurf.org) and used by the National Ocean Service (http://www.ngdc.noaa.gov/mgg/bathymetry/hydro.html) has #HDFInsideThe NeXus Scientific Format has #HDFInside http://www.nexusformat.orgHDF5-FastQuery developed by the Visualization Groupat Lawrence Berkely National Lab (http://www-vis.lbl.gov/Events/SC05/HDF5FastQuery/) has #HDFInsideh5py (http://www.h5py.org) developed at the Colorado Center for Lunar Dust and Atmospheric Studies (http://lasp.colorado.edu/home/ccldas/) and PyTables (http://www.pytables.org/moin) have #HDFInside.GLOBE Claritas (http://www.globeclaritas.com) seismic data processing software and h5vc software for scalable nucleotide tallies (http://www.bioconductor.org/packages/2.14/bioc/html/h5vc.html) have #HDFInside.The KAIRA (Kilpisjärvi Atmospheric Imaging Receiver Array) dual array of omni-directional VHF radio antennas,a project of the Sodankylä Geophysical Observatory and the Low Frequency Array (LOFAR http://www.lofar.org) have #HDFInside.MathWorks (MATLAB, http://www.mathworks.com ) and Exelis (IDL, ENVI, http://www.exelisinc.com/Pages/default.aspx) have #HDFInside.