Dimension Scales
in
HDF-EOS2 & HDF-EOS5
Abe Taaheri, Raytheon IIS
HDF & HDF-EOS Workshop XIV
Champaign, IL
Sep. 29, 2010

...
•
•
•

What is a Dimension Scale ?
Dimension Scales & Metadata
Dimension Scales APIs
–
–

•

Code example
–
–

•

he2
he5
...
What is a Dimension Scale?
It is a sequence of numbers
placed along a dimension to
demarcate intervals along it

• HDF4
- ...
* What is a Dimension Scale?

• HDF5
– A HDF5 dataset.
– With additional metadata that identifies
the dataset as a Dimensi...
Example: 3D dataset
3 Dimension Scales Datasets

5
7

Dataset: 3D Array with
5 x 7 x 10
dimensions

10
Page 5
More on Dimension Scale
in HDF5
• A dimension scale is not required to be a 1-D
array, or to have a specific datatype

• A...
Example: 3D dataset in HDF5
Several Dimension Scales
Datasets

Dataset: 3D Array with
5 x 7 x 10
dimensions

Page 7
HDF5 Dimension Scale Metadata
• When the Dimension Scale is associated with
a dimension of a Dataset, the association is
r...
• HDF4
– Since Version 4.0 ?
– SDsetdimscale()
– SDgetdimscale()
– SDsetdimstrs(): label, unit, format
– SDgetdimstrs()
• ...
HDF Augmentation Tool

Augmentation Tool adds
HDF5 Dimension Scales
inside the file so that
netCDF-4 APIs can access
the d...
•

HDF-EOS

–

Added a few routines to HDF-EOS2 to
create dimension scales like HDF4
 Version 2.17

–

Added a few routin...
HDF-EOS2 APIs

Page 12
• Name:

SWsetdimscale

• Signature:
– intn SWsetdimscale( int32 swathID,
char *fieldname, char *dimname, int32 dimsize,
i...
• Name:

SWgetdimscale

• Signature:
– intn SWgetdimscale( int32 swathID,
char *fieldname, char *dimname,
int32 *dimsize, ...
• Name:

SWsetdimstrs

• Signature:
– intn SWsetdimstrs( int32 swathID,
char *fieldname, char *dimname, char *label,
char ...
• Name:

SWgetdimstrs

• Signature:
– intn SWgetdimstrs( int32 swathID, char
*fieldname, char *dimname, char *label,
char ...
• Similar APIs for Dimension Scales in
– Grid Fields

Page 17
HDF-EOS5 APIs

Page 18
• Name:

HE5_SWsetdimscale

• Signature:
– herr_t HE5_SWsetdimscale( hid_t swathID,
char *fieldname, char *dimname,
const ...
• Name:

HE5_SWgetdimscale

• Signature:
– long HE5_SWgetdimscale( hid_t swathID,
char *fieldname, char *dimname,
hsize_t ...
• Name:

HE5_SWwritedscaleattr

• Signature:
– herr_t HE5_SWwritedscaleattr( hid_t swathID,
const char *dimname, const cha...
• Name:

HE5_SWreaddscaleattr

• Signature:
– herr_t HE5_SWreaddscaleattr( hid_t swathID,
const char *dimname, const char ...
• Name:

HE5_SWinqdscaleattrs

• Signature:
– Long HE5_SWinqdscaleattrs( hid_t swathID,
const char *dimname, char *attrnam...
• Name:

HE5_SWdscaleattrinfo

• Signature:
– herr_t HE5_SWdscaleattrinfo( hid_t swathID,
const char *dimname, const char ...
• Similar APIs for Dimension Scales in
– Grid

– Zonal Average

Page 25
Code Examples

Page 26
Writing Code Example (HDF-EOS2)
#include "hdf.h"
#include "HdfEosDef.h"
int main()
{
intn
status, i;
int32
swfid, SWid;
in...
nGeoTrack = 20;
for (i=0; i<20; i++) { dataGeoTrack[i] = (float) I; }
status = SWsetdimscale ( SWid, "Temperature", "GeoTr...
Writing Code Example (HDF-EOS5)
#include <HE5_HdfEosDef.h>
int main()
{
herr_t
status = FAIL;
hid_t
swfid = FAIL;
hid_t
SW...
nbands = 15;
status = HE5_SWsetdimscale ( SWid1, "Spectra", "Bands", nbands,
H5T_NATIVE_INT, (void *)data );
strcpy(label,...
int datbuf_i1[1] = {-999};
count[0]= 1;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "MissingValue",
H5T_NATIVE_INT, c...
Reading Code Example (HDF-EOS2)
#include "hdf.h"
#include "HdfEosDef.h"
Int main()
{
intn
status, i;
int32
swfid, SWid;
in...
datbuf_int32 = (int32 *) malloc(buffsize);
buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1,
&data_type, (vo...
Reading Code Example (HDF-EOS5)
#include <HE5_HdfEosDef.h>
int main()
{
herr_t
status = FAIL;
hid_t
swfid = FAIL, SWid = F...
bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize,
&numtype, NULL );
databuff = (int *) malloc(bufsize );
b...
for( i = 0; i < nattr; i++)
{
attrname = (char *) calloc(fldnmlen[i] + 1, sizeof(char) );
memmove ( attrname,fldnm[i],fldn...
if( (int)*ntype == 0) {
attr_int = (int *) attr;
for (j = 0; j < count; j++) { printf("tAttrvalue[%d]:
}
if( (int)*ntype =...
Test File content (he2)
HDFView 2.4:
uses plug-in with
hdf-eos5-1.11,
Hdfeos2.15,
HDF4.2r3,
HDF5-1.6.7

Page 38
Test File content (he5)
HDFView 2.4:
uses plug-in with
hdf-eos5-1.11,
Hdfeos2.15,
HDF4.2r3,
HDF5-1.6.7

Page 39
Test File content (he5)
HDFView 2.6:
Without plug-in
HDF4.2r4,
HDF5-1.8.3

Page 40
Test File content (he5)
HDFView 2.6:
uses plug-in with
hdf-eos5-1.12,
Hdfeos2.16,
HDF4.2r4,
HDF5-1.8.3

Page 41
File content (HIRDLS he5)
HIRDLS he5
Modified with
HDF
Augmentation
Tool

Page 42
Suggestions?

– Do we need new APIs ?
– Improvements for existing APIs ?
– What else?

Page 43
Thank You!

Page 44
Upcoming SlideShare
Loading in …5
×

Dimension Scales in HDF-EOS2 and HDF-EOS5

436 views

Published on

Dimension Scales for HDF-EOS2 and HDF-EOS5 field dimensions were added to the new release of HDF-EOS. The new APIs will be presented and sample outputs will be shown. Need for development of new APIs for handling Dimension Scales will be discussed.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
436
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Dimension Scales in HDF-EOS2 and HDF-EOS5

  1. 1. Dimension Scales in HDF-EOS2 & HDF-EOS5 Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshop XIV Champaign, IL Sep. 29, 2010 Page 1
  2. 2. • • • What is a Dimension Scale ? Dimension Scales & Metadata Dimension Scales APIs – – • Code example – – • he2 he5 Writing (he2, he5) Reading (he2, he5) Sample he2 and he5 files with Dimension Scales Page 2
  3. 3. What is a Dimension Scale? It is a sequence of numbers placed along a dimension to demarcate intervals along it • HDF4 - It is an array with size and name similar to its assigned dimension - Stored using a structure similar to the SDS array - One scale is assigned per dimension Page 3
  4. 4. * What is a Dimension Scale? • HDF5 – A HDF5 dataset. – With additional metadata that identifies the dataset as a Dimension Scale – Typically Dimension Scales are logically associated with the dimensions of HDF5 Datasets – The meaning of the association is left to applications. * Pedro Vicente talk, HDF/ HDF-EOS Workshop IX Page 4
  5. 5. Example: 3D dataset 3 Dimension Scales Datasets 5 7 Dataset: 3D Array with 5 x 7 x 10 dimensions 10 Page 5
  6. 6. More on Dimension Scale in HDF5 • A dimension scale is not required to be a 1-D array, or to have a specific datatype • A dataset dimension can have more than 1 associated dimension scale • A Dimension Scale can be shared by two or more Dataset dimensions Page 6
  7. 7. Example: 3D dataset in HDF5 Several Dimension Scales Datasets Dataset: 3D Array with 5 x 7 x 10 dimensions Page 7
  8. 8. HDF5 Dimension Scale Metadata • When the Dimension Scale is associated with a dimension of a Dataset, the association is represented by attributes of the two datasets. • The following dataset attributes are used to describe dimension scale datasets: – Attribute named “CLASS” with the value “DIMENSION_SCALE” – Optional attribute named “NAME” – Attribute references to any associated Dataset Page 8
  9. 9. • HDF4 – Since Version 4.0 ? – SDsetdimscale() – SDgetdimscale() – SDsetdimstrs(): label, unit, format – SDgetdimstrs() • HDF5 – Since Version 1.8 – H5DSset_scale() – H5DSattach_scale(), H5DSdetach_scale() – H5DSset_label(), H5DSget_label() – A few more APIs Page 9
  10. 10. HDF Augmentation Tool Augmentation Tool adds HDF5 Dimension Scales inside the file so that netCDF-4 APIs can access the data Page 10
  11. 11. • HDF-EOS – Added a few routines to HDF-EOS2 to create dimension scales like HDF4  Version 2.17 – Added a few routines to HDF-EOS5 to create dimension scales like those added by the HDF Augmentation Tool  Version 1.13 Page 11
  12. 12. HDF-EOS2 APIs Page 12
  13. 13. • Name: SWsetdimscale • Signature: – intn SWsetdimscale( int32 swathID, char *fieldname, char *dimname, int32 dimsize, int32 numbertype, VOIDP data) • Purpose: – Sets dimension scale for a given dimension Page 13
  14. 14. • Name: SWgetdimscale • Signature: – intn SWgetdimscale( int32 swathID, char *fieldname, char *dimname, int32 *dimsize, int32 *numbertype, VOIDP data) • Purpose: – Gets dimension scale for a given dimension Page 14
  15. 15. • Name: SWsetdimstrs • Signature: – intn SWsetdimstrs( int32 swathID, char *fieldname, char *dimname, char *label, char *unit, char *format) • Purpose: – Sets the label, unit, and format strings for a given dimension Page 15
  16. 16. • Name: SWgetdimstrs • Signature: – intn SWgetdimstrs( int32 swathID, char *fieldname, char *dimname, char *label, char *unit, char *format, intn len) • Purpose: – Gets the label, unit, and format strings for a given dimension Page 16
  17. 17. • Similar APIs for Dimension Scales in – Grid Fields Page 17
  18. 18. HDF-EOS5 APIs Page 18
  19. 19. • Name: HE5_SWsetdimscale • Signature: – herr_t HE5_SWsetdimscale( hid_t swathID, char *fieldname, char *dimname, const hsize_t dimsize, hid_t numbertype, void * data) • Purpose: – Sets dimension scale for a field dimension within the swath Page 19
  20. 20. • Name: HE5_SWgetdimscale • Signature: – long HE5_SWgetdimscale( hid_t swathID, char *fieldname, char *dimname, hsize_t *dimsize, hid_t *numbertype, void * data) • Purpose: – Gets dimension scale for a field dimension within the swath Page 20
  21. 21. • Name: HE5_SWwritedscaleattr • Signature: – herr_t HE5_SWwritedscaleattr( hid_t swathID, const char *dimname, const char *attrname, hid_t ntype, hsize_t count[], void *datbuf) • Purpose: – Writes/Updates a dimension scale attribute in a specific swath Page 21
  22. 22. • Name: HE5_SWreaddscaleattr • Signature: – herr_t HE5_SWreaddscaleattr( hid_t swathID, const char *dimname, const char *attrname, void *datbuf) • Purpose: – Reads a dimension scale attribute from a specific dimension Page 22
  23. 23. • Name: HE5_SWinqdscaleattrs • Signature: – Long HE5_SWinqdscaleattrs( hid_t swathID, const char *dimname, char *attrnames, long *strbufsize) • Purpose: – Retrieve information about the attributes defined Page 23
  24. 24. • Name: HE5_SWdscaleattrinfo • Signature: – herr_t HE5_SWdscaleattrinfo( hid_t swathID, const char *dimname, const char *attrname, hid_t *ntype, hsize_t *count) count : Number of attribute elements • Purpose: – Returns information about attribute(s) in a specific dimension scale Page 24
  25. 25. • Similar APIs for Dimension Scales in – Grid – Zonal Average Page 25
  26. 26. Code Examples Page 26
  27. 27. Writing Code Example (HDF-EOS2) #include "hdf.h" #include "HdfEosDef.h" int main() { intn status, i; int32 swfid, SWid; int32 nbands, nGeoTrack; int32 bands[15]={3,6,9,12,15,18,23,26,29,32,33,34,36,37,39}; char label[16]; char unit[16]; char format[16]; float dataGeoTrack[20]; swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR ); SWid = SWattach ( swfid, "Swath1“ ); Page 27
  28. 28. nGeoTrack = 20; for (i=0; i<20; i++) { dataGeoTrack[i] = (float) I; } status = SWsetdimscale ( SWid, "Temperature", "GeoTrack", nGeoTrack, DFNT_FLOAT32, dataGeoTrack ); nbands = 15; status = SWsetdimscale ( SWid, "Spectra", "Bands", nbands, DFNT_INT32, bands ); strcpy (label, "Bands"); strcpy (unit, "none"); strcpy (format, "I2"); status = SWsetdimstrs ( SWid, "Spectra", "Bands", label, unit, format ); SWdetach ( SWid ); SWclose ( swfid ); return 0; } Page 28
  29. 29. Writing Code Example (HDF-EOS5) #include <HE5_HdfEosDef.h> int main() { herr_t status = FAIL; hid_t swfid = FAIL; hid_t SWid1 = FAIL; int data[15]={1,2,3,6,9,12,15,18,23,26,29,32, 33,34,35}; char label[16], unit[16], format[16]; int nbands; hsize_t count[3]; swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR ); SWid1 = HE5_SWattach ( swfid, "Swath1“ ); Page 29
  30. 30. nbands = 15; status = HE5_SWsetdimscale ( SWid1, "Spectra", "Bands", nbands, H5T_NATIVE_INT, (void *)data ); strcpy(label, "Bands Dim"); strcpy(unit, "None"); strcpy(format, "I2"); count[0]= 12; status = HE5_SWwritedscaleattr ( SWid1, "Bands", "label", H5T_NATIVE_CHAR, count, label ); count[0]= 6; status = HE5_SWwritedscaleattr ( SWid1, "Bands", "unit", H5T_NATIVE_CHAR, count, unit ); count[0]= 4; status = HE5_SWwritedscaleattr ( SWid1, "Bands", "format", H5T_NATIVE_CHAR, count, format ); Page 30
  31. 31. int datbuf_i1[1] = {-999}; count[0]= 1; status = HE5_SWwritedscaleattr ( SWid1, "Bands", "MissingValue", H5T_NATIVE_INT, count, datbuf_i1 ); int datbuf_i2[3] = {-999, 0, 999}; count[0]= 3; status = HE5_SWwritedscaleattr(SWid1, "Bands", "IntValues", H5T_NATIVE_INT, count, datbuf_i2 ); HE5_SWdetach ( SWid1 ); HE5_SWclose ( swfid ); return 0; } Page 31
  32. 32. Reading Code Example (HDF-EOS2) #include "hdf.h" #include "HdfEosDef.h" Int main() { intn status, i; int32 swfid, SWid; int32 *datbuf_int32; int32 nbands, data_type, dimsize, dimsize1; intn len, buffsize; char label[16], unit[16], format[16]; swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR ); SWid = SWattach ( swfid, "Swath1“ ); /* get Bands */ dimsize = SWdiminfo ( SWid, "Bands“ ); buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1, &data_type, NULL ); Page 32
  33. 33. datbuf_int32 = (int32 *) malloc(buffsize); buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1, &data_type, (void *)datbuf_int32 ); for (i=0; i<dimsize; i++) { printf("i = %d datbuf_int32 = %dn", i, datbuf_int32[i]); } /* get str attributes of Bands*/ len = 15; status = SWgetdimstrs ( SWid, "Spectra", "Bands", label, unit, format, len ); printf(" ttlabel = %s unit = %s format = %s nn", label, unit, format); free(datbuf_int32); datbuf_int32 = NULL; SWdetach ( SWid ); SWclose ( swfid ); return 0; } Page 33
  34. 34. Reading Code Example (HDF-EOS5) #include <HE5_HdfEosDef.h> int main() { herr_t status = FAIL; hid_t swfid = FAIL, SWid = FAIL, numtype; int i,j, bufsize; int *databuff; long nattr, strbufsize; char *attrlist, *attrname = (char *)NULL; size_t fldnmlen[HE5_HDFE_NAMBUFSIZE]; char *fldnm[HE5_HDFE_NAMBUFSIZE]; hid_t *ntype; hsize_t count = 0; void *attr; int *attr_int; float *attr_flt; char *attr_char; hsize_t dimsize; swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR ); SWid = HE5_SWattach ( swfid, "Swath1“ ); Page 34
  35. 35. bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize, &numtype, NULL ); databuff = (int *) malloc(bufsize ); bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize, &numtype, (void *)databuff ); for( i = 0; i < dimsize; i++) { printf ("DscaleVal[%d] = %dn",i,databuff[i] ); } nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", NULL, &strbufsize ); attrlist = (char *) calloc(strbufsize + 1, sizeof(char)); nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", attrlist, &strbufsize ); nattr = HE5_EHparsestr(attrlist, ',', fldnm, fldnmlen); Page 35
  36. 36. for( i = 0; i < nattr; i++) { attrname = (char *) calloc(fldnmlen[i] + 1, sizeof(char) ); memmove ( attrname,fldnm[i],fldnmlen[i] ); ntype = (hid_t *) calloc(1, sizeof(hid_t)); if ( strcmp ( attrname, "REFERENCE_LIST") == 0 ) continue; status = HE5_SWdscaleattrinfo ( SWid,"Bands", attrname, ntype, &count ); if( (int)*ntype == 0) { attr_int = (int *) malloc(count*sizeof(int)); attr = (void *) attr_int; } if( (int)*ntype == 10) { attr_flt = (float *) malloc(count*sizeof(float)); attr = (void *) attr_flt; } if( (int)*ntype == 57) { attr_char = (char *) malloc((count+1)*sizeof(char)); attr = (void *) attr_char; } status = HE5_SWreaddscaleattr ( SWid,"Bands", attrname, attr ); printf("tAttribute Name: %s n",attrname); printf("tNumber of attr elem: %lu n", (unsigned long)count); printf("tData type of attribute: %d n", (int)*ntype); Page 36
  37. 37. if( (int)*ntype == 0) { attr_int = (int *) attr; for (j = 0; j < count; j++) { printf("tAttrvalue[%d]: } if( (int)*ntype == 10) { attr_flt = (float *) attr; for (j = 0; j < count; j++) { printf("tAttrvalue[%d]: } } } if( (int)*ntype == 57) { attr_char = (char *) attr; attr_char[count]='0'; printf("tAttrvalue: free ( attr_char ); } %d n", j,attr_int[j]); } %f n", j,attr_flt[j]); } %s n", attr_char); HE5_SWdetach ( SWid ); HE5_SWclose ( swfid ); return 0; Page 37
  38. 38. Test File content (he2) HDFView 2.4: uses plug-in with hdf-eos5-1.11, Hdfeos2.15, HDF4.2r3, HDF5-1.6.7 Page 38
  39. 39. Test File content (he5) HDFView 2.4: uses plug-in with hdf-eos5-1.11, Hdfeos2.15, HDF4.2r3, HDF5-1.6.7 Page 39
  40. 40. Test File content (he5) HDFView 2.6: Without plug-in HDF4.2r4, HDF5-1.8.3 Page 40
  41. 41. Test File content (he5) HDFView 2.6: uses plug-in with hdf-eos5-1.12, Hdfeos2.16, HDF4.2r4, HDF5-1.8.3 Page 41
  42. 42. File content (HIRDLS he5) HIRDLS he5 Modified with HDF Augmentation Tool Page 42
  43. 43. Suggestions? – Do we need new APIs ? – Improvements for existing APIs ? – What else? Page 43
  44. 44. Thank You! Page 44

×