SlideShare a Scribd company logo
1 of 17
Download to read offline
Zhangshi Yin, Jingli Yang, Bob Bane,
Muhammad Rabi, Richard Ullman
Data Usability Group, NASA/GSFC
NASA/GSFC, Code 423
For HDF-EOS Workshop VII
September 23-25, 2003
HDF-EOS Data Extractor (HEEX) is a tool for data users
to extract HDF-EOS data to binary or ASCII data format
in HTML or XML.
Flat binary and ASCII data are required by some users.
They can be imported to IDL, ARC/INFO and used in
any programming.
Extracted flat binary and ASCII data can be another
intermediate data format for storing EOS data.
HDF-EOS Data

HTML

Binary Data

ASCII Data

XML

Binary Data

ASCII Data
heex [-aehx] [-f] [FieldName] -o OutputDirectory HDF-EOS5FileName
HDF-EOS5FileName: HDF-EOS5 file name. Required.
-a Extract data to ASCII format. Default is binary. Optional.
-e Extract every field. Optional.
-f FieldName: extract a field data specified by objec/field. Optional.
If no -e and -f options, only index file, no data.
-h Help message. Optional.
-o OutputDirectory: the directory to put data. Required.
-x Extract IndexFile data to XML format. Default is html. Optional.
HEEX will extract all data into a directory indicated in
-o optional by user. A file called Index.html or
Index.xml in the directory will show all HDF-EOS data
structure and where data are.
-HDFEOS-File Name=grid.he Type="HDFEOS"
-HDFEOS
-GRIDS
-Grid
+Parameters
+Dimension
-GridDataField
+sst_mask_1
-HDFEOS_INFORMATION
-COREMETADATA
+CoreMetadata.0
+ARCHIVEMETADATA
-sst_mask_1
DataType=Integer
Sign=false
Size=1
+DimList
+FieldStorage
Data: ./Grid/sst_mask_1.dat
HDF-EOS Metadata Updater (HEMU) is a tool for
data producers or users to update metadata inside an
HDF-EOS file (HDF-EOS2 and HDF-EOS5).
Users need to read metadata from HDF-EOS data.
Users need to write metadata files to HDF-EOS data replace all
metadata in the HDF-EOS data.
Users need to update metadata in HDF-EOS data by merging
metadata inside HDF-EOS data and metadata from .met files.
1. Extract metadata from a dataset (-x option) to a text file
which can then be modified with any text editor. This option
does not modify the HDF-EOS file.
2. Replace metadata with text from an external file (-f option).
The text is syntax-checked for validity as ODL before
updating.
3. Update metadata with text from an external file (-m option).
This option is designed to update metadata using a .met file.
HDF-EOS Data

Read
CoreMetadata
InventoryMetadata
ArchiveMetadata

Metadata
.Met Data

HDF-EOS Data

Read Metadata

Read Metadata

Check Syntax

Check Syntax

Check Values:
SHORTNAME
PRODUCTIONDATETIME
VERSIONID

If equal, replace
metadata in HDF-EOS
using metadata in .met

If not equal, exit
.Met Data

HDF-EOS Data

Read Metadata

Read Metadata

Check Syntax

Check Syntax

Check Values:
SHORTNAME
PRODUCTIONDATETIME
VERSIONID
If equal, replace metadata
in HDF-EOS using
CollectionMetadata
InventoryMetadata in .met
and ArchiveMetadata in HDF-EOS

If not equal, exit
hemu [-h] HDF-EOSFileName -xmf [MetadataFileName]
-h Print help message.
-x Extract metadata from HDFEOS file.
-m Update metadata with a .met file input.
-f Forcibly replace metadata with metadata from input.
HDF-EOSFileName: HDF-EOS file. Required.
MetadataFileName: Metadata file. Optional
(default is standard input/output).
HEMU will check metadata syntax by ODL program.
For -m and –f options (update metadata with a .met file),
three parameters in INVENTORYMETADATA group
(SHORTNAME, PRODUCTIONDATETIME, and
VERSIONID) must match before the tool will modify the
metadata.
http://hdfeos.gsfc.nasa.gov/hdfeos/index.cfm

More Related Content

What's hot

Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 products
Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 productsInteroperability with netCDF-4 - Experience with NPP and HDF-EOS5 products
Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 productsThe HDF-EOS Tools and Information Center
 
Data export in matlab alvian zainuddin
Data export in matlab alvian zainuddinData export in matlab alvian zainuddin
Data export in matlab alvian zainuddinAlvianzainuddin
 
Presentation on basics of python
Presentation on basics of pythonPresentation on basics of python
Presentation on basics of pythonNanditaDutta4
 
8086 handout for chapter one and two
8086 handout for chapter one and two8086 handout for chapter one and two
8086 handout for chapter one and twohaymanotyehuala
 
1327 Addressing Modes Of 8086
1327 Addressing Modes Of 80861327 Addressing Modes Of 8086
1327 Addressing Modes Of 8086techbed
 
Computer Hardware | 3B
Computer Hardware | 3BComputer Hardware | 3B
Computer Hardware | 3BCMDLMS
 
2015q4_InnerCourse_Presentation
2015q4_InnerCourse_Presentation2015q4_InnerCourse_Presentation
2015q4_InnerCourse_PresentationHung-Wei Liu
 

What's hot (20)

Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 products
Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 productsInteroperability with netCDF-4 - Experience with NPP and HDF-EOS5 products
Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 products
 
Unit 5
Unit 5Unit 5
Unit 5
 
Data export in matlab alvian zainuddin
Data export in matlab alvian zainuddinData export in matlab alvian zainuddin
Data export in matlab alvian zainuddin
 
HDF5 In Support of Database Applications
HDF5 In Support of Database ApplicationsHDF5 In Support of Database Applications
HDF5 In Support of Database Applications
 
Product Designer Hub - Taking HPD to the Web
Product Designer Hub - Taking HPD to the WebProduct Designer Hub - Taking HPD to the Web
Product Designer Hub - Taking HPD to the Web
 
NetCDF and HDF5
NetCDF and HDF5NetCDF and HDF5
NetCDF and HDF5
 
Introductio to c++
Introductio to c++Introductio to c++
Introductio to c++
 
Tools to improve the usability of NASA HDF Data
Tools to improve the usability of NASA HDF DataTools to improve the usability of NASA HDF Data
Tools to improve the usability of NASA HDF Data
 
Standard Library Functions
Standard Library FunctionsStandard Library Functions
Standard Library Functions
 
HDF Update
HDF UpdateHDF Update
HDF Update
 
HDF4 Mapping Project Update
HDF4 Mapping Project UpdateHDF4 Mapping Project Update
HDF4 Mapping Project Update
 
Presentation on basics of python
Presentation on basics of pythonPresentation on basics of python
Presentation on basics of python
 
HDF Product Designer: Using Templates to Achieve Interoperability
HDF Product Designer: Using Templates to Achieve InteroperabilityHDF Product Designer: Using Templates to Achieve Interoperability
HDF Product Designer: Using Templates to Achieve Interoperability
 
8086 handout for chapter one and two
8086 handout for chapter one and two8086 handout for chapter one and two
8086 handout for chapter one and two
 
1327 Addressing Modes Of 8086
1327 Addressing Modes Of 80861327 Addressing Modes Of 8086
1327 Addressing Modes Of 8086
 
Lect 8 updated (1)
Lect 8 updated (1)Lect 8 updated (1)
Lect 8 updated (1)
 
Tree
TreeTree
Tree
 
Status of HDF-EOS, Related Software and Tools
Status of HDF-EOS, Related Software and ToolsStatus of HDF-EOS, Related Software and Tools
Status of HDF-EOS, Related Software and Tools
 
Computer Hardware | 3B
Computer Hardware | 3BComputer Hardware | 3B
Computer Hardware | 3B
 
2015q4_InnerCourse_Presentation
2015q4_InnerCourse_Presentation2015q4_InnerCourse_Presentation
2015q4_InnerCourse_Presentation
 

Similar to HDF-EOS Data Extractor & Metadata Updater

Federated Queries Across Both Different Storage Mediums and Different Data En...
Federated Queries Across Both Different Storage Mediums and Different Data En...Federated Queries Across Both Different Storage Mediums and Different Data En...
Federated Queries Across Both Different Storage Mediums and Different Data En...VMware Tanzu
 
ESDIS Metadata Archive
ESDIS Metadata ArchiveESDIS Metadata Archive
ESDIS Metadata ArchiveTed Habermann
 
Intro to Hadoop
Intro to HadoopIntro to Hadoop
Intro to Hadoopjeffturner
 
Accessing external hadoop data sources using pivotal e xtension framework (px...
Accessing external hadoop data sources using pivotal e xtension framework (px...Accessing external hadoop data sources using pivotal e xtension framework (px...
Accessing external hadoop data sources using pivotal e xtension framework (px...Sameer Tiwari
 

Similar to HDF-EOS Data Extractor & Metadata Updater (20)

Introduction to HDF5
Introduction to HDF5Introduction to HDF5
Introduction to HDF5
 
Proposal for adding Named Dimensions to HDF5 Arrays
Proposal for adding Named Dimensions to HDF5 ArraysProposal for adding Named Dimensions to HDF5 Arrays
Proposal for adding Named Dimensions to HDF5 Arrays
 
Introduction to HDF5
Introduction to HDF5Introduction to HDF5
Introduction to HDF5
 
Federated Queries Across Both Different Storage Mediums and Different Data En...
Federated Queries Across Both Different Storage Mediums and Different Data En...Federated Queries Across Both Different Storage Mediums and Different Data En...
Federated Queries Across Both Different Storage Mediums and Different Data En...
 
UML Representation of NPOESS Data Products in HDF5
UML Representation of NPOESS Data Products in HDF5UML Representation of NPOESS Data Products in HDF5
UML Representation of NPOESS Data Products in HDF5
 
HDF-EOS Maintenance, Current Development and Tools
HDF-EOS Maintenance, Current Development and ToolsHDF-EOS Maintenance, Current Development and Tools
HDF-EOS Maintenance, Current Development and Tools
 
HDF5 Advanced Topics
HDF5 Advanced TopicsHDF5 Advanced Topics
HDF5 Advanced Topics
 
HDF5 Advanced Topics
HDF5 Advanced TopicsHDF5 Advanced Topics
HDF5 Advanced Topics
 
ESDIS Metadata Archive
ESDIS Metadata ArchiveESDIS Metadata Archive
ESDIS Metadata Archive
 
Introduction to HDF5 Data and Programming Models
Introduction to HDF5 Data and Programming ModelsIntroduction to HDF5 Data and Programming Models
Introduction to HDF5 Data and Programming Models
 
HDF5 Advanced Topics - Datatypes and Partial I/O
HDF5 Advanced Topics - Datatypes and Partial I/OHDF5 Advanced Topics - Datatypes and Partial I/O
HDF5 Advanced Topics - Datatypes and Partial I/O
 
Advanced HDF5 Features
Advanced HDF5 FeaturesAdvanced HDF5 Features
Advanced HDF5 Features
 
HDF5 Life cycle of data
HDF5 Life cycle of dataHDF5 Life cycle of data
HDF5 Life cycle of data
 
Intro to Hadoop
Intro to HadoopIntro to Hadoop
Intro to Hadoop
 
Using HDF5 and Python: The H5py module
Using HDF5 and Python: The H5py moduleUsing HDF5 and Python: The H5py module
Using HDF5 and Python: The H5py module
 
Accessing external hadoop data sources using pivotal e xtension framework (px...
Accessing external hadoop data sources using pivotal e xtension framework (px...Accessing external hadoop data sources using pivotal e xtension framework (px...
Accessing external hadoop data sources using pivotal e xtension framework (px...
 
Support for NPP/NPOESS by The HDF Group
Support for NPP/NPOESS by The HDF GroupSupport for NPP/NPOESS by The HDF Group
Support for NPP/NPOESS by The HDF Group
 
Matlab, Big Data, and HDF Server
Matlab, Big Data, and HDF ServerMatlab, Big Data, and HDF Server
Matlab, Big Data, and HDF Server
 
The hadoop file_system
The hadoop file_systemThe hadoop file_system
The hadoop file_system
 
HDF5 iRODS
HDF5 iRODSHDF5 iRODS
HDF5 iRODS
 

More from The HDF-EOS Tools and Information Center

STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...The HDF-EOS Tools and Information Center
 

More from The HDF-EOS Tools and Information Center (20)

Cloud-Optimized HDF5 Files
Cloud-Optimized HDF5 FilesCloud-Optimized HDF5 Files
Cloud-Optimized HDF5 Files
 
Accessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDSAccessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDS
 
The State of HDF
The State of HDFThe State of HDF
The State of HDF
 
Highly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance FeaturesHighly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance Features
 
Creating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 FilesCreating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 Files
 
HDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance DiscussionHDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance Discussion
 
Hyrax: Serving Data from S3
Hyrax: Serving Data from S3Hyrax: Serving Data from S3
Hyrax: Serving Data from S3
 
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLABAccessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
 
HDF - Current status and Future Directions
HDF - Current status and Future DirectionsHDF - Current status and Future Directions
HDF - Current status and Future Directions
 
HDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and FutureHDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and Future
 
HDF - Current status and Future Directions
HDF - Current status and Future Directions HDF - Current status and Future Directions
HDF - Current status and Future Directions
 
H5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only LibraryH5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only Library
 
MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10
 
HDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDFHDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDF
 
HDF5 <-> Zarr
HDF5 <-> ZarrHDF5 <-> Zarr
HDF5 <-> Zarr
 
HDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server FeaturesHDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server Features
 
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
 
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
 
HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?
 
HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020
 

Recently uploaded

Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 

Recently uploaded (20)

Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 

HDF-EOS Data Extractor & Metadata Updater

  • 1. Zhangshi Yin, Jingli Yang, Bob Bane, Muhammad Rabi, Richard Ullman Data Usability Group, NASA/GSFC NASA/GSFC, Code 423 For HDF-EOS Workshop VII September 23-25, 2003
  • 2. HDF-EOS Data Extractor (HEEX) is a tool for data users to extract HDF-EOS data to binary or ASCII data format in HTML or XML.
  • 3. Flat binary and ASCII data are required by some users. They can be imported to IDL, ARC/INFO and used in any programming. Extracted flat binary and ASCII data can be another intermediate data format for storing EOS data.
  • 4. HDF-EOS Data HTML Binary Data ASCII Data XML Binary Data ASCII Data
  • 5. heex [-aehx] [-f] [FieldName] -o OutputDirectory HDF-EOS5FileName HDF-EOS5FileName: HDF-EOS5 file name. Required. -a Extract data to ASCII format. Default is binary. Optional. -e Extract every field. Optional. -f FieldName: extract a field data specified by objec/field. Optional. If no -e and -f options, only index file, no data. -h Help message. Optional. -o OutputDirectory: the directory to put data. Required. -x Extract IndexFile data to XML format. Default is html. Optional.
  • 6. HEEX will extract all data into a directory indicated in -o optional by user. A file called Index.html or Index.xml in the directory will show all HDF-EOS data structure and where data are.
  • 9. HDF-EOS Metadata Updater (HEMU) is a tool for data producers or users to update metadata inside an HDF-EOS file (HDF-EOS2 and HDF-EOS5).
  • 10. Users need to read metadata from HDF-EOS data. Users need to write metadata files to HDF-EOS data replace all metadata in the HDF-EOS data. Users need to update metadata in HDF-EOS data by merging metadata inside HDF-EOS data and metadata from .met files.
  • 11. 1. Extract metadata from a dataset (-x option) to a text file which can then be modified with any text editor. This option does not modify the HDF-EOS file. 2. Replace metadata with text from an external file (-f option). The text is syntax-checked for validity as ODL before updating. 3. Update metadata with text from an external file (-m option). This option is designed to update metadata using a .met file.
  • 13. .Met Data HDF-EOS Data Read Metadata Read Metadata Check Syntax Check Syntax Check Values: SHORTNAME PRODUCTIONDATETIME VERSIONID If equal, replace metadata in HDF-EOS using metadata in .met If not equal, exit
  • 14. .Met Data HDF-EOS Data Read Metadata Read Metadata Check Syntax Check Syntax Check Values: SHORTNAME PRODUCTIONDATETIME VERSIONID If equal, replace metadata in HDF-EOS using CollectionMetadata InventoryMetadata in .met and ArchiveMetadata in HDF-EOS If not equal, exit
  • 15. hemu [-h] HDF-EOSFileName -xmf [MetadataFileName] -h Print help message. -x Extract metadata from HDFEOS file. -m Update metadata with a .met file input. -f Forcibly replace metadata with metadata from input. HDF-EOSFileName: HDF-EOS file. Required. MetadataFileName: Metadata file. Optional (default is standard input/output).
  • 16. HEMU will check metadata syntax by ODL program. For -m and –f options (update metadata with a .met file), three parameters in INVENTORYMETADATA group (SHORTNAME, PRODUCTIONDATETIME, and VERSIONID) must match before the tool will modify the metadata.