Report
Share
Download to read offline
Recommended
Night Water Vapor Borel Spie 8 12 08 White
The document describes a method for retrieving column water vapor at night using mid-wave infrared bands from sensors like MODIS. It adapts an existing algorithm used to retrieve sea surface temperature that relies on the "same temperature criterion" - if the atmosphere is correctly characterized, surface temperatures derived from multiple bands should be the same. The algorithm uses a lookup table approach and can retrieve column water vapor, atmospheric temperature offset, and surface temperature using multiple pixels to reduce ambiguity. It is shown to work well when applied to MODIS data.
Analysis of Black Carbon Data for Ireland - Michael Geever, NUIG
Analysis of Black Carbon Data for Ireland - Michael Geever, NUIGEnvironmental Protection Agency, Ireland
This document analyzes black carbon data collected in Ireland. It begins by defining black carbon as light-absorbing carbon particles produced by incomplete combustion. It then discusses the challenges of measuring black carbon, noting that measurements can vary significantly depending on the instrument and technique used. The document presents data on black carbon levels in Malin Head, Ireland from an Aethalometer over multiple years. It finds that black carbon levels were below 75 ng/m3 59% of the time in 2002 and 62% of the time in 2003. Levels were also frequently below 50 ng/m3, especially in the month of August.MATLAB and HDF
This document provides an overview of MATLAB and its toolboxes for technical computing, modeling, and working with geospatial and scientific data formats like HDF. MATLAB is a technical computing environment used for data analysis, algorithm development, and custom application building. It includes toolboxes for tasks like image processing, mapping, and working with HDF file formats. The Mapping Toolbox allows users to access, visualize, and analyze geospatial data. The Distributed Computing Toolbox enables running MATLAB applications on multiple computers to accelerate processing.
Usage of NCL, IDL, and MATLAB to access NASA HDF4/HDF-EOS2/HDF-EOS5 data
Usage of NCL, IDL, and MATLAB to access NASA HDF4/HDF-EOS2/HDF-EOS5 dataThe HDF-EOS Tools and Information Center
The document discusses providing easy access to HDF data via NCL, IDL, and MATLAB. It presents examples and code snippets for reading HDF data from various NASA data centers like GES DISC, MODAPS, NSIDC, and LP-DAAC into the three software packages. Common issues when working with HDF files like HDF-EOS2 swaths with dimension maps and different ways metadata is stored are also addressed. The overall goal is to help lower the learning curve for users who want to analyze HDF data in their favorite analysis packages.Presentazione_spinetti2.ppt
The document describes an ASI-SRV volcanic risk monitoring system that analyzes Earth observation data from satellites like ASTER and Hyperion to measure volcanic plumes and activity from Mt. Etna. Modules were developed to retrieve aerosol optical thickness, water vapor content, and SO2 flux from the data. Time series analysis of eruptions from 2000-2010 showed high aerosol values correlated with large plumes and SO2 flux, indicating periods of explosive activity. The system provides near real-time monitoring of degassing at Etna to support volcanic risk management.
Eumetsat 09 2005(30mb+) 2
The document summarizes satellite remote sensing data related to clouds, aerosols, and atmospheric composition from NASA satellites including Terra, Aqua, ICESat, and SeaWiFS. It describes algorithms and products for deriving cloud and aerosol optical properties including optical thickness, effective radius, and phase from multi-spectral sensor measurements. Examples of monthly mean cloud properties, aerosol optical thickness, and case studies of dust plumes over Africa are shown.
His Expert's Voice
This document discusses expert systems and conventions for data modeling. It provides an overview of expert systems and how rule-based systems work using examples like traffic lights and Conway's Game of Life. It describes how an expert system could be used for the HDF Product Designer to enforce data modeling conventions. The status, challenges, and future work are outlined for integrating an expert system into the HDF Product Designer to provide convention knowledge and guide the data modeling process.
Recommended
Night Water Vapor Borel Spie 8 12 08 White
The document describes a method for retrieving column water vapor at night using mid-wave infrared bands from sensors like MODIS. It adapts an existing algorithm used to retrieve sea surface temperature that relies on the "same temperature criterion" - if the atmosphere is correctly characterized, surface temperatures derived from multiple bands should be the same. The algorithm uses a lookup table approach and can retrieve column water vapor, atmospheric temperature offset, and surface temperature using multiple pixels to reduce ambiguity. It is shown to work well when applied to MODIS data.
Analysis of Black Carbon Data for Ireland - Michael Geever, NUIG
Analysis of Black Carbon Data for Ireland - Michael Geever, NUIGEnvironmental Protection Agency, Ireland
This document analyzes black carbon data collected in Ireland. It begins by defining black carbon as light-absorbing carbon particles produced by incomplete combustion. It then discusses the challenges of measuring black carbon, noting that measurements can vary significantly depending on the instrument and technique used. The document presents data on black carbon levels in Malin Head, Ireland from an Aethalometer over multiple years. It finds that black carbon levels were below 75 ng/m3 59% of the time in 2002 and 62% of the time in 2003. Levels were also frequently below 50 ng/m3, especially in the month of August.MATLAB and HDF
This document provides an overview of MATLAB and its toolboxes for technical computing, modeling, and working with geospatial and scientific data formats like HDF. MATLAB is a technical computing environment used for data analysis, algorithm development, and custom application building. It includes toolboxes for tasks like image processing, mapping, and working with HDF file formats. The Mapping Toolbox allows users to access, visualize, and analyze geospatial data. The Distributed Computing Toolbox enables running MATLAB applications on multiple computers to accelerate processing.
Usage of NCL, IDL, and MATLAB to access NASA HDF4/HDF-EOS2/HDF-EOS5 data
Usage of NCL, IDL, and MATLAB to access NASA HDF4/HDF-EOS2/HDF-EOS5 dataThe HDF-EOS Tools and Information Center
The document discusses providing easy access to HDF data via NCL, IDL, and MATLAB. It presents examples and code snippets for reading HDF data from various NASA data centers like GES DISC, MODAPS, NSIDC, and LP-DAAC into the three software packages. Common issues when working with HDF files like HDF-EOS2 swaths with dimension maps and different ways metadata is stored are also addressed. The overall goal is to help lower the learning curve for users who want to analyze HDF data in their favorite analysis packages.Presentazione_spinetti2.ppt
The document describes an ASI-SRV volcanic risk monitoring system that analyzes Earth observation data from satellites like ASTER and Hyperion to measure volcanic plumes and activity from Mt. Etna. Modules were developed to retrieve aerosol optical thickness, water vapor content, and SO2 flux from the data. Time series analysis of eruptions from 2000-2010 showed high aerosol values correlated with large plumes and SO2 flux, indicating periods of explosive activity. The system provides near real-time monitoring of degassing at Etna to support volcanic risk management.
Eumetsat 09 2005(30mb+) 2
The document summarizes satellite remote sensing data related to clouds, aerosols, and atmospheric composition from NASA satellites including Terra, Aqua, ICESat, and SeaWiFS. It describes algorithms and products for deriving cloud and aerosol optical properties including optical thickness, effective radius, and phase from multi-spectral sensor measurements. Examples of monthly mean cloud properties, aerosol optical thickness, and case studies of dust plumes over Africa are shown.
His Expert's Voice
This document discusses expert systems and conventions for data modeling. It provides an overview of expert systems and how rule-based systems work using examples like traffic lights and Conway's Game of Life. It describes how an expert system could be used for the HDF Product Designer to enforce data modeling conventions. The status, challenges, and future work are outlined for integrating an expert system into the HDF Product Designer to provide convention knowledge and guide the data modeling process.
ArcGIS and Multi-D: Tools & Roadmap
The document discusses Esri's tools and roadmap for working with multi-dimensional (MD) scientific data in ArcGIS. It outlines Esri's efforts to directly read HDF, GRIB, and netCDF files as raster layers or feature/table views in ArcGIS. MD mosaic datasets allow users to manage variables and dimensions across multiple files and perform on-the-fly computations and visualization of MD data. New functions have been added to improve MD data analysis and visualization, including a vector field renderer to depict raster data as vectors. Esri is also working to better support OPeNDAP data sources.
America Runs on Excel and HDF5 - Glued together by Python
This document discusses the development of PyHexad, a Python-based add-in for Excel that allows users to access and analyze HDF5 data directly in Excel. PyHexad 0.1 allows users to display HDF5 file contents, read arrays and tables into Excel, and read HDF5 images. The developer is seeking feedback on usability, the Python dependency, and interest in helping advance the project to version 1.0. A prototype will be released in early August for testing and further input.
HDF Product Designer
The document describes the HDF Product Designer software tool. It was created to facilitate the design of interoperable scientific data products in HDF5 format. The tool allows intuitive editing of HDF5 objects and supports conventions like CF and ACDD. It also provides validation services to test file compliance. The goal is to help scientists design data products that follow standards and are easy for others to use.
Putting some Spark into HDF5
This presentation discusses putting HDF5 files into Apache Spark for analysis. It describes the differences between traditional file systems and Hadoop's HDFS, and how Spark provides a more accessible way to exploit data parallelism without using MapReduce. The presentation outlines experiments loading HDF5 climate data files into Spark to calculate statistics. It suggests variations like providing a file list instead of traversing directories. The conclusion is that Spark can effectively analyze HDF5 files under the right circumstances but current methods are imperfect, and future work with The HDF Group could build better integrations.
The HDF Group: Community models and outreach
The HDF Group aims to build community around its data platform through engagement goals like improving metadata, developing sharing tools, and hosting workshops. It seeks to bring together disciplines by addressing needs, maintaining websites and forums, publishing a blog, and coordinating networks among NASA missions, data centers, research facilities, and models to facilitate cross-disciplinary data sharing, use, and understanding.
HDF Project Update
The HDF Group provides software for managing large, complex data and services to support users of this technology. It derives most of its revenue from projects related to earth science, including supporting HDF-EOS, JPSS, and other earth science projects. It maintains various tools for working with HDF files and conducts maintenance, support, and development activities to support new versions and capabilities of HDF libraries and software.
HDF Server
The document describes HDF Server (h5serv), which exposes HDF5 files via a RESTful API. H5serv allows full read/write access and supports HDF5 features like compression and hyperslab selection. It uses the Tornado framework to implement a stateless, cacheable API accessed through HTTP requests in JSON. This provides a web interface for HDF5 data while maintaining HDF5 functionality. Future plans include client libraries, authentication/authorization, and improving performance for large repositories.
Indexing HDF5: A Survey
The document discusses indexing methods for HDF5 files to enable efficient searching and access of data based on data values. It describes several existing implementations of HDF5 indexing, including PyTables, FastQuery/FastBit, Alacrity, and prototypes from The HDF Group. It also outlines current and future work to develop indexing and querying capabilities within HDF5 to allow complex multi-dimensional searches across metadata and datasets.
MATLAB and Scientific Data: New Features and Capabilities
This document discusses new features and capabilities in MATLAB for working with scientific data formats and performing technical computing. It highlights enhancements to reading HDF5 and NetCDF files with both high-level and low-level interfaces. It also covers new capabilities for handling dates/times, big data, and accessing web services through RESTful APIs.
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)The HDF-EOS Tools and Information Center
The document discusses enhancing the Geospatial Data Abstraction Library (GDAL) to improve accessibility and interoperability of NASA data products with GIS tools. It developed plugins for three NASA data products to demonstrate reading multidimensional datasets into GIS applications like ArcGIS. Next steps include providing outreach, enhancing the framework to be more flexible and production-ready, and developing guides to help other data centers build GDAL plugins to address their issues with geospatial data. The overall goal is to improve analysis and visualization of NASA scientific data in GIS tools and web applications.More Related Content
Viewers also liked
ArcGIS and Multi-D: Tools & Roadmap
The document discusses Esri's tools and roadmap for working with multi-dimensional (MD) scientific data in ArcGIS. It outlines Esri's efforts to directly read HDF, GRIB, and netCDF files as raster layers or feature/table views in ArcGIS. MD mosaic datasets allow users to manage variables and dimensions across multiple files and perform on-the-fly computations and visualization of MD data. New functions have been added to improve MD data analysis and visualization, including a vector field renderer to depict raster data as vectors. Esri is also working to better support OPeNDAP data sources.
America Runs on Excel and HDF5 - Glued together by Python
This document discusses the development of PyHexad, a Python-based add-in for Excel that allows users to access and analyze HDF5 data directly in Excel. PyHexad 0.1 allows users to display HDF5 file contents, read arrays and tables into Excel, and read HDF5 images. The developer is seeking feedback on usability, the Python dependency, and interest in helping advance the project to version 1.0. A prototype will be released in early August for testing and further input.
HDF Product Designer
The document describes the HDF Product Designer software tool. It was created to facilitate the design of interoperable scientific data products in HDF5 format. The tool allows intuitive editing of HDF5 objects and supports conventions like CF and ACDD. It also provides validation services to test file compliance. The goal is to help scientists design data products that follow standards and are easy for others to use.
Putting some Spark into HDF5
This presentation discusses putting HDF5 files into Apache Spark for analysis. It describes the differences between traditional file systems and Hadoop's HDFS, and how Spark provides a more accessible way to exploit data parallelism without using MapReduce. The presentation outlines experiments loading HDF5 climate data files into Spark to calculate statistics. It suggests variations like providing a file list instead of traversing directories. The conclusion is that Spark can effectively analyze HDF5 files under the right circumstances but current methods are imperfect, and future work with The HDF Group could build better integrations.
The HDF Group: Community models and outreach
The HDF Group aims to build community around its data platform through engagement goals like improving metadata, developing sharing tools, and hosting workshops. It seeks to bring together disciplines by addressing needs, maintaining websites and forums, publishing a blog, and coordinating networks among NASA missions, data centers, research facilities, and models to facilitate cross-disciplinary data sharing, use, and understanding.
HDF Project Update
The HDF Group provides software for managing large, complex data and services to support users of this technology. It derives most of its revenue from projects related to earth science, including supporting HDF-EOS, JPSS, and other earth science projects. It maintains various tools for working with HDF files and conducts maintenance, support, and development activities to support new versions and capabilities of HDF libraries and software.
HDF Server
The document describes HDF Server (h5serv), which exposes HDF5 files via a RESTful API. H5serv allows full read/write access and supports HDF5 features like compression and hyperslab selection. It uses the Tornado framework to implement a stateless, cacheable API accessed through HTTP requests in JSON. This provides a web interface for HDF5 data while maintaining HDF5 functionality. Future plans include client libraries, authentication/authorization, and improving performance for large repositories.
Indexing HDF5: A Survey
The document discusses indexing methods for HDF5 files to enable efficient searching and access of data based on data values. It describes several existing implementations of HDF5 indexing, including PyTables, FastQuery/FastBit, Alacrity, and prototypes from The HDF Group. It also outlines current and future work to develop indexing and querying capabilities within HDF5 to allow complex multi-dimensional searches across metadata and datasets.
MATLAB and Scientific Data: New Features and Capabilities
This document discusses new features and capabilities in MATLAB for working with scientific data formats and performing technical computing. It highlights enhancements to reading HDF5 and NetCDF files with both high-level and low-level interfaces. It also covers new capabilities for handling dates/times, big data, and accessing web services through RESTful APIs.
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)The HDF-EOS Tools and Information Center
The document discusses enhancing the Geospatial Data Abstraction Library (GDAL) to improve accessibility and interoperability of NASA data products with GIS tools. It developed plugins for three NASA data products to demonstrate reading multidimensional datasets into GIS applications like ArcGIS. Next steps include providing outreach, enhancing the framework to be more flexible and production-ready, and developing guides to help other data centers build GDAL plugins to address their issues with geospatial data. The overall goal is to improve analysis and visualization of NASA scientific data in GIS tools and web applications.Viewers also liked (10)
America Runs on Excel and HDF5 - Glued together by Python
America Runs on Excel and HDF5 - Glued together by Python
MATLAB and Scientific Data: New Features and Capabilities
MATLAB and Scientific Data: New Features and Capabilities
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)
Geospatial Data Abstraction Library (GDAL) Enhancement for ESDIS (GEE)