2006-03-14 WG on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives

Work Group Meeting on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives Rudolf B. Husar CAPITA, Washington University, St. Louis, MO Ispra, JRC, March 14. 2004 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Policy Guidance for HTAP IT ,[object Object],[object Object],[object Object],[object Object],[object Object]

Current Air Quality Information ‘Ecosystem’ ,[object Object],[object Object],[object Object],[object Object],Lack of Interoperability Poor data & model utilization Less informed decision making

GEOSS Info Flow Architecture A General Framework Accepted by Members Model Model Data Data

Federated Network for Air Quality Data and Processing Services Project Team: Software Architecture: R. Husar Software Implementation: K. Höijärvi Data and Applications: S. Falke, R. Husar Data Handling Approach

Federated Data System for Air Quality ,[object Object],[object Object],[object Object],AQ Compliance Nowcast/Forecast Status & Trends Find Data Gaps ID New Problems ……… Info Needs Reports ,[object Object],Emission Surface Satellite Model Single Datasets Providers Wrappers Where? What? When? Federate Data Structuring ,[object Object],Slice & Dice Explore Data Viewers ,[object Object],Programs Integrate Understand ,[object Object],Non-intrusive Linking & Mediation Data Users Data Providers

Data Handling Approach: DataFed ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Typical DataFed AQ Analysis Tools Consoles : Data from diverse sources are displayed to create a rich context for exploration and analysis CATT : Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets

Quebec Smoke July, 7, 2002 SeaWiFS Satellite Aerosol Chemical Air Trajectory Map Boarder Web Service Composition

Software Tools: Demo: Networked Data Access, Processing and Fusion AeroCOM – VIEWS SO4 ,[object Object],[object Object],[object Object],Networking

Visualization Goal: 4D, User-selectable layers Image below completed in 1998 Satellite Data Layers: Land Reflectance (SeaWiFS) Fire Pixels (ASTR Night) High Cloud (GOES, Meteosat, GMS2) Aerosol AOT (AVHRR)

4D Dynamic Visualization Demonstrate interactions, allow exploration ‘Google Earth’ for Earth Science is now possible

Earth Science Information Partners Air Quality Cluster 1. Serve as facilitator to Earth Science information community . 2. Promote efficient flow of ES data from collection to end-use . 3. Improve quality and usability of ES data and info systems 4. Expand the use of Earth science information Participant Involvement Partners ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Dissemination & Use – Service Based ,[object Object],[object Object],[object Object],Near Real Time Data Integration Delayed Data Integration Surface Air Quality AIRNOW O3, PM25 ASOS_STI Visibility, 300 sites METAR Visibility, 1200 sites VIEWS_OL 40+ Aerosol Parameters Satellite MODIS_AOT AOT, Idea Project GASP Reflectance, AOT TOMS Absorption Indx, Refl. SEAW_US Reflectance, AOT Model Output NAAPS Dust, Smoke, Sulfate, AOT WRF Sulfate Fire Data HMS_Fire Fire Pixels MODIS_Fire Fire Pixels Surface Meteorology RADAR NEXTRAD SURF_MET Temp, Dewp, Humidity… SURF_WIND Wind vectors ATAD Trajectory, VIEWS locs.

WCS - Interoperable Data Access Service netCDF – Machine independent encoding ncML – XML data description CF – Naming, structure convention OGC Web Coverage Service - Interoperable Data Access Query Data Syntax + Semantics Coverage (parameter) BBOX Time Range netCDF Example Profile ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

GALEON Interoperability Experiment Geo-interface for Atmosphere, Land, Earth, and Ocean netCDF Unify Earth Science & GIS Data Flows Strong European Participation IT – S. Nativi, L. Bigagli UK – Andrew Wolf DE – Peter Bauman) B. Domenico B. Domenico GALEON UNIDATA U Florence/CNR-IMMA WCS Server

OGC WCS Demonstration: THREDDS_GFS 4Dim Dataset ,[object Object],[object Object],[object Object],[object Object],The form of the WCS query is the same for all slices through the data cube (views) The only difference in the views is the thickness of the slices in each dimension Return grid is in multiple formats (NetCDF, CSV, GML, PNG, … ) Map View Services WCS Query Time View Services WCS Query Elevation View Services WCS Query

Summary ,[object Object],[object Object],[object Object],[object Object],[object Object]

OGC WCS Demonstration: Grid, Image, Station Data Types ,[object Object],[object Object],[object Object],[object Object],UNIDATA – THREDDS/GALEON WCS DataFed GALEON WCS U Florence, It GALEON WCS DataFed GALEON WCS Grid Grid Image Station Services WCS Query Services WCS Query Services WCS Query Services WCS Query

Single Data Model for All AQ Data ,[object Object],Multidimensional Data Cube

OGC Web Coverage Service (WCS) Specification ,[object Object],[object Object],[object Object],[object Object],OGC WCS getCoverage Schema Suitable for wrapping with SOAP envelope, WSDL access, loose coupling WCS is for "coverages" – information representing space -time- varying phenomena WCS describes, requests and delivers coverages in spatio-temporal domain WCS version 1.1 is limited to grids/"simple” coverages with homogeneous range sets

AQ Monitoring Network Data Storage and Delivery through OGC Protocols Relational Data Model Star Schema WMS WCS SOS SensorML WFS Observations Station Info. Param/Sensor/Method Data View Services WMS Stations Par-Meth Observations SOS WCS Data Access

Technology Support for ‘Integrated Solutions’ Air Quality Information System

Data Access through Adapters through DataFed SOAP,HTTP Get OGC WCS HTTP Get, Post OGC WMS HTTP Get Station-Point SQL Server, Files… Sequence Image, file nDim Grid OpenDAP NetCDF, … Other Traject., Event, Pic Sources Diverse formats Many data models Data Wrapper Data into geo-cubes Queries to views Virtual Data Cube Global geo-cube data model Makes queries data-neutral Others? e.g. OpenDAP Output Protocol dependent User specified GeoTable CSV,XLS,GML GeoGrid GML,NetCDF.. GeoImage GeoTIFF, PNG.. Other MS Dataset.. Query Adapter Maps query to protocol User selects protocols

2006-03-14 WG on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to 2006-03-14 WG on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives

Similar to 2006-03-14 WG on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives (20)

More from Rudolf Husar

More from Rudolf Husar (20)

Recently uploaded

Recently uploaded (20)

2006-03-14 WG on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives