Air Quality Cluster Technology Track   Earth Science Information Partners Data Flow & Interoperability in DataFed  Service...
Federated Network for Air Quality Data and Processing Services Project Team: Software Architecture:  R. Husar Software Imp...
DataFed in a Nutshell DataFed Vision Aid air quality analysis by effective use of relevant data   DataFed Goals Facilitate...
Typical DataFed AQ Analysis Tools Consoles :  Data from diverse sources are displayed to create a rich context for explora...
Web Services:  Building Blocks of DataFed Programming   Access, Process, Render Data by Service Chaining [Station] [Monito...
Single Data Model for All AQ Data <ul><li>Most  Views  are slices through a cube of data organized by lat, lon, altitude, ...
OGC Abstract Specification –  Coverage (00-106) <ul><li>Abstract Spec.:  Coverages model and visualize spatial relationshi...
Earth Science – GIS Interoperability Shared Data Model
OGC Web Coverage Service (WCS) Specification <ul><li>HTTP GET/POST  based interfaces </li></ul><ul><li>Services have XML  ...
Data Access through Adapters through DataFed  SOAP,HTTP Get OGC WCS   HTTP Get, Post OGC WMS   HTTP Get Station-Point  SQL...
GALEON Interoperability Experiment <ul><li>GALEON  </li></ul><ul><li>Geo-interface for Atmosphere, Land, Earth, and Ocean ...
OGC WCS Demonstration:  THREDDS_GFS  4Dim Dataset <ul><li>Lat/Lon Box  Elev Range  Time Range </li></ul><ul><li>Map:  BBOX...
OGC WCS Demonstration:  AirNOW   Station Dataset Map View Services                   WCS Query   http://webapps.datafed.ne...
OGC WCS Demonstration:  Grid, Image, Station Data Types <ul><li>Coverage=THEEDDS.T&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07...
Benefits of WMS Compatibility: Geospatial One-Stop   Provides Access to DataFed spatial layers through WMS Problem: No Tim...
Summary <ul><li>OGC WCS protocol is suitable for delivering most air quality data: </li></ul>Suitable for most data view q...
Upcoming SlideShare
Loading in...5
×

2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis System

629

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
629
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis System

    1. 1. Air Quality Cluster Technology Track Earth Science Information Partners Data Flow & Interoperability in DataFed Service-based AQ Analysis System R. B. Husar, S. R. Falke and K. Höijärvi Washington University, St. Louis, MO ESIP Federation Winter Meeting 2006 Washington, DC, January 4, 2006 Partners <ul><li>NASA </li></ul><ul><li>NOAA </li></ul><ul><li>EPA </li></ul><ul><li>(?) </li></ul><ul><li>USGS </li></ul><ul><li>DOE </li></ul><ul><li>NSF </li></ul><ul><li>Industry… </li></ul>
    2. 2. Federated Network for Air Quality Data and Processing Services Project Team: Software Architecture: R. Husar Software Implementation: K. Höijärvi Data and Applications: S. Falke, R. Husar The DataFed Project
    3. 3. DataFed in a Nutshell DataFed Vision Aid air quality analysis by effective use of relevant data   DataFed Goals Facilitate access and flow of AQ data from provider to users Support user-driven data processing value chains Participate in specific application projects   Approach: Mediation Between Users and Data Providers DataFed assumes spontaneous, autonomous data providers Non-intrusively wraps datasets for access by web services Mediates, homogenizes data views. e.g. geo-spatial, time...   Applications Browsers and analysis tools for distributed monitoring data    Serve as data gateway for user programs; GIS, science tools DataFed is focused on the mediation of air quality data
    4. 4. Typical DataFed AQ Analysis Tools Consoles : Data from diverse sources are displayed to create a rich context for exploration and analysis CATT : Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets
    5. 5. Web Services: Building Blocks of DataFed Programming Access, Process, Render Data by Service Chaining [Station] [Monitor] Clickable monitor locations layer Map boundary layer
    6. 6. Single Data Model for All AQ Data <ul><li>Most Views are slices through a cube of data organized by lat, lon, altitude, and time (X,Y,Z,T) </li></ul>Multidimensional Data Cube
    7. 7. OGC Abstract Specification – Coverage (00-106) <ul><li>Abstract Spec.: Coverages model and visualize spatial relationships between, and the spatial distribution of, earth phenomena </li></ul><ul><li>Coverage subtypes of Air Quality interest are Image, Grid and Discrete Point </li></ul><ul><li>Has a property (Coverage_Function) that has a spatial domain, and a value set. </li></ul><ul><li>For Image and Grid, the Coverage_Function is an array…Discrete Point coverage is described by a PointC_Function where the spatial domain consists of a collection of points </li></ul><ul><li>Observation: </li></ul><ul><li>The Abstract Spec. foresaw the need for Point Coverage, i.e. fixed monitoring Stations </li></ul>
    8. 8. Earth Science – GIS Interoperability Shared Data Model
    9. 9. OGC Web Coverage Service (WCS) Specification <ul><li>HTTP GET/POST based interfaces </li></ul><ul><li>Services have XML service descriptions (“Capabilities”, “Description”) </li></ul><ul><li>Filter parameters allow selection of subsets of source data </li></ul><ul><li>Output formats advertised by each service instance </li></ul>OGC WCS getCoverage Schema Suitable for wrapping with SOAP envelope, WSDL access, loose coupling WCS is for &quot;coverages&quot; – information representing space -time- varying phenomena WCS describes, requests and delivers coverages in spatio-temporal domain WCS version 1.1 is limited to grids/&quot;simple” coverages with homogeneous range sets
    10. 10. Data Access through Adapters through DataFed SOAP,HTTP Get OGC WCS HTTP Get, Post OGC WMS HTTP Get Station-Point SQL Server, Files… Sequence Image, file nDim Grid OpenDAP NetCDF, … Other Traject., Event, Pic Sources Diverse formats Many data models Data Wrapper Data into geo-cubes Queries to views Virtual Data Cube Global geo-cube data model Makes queries data-neutral Others? e.g. OpenDAP Output Protocol dependent User specified GeoTable CSV,XLS,GML GeoGrid GML,NetCDF.. GeoImage GeoTIFF, PNG.. Other MS Dataset.. Query Adapter Maps query to protocol User selects protocols
    11. 11. GALEON Interoperability Experiment <ul><li>GALEON </li></ul><ul><li>Geo-interface for Atmosphere, Land, Earth, and Ocean netCDF </li></ul><ul><li>Lead by UNIDATA & OGC </li></ul><ul><li>Participants: academia, industry, international </li></ul>Unify Earth Science & GIS Data Flows B. Domenico B. Domenico GALEON UNIDATA
    12. 12. OGC WCS Demonstration: THREDDS_GFS 4Dim Dataset <ul><li>Lat/Lon Box Elev Range Time Range </li></ul><ul><li>Map: BBOX=-180,-90,180,90, 1350,1350& TIME=2005-12-06/2005-12-06/PT3H </li></ul><ul><li>Time: BBOX=-34,49.05,-34,49.05, 1350,1350& TIME=2005-12-05/2005-12-08/PT3H </li></ul><ul><li>Elev: BBOX=-34,49.05,-34,49.05, 0,18000 & TIME=2005-12-06/2005-12-06/PT3H </li></ul>The form of the WCS query is the same for all slices through the data cube (views) The only difference in the views is the thickness of the slices in each dimension Return grid is in multiple formats (NetCDF, CSV, GML, PNG, … ) Map View Services WCS Query Time View Services WCS Query Elevation View Services WCS Query
    13. 13. OGC WCS Demonstration: AirNOW Station Dataset Map View Services WCS Query http://webapps.datafed.net/dvoy_services/ogc.wsfl?SERVICE=wcs&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=AIRNOW.pmfine&FORMAT=CSV&BBOX=-125.3889,22.55111,-61.40667,51.20889,0,0&TIME=2005-06-26T15:00:00Z&WIDTH=999&HEIGHT=999&DEPTH=999 Formats: CSV, NetCDF-table, etc Time View Services http://webapps.datafed.net/dvoy_services/ogc.wsfl?SERVICE=wcs&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:4326&COVERAGE=AIRNOW.pmfine&FORMAT=CSV&BBOX=-90.16028,38.61222,-90.16028,38.61222,0,0&TIME=2005-06-20T00:00:00Z/2005-06-30T00:00:00Z&WIDTH=999&HEIGHT=999&DEPTH=999 Formats: CSV, NetCDF-table, etc
    14. 14. OGC WCS Demonstration: Grid, Image, Station Data Types <ul><li>Coverage=THEEDDS.T& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF </li></ul><ul><li>Coverage=SURF.Bext& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF-table </li></ul><ul><li>Coverage=SEAW.Refl& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=GeoTIFF </li></ul><ul><li>COVERAGE=sst& BBOX=-126,24,-65,52,0,0 &TIME=2001-01-01,2001-01-01&FORMAT=NetCDF </li></ul>UNIDATA – THREDDS/GALEON WCS DataFed GALEON WCS U Florence, It GALEON WCS DataFed GALEON WCS Grid Grid Image Station Services WCS Query Services WCS Query Services WCS Query Services WCS Query
    15. 15. Benefits of WMS Compatibility: Geospatial One-Stop Provides Access to DataFed spatial layers through WMS Problem: No Time Dim! http:// www.datafed.net/DataLinks/OGC/OGC.htm DataFed/VIEWS Asian Dust Concentration (circles) S. Falke
    16. 16. Summary <ul><li>OGC WCS protocol is suitable for delivering most air quality data: </li></ul>Suitable for most data view queries BBOX, TIME Station-Point SQL Server, Files… Sequence Image, file nDim Grid OpenDAP NetCDF, … Applicable to most data types Station, Image, Grid <ul><li>For Air Quality data needs: </li></ul><ul><li>WCS: Add Point coverage to WCS (in addition to grid, polygon) </li></ul><ul><li>WCS: Strongly typed (XML Schema-verifiable) return data types </li></ul><ul><li>Testing: Much fixing and testing for server-client compatibility </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×