DataFed:  View-Based Mediated Web Service Architecture Rudolf B. Husar and Kari Hoijarvi Washington University, St. Louis ...
Information Landscape: Providers  Geography, Content, Agency, Form <ul><li>Data are  distributed  geographically by  auton...
Information Landscape: Users Types, Agency, Info Needs <ul><li>Users are  distributed  geographically </li></ul>EPA NOAA N...
Lets agree on Space-Time-Parameter Data Access Query Protocol
OGS WCS/WMS Protocols Space-Time-Parameter queries T2 T1 Loose Coupling of Servcies GetCapabilities GetData Capabilities, ...
Gio Wiederhold 1992: Mediators
Jeff Ullman, 1998: Answering Queries Using Views Mediator Global Schema Wrapper Wrapper Source 1 Source 2 User query Query...
Wrapper Classes: Point, Image, Grid  5Dim Data Model Common Views
Anatomy of a Wrapper Service: TOMS Satellite Image Data <ul><li>Wrapper Service can access and spatially subset image for ...
Integrated Data System for Air Quality-IDAQ  ESIP AQ Cluster 050510  Draft  [email_address] <ul><li>The challenge is to de...
Model-Data Comparison Workflow Software Workflow Flow Program   Lego-like assembly of component   AeroCom Chemical Models ...
 
DataFed: 100+ Datasets Non-intrusively Federated <ul><li>Data are accessed from autonomous, distributed providers </li></u...
Summary <ul><li>Third-party  mediation can homogenize  distributed ES data </li></ul><ul><li>Agile SOA-based IS can delive...
Upcoming SlideShare
Loading in …5
×

070726 Igarss07 Barcelona

741 views

Published on

<a href="http://datafedwiki.wustl.edu/index.php/2007-07-25_IGARSS07_Barcelona">DataFed Wiki page</a>

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
741
On SlideShare
0
From Embeds
0
Number of Embeds
33
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 070726 Igarss07 Barcelona

    1. 1. DataFed: View-Based Mediated Web Service Architecture Rudolf B. Husar and Kari Hoijarvi Washington University, St. Louis Presented at IGARSS07, Barcelona, ES, July 22, 2007 Mediator Wrapper Wrapper Source 1 Source 2 User query Data View
    2. 2. Information Landscape: Providers Geography, Content, Agency, Form <ul><li>Data are distributed geographically by autonomous providers </li></ul>Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form <ul><li>Data includes emissions </li></ul>Emission Emission Emission Emission Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form <ul><li>Data includes emissions, ambient data, </li></ul>Ambient Ambient Ambient Ambient Emission Emission Emission Emission Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form <ul><li>Data includes emissions, ambient data, satellite data </li></ul>Satellite Satellite Satellite Satellite Ambient Ambient Ambient Ambient Emission Emission Emission Emission Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form <ul><li>Data includes emissions, ambient data, satellite data and model output </li></ul>Model Model Model Model Satellite Satellite Satellite Satellite Ambient Ambient Ambient Ambient Emission Emission Emission Emission Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form <ul><li>Data are provided by multiple agencies : EPA, NOAA, NASA and others </li></ul>NASA Mission NOAA GASP NASA IDEA NASA DAACs NOAA ASOS EPA-AQS DataMart EPA AIRNow RPO VIEWS FS FireInv State/Local Emission EPA NEISGEI EPA NEI NOAA WeaMod EPA AQModel NOAA Forecast Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form NASA DAACs NOAA GASP NASA IDEA NASA Missions EPA NEI EPA NEISGEI FS FireInv State/Local Emission NOAA ASOS RPO VIEWS EPA AIRNow EPA-AQS AIRS NOAA WeaMod EPA AQModel NASA GloModel NOAA Forecast <ul><li>Furthermore, data are provided in varied formats and access protocols </li></ul>Emission Ambient Satellite Model EPA NOAA NASA Other Content | Agency | Form <ul><li>Data on Internet are geography-independent and can be ‘linearized’ </li></ul>Internet NASA DAACs EPA R&D Model EPA AIRNow others
    3. 3. Information Landscape: Users Types, Agency, Info Needs <ul><li>Users are distributed geographically </li></ul>EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist Policy Policy Policy <ul><li>Users includes policy makers </li></ul>EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist <ul><li>Users includes policy makers, the public </li></ul>Policy Policy Policy Public Public EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist <ul><li>Users includes policy makers, the public , AQ managers </li></ul>Policy Policy Policy Public Public Manager Manager EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist and scientist Policy Policy Policy Public Public Manager Manager Scientist Scientist Scientist EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist <ul><li>Users are affiliated with multiple agencies : EPA, NOAA, NASA, as well as others </li></ul>Policy Policy Policy Public Public Manager Manager Scientist Scientist Scientist EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist <ul><li>Furthermore, users need various types of information provided in multiple formats </li></ul>Policy Manager Policy Scientist Manager Scientist Scientist Policy Public Public EPA NOAA NASA Other Stakeholder | Agency | Form Policy Manager Public Scientist <ul><li>Since the users are also on the Internet, their geographic location is irrelevant </li></ul>Public Manager Scientist Internet other
    4. 4. Lets agree on Space-Time-Parameter Data Access Query Protocol
    5. 5. OGS WCS/WMS Protocols Space-Time-Parameter queries T2 T1 Loose Coupling of Servcies GetCapabilities GetData Capabilities, ‘Profile’ Data Where? When? What? Which Format? Server Back End Std. Interface Client Front End Std. Interface CF, EOS, OGC CF OGC, ISO OGC, ISO Standards netCDF, HDF.. Format Temperature What? Time When? BBOX Where? GetData Query
    6. 6. Gio Wiederhold 1992: Mediators
    7. 7. Jeff Ullman, 1998: Answering Queries Using Views Mediator Global Schema Wrapper Wrapper Source 1 Source 2 User query Query Query Query Query Result Result Result Result View Jeff Ullman, Stanford, 1998 Mediator answers queries Collects data from wrappers or mediators Wrapper (adapter) translates between the local and global language, model Heterogeneous sources
    8. 8. Wrapper Classes: Point, Image, Grid 5Dim Data Model Common Views
    9. 9. Anatomy of a Wrapper Service: TOMS Satellite Image Data <ul><li>Wrapper Service can access and spatially subset image for any day (WMS) </li></ul><ul><li>Wrapper Service and mediation is performed by third party </li></ul><ul><li>This makes a non-intrusive, adoptive system for agile networking </li></ul>Image Description for Data Access: image_width=502 image_height=329 margin_bottom=105 margin_left=69 margin_right=69 margin_top=46 lat_min=-70 lat_max=70 lon_min=-180 lon_max=180 Daily TOMS images on FTP archive ftp://toms.gsfc.nasa.gov/pub/eptoms/images/aerosol/y2000/ea000820.gif Template: ftp://toms.gsfc.nasa.gov/pub/eptoms/images/aerosol/y[yyyy]/ea[yy][mm][dd].gif Transparent colors for overlays RGB(89,140,255) RGB(41,117,41) RGB(23,23,23) RGB(0,0,0)
    10. 10. Integrated Data System for Air Quality-IDAQ ESIP AQ Cluster 050510 Draft [email_address] <ul><li>The challenge is to design a general supportive infrastructure </li></ul><ul><li>Simply connecting the relevant provides and users for each info product is messy </li></ul><ul><li>The info system infrastructure needs to facilitate the creation of info products </li></ul>AQ Compliance Nowcast/Forecast Status & Trends Find Data Gaps ID New Problems ……… Info Needs Reports <ul><li>Providers supply the ‘raw material’ (data and models) for ‘refined’ info products </li></ul>Emission Surface Satellite Model Single Datasets Providers Wrappers Where? What? When? Federate Data Structuring <ul><li>Structuring the heterogeneous data into where-when-what ‘cubes’ simplifies the mess </li></ul>Slice & Dice Explore Data Viewers <ul><li>The ‘cubed’ data can be accessed and explored by slicing-dicing tools </li></ul>Programs Integrate Understand <ul><li>More elaborate data integration and fusion can be done by web service chaining </li></ul><ul><li>This infrastructure support for IDAQ can be provided by the ESIP Federation </li></ul>Non-intrusive Linking & Mediation Data Users Data Providers
    11. 11. Model-Data Comparison Workflow Software Workflow Flow Program Lego-like assembly of component AeroCom Chemical Models Paris, FR VIEWS Chemical Data Ft. Collins, CO Model- Data Comparison Workflow Std I/O Std I/O WMS, WCS OGC Services
    12. 13. DataFed: 100+ Datasets Non-intrusively Federated <ul><li>Data are accessed from autonomous, distributed providers </li></ul><ul><li>DataFed ‘wrappers’ provide uniform geo-time referencing </li></ul><ul><li>Tools allow space/time overlay, comparisons and fusion </li></ul>Near Real Time Data Integration Delayed Data Integration Surface Air Quality AIRNOW O3, PM25 ASOS_STI Visibility, 300 sites METAR Visibility, 1200 sites VIEWS_OL 40+ Aerosol Parameters Satellite MODIS_AOT AOT, Idea Project GASP Reflectance, AOT TOMS Absorption Indx, Refl. SEAW_US Reflectance, AOT Model Output NAAPS Dust, Smoke, Sulfate, AOT WRF Sulfate Fire Data HMS_Fire Fire Pixels MODIS_Fire Fire Pixels Surface Meteorology RADAR NEXTRAD SURF_MET Temp, Dewp, Humidity… SURF_WIND Wind vectors ATAD Trajectory, VIEWS locs.
    13. 14. Summary <ul><li>Third-party mediation can homogenize distributed ES data </li></ul><ul><li>Agile SOA-based IS can deliver diverse info products to users </li></ul><ul><li>Since 2005, one such IS, DataFed is used by EPA and in research </li></ul><ul><li>For networking, more data and services need to be federated </li></ul>Parting thoughts Think outside the stovepipe – Think networking Divide and Conquer, NO! Connect and Enable, YES! Thank you

    ×