Current Air Quality Information ‘Ecosystem’  (Draft for Feedback) AQ information includes  emissions ,  ambient  &  satellite  data and  model  outputs The  distributed  data are produced and provided by  agencies , mostly through  portals Providers have  different access protocols , formats, and information usage conditions This  lack of interoperability  causes the  under-utilization  of the rich data resources
Future Integrated AQ information System  (Draft for Feedback) Data are maintained by custodians and exposed through ‘portals’  Mediators uniformly ‘wrap’ data and provide processing services Analysts program the services to create application-specific products Responsibility is shared among data providers and mediator/ integrators ESIPFed can provide the infrastructure and tools for the AQ info system  DataMart VIEWS NEISGEI AIRNow AQMod DAACs ASOS NEI Emission IDEA GASP Missions WeaMod Forecast GloMod FireInv Data Federation  Distributed, Virtual, Uniform AQ Forecasting AQ Compliance Status and Trends Network Assess. Data Processing  Filtering, Aggregation, Fusion Info Products   Reports, Websites  Mediators
Federated Air Quality Data System -   Draft  Text 1 Text 2 ESIP AQ Cluster 050510  Draft  [email_address] Run and click PPT Slideshow to see chart animations Wrappers Where? What? When? Federate Data Structuring Slice & Dice Explore Data Viewers Programs Integrate Understand Inform Public AQ Compliance Forecast AQ Status & Trends Satellite Devel. Network Asses. Manage Hazards ……… Info Needs Reports Emission Surface Satellite Model Single Datasets Providers Networking  Reuse Non-intrusive Linking & Mediation Data Users Data Providers
DataFed Description DataFed Vision Better air quality management and science through by effective use of relevant data   DataFed Goals Facilitate the access and flow of atmospheric data from provider to users Support the development of user-driven data processing value chains P articipate in specific application projects    Approach: Mediation Between Users and Data Providers DataFed assumes spontaneous, autonomous emergence of AQ data ( a la  Internet) Non-intrusively  wraps  datasets for access by web services WS-based  mediators  provide homogeneous data views e.g. geo-spatial, time... End-user programming  of data access and processing through WS composition (limited)    Applications Building browsers and analysis tools for distributed monitoring data     Serve as data gateway for user programs; web pages, GIS, science tools DataFed is currently focused on the mediation of air quality data
Anatomy of a Wrapper Service: TOMS Satellite Image Data Given the URL template and the image description, the wrapper service can access the image for any day, any spatial subset using a HTTP URL or SOAP protocol: Wrapper classes are available for geo-spatial (incl. satellite) images, SQL servers, text files,etc. The mediator classes are implemented as web services for uniform data access, transformation and portrayal.   src_img_width src_img_height src_margin_right src_margin_left src_margin_top src_margin_bottom src_lon_min src_lat_max src_lat_min src_lon_max Image Description for Data Access: src_image_width=502 src_image_height=329 src_margin_bottom=105 src_margin_left=69 src_margin_right=69 src_margin_top=46 src_lat_min=-70 src_lat_max=70 src_lon_min=-180 src_lon_max=180 The daily TOMS images reside on the FTP archive, e.g.  ftp://toms. gsfc . nasa . gov /pub/ eptoms /images/aerosol/y2000/ea000820.gif URL template:  ftp://toms.gsfc.nasa.gov/pub/eptoms/images/aerosol/y[yyyy]/ea[yy][mm][dd].gif Transparent colors for overlays RGB(89,140,255) RGB(41,117,41) RGB(23,23,23) RGB(0,0,0)
SeaWiFS Satellite SeaWiFS Satellite Aerosol Chemical Air Trajectory Map Boarder VIEW by Web Service Composition
Datasets Used in FASTNET Data are accessed from autonomous, distributed providers DataFed ‘wrappers’ provide uniform geo-time referencing Tools allow space/time overlay, comparisons and fusion Near Real Time Data Integration Delayed Data Integration Surface Air Quality  AIRNOW O3, PM25  ASOS_STI Visibility, 300 sites METAR Visibility, 1200 sites VIEWS_OL 40+ Aerosol Parameters Satellite MODIS_AOT AOT, Idea Project GASP Reflectance, AOT TOMS Absorption Indx, Refl. SEAW_US Reflectance, AOT Model Output NAAPS Dust, Smoke, Sulfate, AOT WRF Sulfate Fire Data HMS_Fire Fire Pixels MODIS_Fire Fire Pixels Surface Meteorology RADAR NEXTRAD SURF_MET Temp, Dewp, Humidity… SURF_WIND Wind vectors ATAD Trajectory, VIEWS locs.
A Sample of Datasets Accessible through ESIP Mediation Near Real Time (~ day) It has been demonstrated (project FASTNET) that these and other datasets can be accessed, repackaged and delivered by AIRNow through ‘Consoles’ MODIS Reflectance MODIS AOT TOMS Index GOES  AOT GOES  1km Reflec NEXTRAD Radar MODIS  Fire Pix NRL MODEL NWS Surf Wind, Bext
FASTNET: Inter-RPO pilot project, through NESCAUM, 2004 Web-based data, tools for community use Built on DataFed infra-structure, NSF, NASA Project fate depends on sponsor, user evaluation
Some of the Tools Used in FASTNET Data Catalog Data Browser PlumeSim, Animator Combined Aerosol Trajectory Tool (CATT) Consoles:  Data from diverse sources are displayed to create a rich context for exploration and analysis CATT:  Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions  Viewer:  General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets
Midwest HazeCam Image Console Image Archive and Browser Hourly Midwest HazeCam Images are archived by DataFed data access system Archived images for all cameras can be browsed through this console HazeCam URL for a day:  http://www.datafed.net/consoles/MWH_WebCams.asp?image_width=400&image_height=300&datetime=2005-01-31T13:00:00 URL for a site and day:  http:// webapps . datafed .net/datasets/webcam/ cincinnati /20050131-13mwhcincinnati.jpg URLs can be embedded as links into emails, bookmarks, web pages, PPT and PDF files.  Midwest HazeCam Image Browser Select date and time Set image size and time MW HazeCam Console Other FASTNET Consoles
Aerosol Event Catalog: Web pages Catalog of generic ‘web objects’ – pages, images, animations that relate to aerosol events Each ‘web object’ is cataloged by location, time and aerosol type.
Distributed  Responsibility  in OpenDAP The  data  lies with the data  providers The  data  access protocol  lies with  OPeNDAP Application programs  with the  developers  (Matlab, .. Excel…) Data  discovery  with the  GCMD  and NVODS Distributed Responsibility DataFed(??)  The  data  lies with the data  providers The  wrappers  and  mediators  with  DataFed community Application programs  with  end user   Data  discovery  with  data & service registries Distribution of Responsibility
0504142345 HMS + GOES
0504142345 HMS + GOES

050510 Esip Data Fed Smoke

  • 1.
    Current Air QualityInformation ‘Ecosystem’ (Draft for Feedback) AQ information includes emissions , ambient & satellite data and model outputs The distributed data are produced and provided by agencies , mostly through portals Providers have different access protocols , formats, and information usage conditions This lack of interoperability causes the under-utilization of the rich data resources
  • 2.
    Future Integrated AQinformation System (Draft for Feedback) Data are maintained by custodians and exposed through ‘portals’ Mediators uniformly ‘wrap’ data and provide processing services Analysts program the services to create application-specific products Responsibility is shared among data providers and mediator/ integrators ESIPFed can provide the infrastructure and tools for the AQ info system DataMart VIEWS NEISGEI AIRNow AQMod DAACs ASOS NEI Emission IDEA GASP Missions WeaMod Forecast GloMod FireInv Data Federation Distributed, Virtual, Uniform AQ Forecasting AQ Compliance Status and Trends Network Assess. Data Processing Filtering, Aggregation, Fusion Info Products Reports, Websites Mediators
  • 3.
    Federated Air QualityData System - Draft Text 1 Text 2 ESIP AQ Cluster 050510 Draft [email_address] Run and click PPT Slideshow to see chart animations Wrappers Where? What? When? Federate Data Structuring Slice & Dice Explore Data Viewers Programs Integrate Understand Inform Public AQ Compliance Forecast AQ Status & Trends Satellite Devel. Network Asses. Manage Hazards ……… Info Needs Reports Emission Surface Satellite Model Single Datasets Providers Networking Reuse Non-intrusive Linking & Mediation Data Users Data Providers
  • 4.
    DataFed Description DataFedVision Better air quality management and science through by effective use of relevant data   DataFed Goals Facilitate the access and flow of atmospheric data from provider to users Support the development of user-driven data processing value chains P articipate in specific application projects   Approach: Mediation Between Users and Data Providers DataFed assumes spontaneous, autonomous emergence of AQ data ( a la Internet) Non-intrusively wraps datasets for access by web services WS-based mediators provide homogeneous data views e.g. geo-spatial, time... End-user programming of data access and processing through WS composition (limited)   Applications Building browsers and analysis tools for distributed monitoring data    Serve as data gateway for user programs; web pages, GIS, science tools DataFed is currently focused on the mediation of air quality data
  • 5.
    Anatomy of aWrapper Service: TOMS Satellite Image Data Given the URL template and the image description, the wrapper service can access the image for any day, any spatial subset using a HTTP URL or SOAP protocol: Wrapper classes are available for geo-spatial (incl. satellite) images, SQL servers, text files,etc. The mediator classes are implemented as web services for uniform data access, transformation and portrayal. src_img_width src_img_height src_margin_right src_margin_left src_margin_top src_margin_bottom src_lon_min src_lat_max src_lat_min src_lon_max Image Description for Data Access: src_image_width=502 src_image_height=329 src_margin_bottom=105 src_margin_left=69 src_margin_right=69 src_margin_top=46 src_lat_min=-70 src_lat_max=70 src_lon_min=-180 src_lon_max=180 The daily TOMS images reside on the FTP archive, e.g. ftp://toms. gsfc . nasa . gov /pub/ eptoms /images/aerosol/y2000/ea000820.gif URL template: ftp://toms.gsfc.nasa.gov/pub/eptoms/images/aerosol/y[yyyy]/ea[yy][mm][dd].gif Transparent colors for overlays RGB(89,140,255) RGB(41,117,41) RGB(23,23,23) RGB(0,0,0)
  • 6.
    SeaWiFS Satellite SeaWiFSSatellite Aerosol Chemical Air Trajectory Map Boarder VIEW by Web Service Composition
  • 7.
    Datasets Used inFASTNET Data are accessed from autonomous, distributed providers DataFed ‘wrappers’ provide uniform geo-time referencing Tools allow space/time overlay, comparisons and fusion Near Real Time Data Integration Delayed Data Integration Surface Air Quality AIRNOW O3, PM25 ASOS_STI Visibility, 300 sites METAR Visibility, 1200 sites VIEWS_OL 40+ Aerosol Parameters Satellite MODIS_AOT AOT, Idea Project GASP Reflectance, AOT TOMS Absorption Indx, Refl. SEAW_US Reflectance, AOT Model Output NAAPS Dust, Smoke, Sulfate, AOT WRF Sulfate Fire Data HMS_Fire Fire Pixels MODIS_Fire Fire Pixels Surface Meteorology RADAR NEXTRAD SURF_MET Temp, Dewp, Humidity… SURF_WIND Wind vectors ATAD Trajectory, VIEWS locs.
  • 8.
    A Sample ofDatasets Accessible through ESIP Mediation Near Real Time (~ day) It has been demonstrated (project FASTNET) that these and other datasets can be accessed, repackaged and delivered by AIRNow through ‘Consoles’ MODIS Reflectance MODIS AOT TOMS Index GOES AOT GOES 1km Reflec NEXTRAD Radar MODIS Fire Pix NRL MODEL NWS Surf Wind, Bext
  • 9.
    FASTNET: Inter-RPO pilotproject, through NESCAUM, 2004 Web-based data, tools for community use Built on DataFed infra-structure, NSF, NASA Project fate depends on sponsor, user evaluation
  • 10.
    Some of theTools Used in FASTNET Data Catalog Data Browser PlumeSim, Animator Combined Aerosol Trajectory Tool (CATT) Consoles: Data from diverse sources are displayed to create a rich context for exploration and analysis CATT: Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets
  • 11.
    Midwest HazeCam ImageConsole Image Archive and Browser Hourly Midwest HazeCam Images are archived by DataFed data access system Archived images for all cameras can be browsed through this console HazeCam URL for a day: http://www.datafed.net/consoles/MWH_WebCams.asp?image_width=400&image_height=300&datetime=2005-01-31T13:00:00 URL for a site and day: http:// webapps . datafed .net/datasets/webcam/ cincinnati /20050131-13mwhcincinnati.jpg URLs can be embedded as links into emails, bookmarks, web pages, PPT and PDF files. Midwest HazeCam Image Browser Select date and time Set image size and time MW HazeCam Console Other FASTNET Consoles
  • 12.
    Aerosol Event Catalog:Web pages Catalog of generic ‘web objects’ – pages, images, animations that relate to aerosol events Each ‘web object’ is cataloged by location, time and aerosol type.
  • 13.
    Distributed Responsibility in OpenDAP The data lies with the data providers The data access protocol lies with OPeNDAP Application programs with the developers (Matlab, .. Excel…) Data discovery with the GCMD and NVODS Distributed Responsibility DataFed(??) The data lies with the data providers The wrappers and mediators with DataFed community Application programs with end user Data discovery with data & service registries Distribution of Responsibility
  • 14.
  • 15.