061206 Ua Huntsville Seminar


Published on


Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • We will start out by highlighting significant trends in air pollution monitoring and management Next we will describe an agile information system architecture for air quality decision support Finally, we will show the application of this architecture and technologies for two use cases: a real time smoke event analysis and a long-term model evaluation
  • 061206 Ua Huntsville Seminar

    1. 1. Part I DataFed An Agile Distributed Air Quality Data System Rudolf B. Husar Washington University, St. Louis Seminar Presented at University of Alabama, Dept. of Atmospheric Sciences December 6, 2006
    2. 2. Revolution in Sensing (1990s) & Delivery (2000+) Real-time Air Pollution Sensing and Reporting High Resolution Satellite Data Surface PM25 and Ozone Data Smoke Plumes
    3. 3. Changes in Air Quality Management Command & Control Weight of Evidence Flexible NAAMS Rigid Monitoring
    4. 4. Data Acquisition and Usage Activities (Select View Show, click to step through PPT) <ul><li>The data life cycle consists of the acquisition and the usage parts </li></ul>Usage Activities Data Acquisition <ul><li>The acquisition part processes the sensory data by firmly linked procedures </li></ul>The focus is on data usage activities <ul><li>The usage activities are more iterative, dynamic procedures </li></ul><ul><li>The collected and cleaned data are stored in the repository </li></ul>Data Repository <ul><li>The usage cycle transform data into knowledge for decision making </li></ul>Decisions
    5. 5. Stages of AQ Data Flow and Value-Adding Processes Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System Analyzing Filter/Integrate Aggregate/Fuse Custom Analysis Organizing Document Structure/Format Interfacing Characterizing Display/Browse Compare/Fuse Characterize Value-Adding Processes Reporting Inclusiveness Iterative/Agile Dynamic Report
    6. 6. Generic Decision Support for Air Quality Decisions Global Earth Observing System of Systems GEOSS Architecture Framework Knowledge into the Minds of Regulatory Analysts Knowledge into the Minds of Technical Analysts Observations Reports: Model Forecasts, Obs. Evidence Models Decisions Knowledge into the Minds of Decision- making managers Decision Support System
    7. 7. Characteristics of System of Systems (SoS) <ul><li>Autonomous constituents managed/operated independently </li></ul><ul><li>Independent evolution of each constituent </li></ul><ul><li>Displays emergent behavior </li></ul><ul><li>Must recognize, manage, exploit the characteristics: </li></ul><ul><li>No stakeholder has complete SoS insight </li></ul><ul><li>Central control is limited; distributed control is essential </li></ul><ul><li>Users, must be involved throughout the life of a SoS </li></ul>
    8. 8. Differentiating System Engineering and System of Systems Engineering “ System of Systems Engineering for Air Force Capability Development”, 2005
    9. 9. Air Quality Cluster TechTrack Earth Science Information Partners Air Quality Information System Architecture Air Quality Data Meteorology Data Emissions Data Informing Public AQ Compliance Status and Trends Network Assess. Tracking Progress Data to Knowledge Transformation Data Products Knowledge Products Mediators Partners <ul><li>NASA </li></ul><ul><li>NOAA </li></ul><ul><li>EPA </li></ul><ul><li>(?) </li></ul><ul><li>USGS </li></ul><ul><li>DOE </li></ul><ul><li>NSF </li></ul><ul><li>Industry… </li></ul>Flow of Data Flow of Control
    10. 10. Loosely Coupled Data Access through Standard Protocols The next three slides describe the key technologies used in the creation of an adaptable and responsive air quality information system. OGC data access protocols and standard formats facilitate loose coupling between data on the internet and processing services. For air quality, the Web Coverage Service (WCS), provides a universal simple query language for requesting data as where, when, what. That is: geographic (3D bounding box), time range and parameter. The Web Map Service (WMS) and Web Feature Service (WFS) are also useful. The use of standard data physical data formats and naming conventions elevates the syntactic and semantic interoperability. Within DataFed all data access services are implemented as WCS or WMS and optionally WFS. General format adapter components permit data request in a variety of standard formats. GetCapabilities GetData Capabilities, ‘Profile’ Data Where? When? What? Which Format? Server Back End Std. Interface Client Front End Std. Interface T2 T1 Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System CF, EOS, OGC CF OGC, ISO OGC, ISO Standards netCDF, HDF.. Format Temperature What? Time When? BBOX Where? GetData Query
    11. 11. Web Services and Workflow for Loose Coupling Service Broker Service Provider Publish Find Bind Service User Web Service Interaction Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System Web Services Triad: Publish – Find – Bind Workflow Software: Dynamic Programming Service Chaining & Workflow
    12. 12. Collaborative Reporting and Dynamic Delivery Analysis Reports: Information supplied by many Needs continuous program feedback Report needs many authors Wiki technologies are for collaborative writing Dynamic Delivery: Much of the content is dynamic Animated presentations are compelling Movies and screencasts are for dynamic delivery Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System Co Writing - Wiki ScreenCast
    13. 13. Agile Information System: Data Access, Processing and Products Providers NASA DAACs EPA R&D Model EPA AIRNow others Public Manager Scientist Users other <ul><li>The info system transforms the data into info products for each user </li></ul><ul><li>In the first stage the heterogeneous data are prepared for uniform access </li></ul>Uniform Access <ul><li>The second stage performs filtering, aggregation, fusion and other operations </li></ul>Data Processing Web Service Chain Custom Processing SciFlo DataFed Info Products Reports, Websites Forecasting Compliance Other Sci. Reports <ul><li>The third stage prepares and delivers the needed info products </li></ul>
    14. 14. Approach: Mediation Between Users and Data Providers DataFed assumes spontaneous, autonomous data providers Non-intrusively wraps datasets for access by web services Mediates, homogenizes data views. e.g. geo-spatial, time...
    15. 15. ‘ Stovepipe’ and Federated Usage Architectures Landscape Scientist Science DAACs <ul><li>Current info systems are project/program oriented and provide end-to-end solutions </li></ul>Info Users Data Providers Info System AIRNow Public AIRNow Model Compliance Manager <ul><li>Part of the data resources of any project can be shared for re-use through DataFed </li></ul><ul><li>Through the Federation, the data are homogenized into multi-dimensional cubes </li></ul><ul><li>Data processing and rendering can then be performed through web services </li></ul><ul><li>Each project/program can be augmented by Federation data and services </li></ul>
    16. 16. DataFed User Tools <ul><ul><li>Data Catalog </li></ul></ul><ul><ul><li>Data Browser </li></ul></ul><ul><ul><li>PlumeSim, Animator </li></ul></ul><ul><ul><li>Combined Aerosol Trajectory Tool (CATT) </li></ul></ul>Consoles: Data from diverse sources are displayed to create a rich context for exploration and analysis CATT: Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets
    17. 17. Federated Datasets <ul><li>Data are accessed from autonomous, distributed providers </li></ul><ul><li>DataFed ‘wrappers’ provide uniform geo-time referencing </li></ul><ul><li>Tools allow space/time overlay, comparisons and fusion </li></ul>Near Real Time Data Integration Delayed Data Integration Surface Air Quality AIRNOW O3, PM25 ASOS_STI Visibility, 300 sites METAR Visibility, 1200 sites VIEWS_OL 40+ Aerosol Parameters Satellite MODIS_AOT AOT, Idea Project GASP Reflectance, AOT TOMS Absorption Indx, Refl. SEAW_US Reflectance, AOT Model Output NAAPS Dust, Smoke, Sulfate, AOT WRF Sulfate Fire Data HMS_Fire Fire Pixels MODIS_Fire Fire Pixels Surface Meteorology RADAR NEXTRAD SURF_MET Temp, Dewp, Humidity… SURF_WIND Wind vectors ATAD Trajectory, VIEWS locs.
    18. 18. A Sample of Datasets Accessible through ESIP Mediation Near Real Time (~ day) <ul><li>It has been demonstrated (project FASTNET) that these and other datasets can be accessed, repackaged and delivered by AIRNow through ‘Consoles’ </li></ul>MODIS Reflectance MODIS AOT TOMS Index GOES AOT GOES 1km Reflec NEXTRAD Radar MODIS Fire Pix NRL MODEL NWS Surf Wind, Bext
    19. 19. Web Services: Building Blocks of DataFed Programming Access, Process, Render Data by Service Chaining Web Service Composition NASA SeaWiFS Satellite NOAA ATAD Trajectory OGC Map Boundary RPO VIEWS Chemistry Data Access Data Processing Layer Overlay LAYERS
    20. 20. Interoperability Wrappers and Adapters