Your SlideShare is downloading. ×

050405 Epa Info System

785

Published on

http://capitawiki.wustl.edu/index.php/20050421_Data_Flow_and_Flow_Control_in_AQ_Management

http://capitawiki.wustl.edu/index.php/20050421_Data_Flow_and_Flow_Control_in_AQ_Management

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
785
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • AQ data arise from diverse sources, each having specific history, driving forces, formats, quality, etc. Data analysis, i.e. turning the raw data into ‘actionable’ knowledge, requires combining data from these sources The three major data ‘processing’ operations (services) are filtering, aggregation and fusion
  • Transcript

    • 1. Data Flow and Flow Control in AQ Management
      • Data are supplied by the provider and exposed on the ‘smorgasbord’
      • However, the choice of data and processes is made by the user
      • Thus, the autonomous data consumers, providers and mediators form the info system
      Provider Push User Pull Flow of Data Flow of Control AQ DATA METEOROLOGY EMISSIONS DATA Informing Public AQ Compliance Status and Trends Network Assess. Tracking Progress Data to Knowledge ‘Refinery’
      • The data ‘refining’ process is not a chain but network connection processing nodes.
      • Like on the Internet, new nodes and connections are added continuously
      • Thus, the infosystem needs to support the dynamic addition of new nodes and connections
      • Hence – there is a need for loosely coupled ‘plug-and-play’ architecture
    • 2. CATT: A Community Tool! Part of an Analysis Value Chain Next Process Next Process Why? How? When? Where? Aerosol Data Collection IMP. EPA Aerosol Sensors Integration VIEWS Integrated AerData AEROSOL Weather Data Assimilate NWS Gridded Meteor. Trajectory ARL Traject.Data TRANSPORT TrajData Cube Aggreg. Traject. AerData Cube CATT Aggreg.Aerosol CATT-In CAPITA CATT-In CAPITA There! Not There! Further Analysis GIS Grid Processing Emission Comparison
    • 3. Direction of Dust Origin at 5 IMPROVE Sites Ad hoc Data Processing Value Chain High ‘dust’ concentration at 5 sites indicate the same airmass pathway from the tropical Atlantic Weather Serv. Upper Air Data NOAA ARL ATAD ATAD Traject Gebhart (2002) NPS-CIRA IMPROVEData PMF Tool Pareto (2001) PMF “Sources” Coutant (2002) CATT Tool Husar (2003) Aggregation Poirot (2003)
    • 4. Background
      • Atmospheric aerosol system has three extra dimensions (red), compared to gases (blue) :
        • Spatial dimensions (X, Y, Z)
        • Temporal Dimensions (T)
        • Particle size (D)
        • Particle Composition ( C )
        • Particle Shape (S)
      • Bad news: The mere characterization of the 7D aerosol system is a challenge
        • Spatially dense network -X, Y, Z(??)
        • Continuous monitoring (T)
        • Size segregated sampling (D)
        • Speciated analysis ( C )
        • Shape (??)
      • Good news: The aerosol system is self-describing.
        • Once the aerosol is characterized (Speciated monitoring) and multidimensional aerosol data are organized, (see RPO VIEWS effort), unique opportunities exists for extracting information about the aerosol system (sources, transformations) from the data directly.
      • Analysts challenge: Deciphering the handwriting contained in the data
        • Chemical fingerprinting/source apportionment
        • Meteorological back-trajectory analysis
        • Dynamic modeling
    • 5.
      • SeaWiFS Satellite
      SeaWiFS Satellite Aerosol Chemical Air Trajectory Map Boarder VIEW by Web Service Composition
    • 6. The Researcher’s Challenge “ The researcher cannot get access to the data; if he can, he cannot read them; if he can read them, he does not know how good they are; and if he finds them good he cannot merge them with other data.” Information Technology and the Conduct of Research: The Users View National Academy Press, 1989
      • These resistances can be overcome through
      • A catalog of distributed data resources for easy data ‘ discovery ’
      • Uniform data coding and formatting for easy access, transfer and merging
      • Rich and flexible metadata structure to encode the knowledge about data
      • Powerful shared tools to access, merge and analyze the data
    • 7. Earth Science Data to Knowledge Transformation: Value-Adding Processes Petabytes 10 15 Terabytes 10 12 Gigabytes 10 9 Megabytes 10 6 Calibration, Transformation To Characterized Geophysical Parameters Filtering, Aggregation, Fusion, Modeling, Trends, Forecasting Interactive Dissemination ACCESS Multi-platform/parameter, high space/time resolution, remote & in-situ sensing Sensing Analysis & Synthesis Data Acquisition Value Chain (Network) InfoSystem Goal: Add as much value to the data as possible to benefit all users Data Usage Value Network Flexible data selection, and processing to to deliver right knowledge, right place right time Data - L1 Information – L2 Knowledge – L3-6? Usable Knowledge Query Data Distributed, Dynamic More Local, DAAC Processing Knowledge Use
    • 8. Assertions on Web Services Technology
      • Currently Web Services are the leading (and only?) technologies for building software applications in autonomous, networked, dynamic environment
      • The future is promising since businesses are driving the WS technologies and the community is benefiting from the increasingly ‘semantic web’
      • A growing resource pool is exposed as ‘services’ and WS-based ES applications development frameworks are being developed/evaluated (e.g. SciFlo, DataFed)
      • WS Adaptation Issues
      • Catalogs for finding and using services are grossly inadequate
      • The semantic layers of the interoperability stack are not yet available
      • General ‘ fallacies of distributed computing’ :
        • Network is reliable
        • Latency is zero
        • Bandwidth infinite
        • Network is secure
        • Topology stable
        • One administrator
        • No transport costs
        • Network uniform
    • 9. Interoperability Stack
      • Kickoff Questions
      • What is a Web Service?
        • e.g. 'A programming module with a well-defined, web-based I/O interface' (operating on well structured data??)
        • Examples of what is/is not a WS
      • WS Classification by Interoperability Layer
        • Transport
        • Interface Syntax
          • Strongly typed interface (e.g. SOAP, WSDL)
          • Weakly typed interface (e.g. arbitrary CGI? URL interface)
        • Protocol/Data
        • Semantics
      • WS Classification by Architecture
        • Services for Tightly Coupled applications (e.g. URL service called from IDL)
        • Services for Loosely Coupled (e.g. application composed from SOAP services)
      HTTP, SMTP Addressing, Data flow Transport XML Data format Syntax SOAP, WS-* ext. Communication behavior Protocol Schema, WSDL Types Data WSDL ext., Policy, RDF Meaning Semantics Standards Description Layer
    • 10. Data Flow and Flow Control in AQ Management
      • Relationship between different information activities
      States Regions AIRS AQS EPA Air Portal EPA Science Portal VIEWS AIRNOW
    • 11. Information Techology Vision Scenario: Smoke Impact REASoN Project: Application of NASA ESE Data and Tools to Particulate Air Quality Management ( PPT/PDF )
      • Scenario:
      • Smoke form Mexico causes record PM over the Eastern US.
      • Goal:
      • Detect smoke emission and predict PM and ozone concentration
      • Support air quality management and transportation safety
      • Impacts:
      • PM and ozone air quality episodes, AQ standard exceedance
      • Transportation safety risks due to reduced visibility
      • Timeline:
      • Routine satellite monitoring of fire and smoke
      • The smoke event triggers intensified sensing and analysis
      • The event is documented for science and management use
      • Science/Air Quality Information Needs:
      • Quantitative real-time fire & smoke emission monitoring
      • PM, ozone forecast (3-5 days) based on smoke emissions data
      • Information Technology Needs:
      • Real-time access to routine and ad-hoc data and models
      • Analysis tools: browsing, fusion, data/model integration
      • Delivery of science-based event summary/forecast to air quality and aviation safety managers and to the public
      Record Smoke Impact on PM Concentrations [email_address] , stefan @me. wustl . edu Smoke Event
    • 12. Smoke Scenario: IT needs and Capabilities Community interaction during events through virtual workgroup sites; quantitative now-casting and observation-augmented forecasting Smoke event summary and forecast suitably packaged and delivered for agency and public decision makers Uncoordinated event monitoring, serendipitous and limited analysis. Event summary by qualitative description and illustration Smoke event summary and forecast for managers (air quality, aviation safety) and the public Services linking tools Service chaining languages for building web applications; Data browsers, data processing chains; Tools for navigating spatio-temporal data; User-defined views of the smoke; Conceptual framework for merging satellite, surface and modeling data Most tools are personal, dataset specific and ‘hand made’ Analysis tools for data browsing, fusion and data/model integration Web services for data registration, geo-time-parameter referencing, non-intrusive addition of ad hoc data; communal tools for data finding, extracting Agents (services) to seamlessly access distributed data and provide uniformly presented views of the smoke. Human analysts access a fraction of a subset of qualitative satellite images and some surface monitoring data Limited real-time datasets are downloaded from providers, extracted, geo-time-param-coded, etc. by each analyst Real-time access to routine and ad-hoc fire, smoke, transport data/ and models How to get there New capabilities Current state IT need vision
    • 13. Data Analysis and Decision Support   Retrospective Anal. Months-years Now Analysis Days Predictive Analysis Days-years Data Sources & Types All the Real-Time data + NPS IMPROVE Aer. Chem. EPA Speciation EPA PM10/PM2.5 EPA CMAQ Full Chem. Model EPA PM2.5Mass NWS ASOS Visibility, WEBCAMs NASA MODIS, GOES, TOMS, MPL NOAA Fire, Weather & Wind NAAPS MODEL Simulation NAAPS MODEL Forecast NOAA/EPA CMAQ? Data Analysis Tools & Methods Full chemical model simulation Diagnostic & inverse modeling Chemical source apportionment Multiple event statistics Spatio-temporal overlays Multi-sensory data integration Back & forward trajectories, CATT Pattern analysis Emission and met. forecasts Full chemical model Data assimilation Parcel tagging, tracking Communication Collab. & Coord. Methods Tech Reports for reg. support Peer reviewed scientific papers Science-AQ mgmt. interaction Reconciliation of perspectives Analyst and managers consoles Open, inclusive communication Data assimilation methods Community data & idea sharing Open, public forecasts Model-data comparison Modeler-data analyst comm. Analysis Products Quantitative natural aer. concr. Natural source attribution Comparison to manmade aer. Current Aerosol Pattern Evolving Event Summary Causality (dust, smoke, sulfate) Future natural emissions Simulated conc. pattern Future location of high conc. Decision Support Jurisdiction: nat./manmade State Implementation Plans, (SIP) PM/Haze Crit. Documents, Regs Jurisdiction: nat./manmade Triggers for management action Public information & decisions Statutory & policy changes Management action triggers Progress tracking
    • 14. Data Acquisition and Usage Value Chain Monitor Store Data 1 Monitor Store Data 2 Monitor Store Data n Monitor Store Data m IntData 1 IntData n IntData 2 Virtual Int. Data
    • 15. Information ‘Refinery’ Value Chain (Taylor, 1985) Organizing Grouping Classifying Formatting Displaying Analyzing Separating Evaluating Interpreting Synthesizing Judging Options Quality Advantages Disadvantages Deciding Matching goals, Compromising Bargaining Deciding e.g. CIRA VIEWS e.g. Langley IDEA FASTNET Summary Rpt e.g. RPO Manager Informing Knowledge Action Productive Knowledge Information Data

    ×