2008-02-11: EPA DataFed Presentation

The Federated Data System DataFed
Non- intrusive data integration infrastructure
Based on standards-based web services
Processing tools created from reusable components

Local, Regional, Global Pollution
Before 1950s:
Local
Smoke, Fly ash
Post- 2000s: Global, HTAP Ozone, PM,Global Change 1970s-1990s: Regional, LRTP Acid Rain, Haze
The LRTP/HTAP flow of air pollutants is likely to increase as overseas economies grow.
Pollutant influx leads to significant exceedances of O3 PM NAAQS in some regions
Even after domestic controls, some US areas will be no-compliant because of LRTP

Terrestrial Airborne Near-Space LEO/MEO Commercial Satellites and Manned Spacecraft Far-Space L1/HEO/GEO TDRSS & Commercial Satellites Deployable Permanent Coordinating Earth Observing Systems Forecasts & Predictions Aircraft/Balloon Event Tracking and Campaigns User Community Vantage Points Capabilities

` Products Products State & Local Canadian Providences NOAA NWS HHS CDC-EPHTN Aerosol Optical Depth (GASP) TERRA MODIS AQUA MODIS Products CMAQ Forecast Data US EPA AQS Products SLAMS/NAMS SURACE PM 2.5 Data Air Quality/Public Health NTO Integrated Observed-Modeled Air Quality Fields Products Spatial surface Predictions Satellite/Model/Surface Data Fusion State Public Health Departments ~10:30 local overpass ~1:30 local overpass Algorithms/QA NASA GFSC Science Team NASA GFSC DACC *Note: Regional East Atmospheric Lidar Mesonet (REALM) is university led federated network by UMBC and is identified as a NTO in the implementation plan. Products Aerosol Optical Depth (MOD04_L2) NOAA NESDIS NOAA NESDIS/ORA & CREST Institutes ? UMBC CREST Institute GEOS-12 CONUS every 30 minutes REALM Continuous Vertical Resolution Data Products Algorithms/QA EPA OAR & ORD Products CMAQ Assessment Data Products Studies and Impacts to human health US EPA OAQPS/ORD/OEI RSI Gateway

P. Dickerson, EPA

http://www.igospartners.org/ http://earthobservations.org/ http://www.epa.gov/ttn/amtic/monstratdoc.html National Ambient Air Monitoring Strategy Office of Air Quality Planning and Standards Research Triangle Park, NC December 2005 http://www.al.noaa.gov/AQRS/ reports/monitoring.html http://www.empa.ch/gaw/gawsis/ http://www.nesdis.noaa.gov/ http://www.emep.int/ CENR/AQRS GEOSS NOAA CMDL NOAA NESDIS EMEP R. Scheffe http://www.fz-juelich.de/icg/icg-ii/iagos/ http://www.fz-juelich.de/icg/icg-ii/mozaic/home http://www.cmdl.noaa.gov/ Barrow Mauna Loa Trinidad Head A. Samoa S. Pole L2 NCORE L3 L1 GAW

The Scheffe Challenge: Organizations - Programs – Data: A Mess Info System Challenges: What’s the overall dependency Information Flow Forces and Controls on Data Flow Cooperation, Competition, Co-Opetition GEOSS Eco-informatics Accountability/ indicators SIPs, nat.rules designations PHASE PM research Risk/exposure assessments AQ forecasting Programs NAAQS setting EPA NOAA NASA NPS USDA DOE Private Sector States/Tribes RPO’s/Interstate Academia NARSTO NAS, CAAAC CASAC, OMB Enviros Organizations CDC Supersites IMPROVE, NCore PM monit, PAMS CASTNET Lidar systems NADP Satellite data Intensive studies PM centers Other networks: SEARCH, IADN.. Data sources CMAQ GEOS-CHEM Emissions Meteorology Health/mort. records

Relationship Between Organizations - Programs – Data Version 0.1 Goals $$ Info needs, $$ Data need, $$ Judge, Decide, Act Analyze, Report Actionable Knowledge Decision, Action Public Measure, Organize Organized Data Flow of Information Data systems organize the measurements and models and provide them to programs. Programs analyze the data and provide actionable knowledge to organizations. Organizations evaluate multiple information sources, make decisions and act. Flow of Control Public and special interest groups set up organizations and provides them with funding Organizations develop programs, define their scope, governance and funding Programs satisfy their information needs by monitoring or by using other’s data Data sources acquire the data for their parent programs and also expose them for reuse

System of Systems Global Earth Observing System of Systems - GEOSS
Characteristics of System of Systems (SoS)
Autonomous constituents managed/operated independently
Independent evolution of each constituent
SoS displays emergent behavior
Must recognize, manage, exploit the characteristics:
No stakeholder has complete SoS insight
Central control is limited; distributed control is essential
Users, must be involved throughout the life of a SoS

GEOSS Architecture and Interoperability

Screencast: Information Landscape

Screencast: Info System Screencast

Screencast: DataFed Technologies

Screencast: DataFed Tools

KMZ: Google Earth-DataFed Mashup GA Smoke Global Chem

The Transformational Effect of Networking
Information has become the main driver of progress
Time and place are no longer barriers to participation and interaction
The Web has become a medium participation - ‘Web 2.0’ phenomenon
“ Networking has led to an unprecedented surge of productivity” Time Magazine, Person of the Year 2006, YOU
These are opportunities to enable Earth Science through more networking
But many resistances to networking exist that need to be overcome

Networking Multiplies Value Creation Application Data 1 User Stovepipe Value = 1 1 Data x 1 Program = 1 Enclosed Value-Creating Process - ‘Stovepipe’

Application Data Application Application Application Application Stovepipe 1 User Stovepipe Value = 1 1 Data x 1 Program = 1 5 Uses of Data Value = 5 1 Data x 5 Program = 5 Networking Multiplies Value Creation

Networking Multiplies Value Creation Merging data may creates new, unexpected opportunities Not all data are equally valuable to all programs 1 User Stovepipe Value = 1 1 Data x 1 Program = 1 5 Uses of Data Value = 5 1 Data x 5 Program = 5 Open Network Value = 25 5 Data x 5 Program = 25 Data Data Data Data Data Stovepipe Application Application Application Application Application

The Future
AQ Science, Management
Pollutant Characterization (Obs-Model-Emission Integration)
Agile monitoring and assessment
GEOSS, Collaboration, Informatics
The future is bright, too bright?
So many new things, so little time
DataFed
Continue promoting standards-based networking
Enabling IS users create new, actionable knowledge faster
Move data flow maintenance from R/D to operational

Integrated observation-modeling complex – R. Scheffe Optimized PM2.5, O3 Characterizations Health Air management ecosystems Land AQ Monitors Total column depth (through Satellites) AQ model results Vertical Profiles Integrated Observation- Modeling

Pollutant Characterization, Understanding
Characterization – creating the best available pollutant pattern as distributed in space-time-parameter
Characterization - achievable by Reanalysis with the ‘best available’ model and assimilated observations
Understanding gained from the model processes and applying previous/tacit knowledge
Goal: Pollutant Characterization and Understanding
Models Observations Emissions Reanalysis Forward model with assimilated observations Data Interpretation Use of previous & tacit knowledge to explain data GOAL: Knowledge Creation Characterization of pattern; understating of processes Characterization

NAAMS: National Ambient Air Monitoring Strategy and NCore … coordinated multi-pollutant real-time monitoring network

Public Information Health/Exp. Assessment Emissions Planning AQ Trends and Accountability Science Support NAAQS National Air Quality Information Integration AQ Data Pool National Air Quality Info Network Re-examination of Data Access and processing Systems Pooling of data/info resources for re-use in multiple applications; a la GEOSS

Sensing Revolution Web 2.0

Summary
There is a slow ‘ aligning of stars ’ for integrating heterogeneous data
System of Systems architecture is suitable for integrating data
Standard data access is a key interoperability protocol
Heterogeneous data can be non-intrusively standardized by mediators
Service-based software architecture delivers tailored products to diverse uses
Federated data and shared web-based tools are in use
DataFed already includes over 100 datasets (emissions, ground, satellite)
The system has been applied to EPA policy, regulatory and science development
However,
DataFed is just one of the many mediator nodes, but these need to be connected
Much more data would need to be federated
HTAP model-data comparison would be an attractive use case

DataFed Applications (2002-2007) Science Mystery (Nitrate?) Events Data Integration (PM-Bext; NO2 Sat-Surf; AQ Event Detection Algorithms AQ Management Exceptional Event Analysis (EPA – N. Frank) Network Assessment (EPA – R. Scheffe) Fire-Smoke, Global Emissions (EPA – T. Keating) FASTNET, CATT Tools, S/R Analysis (RPO – R. Poirot) IS Networking Infrastructure NASA/ESIP Web Services, SAO (NASA – L. Friedl, K. Moe) GEOSS Interoperability Demos (Wash. U) HTAP Network, Integration (EPA – Keating)

FASTNET Report: 0409FebMystHaze (RPO – R. Poirot) Mystery Winter Haze: Natural? Nitrate/Sulfate? Stagnation? Contributed by the FASNET Community, Sep. 2004 Correspondence to R Husar , R Poirot Coordination Support by Inter-RPO WG Fast Aerosol Sensing Tools for Natural Event Tracking, FASTNET NSF Collaboration Support for Aerosol Event Analysis NASA REASON Coop EPA -OAQPS AIRNOW PM25 - February Sulfate-driven Jul-Aug peak Feb-Mar peak, of unknown origin

Data Fusion: AIRNOW PM25 - ASOS Bext
2004 July 20 14:00
July 21, 2004 July 22, 2004 July 23, 2004 ARINOW PM25 ARINOW PM25 ARINOW PM25 ASOS RHBext ASOS RHBext ASOS RHBext

PM Event Detection from Time Series
Contributed by the FASNET Community, Sep. 2004
Correspondence to R Husar , R Poirot
Coordination Support by
Inter-RPO WG Fast Aerosol Sensing Tools for Natural Event Tracking, FASTNET
NSF Collaboration Support for Aerosol Event Analysis
NASA REASON Coop
EPA -OAQPS
Event : Deviation > x*percentile

Speciated PM Network Assessment (EPA – R. Scheffe) CIRA/ VIEWS Database CAPITA/ DataFed Database Network Assessment PPT IMPROVE EPA SPEC CIRA Tools and Processes DataFed Tools and Processes Analysis Tools and Processes Speciated Data Flow and Processing EPA NCore Process Evaluation, Feedback

Distributed Fire Data Sources (S. Falke, EPA, NASA) Numerous state, regional, and national fire related databases and online access applications exist. The challenge is to bring them together, on-the-fly, without requiring substantial changes to the underlying systems. Also need to access data sources that are not “Web-ready”. BlueSkyRAINS GeoMAC WFAS USGS NOAA UMaryland

Combined Aerosol Trajectory Tool, CATT (RPO – R. Poirot) Next Process Next Process Aerosol Data Collection IMP. EPA Aerosol Sensors Integration VIEWS Integrated AerData AEROSOL Weather Data Assimilate NWS Gridded Meteor. Trajectory ARL Traject.Data TRANSPORT TrajData Cube Aggreg. Traject. AerData Cube CATT Aggreg.Aerosol CATT-In CAPITA CATT-In CAPITA Trajectory Browser Kitty: Simple CATT CATT Transport Analyzer

HTAP Data Network (EPA – T. Keating) TF HTAP Workshop Forshungszentrum Juelich, Oct 17-19, 2007, Juelich, Germay Application Examples for NOx Analysis Collaborators: Rudolf Husar , Washington U. St. Louis Stefan Falke , Northrop, Wash U. Greg Leptoukh , NASA, Goddard Martin Schultz , FZJ, Juelich

GEOSS Interoperability Demos (Washington Univ.) Beijing Barcelona Denver

Origin of Fine Dust Events over the US
Sulfate is local, no major spikes
Gobi dust transport in spring Sahara dust import in summer Fine dust spikes over the entire US are mainly from intercontinental transport

Air Quality Management System: Components and Functions Public Analyzing Interpreting Evaluating Separating Synthesizing Organizing Quality control Formatting Documenting Displaying Deciding Evaluate options Matching goals Compromising Choosing Data Manager, Organizer Technical Analysts, Program Manager Policy Analysts, Decision Maker Value Adding Processes Human Agents Decision Support System (DSS) The primary purpose of data systems is to mediate between data providers and programs/projects Programs perform analysis for Orgs., the DSS is within programs The big decisions of societal importance are done by Organizations (This needs more wisdom from the practioners)

Flow of Data and Usage Control Data Control Requesting Information Providing Information Sensors Acquisition processing User Programs NAAQS SIPs Forecast GEOSS … Info System Negotiating Space Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System User Agencies

DataFed Tools - Subset Consoles: Data from diverse sources are displayed to create a rich context for exploration and analysis CATT: Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets

Summary
Global Monitoring - Modeling Revolution – ‘May you live in interesting times’
We are in the midst of an observational revolution (satellites, monitoring networks).
The global distribution and transport of some pollutants can be monitored daily
Global models are maturing into effective analytical and predictive tools
Results to Date:
Compelling evidence for global-scale transport of PM and Ozone
Qualitative evidence of ‘extra-jurisdictional’ impact on the US air quality
Potential for quantification of natural and non-US impact
Such massive job will require:
International, interagency, interdisciplinary collaboration.
Open flow of data/knowledge
Scientific ‘value-adding chains’

FASTNET and DataFed FASTNET (Fast Aerosol Sensing Tools for Natural Event Tracking) an open communal information sharing facility to study aerosol events , including detection, tracking and impact on PM and haze. The main asset of FASTNET is the community of data analysts, modelers, managers participating in the production of actionable knowledge from data and models The community is supported by a non-intrusive data integration infrastructure based on Internet standards (web services) and a set of web-tools evolving under the federated data system, DataFed DataFed is supported by its community and is under the umbrella of the interagency Earth Science Information Partners, ESIP (NASA, NOAA and EPA)

Emerging Air Quality Data Flow Network OGC WCS Data Access Protocol GEOSS Provides SOA for Coupling for Autonomous Nodes Facilitates Publishing, Finding and Accessing Data

Application of OGC WCS Data Access Protocol
Regardless of the data location, data type and format,
the parameter-space-time query is the same
the return is in user selectable format from the offerings
Coverage=THEEDDS.T& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07 &FORMAT=NetCDF Coverage=SEAW.Refl& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07 &FORMAT=GeoTIFF Coverage=SURF.Bext& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07 &FORMAT=NetCDF-table Grid Image Station Data Parameter Bounding Box Time Range Out Format

Web 1.0 -> Web 2.0 Transformation
The Web is being transformed: It is becoming more participatory
Its content is increasingly generated and distributed by individuals
See the explosive growth of wikies, picture-sharing, blogs, Facebook
This architectural, technological and cultural change is Web 2.0
Web 2.O is good for AtmosphericScience community since it allows
Better harvesting of current knowledge
Collaborative creation new knowledge.

Distributed Responsibility DataFed
The data lies with the data providers
The wrappers and mediators with DataFed community
Application programs with end user
Data discovery with data & service registries
Distribution of Responsibility

The Information Interoperability Stack

Imagine…More Shared Obs & Models…. On Your Fingertips or Google Earth.. 2007++ More Global Data & Models 2007 Global Data & Models

Regional Haze Rule: Natural Aerosol
Looking ahead to reach natural conditions
… in 60+ years!!!

Asian Dust Cloud over N. America On April 27, 1998 the dust cloud arrived in North America. Regional average PM10 concentrations increased to 65 mg/m 3 In Washington State, PM10 concentrations exceeded 100 mg/m 3 Asian Dust 100  g/m 3 Hourly PM10

Aircraft Detection of Siberian Forrest Smoke near Seattle, WA Jaffe et. al., 2003

2008-02-11: EPA DataFed Presentation

by Rudolf Husar

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 20

Statistics