2004-09-21 Natural Aerosol Event Detection and Characterization
110823 solta11 intro
1. What is the workshop theme?
Air Quality Data Network
(ADN)
2. What is the Air Quality Data Network?
A non-intrusive standards-based network infrastructure consisting of
distributed servers, clients, catalogs and governance
Data Network Pilot
7 Active AQ WCS Servers
Data Catalog
Prototype
3. Purpose of the Air Quality Data Network?
To facilitate integration of Earth Observation and Models
and re-use in multiple Societal Benefit Areas
Earth Ob- Societal
servations Benefit
Monitorig Informing
Network the Public
Data Network Pilot Protecting
Satellite 7 Active AQ WCS Servers Health
Atmosph.
Model Science
Global
Emission Data Catalog Policies
Prototype
Any Dataset Can Serve Any Single Problem
Many Communities Requires Many Datasets
4. What is the tangible contribution of the Network?
Earth Ob- Societal
servations Benefit
Monitorig Informing
Network the Public
Protecting
Satellite
Data Pool Health
Atmosph.
Model Science
Global
Emission Policies
5. Who participates in the AQ Data Network?
Earth Ob- Societal
Data Portal Faciliator Decision SupportBenefit
servations
ABC | AQ_CoP | B.GN | CHIST |CIERA | COST |
CyAir DataONE | EANET | EGIDA | ESIP | AIRNow
CIERA EuroGEOSS | INSPIRE | MaleDecl | QA4EO BlueSky
Monitorig DataFed CIAM Informing
Network EBAS INSPIRE
PEGASOS the Public
MACC SDS-WAS
VIEWS VIEWS …
.... Others
Protecting
Satellite ACDISC Data Pool Health
AirBase Science Team
AIRNow
AQS ABC Atmosph.
Model AC&C Science
DLR AeroCOM
GISC AQAST
LANCE AQMEII CCI-
Aerosol Global
RSIG PEGASOS
Emission ++ others TF-HTAP Policies
Others ...
Integrating Initiatives/Organizations
…are the main clients and users of the AQ Network
6. What is/Who are the Air Quality CoP?
A self-organized, voluntary group promoting the use of
Earth observations for air quality management and science
Earth Ob- Societal
Data Faciltators Decision
servations Benefit
Hubs ABC | AQ_CoP | B.GN | CHIST |CIERA | COST |
Support
CyAir DataONE | EANET | EGIDA | ESIP |
CIERA EuroGEOSS | INSPIRE | MaleDecl | QA4EO AIRNow
BlueSky
Monitorig DataFed CIAM
EBAS Informing
Network INSPIRE
the Public
MACC PEGASOS
SDS-WAS
VIEWS VIEWS …
.... Others
Protecting
Satellite
ACDISC
Data Pool Science
Health
AirBase Teams
AIRNow ABC Atmosph.
Model AQS AC&C Science
AeroCOM
DLR AQAST
GISC AQMEII CCI-
Aerosol Global
LANCE PEGASOS
Emission RSIG TF-HTAP Policies
+++ others Others ...
Members of AQ CoP are from the the Motto: Other Integrating Initiatives
AQ CoP Motto: Connecting and Enabling Other Integrating Initiatives
7. Workshop Goals (Martin!)
•Get to know each other
•Identify core AQ applications and sharable data sets
•Exchange state-of-the-art, interoperability practices
•Address technical and programmatic issues
•Have fun
8. Preliminary Agenda
•Wednesday, 25 Aug: State of AQ Interoperability
– Participant presentations
– Integrating Initiatives, Agencies, ‘End Users’, their needs, roles
– GEO AQ CoP role, relationships, contributions
•Thursday, 26 Aug: Technical Realization of Interoperability
– Syntactic issues (Improved standards? WCS, WMS, CSW…; netcdf, other formats)
– Semantic issues (Metadata for finding and understanding data? CF? )
– Programmatic issues (Which data can be shared by whom? Data versions, etc.?)
•Friday, 27 Aug: Outcomes. Consensus? Next?
– What few things must be the SAME, so that everything else can be DIFFERENT?
– How can we collaborate? Interact with other Initiatives, Agencies, Users?
– Do we have common long-term goals? Next steps?
10. The era of a single data provider, the individual scientist or the lone
decision supporter is over.
There is a new pattern emerging in the maturing information ecosystem
Data Portals aggregators of many data streams
Data Portals AQ Science Teams AQ Decision Support Systems
Data Aggregators for Science Collectives and Data Driven Analysis and
Easier Access Collaboratives Prognosis
VIEWS - Visibility Information Exchange Web System TF HTAP Task Force, Hemispheric Transport of AirVIEWS - Visibility Information Exchange Web System
Pollutants
EPA RSIG - Remote Sensing Information Gateway
NASA AQAST - Quality Applied Science Team TF HTAP Task Force, Hemispheric Transport of Air Pollutan
EPA AQS - Air Quality System GMES MACC: Monitoring Atmospheric Composition and Cli
NASA GIOVANNI - Data & Information Services Center
ESA CCI Aerosol: Climate Change Initiative for AerosolsExceptional Air Pollution Event Analysis
EPA -
EBAS - Norwegian Institute of Air Research GMES MACC: Monitoring Atmospheric Composition and Climate Data Systems for Air Quality Data
DataFed - Federated
CEOS ACP: Atmospheric Composition Portal EPA - Exceptional Air Pollution Event Analysis
CIERA: Comm. Initiative for Emissions Res. & Apps. AeroCom - Aerosol Model Inter-comparison Project
GMES MACC: Monitoring Atm. Composition and Climate
This is a good thin for the Earth System & GEOSS since it does not
have to support individuals but can fucus on fewer groupings
Implicit structure Cascading multi-stage process al la the economy-of-
scale
11. From Observations to Decisions
Spectrum of User Communities
Earth Obs. EO Service Discipline EnvHealth EnvPolicy Public &
& Modeler Provider Scientist Model, Res & Manager Officials
Spectrum of Activities and Approach
Measuring InfoProc. & SciKnow Decision Decision Assessing
& Modeling Distributi Creation Support Making Benefit
Science Dec. Sup.
Data Teams Systems
Portals
Felicitators of Collaboration
GEOSS AIP Architecture Implementation Pilot | OGC Network
EPA Data Summit
GO-ESSP: Global Org. for Earth System Science Portals
EU EGIDA - Coord. Cross-Disciplinary Projects to Promote GEOSS
EPA CyAir: Cyberinfratructure for Air Quality Managment
CIERA: Community Initiative for Emissions Research and Applications
GEO AQ CoP: GEO Air Quality Community of Practice
COST ES0602: EU Network on Chemical Weather Forecasting..
AeroCom - Aerosol Model Inter-comparison Project
ESIP Earth Science Information Partners, AQ Work Group
12. Value Adding Groups
Felicitators, Connectors
GEOSS AIP Architecture Implementation Pilot | OGC Network
EPA Data Summit
GO-ESSP: Global Org. for Earth System Science Portals
EU EGIDA - Coord. Cross-Disciplinary Projects to Promote GEOSS
EPA CyAir: Cyberinfratructure for Air Quality Managment
CIERA: Community Initiative for Emissions Research and Applications
GEO AQ CoP: GEO Air Quality Community of Practice
COST ES0602: EU Network on Chemical Weather Forecasting..
AeroCom - Aerosol Model Inter-comparison Project
ESIP Earth Science Information Partners, AQ Work Group
GEO Air Quality
Data Portals
VIEWS - Visibility Information Exchange Web System Community of Practice AQ Decision Support Systems
VIEWS - Visibility Information Exchange Web System
EPA RSIG - Remote Sensing Information Gateway
TF HTAP Task Force, Hemispheric Transport of Air Pollutants
EPA AQS - Air Quality System GMES MACC: Monitoring Atmospheric Composition and Clima
Main Value Adding Activity:
NASA GIOVANNI - Data & Information Services Center EPA - Exceptional Air Pollution Event Analysis
EBAS - Norwegian Institute of Air Research DataFed - Federated Data Systems for Air Quality Data
CEOS ACP: Atmospheric Composition Portal
CIERA: Comm. Initiative for Emissions Res. & Apps. Interoperable Network
GMES MACC: Monitoring Atm. Composition and Climate
AQ Science Teams
TF HTAP Task Force, Hemispheric Transport of Air Pollutants
NASA AQAST - Quality Applied Science Team
ESA CCI Aerosol: Climate Change Initiative for Aerosols
GMES MACC: Monitoring Atmospheric Composition and Climate
EPA - Exceptional Air Pollution Event Analysis
AeroCom - Aerosol Model Inter-comparison Project
13. • The GEO AQ CoP is a self-organized voluntary group that fosters
the use of Earth observations to air quality management and science
by leveraging synergies of sharing and collaboration.
• Community activities include collaborative website; workshops;
gathering user requirements; sharing tools and best practices as well
as aiding other communities: data portals, science teams and
decision support activities.
• A persistent output facilitated by the AQ CoP is a network of air
quality/atmospheric composition data servers that use OGC-
standard protocols and the GEOSS interoperability process.
• Searching, browsing and accessing data from monitoring networks,
satellite sensors, emissions, as well as from AQ models is through a
Community Catalog.
14. • The creation of an information system for Earth Sciences
has been a grand challenge for decades
• It has been pursued by innumerable workshops, vision
statements, pilot projects etc…
• There were many tries Progress has been slow …
• “The road toward integrated Earth Science info systems
is littered with wrecks of well-intended attempts..”
•
15. • We know that info systems like libraries and the internet
provide useful services
• The info technologies and the engineering design of these
systems is understood.
• However, the science, the explanations of how these info
systems work is not in hand. .
– What causes information to be generated, transmitted and
‘consumed’?
– What are the driving forces to the flow of information? Provider
push, consumer pull? Impediments.. …Can you push a rope?
– Are there suitable ‘laws of nature’, equivalent to conservation of
mass and energy?
16. What will be discussed?
The state of ADN including key impediments; co-development of server
software and tools; sharing best practices
Workshop Focus:
Technical Realization of AQ Network.
Selected Server Topics Selected Network Topics
Interoperability Standards AQ Network Characteristics
Server for Grid & Point Data Publish, Find, Access
Server Maintenance-Support Catalog & Metadata Issues
Server Performance ADN Operation, Governance
“What few things must be the SAME,
so that everything else can be DIFFERENT?”
17. The GEOSS goal:
To create more societal benefit from Earth Observations and Models and
since effective value creation hinges on seamless data flow and re-use
The AQ CoP chose to focus is on facilitating the interoperable data
systems, i.e. and quality data network.
Show Nework of seven servers
Somber note
19. From Observations to Decisions
Spectrum of User Communities and their Activites
Earth Obs. & EO Service Discipline EnvHealth EnvPolicy & Public &
Modeler Provider Scientist Model, Res Manager Officials
Measuring& InfoProc. & SciKnow Decision Decision Assessing
Modeling Distribution Creation Support Making Benefit
Data Portals AQ Science Teams
VIEWS - Visibility Information Exchange Web System
TF HTAP Task Force, Hemispheric Transport of Air Pollutants
AQ Decision Support Sys. VI
EPA RSIG - Remote Sensing Information Gateway EWS - Visibility Information Exchange Web System
EPA AQS - Air Quality System NASA AQAST - Quality Applied Science Team TF HTAP Task Force, Hemispheric Transport of Air Pollutants
NASA GIOVANNI - Data & Information Services Center ESA CCI Aerosol: Climate Change Initiative for Aerosols MACC: Monitoring Atmospheric Composition and Climate
GMES
EBAS - Norwegian Institute of Air Research EPA - Exceptional Air Pollution Event Analysis
GMES MACC: Monitoring Atmospheric Composition and Climate
DataFed - Federated Data Systems for Air Quality Data
CEOS ACP: Atmospheric Composition Portal
CIERA: Comm. Initiative for Emissions Res. & Apps. EPA - Exceptional Air Pollution Event Analysis
GMES MACC: Monitoring Atm. Composition and Climate AeroCom - Aerosol Model Inter-comparison
Project
20. The of GEOSS (and most other EO integrating initiatives) is create more
societal benefit from Earth Observations and Models
Effective creation of societal benefit hinges on seamless data flow and re-
use, so a universal value is the ability the share and re-use data.
Hence AQ CoP chose to focus on facilitating Air Quality Data Network.
We also understand that the creation of an Earth Science information
system has been a grand challenge for decades….
…and that the “The road toward integrated Earth Science info systems is
littered with wrecks of well-intended attempts..”
So, challenge is how to make more progress this time..
21. Spectrum of User Communities and their Activities
Earth Obs. & EO Service Discipline EnvHealth EnvPolicy & Public &
Modeler Provider Scientist Model, Res Manager Officials
Measuring& InfoProc. & SciKnow Decision Decision Assessing
Modeling Distribution Creation Support Making Benefit
Earth Ob-
servations Societal
Benefit
Monitorig
Network Informing
the Public
Satellite Protecting
Health
Atmosph.
Model Science
Global
Emission Policies
22. Networking Air Quality Observations and Models:
From Virtual to Real
Practitioners from Europe and North America will assess the current state of the network,
address key interoperability challenges, share best practices. and work on turning the
network from virtual to real.
Agenda, August 23, 24, 25 Participants
Tue. AM: Reports of Participants, Net.-Nodes ~30 Participants
PM: State of AQ Data Network, Catalog ~20 Organizations
Wed. AM: Technical Realization of AQ Network CNR, DHZ, DLR, EEA, EnvCanada,
PM: Technical Realization of AQ Network USEPA, FZ Juelich, GEO, JRC,
Thu. AM: Technical Realization of AQ Network MetNO, NASA, NGC, NILU, NOAA,
PM: Outcomes, Next? NPS, RSIG, STI, UNIDATA, WUStL
Aug 23-25, 2011, Stomorska, Šolta, Croatia
Logistics
WS Intro slides
Workshop Schedule
Intro of Individual
See Workshop Wiki for Details, Contacts: Rudolf Husar, rhusar@wustl.edu; Martin Schultz, m.schultz@fz-juelich.de
23. Workshop Focus: Technical Realization of AQ Network.
AQ Data Network Status Selected Server Topics AQ Network Future?
Selection and use of Standards
WCS version 1.0,
1.1.2, 2.0?
Combining WMS, WFS,
WCS?
AQ Data Network Pilot Server for Different Data Types
Grid data (model, emiss., sat.)
7 Active AQ WCS Servers Point-Station (surf. Netw.)
7 Near-Term New WCS Nodes Other data types?
Server Maintenance-Support
SourceForge, Docum. Guids
Server code governance
Server Performance
Remote access or cache
Streaming vs file transfer
AQ Data Network Selected Network
Topics
Catalog - Prototype AQ Network Design Issues
Autonomy-interop. balance
Network Catalog(s)
AQ Community Catalog
Domain/Application Catalog(s)
Network Metadata Issues
Discovery metadata for AQ
Provenance, quality, security
Network Operation, Maintenance
Governance,
See Workshop Wiki for Details, Contacts: Rudolf Husar, rhusar@wustl.edu; Martin Schultz, m.schultz@fz-juelich.de
Legitimacy
Editor's Notes
The current Pilot AQ Network consists of about about half a dozen data hubs offer part of their data holdings through OGC-standard WCS web services. Interoperability is among these servers is assured by a jointly developed, open-source server software that is installed at each data hub. The offered data are published and can be found through the prototype Community catalog and then accessed by OGC-compliant client software, regardless where the data are located. Together, the distributed servers, the data catalog and the clients are the three components of the service-oriented, loosely coupled AQ Network. The AQ data network Pilot resulted from the close collaboration of a few groups and it is still small, fragile and limited in many ways. The server implementation required to make numerous arbitrary decisions. The technical realization of an extended, inclusive and robust network will be the focus of the broader community represented at the workshop. The server-level topics include Selection and use of standards, conventions and community arrangements. Server issues for gridded data for satellite, emissions and model data Server issues for station-point monitoring data Server maintenance and software support Server performance issue and prectices Network level topics include
The current Pilot AQ Network consists of about about half a dozen data hubs offer part of their data holdings through OGC-standard WCS web services. Interoperability is among these servers is assured by a jointly developed, open-source server software that is installed at each data hub. The offered data are published and can be found through the prototype Community catalog and then accessed by OGC-compliant client software, regardless where the data are located. Together, the distributed servers, the data catalog and the clients are the three components of the service-oriented, loosely coupled AQ Network. The AQ data network Pilot resulted from the close collaboration of a few groups and it is still small, fragile and limited in many ways. The server implementation required to make numerous arbitrary decisions. The technical realization of an extended, inclusive and robust network will be the focus of the broader community represented at the workshop. The server-level topics include Selection and use of standards, conventions and community arrangements. Server issues for gridded data for satellite, emissions and model data Server issues for station-point monitoring data Server maintenance and software support Server performance issue and prectices Network level topics include
The vision is to have these data available through an interoperability framework that allows them to be used via various subsets and combinations to support specific research and decision applications There are numerous Earth Observations that are available and in principle useful for air quality applications such as informing the public and enforcing AQ standards. However, connecting a user to the right observations or models is accompanied by an array of hurdles. The GEOSS Common Infrastructure allows the reuse of observations and models for multiple purposes
The current Pilot AQ Network consists of about about half a dozen data hubs offer part of their data holdings through OGC-standard WCS web services. Interoperability is among these servers is assured by a jointly developed, open-source server software that is installed at each data hub. The offered data are published and can be found through the prototype Community catalog and then accessed by OGC-compliant client software, regardless where the data are located. Together, the distributed servers, the data catalog and the clients are the three components of the service-oriented, loosely coupled AQ Network. The AQ data network Pilot resulted from the close collaboration of a few groups and it is still small, fragile and limited in many ways. The server implementation required to make numerous arbitrary decisions. The technical realization of an extended, inclusive and robust network will be the focus of the broader community represented at the workshop. The server-level topics include Selection and use of standards, conventions and community arrangements. Server issues for gridded data for satellite, emissions and model data Server issues for station-point monitoring data Server maintenance and software support Server performance issue and prectices Network level topics include
The current Pilot AQ Network consists of about about half a dozen data hubs offer part of their data holdings through OGC-standard WCS web services. Interoperability is among these servers is assured by a jointly developed, open-source server software that is installed at each data hub. The offered data are published and can be found through the prototype Community catalog and then accessed by OGC-compliant client software, regardless where the data are located. Together, the distributed servers, the data catalog and the clients are the three components of the service-oriented, loosely coupled AQ Network. The AQ data network Pilot resulted from the close collaboration of a few groups and it is still small, fragile and limited in many ways. The server implementation required to make numerous arbitrary decisions. The technical realization of an extended, inclusive and robust network will be the focus of the broader community represented at the workshop. The server-level topics include Selection and use of standards, conventions and community arrangements. Server issues for gridded data for satellite, emissions and model data Server issues for station-point monitoring data Server maintenance and software support Server performance issue and prectices Network level topics include
At the end of August 2011, the GEO Air Quality Community of Practice is organizing a small topical workshop on the Networking of Air Quality Observations and Models. Data system practitioners from Europe and North America will address interoperability challenges, share best practices and seek ways to expand the network. The central theme of the workshop will be on the implementation of interoperable data server nodes, so the majority of the three-day agenda will be devoted to the technical aspects of server design and the realization of the interoperable AQ Data Network. The majority of the 30 or so workshop participants are data system developers and managers. They represent about 20 organizations including data hubs that provide access to data as well as science teams and decision support systems that are users of observations, emissions and model data. The workshop venue is located in Stomorska, adjacent to Split, Croaotia.
The current Pilot AQ Network consists of about about half a dozen data hubs offer part of their data holdings through OGC-standard WCS web services. Interoperability is among these servers is assured by a jointly developed, open-source server software that is installed at each data hub. The offered data are published and can be found through the prototype Community catalog and then accessed by OGC-compliant client software, regardless where the data are located. Together, the distributed servers, the data catalog and the clients are the three components of the service-oriented, loosely coupled AQ Network. The AQ data network Pilot resulted from the close collaboration of a few groups and it is still small, fragile and limited in many ways. The server implementation required to make numerous arbitrary decisions. The technical realization of an extended, inclusive and robust network will be the focus of the broader community represented at the workshop. The server-level topics include Selection and use of standards, conventions and community arrangements. Server issues for gridded data for satellite, emissions and model data Server issues for station-point monitoring data Server maintenance and software support Server performance issue and prectices Network level topics include
