Ellsworth LeDrew (UIC) Gary Foley (UIC) Jay Pearlman (UIC and ADC) Hans-Peter Plag (UIC and ADC) Rudolf Husar (UIC and ADC) David McCabe (ADC)
Concept (LeDrew) Past AQ at User and the GEOSS Architecture Workshop Series: ESIP_Jan06: First public web service chaining demo  IGARSS06 Denver: Demo of web service chaining; GMU, JPL, WashU IGARSS07 Barcelona: Sahara Dust; WashU with 12 NAM & EU data partners Outcome: Successful Tech Demos but ‘SoS’ constructs fragile & uncontained
Application of OGC WCS Data Access Protocol Regardless of the data location, data type and format,  the  parameter-space-time  query is the same the return is in  user selectable format  from the offerings Coverage=THEEDDS.T&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF Coverage=SEAW.Refl&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=GeoTIFF Coverage=SURF.Bext&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF-table Grid Image Station Data Parameter Bounding Box Time Range Out Format
Current SOA Air Quality Applications:  Analysis Consoles
Emerging Air Quality Data Flow Network OGC WCS Data Access Protocol GEOSS Provides SOA for Coupling for Autonomous Nodes  Facilitates Publishing, Finding and Accessing Data
Concept (con’t) Each  role   identifies what data is required at that level Determines how data should be packaged by GEO Determines what is the customer base, what is the  ‘chain of information’ Includes data type, format, unit conversion, time resolution, quality control process, required fields, optional fields (Hans-Peter Plag:  GEOSS User Needs and Performance Utility: Functional Spec and Database Structure) Value-Adding Processes SBA Info Flow
Stages of AQ Data Flow and Value-Adding Processes  Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System Analyzing Filter/Integrate Aggregate/Fuse Custom Analysis Organizing Document Structure/Format Interfacing Characterizing Display/Browse Compare/Fuse  Characterize Value-Adding Processes Reporting Inclusiveness Iterative/Agile  Dynamic Report Data Control Requesting Information Providing Information Sensors Acquisition processing User Programs NAAQS  SIPs  Forecast GEOSS  … Info System Negotiating Space User Agencies
Generic Decision Support for Air Quality Decisions Global Earth Observing System of Systems  GEOSS Architecture Framework Knowledge  into the Minds of Regulatory  Analysts Knowledge  into  the Minds of  Technical  Analysts Observations Reports: Model Forecasts,  Obs. Evidence Models Decisions Knowledge  into the Minds of  Decision- making  managers Decision Support System
GEOSS Framework Policy Dec. Management Personal Dec Policy Dec. Management Personal Dec Policy Dec. Management Personal Dec
GEOSS Framework User Application Erquirements
System of Systems Global Earth  Observing System of Systems - GEOSS Characteristics of System of Systems (SoS) Autonomous constituents managed/operated independently Independent evolution of each constituent SoS displays emergent behavior Must recognize, manage, exploit the characteristics: No stakeholder has complete SoS insight Central control is limited; distributed control is essential Users, must be involved throughout the life of a SoS
System of Systems Issues: From  User  Engagement to  Stakeholder  Engagement?   From: Air Fore SoS Document
The Scheffe Challenge:  Organizations - Programs – Data:  A Mess Info System Challenges: What’s the overall dependency Information Flow Forces and Controls on Data Flow Cooperation, Competition, Co-Opetition GEOSS Eco-informatics Accountability/ indicators SIPs, nat.rules designations PHASE PM research Risk/exposure assessments AQ forecasting Programs NAAQS setting EPA NOAA NASA NPS USDA DOE Private Sector States/Tribes RPO’s/Interstate Academia NARSTO NAS, CAAAC CASAC, OMB Enviros Organizations CDC Supersites IMPROVE, NCore PM monit, PAMS CASTNET Lidar systems NADP Satellite data Intensive studies PM centers Other networks: SEARCH, IADN.. Data sources CMAQ GEOS-CHEM Emissions Meteorology Health/mort. records
Relationship Between Organizations - Programs – Data Version 0.1 Goals $$ Info needs, $$ Data need, $$ Judge, Decide, Act Analyze, Report Actionable Knowledge Decision, Action Public  Measure, Organize Organized Data Flow of Information Data systems organize the measurements and models and provide them to programs.  Programs analyze the data and provide actionable knowledge to organizations. Organizations evaluate multiple information sources, make decisions and act.  Flow of Control Public and special interest groups  set up organizations and provides them with funding  Organizations develop programs, define their scope, governance and funding Programs satisfy their information needs by monitoring or by using other’s data Data sources acquire the data for their parent programs and also expose them for reuse
GEOSS Research: System of Systems Science Network Science Informatics Science Science and understanding is poor in each area GEOSS requires convergence of these sciences  Leave science and convergence to serendipity?
 
Concept (con’t) Team works through possible decision process to determine whether, if GEO provided the information, the system will work as required. Product will be entries in tables and a narrative of processes A product would be a list of best practices, one or more of which will be entered into the GEO Best Practice Wiki (wiki.ieee-earth.org) .
GEO Best Practice Wiki (wiki.ieee-earth.org) To be modified by the group at the workshop? Practice Example Structure by Template
The Information Interoperability Stack Web 2.0
EE Analysis Community Workspace
Community Data Sharing - ‘DataSpaces’ Catalog - Find Dataset Describe Dataset Discuss Dataset Earth Science Information Partners (ESIP) Semantic Wiki: Structured (RDF and Unstructured Content Open, Standard Matadata - RDF Ready for Export/Harvesting  by Registries, Catalogs
Communal Event Analysis  Southern California Fire Smoke Approach (Blogs, Flickr, YOuTube): Use app to perform coarse filtering  Controlled tagging in del.icio.us RSS feed from del.icio.us
Community Workspaces The Web is being transformed: It is becoming more participatory See the explosive growth of wikies, picture-sharing, blogs, Facebook  This architectural, technological and cultural change is  Web 2.0 Web 2.O is good for Earth Science community since it allows Better harvesting of current knowledge However, many cultural, legal and other berries remain  Workspaces may connect info providers, analysts, decision makers The challenge is to learn how to apply these tools GEOSS
Concept (LeDrew) Architecture Data Committee/User Engagement Committee Interaction in the GEOSS Architecture Implementation – User Engagement Pilot
AIP Process improvement ideas Comments on AIP Process Process Improvement Functionality defined by developers not users UIC/ADC collab., Workshops;  Operational SBA functionality  AIP Process is closed Make development iterative; Anyone can join at anytime Process focused on demonstrations More operational persistence; Deliver AR-07-01 Operations Too structured: milestones, deliverables Retain structure within phase; More iterative, open approach Adopted from  Percivall , Feb 2008  by R. Husar Contact with UIC; Toronto, Workshop; Open Invitations, ESIP, CoP wiki Expand AQ Data Netw; Seek link to AR-07-01 One phase at a time; Feedback, feedback Air Quality CoP Approach
Core GEOSS Architecture GEOSS Component,  Service registry Standards, Special  Arrangements Registries references GEO Web Portal  and other Clients searches Offerors contribute to register GEOSS Clearinghouse User   accesses list of catalogues accesses searches invokes reference From  Percivall , Feb 2008 Catalogues Services
GEOSS Architecture  for the Air Quality Community (March 2008) GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests bind, invoke references publish provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Catalog User find,  select links to GEOSS Community Service Offerors, Users Services
GEOSS Architecture  for the Air Quality Community (March 2008) GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find GEOSS Community Service Offerors, Users Services
GEOSS Architecture  for the Air Quality Community (March 2008) GEOSS Comp. Registry Community AQ Portal(s) extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog(s) Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find GEOSS Community Service Offerors, Users, Community of Practice links to Services Community Of Practice
Questions on Global Viewpoint Right Level of Networking? Public Information Health/Exp. Assessment Emissions Planning AQ Trends and Accountability Science Support NAAQS National Air Quality Information Integration AQ Data Pool Re-examination of Data Access and processing Systems Product Value (Chain) Global Regional National Theme Adopted from FedEO Local
Collaboration, Are We Ready? Given ?

2008-05-05 GEOSS UIC-ADC AQ Scen W shop Toronto

  • 1.
    Ellsworth LeDrew (UIC)Gary Foley (UIC) Jay Pearlman (UIC and ADC) Hans-Peter Plag (UIC and ADC) Rudolf Husar (UIC and ADC) David McCabe (ADC)
  • 2.
    Concept (LeDrew) PastAQ at User and the GEOSS Architecture Workshop Series: ESIP_Jan06: First public web service chaining demo IGARSS06 Denver: Demo of web service chaining; GMU, JPL, WashU IGARSS07 Barcelona: Sahara Dust; WashU with 12 NAM & EU data partners Outcome: Successful Tech Demos but ‘SoS’ constructs fragile & uncontained
  • 3.
    Application of OGCWCS Data Access Protocol Regardless of the data location, data type and format, the parameter-space-time query is the same the return is in user selectable format from the offerings Coverage=THEEDDS.T& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF Coverage=SEAW.Refl& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=GeoTIFF Coverage=SURF.Bext& BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF-table Grid Image Station Data Parameter Bounding Box Time Range Out Format
  • 4.
    Current SOA AirQuality Applications: Analysis Consoles
  • 5.
    Emerging Air QualityData Flow Network OGC WCS Data Access Protocol GEOSS Provides SOA for Coupling for Autonomous Nodes Facilitates Publishing, Finding and Accessing Data
  • 6.
    Concept (con’t) Each role identifies what data is required at that level Determines how data should be packaged by GEO Determines what is the customer base, what is the ‘chain of information’ Includes data type, format, unit conversion, time resolution, quality control process, required fields, optional fields (Hans-Peter Plag: GEOSS User Needs and Performance Utility: Functional Spec and Database Structure) Value-Adding Processes SBA Info Flow
  • 7.
    Stages of AQData Flow and Value-Adding Processes Domain Processing Data Sharing Std. Interface Gen. Processing Std. Interface Data Control Reports Reporting Obs. & Models Decision Support System Analyzing Filter/Integrate Aggregate/Fuse Custom Analysis Organizing Document Structure/Format Interfacing Characterizing Display/Browse Compare/Fuse Characterize Value-Adding Processes Reporting Inclusiveness Iterative/Agile Dynamic Report Data Control Requesting Information Providing Information Sensors Acquisition processing User Programs NAAQS SIPs Forecast GEOSS … Info System Negotiating Space User Agencies
  • 8.
    Generic Decision Supportfor Air Quality Decisions Global Earth Observing System of Systems GEOSS Architecture Framework Knowledge into the Minds of Regulatory Analysts Knowledge into the Minds of Technical Analysts Observations Reports: Model Forecasts, Obs. Evidence Models Decisions Knowledge into the Minds of Decision- making managers Decision Support System
  • 9.
    GEOSS Framework PolicyDec. Management Personal Dec Policy Dec. Management Personal Dec Policy Dec. Management Personal Dec
  • 10.
    GEOSS Framework UserApplication Erquirements
  • 11.
    System of SystemsGlobal Earth Observing System of Systems - GEOSS Characteristics of System of Systems (SoS) Autonomous constituents managed/operated independently Independent evolution of each constituent SoS displays emergent behavior Must recognize, manage, exploit the characteristics: No stakeholder has complete SoS insight Central control is limited; distributed control is essential Users, must be involved throughout the life of a SoS
  • 12.
    System of SystemsIssues: From User Engagement to Stakeholder Engagement? From: Air Fore SoS Document
  • 13.
    The Scheffe Challenge: Organizations - Programs – Data: A Mess Info System Challenges: What’s the overall dependency Information Flow Forces and Controls on Data Flow Cooperation, Competition, Co-Opetition GEOSS Eco-informatics Accountability/ indicators SIPs, nat.rules designations PHASE PM research Risk/exposure assessments AQ forecasting Programs NAAQS setting EPA NOAA NASA NPS USDA DOE Private Sector States/Tribes RPO’s/Interstate Academia NARSTO NAS, CAAAC CASAC, OMB Enviros Organizations CDC Supersites IMPROVE, NCore PM monit, PAMS CASTNET Lidar systems NADP Satellite data Intensive studies PM centers Other networks: SEARCH, IADN.. Data sources CMAQ GEOS-CHEM Emissions Meteorology Health/mort. records
  • 14.
    Relationship Between Organizations- Programs – Data Version 0.1 Goals $$ Info needs, $$ Data need, $$ Judge, Decide, Act Analyze, Report Actionable Knowledge Decision, Action Public Measure, Organize Organized Data Flow of Information Data systems organize the measurements and models and provide them to programs. Programs analyze the data and provide actionable knowledge to organizations. Organizations evaluate multiple information sources, make decisions and act. Flow of Control Public and special interest groups set up organizations and provides them with funding Organizations develop programs, define their scope, governance and funding Programs satisfy their information needs by monitoring or by using other’s data Data sources acquire the data for their parent programs and also expose them for reuse
  • 15.
    GEOSS Research: Systemof Systems Science Network Science Informatics Science Science and understanding is poor in each area GEOSS requires convergence of these sciences Leave science and convergence to serendipity?
  • 16.
  • 17.
    Concept (con’t) Teamworks through possible decision process to determine whether, if GEO provided the information, the system will work as required. Product will be entries in tables and a narrative of processes A product would be a list of best practices, one or more of which will be entered into the GEO Best Practice Wiki (wiki.ieee-earth.org) .
  • 18.
    GEO Best PracticeWiki (wiki.ieee-earth.org) To be modified by the group at the workshop? Practice Example Structure by Template
  • 19.
  • 20.
  • 21.
    Community Data Sharing- ‘DataSpaces’ Catalog - Find Dataset Describe Dataset Discuss Dataset Earth Science Information Partners (ESIP) Semantic Wiki: Structured (RDF and Unstructured Content Open, Standard Matadata - RDF Ready for Export/Harvesting by Registries, Catalogs
  • 22.
    Communal Event Analysis Southern California Fire Smoke Approach (Blogs, Flickr, YOuTube): Use app to perform coarse filtering Controlled tagging in del.icio.us RSS feed from del.icio.us
  • 23.
    Community Workspaces TheWeb is being transformed: It is becoming more participatory See the explosive growth of wikies, picture-sharing, blogs, Facebook This architectural, technological and cultural change is Web 2.0 Web 2.O is good for Earth Science community since it allows Better harvesting of current knowledge However, many cultural, legal and other berries remain Workspaces may connect info providers, analysts, decision makers The challenge is to learn how to apply these tools GEOSS
  • 24.
    Concept (LeDrew) ArchitectureData Committee/User Engagement Committee Interaction in the GEOSS Architecture Implementation – User Engagement Pilot
  • 25.
    AIP Process improvementideas Comments on AIP Process Process Improvement Functionality defined by developers not users UIC/ADC collab., Workshops; Operational SBA functionality AIP Process is closed Make development iterative; Anyone can join at anytime Process focused on demonstrations More operational persistence; Deliver AR-07-01 Operations Too structured: milestones, deliverables Retain structure within phase; More iterative, open approach Adopted from Percivall , Feb 2008 by R. Husar Contact with UIC; Toronto, Workshop; Open Invitations, ESIP, CoP wiki Expand AQ Data Netw; Seek link to AR-07-01 One phase at a time; Feedback, feedback Air Quality CoP Approach
  • 26.
    Core GEOSS ArchitectureGEOSS Component, Service registry Standards, Special Arrangements Registries references GEO Web Portal and other Clients searches Offerors contribute to register GEOSS Clearinghouse User accesses list of catalogues accesses searches invokes reference From Percivall , Feb 2008 Catalogues Services
  • 27.
    GEOSS Architecture for the Air Quality Community (March 2008) GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests bind, invoke references publish provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Catalog User find, select links to GEOSS Community Service Offerors, Users Services
  • 28.
    GEOSS Architecture for the Air Quality Community (March 2008) GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find GEOSS Community Service Offerors, Users Services
  • 29.
    GEOSS Architecture for the Air Quality Community (March 2008) GEOSS Comp. Registry Community AQ Portal(s) extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog(s) Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find GEOSS Community Service Offerors, Users, Community of Practice links to Services Community Of Practice
  • 30.
    Questions on GlobalViewpoint Right Level of Networking? Public Information Health/Exp. Assessment Emissions Planning AQ Trends and Accountability Science Support NAAQS National Air Quality Information Integration AQ Data Pool Re-examination of Data Access and processing Systems Product Value (Chain) Global Regional National Theme Adopted from FedEO Local
  • 31.
    Collaboration, Are WeReady? Given ?