Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems

Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems Janet J. redericks F Applied Ocean Physics & Engineering Woods Hole Oceanographic Ins=tu=on Carlos Rueda Monterey Bay Aquarium Research Ins=tute Workshop on Sensor Web Enablement 2011 (SWE 2011) As part of The 2011 Cybera Summit on Data For All -‐ Opening up the Cloud October 6-‐7, 2011, Banﬀ, AB, Canada 1

Data Provider NOAA/NDBC provides 24/7 QC; Nightmare! Feeds National IOOS backbone; NOAA/NODC provides national archival for valued data sets (they can determine the value) NSF/OOI; NSF/R2R; NSF/BCODMO Sensor Manufacturers provides community-‐based integration <html/> and manuals with tools and QC, along with discovery and mapping opportunities Real-‐time Rapid Response integration can be accomplished quickly and reliably by communicating metadata Research and survey in standards-‐based systems data served with associated metadata in a few speci5ic Modeling formats with using translation tools from the cloud, associated software modelers have access to a broader installations source of information ANYONE By fully describing data, sensors and processing with associated provenance, data can be discovered and explored for any program User-‐based Output 2

Data Provider IOOS (and Consumer) Nightmare! GEOSS Sensor Manufacturers NOAA/NDBC <html/> and manuals provides 24/7 QC; Feeds National IOOS backbone; Research and survey data served with associated metadata in a few speci5ic formats with Research and survey Research and survey associated software data served with data served with installations associated metadata associated metadata in a few speci5ic in a few speci5ic formats with formats with associated software associated software installations installations User-‐based Output 3

Data Provider NOAA/NDBC (and Consumer) provides 24/7 QC; Feeds National IOOS backbone; Nightmare! NOAA/NODC provides national archival for valued data sets (they can determine the value) NSF/OOI; NSF/R2R; NSF/BCODMO Sensor Manufacturers provides community-‐based integration <html/> and manuals with tools and QC, along with discovery and mapping opportunities Real-‐time Rapid Response integration can be accomplished quickly and reliably by communicating metadata Research and survey in standards-‐based systems data served with associated metadata in a few speci5ic Modeling formats with using translation tools from the cloud, associated software modelers have access to a broader installations source of information ANYONE By fully describing data, sensors and processing with associated provenance, data can be discovered and explored for any program User-‐based Output 4

GOAL: two paths Described well enough for assessment of NOAA/NDBC data for speciﬁed use and for a provides 24/7 QC; repurposed applica<on Feeds National IOOS backbone; NOAA/NODC provides national archival for valued Sensor Manufacturers data sets (they can determine the value) and domain experts develop sensor and NSF/OOI; NSF/R2R; NSF/BCODMO processing provides community-‐based integration descriptions in with tools and QC, along with discovery standards-‐based Converters; and mapping opportunities encodings QC algorithms; vocabularies & Real-‐time Rapid Response ontologies; integration can be accomplished quickly analysis and and reliably by communicating metadata visualization tools in standards-‐based systems Research and survey data served with associated metadata Modeling in a community-‐ using translation tools from the cloud, adopted, standards-‐ modelers have access to a broader based framework source of information ANYONE Standards-‐based By fully describing data, sensors and processing with associated provenance, (machine-‐to-‐machine harves=ng) data can be discovered and explored for any program User-‐based Frameworks 5

Data Provider needs to communicate how the sensible proper=es were turned into observa=ons! Logging/ Web Sensor Processing Service 6

Project to Address Data Quality in Sensor Web Enablement Frameworks BACKGROUND Quality Assurance Guides/Implementa=on (QARTOD) Seman=c Tools (MMI) Standards (OGC) (OOStethys/OGC-‐OIE/OpenIOOS) Vocabulary Registry & Term Syntactac=c Interoperabilty So_ware Packages/ QC Tests Mapping (SensorML/O&M) cookbooks Ontology Development & Standards-‐based web MetaData Requirements Registry services (SOS) Observa=ons Based SOS Quality – to – OGC (Q2O) -‐ IntegraKon of sensor & processing descripKons aimed towards the ability to assess quality of observaKons 7

Community-‐based Development Domain Experts Sensor Mfgrs Content Speciﬁca=ons/ SWE Implementa=on Model Operators What informaKon is needed to assess quality of data and how do we implement it into an Sensor ObservaKon Services (SOS)? IT Specialists 8

Data CL SensorML HARVESTS I DescribeSensor (SensorML) E SOS N GetObserva@on (O&M) T REFERENCES RESOLVES OWL/RDF Vocabularies/ ontologies 9

Data CL SensorML HARVESTS I DescribeSystem (SensorML) E SOS N GetObserva@on (O&M) T <sml:output name="swell"> <swe:Quan=ty deﬁni=on="hkp://mmisw.org/ont/mvco/proper=es/swell"> REFERENCES <swe:uom code="cm"/> RESOLVES </swe:Quan=ty> </sml:output> OWL/RDF Vocabularies/ ontologies 10

Data CL SensorML HARVESTS I DescribeSystem (SensorML) E SOS N GetObserva@on (O&M) T REFERENCES RESOLVES OWL/RDF Vocabularies/ ontologies 11

Building ontologies 12

Five Role-‐based Categories of SensorML Observable Proper=es SML system Sensor/Deployment Files (SensorML -‐> DescribeSensor) Configura=on/Ownership/Deployment (CONDEP)File Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Configura=on, Deployment and Descrip=on of Sensor Model Event History Details Process Files (SensorML -‐> DescribeSensor) QC Tests -‐ are classified as QC tests (QcCategory) and Processing Descrip=ons -‐ to describe how an may have associated QC flags as output observa=on is derived from sensor output Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on) 13

Five Role-‐based Categories of SensorML Observable Proper=es SML system OEM Model DescripKon Sensor/Deployment Files (SensorML -‐> DescribeSensor) Created by Original Equipment Manufacturer (OEM) File manufacturer and Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and Event History Details anyone available for Descrip=on of Sensor Model using the par<cular model – accuracy; error analysis etc specific to the model Process Files (SensorML -‐> DescribeSensor) QC Tests -‐ are classified as QC tests (QcCategory) and Processing Descrip=ons -‐ to describe how an may have associated QC flags as output observa=on is derived from sensor output Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on) 14

Five Role-‐based Categories of SensorML ConfiguraKon and Observable Proper=es Deployment File Working with OEM SML system file/Sensor Manufacturers and Marine Operator Sensor/Deployment Files (SensorML -‐> DescribeSensor) describe this instance: Original Equipment Manufacturer (OEM) File Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and contacts (operator), Model Descrip=on of Sensor Event History Details parameters (set-‐up specifica=on that can affect accuracy or relevance to repurposed Process Files (SensorML -‐> DescribeSensor) applica=on), posi=ons, QC Tests -‐ are classified as QC tests (QcCategory) and Processing Descrip=ons -‐ to describe how an may have arela=ng to ags as output events ssociated QC fl observa=on is derived from sensor output sensor health Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on) 15

Five Role-‐based Categories of SensorML Observable Proper=es SML system Sensor/Deployment Files (SensorML -‐> DescribeSensor) Configura=on/Ownership/Deployment (CONDEP)File Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Configura=on, Deployment and Descrip=on of Sensor Model Event History Details QC Tests Data manager Process Files (SensorML -‐> DescribeSensor) describes QC tests and associated flags; inputs QC Tests -‐ are classified as QC tests (QcCategory) and Processing Descrip=ons -‐ to describe how an may have associated QC flags as output to tests and observa=on is derived from sensor output parameters are specified – the parameters can be =me-‐stamped Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on) 16

Five Role-‐based Categories of SensorML Observable Proper=es SML system Sensor/Deployment Files (SensorML -‐> DescribeSensor) Configura=on/Ownership/Deployment (CONDEP)File Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Configura=on, Deployment and Descrip=on of Sensor Model Event History Details Processing DescripKons Process Files (SensorML -‐> DescribeSensor) Data managers and domain are classified as QC tests (QcCategory) and QC Tests -‐ experts provide Processing Descrip=ons -‐ to describe how an may have associated QC flags as output observa=on is derived from sensor output authorita<ve reference and descrip<ons of processing used for derived proper=es Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on) 17

Five Role-‐based Categories of SensorML Observable Proper=es SML system Sensor/Deployment Files (SensorML -‐> DescribeSensor) Configura=on/Ownership/Deployment (CONDEP)File Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Configura=on, Deployment and Descrip=on of Sensor Model Event History Details Process Files (SensorML -‐> DescribeSensor) QC Tests -‐ are classified as QC tests (QcCategory) and Processing Descrip=ons -‐ to describe how an may have associated QC flags as output observa=on is derived from sensor output Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on) 18

How does this model enable dynamic quality assessment? 1)  ROLES -‐ Provides a template for instrument manufacturers/data managers/ marine operators to describe details that describe quality related informa=on in a standards-‐based encoding 2)  CONNECTIONS -‐ Through the connec=ons list in SensorML, the QC ﬂags can be associated with the QC tests with associated parameters 3)  ENABLING SEMANTIC MAPPINGS -‐ Through inclusion of associated URLs encoded with each term, ontologies and mappings can be built to deﬁne rela=onships across poli=cal and research domains promo=ng interoperability and interdisciplinary research for all geospa=al, sensor-‐based observa=ons. 4)  Encoding thorough descrip=ons of processing and process lineage promotes beker understanding of the observa=ons, which enhances the value and reliability of the data. The original provider has no knowledge of how the data may be used! We need to communicate enough informa=on to enable assessment for a par=cular use beyond the project design! Did they sample fast enough for the new applica=on? Or long enough? Is the repor=ng frequency adequate? 19

Next Steps •  Build beker SensorML editors and registries -‐-‐ making things easier and promo=ng fully-‐described sensor and processing lineage. This will promote adop<on of the use of standards and more fully-‐described systems! •  Encourage manufactures, data managers and domain experts to create meaningful vocabularies including authorita=ve references to processing algorithms, with figures, equa=ons, etc. and to register the vocabularies, providing resolvable links in a standards-‐ based encoding (OWL) •  Provide tools and opportuni=es for domain experts to create and register ontologies, associa=ng terms in RDF (Is ThisQCtest the same as ThatQCtest? Does this QC flag have th same meaning as thatQCflag) 20

Conclusions •  Structured Q2O (hkp://q2o.whoi.edu) SensorML serves as a model for any sensor-‐based, in situ observa=ons; each component can be implemented by the responsible party and adop=on of the model can happen in stages. •  By associa=ng QC ﬂags with qc tests, processing methods with observa=ons, and fully-‐describing how observable proper=es become observa=ons knowledge about quality will be shared. •  By referencing (encoding) resolvable terms, ontologies can be built and registered to foster interoperability across-‐domains and poli=cal boundaries. The ability to dynamically assess data quality will provide a trusted founda<on for observing systems 21

Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems

by Cybera Inc.

Accessibility

Categories

Upload Details

Usage Rights

Statistics