Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Observations to Information

560 views

Published on

2013 Leptoukh Lecture, presented at American Geophysical Union - Fall Meeting, San Francisco, 2013-12-12

Published in: Science
  • Login to see the comments

Observations to Information

  1. 1. Observations to Information Simon Cox | Research Scientist | Environmental Information Systems 12 December 2013 LAND AND WATER 2013 Leptoukh Lecture
  2. 2. Standards for observation data • Motivation and development • Earth-science and environmental applications • Renewal AGU Fall 2013 | IN42A-01 | Cox | Leptoukh2 |
  3. 3. Origins AGU Fall 2013 | IN42A-01 | Cox | Leptoukh3 |
  4. 4. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1995 4 |
  5. 5. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX • Exploration data standards 5 |
  6. 6. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX • Exploration data standards 6 |
  7. 7. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards 7 |
  8. 8. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards • 2002-2005 OGC SWE • Sensors anywhere 8 |
  9. 9. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards • 2002-2005 OGC SWE • Sensors anywhere • 2005-2013 Fluid Earth • Water informatics 9 |
  10. 10. A bit of history AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards • 2002-2005 OGC SWE • Sensors anywhere • 2005-2013 WIRADA • Water informatics 10 |
  11. 11. Motivation for a standard All of society’s grand challenges require data to be shared and integrated across cultures, scales and technologies AGU Fall 2013 | IN42A-01 | Cox | Leptoukh11 |
  12. 12. Motivation for a standard • Integrated analysis and modelling • Discovery & data integration a significant challenge • Standard vocabulary Many private contracts one public agreement! AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Remote sensing Sensor Value Parameter Scene Earth science Algorithm, code, simulator Model, field Variable Volume, grid Metrology Instrument Value Measurand Sample Chemistry Instrument, analytical process Analysis Analyte Sample Environmental monitoring Gauge, sensor Value, time-series Parameter Station Observations & Measurements procedure result observed property feature of interest 12 |
  13. 13. SWE/O&M AGU Fall 2013 | IN42A-01 | Cox | Leptoukh13 |
  14. 14. O&M OM_Observation + phenomenonTime + resultTime + validTime [0..1] + resultQuality [0..*] + parameter [0..*] GF_PropertyType GFI_Feature OM_Process Any +observedProperty 1 0..* +featureOfInterest 1 0..* +procedure1 +result An Observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Cox, OGC Abstract Specification – Topic 20: Observations and Measurements 2.0 ISO 19156:2011 Geographic Information – Observations and measurements14 |
  15. 15. Scope • In situ observations • Remote sensing • Ex-situ observations • Numerical models/simulations • Forecasts Any action whose result is an estimate of a property value AGU Fall 2013 | IN42A-01 | Cox | Leptoukh15 |
  16. 16. Specimen Sampling features Observation SamplingFeature + Parameter + lineage Feature 0..* SpatialSamplingFeature + positionalAccuracy +relatedObservation 0..* SamplingSolidSamplingPoint SamplingCurve SamplingSurface Intention +sampledFeature SamplingFeatureComplex + role 0..* +relatedSamplingFeature 0..* +relatedObservation 0..* Profile Section Station Borehole MapHorizonScene Mine AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Cox, OGC Abstract Specification – Topic 20: Observations and Measurements 2.0 ISO 19156:2011 Geographic Information – Observations and measurements GM_Object +shape 16 | Harmonized with CSML, NCAR
  17. 17. OGC Sensor Web Enablement • SensorML • O&M • Sensor Observation Service • Sensor Planning Service AGU Fall 2013 | IN42A-01 | Cox | Leptoukh17 |
  18. 18. AGU Fall 2013 | IN42A-01 | Cox | Leptoukh SOS getObservation getResult describeSensor getFeatureOfInterest Sensor Observation Service Feature service getFeature, type=Observation Gridded data service getCoverage getCoverage (result) Sensor Register getRecordById Feature service getFeature 18 |
  19. 19. Observations & Measurements procedure result observed property feature of interest result time phenomenon time valid time procedure Provider or consumer? AGU Fall 2013 | IN42A-01 | Cox | Leptoukh19 |
  20. 20. Integrated into ISO 19100 framework AGU Fall 2013 | IN42A-01 | Cox | Leptoukh ISO dependencies • gf - Feature ISO 19109 • cv - Coverage (fields) ISO 19123 • md - Metadata ISO 19115 • gm - Geometry ISO 19107 • tm - Temporal ISO 19108 • basic - Datatypes ISO 19103 20 |
  21. 21. OM_Observation + phenomenonTime + resultTime + validTime [0..1] + resultQuality [0..*] + parameter [0..*] GFI_PropertyType GFI_Feature OM_ProcessGFI_DomainFeature Any +observedProperty 1 +propertyValueProvider 0..* +featureOfInterest 1 +generatedObservation 0..* +procedure1 +result Range observed property Parameter dictionary procedure Register of sensors, processes & algorithms feature of interest Feature-type catalogue Feature service result format: GML, SWE, netCDF, JSON, SQLite ... Domain specialization AGU Fall 2013 | IN42A-01 | Cox | Leptoukh21 |
  22. 22. Linked vocabularies can be shared and re-used AGU Fall 2013 | IN42A-01 | Cox | Leptoukh22 |
  23. 23. Applications AGU Fall 2013 | IN42A-01 | Cox | Leptoukh23 |
  24. 24. WaterML2 specializes O&M AGU Fall 2013 | IN42A-01 | Cox | Leptoukh «FeatureType» TimeseriesObservation OM_Process «FeatureType» Procedures:: ObservationProcess MD_Metadata «Type» ObservationMetadata «FeatureType» observation::OM_Observation + phenomenonTime :TM_Object + resultTime :TM_Instant + validTime :TM_Period [0..1] + resultQuality :DQ_Element [0..*] + parameter :NamedValue [0..*] «FeatureType» coverageObservation:: OM_DiscreteCoverageObservation «FeatureType» Timeseries (TVP) Observation:: TimeseriesTVPObservation «FeatureType» TimeseriesDomainRangeObservation «FeatureTyp... General Feature Instance:: GFI_Feature «metaclass» General Feature Model:: GF_PropertyType {root} + memberName :LocalName + definition :CharacterString «FeatureType» Timeseries::Timeseries +metadata 0..1 +generatedObservation 0..* +procedure 1 +propertyValueProvider 0..* Domain +featureOfInterest 1 0..* +relatedObservation 0..* Phenomenon +observedProperty 1 Range +result Name: Package: Version: Author: Context diagram: TimeseriesObservation «RequirementsClass» Timeseries Observation 1.0 CSIRO • Result is a time-series • Observed-property relates to water 24 | • Key predecessors: • CUAHSI WaterML • WDTF
  25. 25. AGU Fall 2013 | IN42A-01 | Cox | Leptoukh25 |
  26. 26. 26 | AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
  27. 27. WaterML-WQ constrains O&M and WaterML AGU Fall 2013 | IN42A-01 | Cox | Leptoukh27 | «FeatureType» measurement:: OM_Measurement AnyFeature «FeatureType» observ ation::OM_Observ ation + parameter :NamedValue [0..*] + phenomenonTime :TM_Object + resultQuality :DQ_Element [0..*] + resultTime :TM_Instant + validTime :TM_Period [0..1] constraints {observedProperty shall be a phenomenon associated with the feature of interest} {procedure shall be suitable for observedProperty} {result type shall be suitable for observedProperty} {a parameter.name shall not appear more than once} Units of Measure::Measure {root} + value :Number + convert(UnitOfMeasure*) :Measure «FeatureType» General Feature Instance::GFI_Feature «metaclass» General Feature Model:: GF_PropertyType {root} «metaclass» General Feature Model:: GF_FeatureType «FeatureType» observation:: OM_Process Metadata entity set information:: MD_Metadata «FeatureType» cov erageObserv ation:: OM_DiscreteCov erageObserv ation «FeatureType» Timeseries Observ ation:: TimeseriesObserv ation «FeatureType» Timeseries (TVP) Observ ation:: TimeseriesTVPObserv ation The XML element om:result SHALL have a uom property that is an instance of the owl:Class http://qudt.org/schema/qudt#Unit as defined in http://resources.data.gov.au/water/def/water-quality/wq- quantity. The XML element om:observedProperty SHALL have an xlink:href property that is an instance of the http://qudt.org/schema/qudt#Quantity scheme as defined in http://resources.data.gov.au/water/def/water- quality/wq-quantity. The XML element om:featureOfInterest SHOULD have an xlink:href property that is an instance of a GroundWaterML 1 GroundWaterBody feature or sub-type of HydrologicUnit feature as specified in the XML schema at http://ngwd-bdnes.cits.nrcan.gc.ca/service/gwml/schemas/gwml.xsd OR The XML element om:featureOfInterest SHOULD have an xlink:href property that is an instance of an OGC HY_Features HY_HydroFeature or sub-type as specified at "HY_Features: a Common Hydrologic Feature Model Discussion Paper OGC 11-039r2" «Type» Measurement (TVP) Timeseries:: MeasurementTimeseriesTVP Timeseries «FeatureType» Interleav ed (TVP) Timeseries:: TimeseriesTVP TimeValuePair «Type» Measurement (TVP) Timeseries:: MeasureTimeValuePair + value :Measure «FeatureType» WQ_Measurement:: WQ_Measurement «FeatureType» WQ_MeasurementTimeSeriesTVPObserv ation:: WQ_MeasurementTimeSeriesTVPObserv ation «FeatureType» WQ_MeasurementTimeseriesTVP:: WQ_MeasurementTimeSeriesTVP «metaclass» WQ_Observ ation:: WQ_PropertyType O&M Classes WaterML 2 Classes Water Quality Classes Legend +generatedObservation 0..* ProcessUsed +procedure 1 Phenomenon +observedProperty 1 Metadata +metadata 0..1 +result Range +collection 0..*CoverageFunction +element 0..* +result 0..* +relatedObservation 0..* +propertyValueProvider 0..* Domain +featureOfInterest 1 +carrierOfCharacteristics 0..* +theGF_FeatureType 1 «instanceOf» • Subject is a groundwater or geofabric feature • observed property is water-quality property • units of measure must match
  28. 28. Observables vocabulary AGU Fall 2013 | IN42A-01 | Cox | Leptoukh dissolved nitrogen concentration nitrogen elemental nitrogen (CHEBI_33267) Concentration MolePercent MilliGramsPerLitre AmountOfSubstancePerUnitVolume nitrogen concentration +qudt:generalization +objectOfInterest +exactMatch +qudt:quantityKind +qudt:quantityKind +qudt:generalization +qudt:unit +qudt:unit +qudt:generalization 28 | QUDT WQOP ChEBI
  29. 29. AGU Fall 2013 | IN42A-01 | Cox | Leptoukh29 | Environmental Information Standards | Simon Cox29 | SamplingFeature Spatial SamplingFeature Specimen Geometry SoilSample SoilSpecimen Domain Feature Observation ANZSoilML extends O&M Sampling Features SoilThematic Object Soil SoilLandscape SoilProfile Element SoilSurface Stream Channels Climate Vegetation LandSurface Anthropogenic Activity SoilFeature AbstractSoil Landform Landscape Feature ... also GeoSciML
  30. 30. Motivation From instrument to User Existing Activity 30
  31. 31. Existing Activity SWE Adoption Seismic Profile O&M Diviacco, P. et al. 2011 Marine Seismic Metadata for an Integrated European Scale Data Infrastructure 31
  32. 32. Integration with existing SeaDataNet Standards Existing Activity SWE Adoption Seismic 32
  33. 33. EU Air Quality Directive 33
  34. 34. INSPIRE – OM 34
  35. 35. 35
  36. 36. 36 12Z 7-May 9-May8-May6-May5-May 00Z00Z12Z00Z12Z12Z00Z12Z 00Z result forecast : OM_Observation parameter.name = “analysisTime” parameter.value = 2010-05-06T00:00Z phenomenonTime.begin = 2010-05-06T00:00Z phenomenonTime.end = 2010-05-09T12:00Z resultTime = 2010-05-06T04:30Z validTime [optional – not specified] resultQuality [optional – not specified] ISO19156 Observations and measurements: also suitable for numerical simulations – including forecasts
  37. 37. 37 Common constraints applicable to all WMO METCE Observation types WMO METCE «FeatureType» OM_ComplexObservation (ISO 19156) «Type» Record (ISO 19103) +result «FeatureType» ComplexSamplingMeasurement «FeatureType» Process +procedure «FeatureType» OM_Observation (ISO 19156) «FeatureType» Process +procedure (ISO 19156) «FeatureType» GFI_FeatureType +featureOfInterest (ISO 19156) «FeatureType» SF_SpatialSamplingFeature (ISO 19156) +featureOfInterest All specialisations of OM_Observation defined in WMO METCE require: • association role ‘featureOfInterest’ shall be of type SF_SpatialSamplingFeature • association role ‘procedure’ shall be of type Process (from WMO METCE)
  38. 38. SamplingFeature as OM_Observation.featureOfInterest 38 In meteorology, we define a sampling regime that enables us to observe, measure or simulate the real-world. Sampling Features (from ISO 19156 ‘Observations and measurements’) provide a way to characterise this sampling regime and the relationship to the real-world. :ComplexSamplingMeasurement03839:SF_SamplingPoint +featureOfInterest :Point @srsName “EPSG:4326” pos = “50.737 -3.405” +shape 84579:NamedPlace name = “Exeter Airport” +sampledFeature
  39. 39. National Environmental Information Infrastructure 39 |AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
  40. 40. NEII components Metadata Catalog Monitoring Sites Register Vocabulary Service Observing Methods Register Data Services 40 |AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
  41. 41. Metadata Catalog Observing Methods Register Monitoring Sites Register Vocabulary Service Data Service (observations) Data Service (geographies) Dataset Site Geography Parameter Instrument / Method Service … … … … … … … … … … … … … … … Integration through linking table Observation observed property feature of interest procedure result sampling feature 41 |AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
  42. 42. ODM2 Procedure Observed property Feature of Interest result Observation OM_Observation + phenomenonTime + resultTime + validTime [0..1] + resultQuality [0..*] + parameter [0..*] GF_PropertyType GFI_Feature OM_Process Any +observedProperty 1 0..* +featureOfInterest 1 0..* +procedure1 +result 42
  43. 43. - IGSN - ResourceURI - Registrant ID - timeStamp - status IGSN Registry - IGSN - Registrant - MetadataTimeStamp - Title - Description - SamplingLocation - SamplingTime - Distributor - Originator - SpecimenType - MaterialClass - SamplingMethod SESAR Catalog - IGSN - SampleEvent - SamplePhysicalSize - RelatedResource - SamplingMethodDetails - ProcessingHistory - CurationHistory - More local detail… Allocating Agent CatalogISO & OGC O&M compliant AGU Fall 2013 | IN42A-01 | Cox | Leptoukh 43
  44. 44. <spec:SF_Specimen gml:id="abc123"> <gml:identifier codeSpace="http://igsn.org">IGSN.SIOabc123</gml:identifier> <sa:sampledFeature xlink:href="http://example.org/midAtlanticRidge"/> <sa:relatedSamplingFeature> <sa:SamplingFeatureComplex> <sa:role xlink:href="http://example.org/parent"/> <sa:relatedSamplingFeature xlink:href="http://handle.net/10273/IGSN.SIOxyz456"/> </sa:SamplingFeatureComplex> </sa:relatedSamplingFeature> <spec:materialClass xlink:href="http://example.org/rock"/> <spec:samplingTime> <gml:TimeInstant gml:id="tim123"> <gml:timePosition>2009-06-30T08:00:00.00-02:00</gml:timePosition> </gml:TimeInstant> </spec:samplingTime> <spec:samplingMethod xlink:href="http://example.org/ghostbuster"/> <spec:samplingLocation> <gml:Point gml:id="loc123" srsName="http://www.opengis.net/def/crs/EPSG/0/4326"> <gml:pos>45.12 -81.78</gml:pos> </gml:Point> </spec:samplingLocation> <spec:size uom="kg">0.46</spec:size> <spec:currentLocation xlink:href="http://example.org/Warehouse3/shelf9/box67"/> <spec:specimenType xlink:href="http://example.org/splitCore"/> </spec:SF_Specimen> AGU Fall 2013 | IN42A-01 | Cox | Leptoukh IGSN Sample in OMXML
  45. 45. AGU Fall 2013 | IN42A-01 | Cox | Leptoukh <http://handle.net/10273/IGSN.SIOabc123> a sam:Specimen ; rdfs:label "SIO specimen abc123"^^xsd:string ; sam:currentLocation <http://example.org/various/Warehouse3/shelf9/box67> ; sam:materialClass p1:rock ; sam:preparationStep [ sam:processOperator p1:JohnDoe ; sam:processingDetails <http://example.org/various/sf-process/jkl987> ; sam:time <http://handle.net/10273/IGSN.SIOabc123/tim2> ] ; sam:sampledFeature p1:midAtlanticRidge ; sam:samplingFeatureComplex [ sam:relatedSamplingFeature <igsn:SIOxyz456> ; sam:role p1:parent ] ; sam:samplingLocation p1:loc123 ; sam:samplingMethod <http://ldeo.columbia.edu/sampling/ghostbuster> ; sam:samplingTime "2013-06-12T09:25:00.00+11:00"^^xsd:dateTime sam:size [ basic:uom unit:kg ; basic:value "0.46"^^basic:Number ] ; sam:specimenType p1:splitCore . IGSN Sample in RDF
  46. 46. AGU Fall 2013 | IN42A-01 | Cox | Leptoukh class Specimen SF_Specimen + currentLocation :Location [0..1] + materialClass :GenericName + samplingMethod :SF_Process [0..1] + samplingTime :TM_Object + specimenType :GenericName [0..1] «estimatedProperty» + samplingLocation :GM_Object [0..1] + size :Measure [0..1] SF_SamplingFeature + lineage :LI_Lineage [0..1] + parameter :NamedValue [0..*] GFI_Feature SamplingFeatureComplex + role :GenericName PreparationStep + processOperator :CI_ResponsibleParty [0..1] + time :TM_Object SF_Process OM_Observation SF_SamplingFeatureCollection Design +relatedObservation 0..* +processingDetails 0..* Intention +sampledFeature 1..* 0..* +relatedSamplingFeature 0..* +member 1..* Single model underlies different implementations
  47. 47. Application of O&M • Direct • SOS + OMXML – [Metadata for values from] data services • Specializations – WaterML2, WaterML-WQ, SoilML, SeaDataNet, INSPIRE, WMO ... • Bridging vocabulary for integration • TERENO (GFZ), NEII, ODM2, IGSN/SESAR, OBOE ... • Checklist ... • Slots for implicit  explicit for x-domain use AGU Fall 2013 | IN42A-01 | Cox | Leptoukh47 |
  48. 48. Modernizing AGU Fall 2013 | IN42A-01 | Cox | Leptoukh48 |
  49. 49. XSD OWL from UML AGU Fall 2013 | IN42A-01 | Cox | Leptoukh UML Model Output in OWL 49 |
  50. 50. om:Observation a :Class ; rdfs:label "Observation"@en ; rdfs:subClassOf gf:AnyFeature , h2o:FeatureType ; :disjointWith om:Process ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:result ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:observedProperty ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:featureOfInterest ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:phenomenonTime ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:procedure ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:resultTime ] . http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation AGU Fall 2013 | IN42A-01 | Cox | Leptoukh50 | O&M in OWL2
  51. 51. sam:Specimen AGU Fall 2013 | IN42A-01 | Cox | Leptoukh51 |
  52. 52. AGU Fall 2013 | IN42A-01 | Cox | Leptoukh OM_Observation + phenomenonTime + resultTime + validTime [0..1] + resultQuality [0..*] + parameter [0..*] GF_PropertyType GFI_Feature OM_Process Any +observedProperty 1 0..* +featureOfInterest 1 0..* +procedure1 +result 52 | SSN ontology
  53. 53. Alignment AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Is Observation a Social Object, or an Event? 53 |
  54. 54. Lessons AGU Fall 2013 | IN42A-01 | Cox | Leptoukh54 |
  55. 55. Lessons • Ontology for observations, models and forecasts • X-domain interoperability • Extensible, specialize with vocabularies, link to other ontologies • Standards • A generic model can provide a checklist for design, and a basis for harmonization and incremental design • Consensus introduces an overhead • Change • Common conceptual model supports multiple implementations • Widely used AGU Fall 2013 | IN42A-01 | Cox | Leptoukh55 |
  56. 56. Additional credits O&M: Fowler & Odell; GeoSciML team; OGC; ISO/TC 211; Rob Atkinson, Rob Woodcock(CSIRO) WaterML: CUAHSI; Gavin Walker, Pete Taylor, Laurent Lefort (CSIRO); Paul Sheahan (BOM) Soil, WQ: Bruce Simons, Peter Wilson, Jonathan Yu (CSIRO); Alistair Ritchie (Landcare NZ) SeaDataNet: Jordi Sorribas, Paolo Diviacco AQD: Michael Lutz, Ale Sarretta (JRC); Kathi Schleidt (Env. Agency Austria) WMO: Jeremy Tandy (UK Met Office); Aaron Braeckel (UCAR) NEII: Andrew Woolf, Dom Lowe (BOM) ODM2: Jeff Horsburgh IGSN: Kerstin Lehnert (LDEO), Jens Klump (GFZ) Vocabulary services: Stuart Williams (Epimorphics), Roy Lowry, Adam Leadbetter (BODC) AGU Fall 2013 | IN42A-01 | Cox | Leptoukh56 |
  57. 57. LAND AND WATER Thank youCSIRO Land and Water Simon Cox Research Scientist t +61 3 9252 6342 e simon.cox@csiro.au w www.csiro.au/people/simon.cox

×