Observations to Information
Simon Cox | Research Scientist | Environmental Information Systems
12 December 2013
LAND AND WATER
2013 Leptoukh Lecture
Standards for observation data
• Motivation and development
• Earth-science and environmental applications
• Renewal
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh2 |
Origins
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh3 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1995
4 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1999-2004 XMML, ADX
• Exploration data standards
5 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1999-2004 XMML, ADX
• Exploration data standards
6 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1999-2004 XMML, ADX
-2010 AuScope
• Exploration data standards
7 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1999-2004 XMML, ADX
-2010 AuScope
• Exploration data standards
• 2002-2005 OGC SWE
• Sensors anywhere
8 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1999-2004 XMML, ADX
-2010 AuScope
• Exploration data standards
• 2002-2005 OGC SWE
• Sensors anywhere
• 2005-2013 Fluid Earth
• Water informatics
9 |
A bit of history
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
• 1993-2000 AGCRC
• Web-mapping
• Reporting research online
• 1999-2004 XMML, ADX
-2010 AuScope
• Exploration data standards
• 2002-2005 OGC SWE
• Sensors anywhere
• 2005-2013 WIRADA
• Water informatics
10 |
Motivation for a standard
All of society’s grand challenges require data to
be shared and integrated across cultures, scales
and technologies
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh11 |
Motivation for a standard
• Integrated analysis and modelling
• Discovery & data integration a
significant challenge
• Standard vocabulary
Many private contracts
one public agreement!
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
Remote sensing
Sensor
Value
Parameter
Scene
Earth science
Algorithm, code,
simulator
Model, field
Variable
Volume, grid
Metrology
Instrument
Value
Measurand
Sample
Chemistry
Instrument,
analytical process
Analysis
Analyte
Sample
Environmental
monitoring
Gauge, sensor
Value, time-series
Parameter
Station
Observations &
Measurements
procedure
result
observed property
feature of interest
12 |
SWE/O&M
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh13 |
O&M
OM_Observation
+ phenomenonTime
+ resultTime
+ validTime [0..1]
+ resultQuality [0..*]
+ parameter [0..*]
GF_PropertyType
GFI_Feature
OM_Process Any
+observedProperty
1
0..*
+featureOfInterest 1
0..*
+procedure1 +result
An Observation is an action whose result is an estimate of the value
of some property of the feature-of-interest, obtained using a specified procedure
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
Cox, OGC Abstract Specification – Topic 20: Observations and Measurements 2.0
ISO 19156:2011 Geographic Information – Observations and measurements14 |
Scope
• In situ observations
• Remote sensing
• Ex-situ observations
• Numerical models/simulations
• Forecasts
Any action whose result is an estimate of a property value
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh15 |
Specimen
Sampling features
Observation
SamplingFeature
+ Parameter
+ lineage
Feature
0..*
SpatialSamplingFeature
+ positionalAccuracy
+relatedObservation 0..*
SamplingSolidSamplingPoint SamplingCurve SamplingSurface
Intention
+sampledFeature
SamplingFeatureComplex
+ role
0..*
+relatedSamplingFeature
0..*
+relatedObservation
0..*
Profile
Section
Station
Borehole
MapHorizonScene
Mine
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
Cox, OGC Abstract Specification – Topic 20: Observations and Measurements 2.0
ISO 19156:2011 Geographic Information – Observations and measurements
GM_Object
+shape
16 |
Harmonized with
CSML, NCAR
OGC Sensor Web Enablement
• SensorML
• O&M
• Sensor Observation Service
• Sensor Planning Service
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh17 |
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
SOS
getObservation
getResult
describeSensor
getFeatureOfInterest
Sensor Observation Service
Feature
service
getFeature,
type=Observation
Gridded data
service
getCoverage
getCoverage
(result)
Sensor
Register
getRecordById
Feature
service
getFeature
18 |
Observations &
Measurements
procedure
result
observed property
feature of interest
result time
phenomenon time
valid time
procedure
Provider or
consumer?
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh19 |
Integrated into ISO 19100 framework
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
ISO dependencies
• gf - Feature ISO 19109
• cv - Coverage (fields) ISO 19123
• md - Metadata ISO 19115
• gm - Geometry ISO 19107
• tm - Temporal ISO 19108
• basic - Datatypes ISO 19103
20 |
OM_Observation
+ phenomenonTime
+ resultTime
+ validTime [0..1]
+ resultQuality [0..*]
+ parameter [0..*]
GFI_PropertyType
GFI_Feature
OM_ProcessGFI_DomainFeature Any
+observedProperty
1
+propertyValueProvider
0..*
+featureOfInterest
1
+generatedObservation
0..*
+procedure1 +result
Range
observed property
Parameter dictionary
procedure
Register of sensors,
processes & algorithms
feature of interest
Feature-type catalogue
Feature service
result format:
GML, SWE,
netCDF, JSON, SQLite
...
Domain specialization
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh21 |
Linked vocabularies can be shared and re-used
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh22 |
Applications
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh23 |
WaterML2 specializes O&M
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
«FeatureType»
TimeseriesObservation
OM_Process
«FeatureType»
Procedures::
ObservationProcess
MD_Metadata
«Type»
ObservationMetadata
«FeatureType»
observation::OM_Observation
+ phenomenonTime :TM_Object
+ resultTime :TM_Instant
+ validTime :TM_Period [0..1]
+ resultQuality :DQ_Element [0..*]
+ parameter :NamedValue [0..*]
«FeatureType»
coverageObservation::
OM_DiscreteCoverageObservation
«FeatureType»
Timeseries (TVP) Observation::
TimeseriesTVPObservation
«FeatureType»
TimeseriesDomainRangeObservation
«FeatureTyp...
General Feature
Instance::
GFI_Feature
«metaclass»
General Feature Model::
GF_PropertyType
{root}
+ memberName :LocalName
+ definition :CharacterString
«FeatureType»
Timeseries::Timeseries
+metadata
0..1
+generatedObservation
0..*
+procedure
1
+propertyValueProvider
0..*
Domain
+featureOfInterest
1
0..*
+relatedObservation 0..*
Phenomenon
+observedProperty
1
Range +result
Name:
Package:
Version:
Author:
Context diagram: TimeseriesObservation
«RequirementsClass» Timeseries Observation
1.0
CSIRO
• Result is a time-series
• Observed-property
relates to water
24 |
• Key predecessors:
• CUAHSI WaterML
• WDTF
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh25 |
26 | AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
WaterML-WQ constrains
O&M and WaterML
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh27 |
«FeatureType»
measurement::
OM_Measurement
AnyFeature
«FeatureType»
observ ation::OM_Observ ation
+ parameter :NamedValue [0..*]
+ phenomenonTime :TM_Object
+ resultQuality :DQ_Element [0..*]
+ resultTime :TM_Instant
+ validTime :TM_Period [0..1]
constraints
{observedProperty shall be a phenomenon associated with the feature of interest}
{procedure shall be suitable for observedProperty}
{result type shall be suitable for observedProperty}
{a parameter.name shall not appear more than once}
Units of Measure::Measure
{root}
+ value :Number
+ convert(UnitOfMeasure*) :Measure
«FeatureType»
General Feature
Instance::GFI_Feature
«metaclass»
General Feature Model::
GF_PropertyType
{root}
«metaclass»
General Feature Model::
GF_FeatureType
«FeatureType»
observation::
OM_Process
Metadata entity set
information::
MD_Metadata
«FeatureType»
cov erageObserv ation::
OM_DiscreteCov erageObserv ation
«FeatureType»
Timeseries Observ ation::
TimeseriesObserv ation
«FeatureType»
Timeseries (TVP) Observ ation::
TimeseriesTVPObserv ation
The XML element om:result SHALL have a uom property
that is an instance of the owl:Class
http://qudt.org/schema/qudt#Unit as defined in
http://resources.data.gov.au/water/def/water-quality/wq-
quantity.
The XML element om:observedProperty SHALL have
an xlink:href property that is an instance of the
http://qudt.org/schema/qudt#Quantity scheme as
defined in
http://resources.data.gov.au/water/def/water-
quality/wq-quantity.
The XML element om:featureOfInterest SHOULD have an xlink:href property that is an instance of
a GroundWaterML 1 GroundWaterBody feature or sub-type of HydrologicUnit feature as specified
in the XML schema at http://ngwd-bdnes.cits.nrcan.gc.ca/service/gwml/schemas/gwml.xsd
OR
The XML element om:featureOfInterest SHOULD have an xlink:href property that is an instance of
an OGC HY_Features HY_HydroFeature or sub-type as specified at "HY_Features: a Common
Hydrologic Feature Model Discussion Paper OGC 11-039r2"
«Type»
Measurement (TVP) Timeseries::
MeasurementTimeseriesTVP
Timeseries
«FeatureType»
Interleav ed (TVP) Timeseries::
TimeseriesTVP
TimeValuePair
«Type»
Measurement (TVP) Timeseries::
MeasureTimeValuePair
+ value :Measure
«FeatureType»
WQ_Measurement::
WQ_Measurement
«FeatureType»
WQ_MeasurementTimeSeriesTVPObserv ation::
WQ_MeasurementTimeSeriesTVPObserv ation
«FeatureType»
WQ_MeasurementTimeseriesTVP::
WQ_MeasurementTimeSeriesTVP
«metaclass»
WQ_Observ ation::
WQ_PropertyType
O&M Classes
WaterML 2 Classes
Water Quality Classes
Legend
+generatedObservation 0..*
ProcessUsed
+procedure
1
Phenomenon
+observedProperty
1
Metadata
+metadata 0..1
+result
Range
+collection
0..*CoverageFunction
+element
0..*
+result
0..*
+relatedObservation 0..*
+propertyValueProvider
0..*
Domain
+featureOfInterest
1
+carrierOfCharacteristics
0..*
+theGF_FeatureType
1
«instanceOf»
• Subject is a groundwater
or geofabric feature
• observed property is
water-quality property
• units of measure must match
Observables vocabulary
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
dissolved nitrogen
concentration
nitrogen
elemental nitrogen
(CHEBI_33267)
Concentration
MolePercent
MilliGramsPerLitre
AmountOfSubstancePerUnitVolume
nitrogen concentration
+qudt:generalization
+objectOfInterest
+exactMatch
+qudt:quantityKind
+qudt:quantityKind
+qudt:generalization
+qudt:unit
+qudt:unit
+qudt:generalization
28 |
QUDT
WQOP
ChEBI
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh29 | Environmental Information Standards | Simon Cox29 |
SamplingFeature
Spatial
SamplingFeature
Specimen
Geometry
SoilSample SoilSpecimen
Domain Feature
Observation
ANZSoilML extends O&M
Sampling Features
SoilThematic
Object
Soil
SoilLandscape
SoilProfile
Element
SoilSurface
Stream
Channels
Climate
Vegetation
LandSurface
Anthropogenic
Activity
SoilFeature
AbstractSoil
Landform
Landscape
Feature
... also GeoSciML
Motivation
From instrument to User
Existing Activity
30
Existing Activity
SWE Adoption
Seismic Profile O&M
Diviacco, P. et al. 2011 Marine Seismic Metadata for an Integrated European Scale Data Infrastructure
31
Integration with existing SeaDataNet Standards
Existing Activity
SWE Adoption
Seismic
32
EU Air Quality Directive
33
INSPIRE – OM
34
35
36
12Z
7-May 9-May8-May6-May5-May
00Z00Z12Z00Z12Z12Z00Z12Z 00Z
result
forecast : OM_Observation
parameter.name = “analysisTime”
parameter.value = 2010-05-06T00:00Z
phenomenonTime.begin = 2010-05-06T00:00Z
phenomenonTime.end = 2010-05-09T12:00Z
resultTime = 2010-05-06T04:30Z
validTime [optional – not specified]
resultQuality [optional – not specified]
ISO19156 Observations and measurements:
also suitable for numerical simulations – including forecasts
37
Common constraints applicable to all WMO
METCE Observation types
WMO METCE
«FeatureType»
OM_ComplexObservation
(ISO 19156)
«Type»
Record
(ISO 19103)
+result
«FeatureType»
ComplexSamplingMeasurement
«FeatureType»
Process
+procedure
«FeatureType»
OM_Observation
(ISO 19156)
«FeatureType»
Process
+procedure
(ISO 19156)
«FeatureType»
GFI_FeatureType
+featureOfInterest
(ISO 19156)
«FeatureType»
SF_SpatialSamplingFeature
(ISO 19156)
+featureOfInterest
All specialisations of OM_Observation defined in WMO METCE require:
• association role ‘featureOfInterest’ shall be of type SF_SpatialSamplingFeature
• association role ‘procedure’ shall be of type Process (from WMO METCE)
SamplingFeature as OM_Observation.featureOfInterest
38
In meteorology, we define a sampling regime that enables us to observe,
measure or simulate the real-world. Sampling Features (from ISO 19156
‘Observations and measurements’) provide a way to characterise this sampling
regime and the relationship to the real-world.
:ComplexSamplingMeasurement03839:SF_SamplingPoint
+featureOfInterest
:Point
@srsName “EPSG:4326”
pos = “50.737 -3.405”
+shape
84579:NamedPlace
name = “Exeter Airport” +sampledFeature
National Environmental
Information Infrastructure
39 |AGU Fall 2013 | IN42A-01 |
Cox | Leptoukh
NEII components
Metadata
Catalog
Monitoring
Sites Register
Vocabulary
Service
Observing
Methods
Register
Data Services
40 |AGU Fall 2013 | IN42A-01 |
Cox | Leptoukh
Metadata
Catalog
Observing
Methods
Register
Monitoring
Sites Register
Vocabulary
Service
Data Service
(observations)
Data Service
(geographies)
Dataset Site Geography Parameter Instrument / Method Service …
… … … … … … …
… … … … … … …
Integration through linking table
Observation
observed
property
feature of
interest
procedure result
sampling
feature
41 |AGU Fall 2013 | IN42A-01 |
Cox | Leptoukh
ODM2
Procedure
Observed
property
Feature of Interest
result
Observation
OM_Observation
+ phenomenonTime
+ resultTime
+ validTime [0..1]
+ resultQuality [0..*]
+ parameter [0..*]
GF_PropertyType
GFI_Feature
OM_Process Any
+observedProperty
1
0..*
+featureOfInterest
1
0..*
+procedure1 +result
42
- IGSN
- ResourceURI
- Registrant ID
- timeStamp
- status
IGSN Registry
- IGSN
- Registrant
- MetadataTimeStamp
- Title
- Description
- SamplingLocation
- SamplingTime
- Distributor
- Originator
- SpecimenType
- MaterialClass
- SamplingMethod
SESAR Catalog
- IGSN
- SampleEvent
- SamplePhysicalSize
- RelatedResource
- SamplingMethodDetails
- ProcessingHistory
- CurationHistory
- More local detail…
Allocating Agent CatalogISO & OGC O&M
compliant
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh 43
<spec:SF_Specimen gml:id="abc123">
<gml:identifier codeSpace="http://igsn.org">IGSN.SIOabc123</gml:identifier>
<sa:sampledFeature xlink:href="http://example.org/midAtlanticRidge"/>
<sa:relatedSamplingFeature>
<sa:SamplingFeatureComplex>
<sa:role xlink:href="http://example.org/parent"/>
<sa:relatedSamplingFeature xlink:href="http://handle.net/10273/IGSN.SIOxyz456"/>
</sa:SamplingFeatureComplex>
</sa:relatedSamplingFeature>
<spec:materialClass xlink:href="http://example.org/rock"/>
<spec:samplingTime>
<gml:TimeInstant gml:id="tim123">
<gml:timePosition>2009-06-30T08:00:00.00-02:00</gml:timePosition>
</gml:TimeInstant>
</spec:samplingTime>
<spec:samplingMethod xlink:href="http://example.org/ghostbuster"/>
<spec:samplingLocation>
<gml:Point gml:id="loc123" srsName="http://www.opengis.net/def/crs/EPSG/0/4326">
<gml:pos>45.12 -81.78</gml:pos>
</gml:Point>
</spec:samplingLocation>
<spec:size uom="kg">0.46</spec:size>
<spec:currentLocation xlink:href="http://example.org/Warehouse3/shelf9/box67"/>
<spec:specimenType xlink:href="http://example.org/splitCore"/>
</spec:SF_Specimen>
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
IGSN Sample in OMXML
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
<http://handle.net/10273/IGSN.SIOabc123>
a sam:Specimen ;
rdfs:label "SIO specimen abc123"^^xsd:string ;
sam:currentLocation <http://example.org/various/Warehouse3/shelf9/box67> ;
sam:materialClass p1:rock ;
sam:preparationStep
[ sam:processOperator p1:JohnDoe ;
sam:processingDetails
<http://example.org/various/sf-process/jkl987> ;
sam:time <http://handle.net/10273/IGSN.SIOabc123/tim2> ] ;
sam:sampledFeature p1:midAtlanticRidge ;
sam:samplingFeatureComplex
[ sam:relatedSamplingFeature <igsn:SIOxyz456> ;
sam:role p1:parent ] ;
sam:samplingLocation
p1:loc123 ;
sam:samplingMethod <http://ldeo.columbia.edu/sampling/ghostbuster> ;
sam:samplingTime "2013-06-12T09:25:00.00+11:00"^^xsd:dateTime
sam:size [ basic:uom unit:kg ;
basic:value "0.46"^^basic:Number ] ;
sam:specimenType p1:splitCore .
IGSN Sample in RDF
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
class Specimen
SF_Specimen
+ currentLocation :Location [0..1]
+ materialClass :GenericName
+ samplingMethod :SF_Process [0..1]
+ samplingTime :TM_Object
+ specimenType :GenericName [0..1]
«estimatedProperty»
+ samplingLocation :GM_Object [0..1]
+ size :Measure [0..1]
SF_SamplingFeature
+ lineage :LI_Lineage [0..1]
+ parameter :NamedValue [0..*]
GFI_Feature
SamplingFeatureComplex
+ role :GenericName
PreparationStep
+ processOperator :CI_ResponsibleParty [0..1]
+ time :TM_Object
SF_Process
OM_Observation
SF_SamplingFeatureCollection
Design
+relatedObservation
0..*
+processingDetails
0..*
Intention
+sampledFeature
1..*
0..*
+relatedSamplingFeature
0..*
+member 1..*
Single model underlies
different implementations
Application of O&M
• Direct
• SOS + OMXML
– [Metadata for values from] data services
• Specializations
– WaterML2, WaterML-WQ, SoilML, SeaDataNet, INSPIRE, WMO ...
• Bridging vocabulary for integration
• TERENO (GFZ), NEII, ODM2, IGSN/SESAR, OBOE ...
• Checklist ...
• Slots for implicit  explicit for x-domain use
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh47 |
Modernizing
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh48 |
XSD OWL from UML
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
UML Model
Output in OWL
49 |
om:Observation
a :Class ;
rdfs:label "Observation"@en ;
rdfs:subClassOf gf:AnyFeature , h2o:FeatureType ;
:disjointWith om:Process ;
rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:result ] ;
rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:observedProperty ] ;
rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:featureOfInterest ] ;
rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:phenomenonTime ] ;
rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:procedure ] ;
rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:resultTime ] .
http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh50 |
O&M in OWL2
sam:Specimen
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh51 |
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
OM_Observation
+ phenomenonTime
+ resultTime
+ validTime [0..1]
+ resultQuality [0..*]
+ parameter [0..*]
GF_PropertyType
GFI_Feature
OM_Process Any
+observedProperty
1
0..*
+featureOfInterest
1
0..*
+procedure1 +result
52 |
SSN ontology
Alignment
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
Is Observation a Social Object, or an Event?
53 |
Lessons
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh54 |
Lessons
• Ontology for observations, models and forecasts
• X-domain interoperability
• Extensible, specialize with vocabularies, link to other ontologies
• Standards
• A generic model can provide a checklist for design, and a basis for
harmonization and incremental design
• Consensus introduces an overhead
• Change
• Common conceptual model supports multiple implementations
• Widely used
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh55 |
Additional credits
O&M: Fowler & Odell; GeoSciML team; OGC; ISO/TC 211;
Rob Atkinson, Rob Woodcock(CSIRO)
WaterML: CUAHSI; Gavin Walker, Pete Taylor, Laurent Lefort (CSIRO);
Paul Sheahan (BOM)
Soil, WQ: Bruce Simons, Peter Wilson, Jonathan Yu (CSIRO); Alistair Ritchie
(Landcare NZ)
SeaDataNet: Jordi Sorribas, Paolo Diviacco
AQD: Michael Lutz, Ale Sarretta (JRC); Kathi Schleidt (Env. Agency Austria)
WMO: Jeremy Tandy (UK Met Office); Aaron Braeckel (UCAR)
NEII: Andrew Woolf, Dom Lowe (BOM)
ODM2: Jeff Horsburgh
IGSN: Kerstin Lehnert (LDEO), Jens Klump (GFZ)
Vocabulary services: Stuart Williams (Epimorphics), Roy Lowry, Adam Leadbetter
(BODC)
AGU Fall 2013 | IN42A-01 | Cox | Leptoukh56 |
LAND AND WATER
Thank youCSIRO Land and Water
Simon Cox
Research Scientist
t +61 3 9252 6342
e simon.cox@csiro.au
w www.csiro.au/people/simon.cox

Observations to Information

  • 1.
    Observations to Information SimonCox | Research Scientist | Environmental Information Systems 12 December 2013 LAND AND WATER 2013 Leptoukh Lecture
  • 2.
    Standards for observationdata • Motivation and development • Earth-science and environmental applications • Renewal AGU Fall 2013 | IN42A-01 | Cox | Leptoukh2 |
  • 3.
    Origins AGU Fall 2013| IN42A-01 | Cox | Leptoukh3 |
  • 4.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1995 4 |
  • 5.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX • Exploration data standards 5 |
  • 6.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX • Exploration data standards 6 |
  • 7.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards 7 |
  • 8.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards • 2002-2005 OGC SWE • Sensors anywhere 8 |
  • 9.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards • 2002-2005 OGC SWE • Sensors anywhere • 2005-2013 Fluid Earth • Water informatics 9 |
  • 10.
    A bit ofhistory AGU Fall 2013 | IN42A-01 | Cox | Leptoukh • 1993-2000 AGCRC • Web-mapping • Reporting research online • 1999-2004 XMML, ADX -2010 AuScope • Exploration data standards • 2002-2005 OGC SWE • Sensors anywhere • 2005-2013 WIRADA • Water informatics 10 |
  • 11.
    Motivation for astandard All of society’s grand challenges require data to be shared and integrated across cultures, scales and technologies AGU Fall 2013 | IN42A-01 | Cox | Leptoukh11 |
  • 12.
    Motivation for astandard • Integrated analysis and modelling • Discovery & data integration a significant challenge • Standard vocabulary Many private contracts one public agreement! AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Remote sensing Sensor Value Parameter Scene Earth science Algorithm, code, simulator Model, field Variable Volume, grid Metrology Instrument Value Measurand Sample Chemistry Instrument, analytical process Analysis Analyte Sample Environmental monitoring Gauge, sensor Value, time-series Parameter Station Observations & Measurements procedure result observed property feature of interest 12 |
  • 13.
    SWE/O&M AGU Fall 2013| IN42A-01 | Cox | Leptoukh13 |
  • 14.
    O&M OM_Observation + phenomenonTime + resultTime +validTime [0..1] + resultQuality [0..*] + parameter [0..*] GF_PropertyType GFI_Feature OM_Process Any +observedProperty 1 0..* +featureOfInterest 1 0..* +procedure1 +result An Observation is an action whose result is an estimate of the value of some property of the feature-of-interest, obtained using a specified procedure AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Cox, OGC Abstract Specification – Topic 20: Observations and Measurements 2.0 ISO 19156:2011 Geographic Information – Observations and measurements14 |
  • 15.
    Scope • In situobservations • Remote sensing • Ex-situ observations • Numerical models/simulations • Forecasts Any action whose result is an estimate of a property value AGU Fall 2013 | IN42A-01 | Cox | Leptoukh15 |
  • 16.
    Specimen Sampling features Observation SamplingFeature + Parameter +lineage Feature 0..* SpatialSamplingFeature + positionalAccuracy +relatedObservation 0..* SamplingSolidSamplingPoint SamplingCurve SamplingSurface Intention +sampledFeature SamplingFeatureComplex + role 0..* +relatedSamplingFeature 0..* +relatedObservation 0..* Profile Section Station Borehole MapHorizonScene Mine AGU Fall 2013 | IN42A-01 | Cox | Leptoukh Cox, OGC Abstract Specification – Topic 20: Observations and Measurements 2.0 ISO 19156:2011 Geographic Information – Observations and measurements GM_Object +shape 16 | Harmonized with CSML, NCAR
  • 17.
    OGC Sensor WebEnablement • SensorML • O&M • Sensor Observation Service • Sensor Planning Service AGU Fall 2013 | IN42A-01 | Cox | Leptoukh17 |
  • 18.
    AGU Fall 2013| IN42A-01 | Cox | Leptoukh SOS getObservation getResult describeSensor getFeatureOfInterest Sensor Observation Service Feature service getFeature, type=Observation Gridded data service getCoverage getCoverage (result) Sensor Register getRecordById Feature service getFeature 18 |
  • 19.
    Observations & Measurements procedure result observed property featureof interest result time phenomenon time valid time procedure Provider or consumer? AGU Fall 2013 | IN42A-01 | Cox | Leptoukh19 |
  • 20.
    Integrated into ISO19100 framework AGU Fall 2013 | IN42A-01 | Cox | Leptoukh ISO dependencies • gf - Feature ISO 19109 • cv - Coverage (fields) ISO 19123 • md - Metadata ISO 19115 • gm - Geometry ISO 19107 • tm - Temporal ISO 19108 • basic - Datatypes ISO 19103 20 |
  • 21.
    OM_Observation + phenomenonTime + resultTime +validTime [0..1] + resultQuality [0..*] + parameter [0..*] GFI_PropertyType GFI_Feature OM_ProcessGFI_DomainFeature Any +observedProperty 1 +propertyValueProvider 0..* +featureOfInterest 1 +generatedObservation 0..* +procedure1 +result Range observed property Parameter dictionary procedure Register of sensors, processes & algorithms feature of interest Feature-type catalogue Feature service result format: GML, SWE, netCDF, JSON, SQLite ... Domain specialization AGU Fall 2013 | IN42A-01 | Cox | Leptoukh21 |
  • 22.
    Linked vocabularies canbe shared and re-used AGU Fall 2013 | IN42A-01 | Cox | Leptoukh22 |
  • 23.
    Applications AGU Fall 2013| IN42A-01 | Cox | Leptoukh23 |
  • 24.
    WaterML2 specializes O&M AGUFall 2013 | IN42A-01 | Cox | Leptoukh «FeatureType» TimeseriesObservation OM_Process «FeatureType» Procedures:: ObservationProcess MD_Metadata «Type» ObservationMetadata «FeatureType» observation::OM_Observation + phenomenonTime :TM_Object + resultTime :TM_Instant + validTime :TM_Period [0..1] + resultQuality :DQ_Element [0..*] + parameter :NamedValue [0..*] «FeatureType» coverageObservation:: OM_DiscreteCoverageObservation «FeatureType» Timeseries (TVP) Observation:: TimeseriesTVPObservation «FeatureType» TimeseriesDomainRangeObservation «FeatureTyp... General Feature Instance:: GFI_Feature «metaclass» General Feature Model:: GF_PropertyType {root} + memberName :LocalName + definition :CharacterString «FeatureType» Timeseries::Timeseries +metadata 0..1 +generatedObservation 0..* +procedure 1 +propertyValueProvider 0..* Domain +featureOfInterest 1 0..* +relatedObservation 0..* Phenomenon +observedProperty 1 Range +result Name: Package: Version: Author: Context diagram: TimeseriesObservation «RequirementsClass» Timeseries Observation 1.0 CSIRO • Result is a time-series • Observed-property relates to water 24 | • Key predecessors: • CUAHSI WaterML • WDTF
  • 25.
    AGU Fall 2013| IN42A-01 | Cox | Leptoukh25 |
  • 26.
    26 | AGUFall 2013 | IN42A-01 | Cox | Leptoukh
  • 27.
    WaterML-WQ constrains O&M andWaterML AGU Fall 2013 | IN42A-01 | Cox | Leptoukh27 | «FeatureType» measurement:: OM_Measurement AnyFeature «FeatureType» observ ation::OM_Observ ation + parameter :NamedValue [0..*] + phenomenonTime :TM_Object + resultQuality :DQ_Element [0..*] + resultTime :TM_Instant + validTime :TM_Period [0..1] constraints {observedProperty shall be a phenomenon associated with the feature of interest} {procedure shall be suitable for observedProperty} {result type shall be suitable for observedProperty} {a parameter.name shall not appear more than once} Units of Measure::Measure {root} + value :Number + convert(UnitOfMeasure*) :Measure «FeatureType» General Feature Instance::GFI_Feature «metaclass» General Feature Model:: GF_PropertyType {root} «metaclass» General Feature Model:: GF_FeatureType «FeatureType» observation:: OM_Process Metadata entity set information:: MD_Metadata «FeatureType» cov erageObserv ation:: OM_DiscreteCov erageObserv ation «FeatureType» Timeseries Observ ation:: TimeseriesObserv ation «FeatureType» Timeseries (TVP) Observ ation:: TimeseriesTVPObserv ation The XML element om:result SHALL have a uom property that is an instance of the owl:Class http://qudt.org/schema/qudt#Unit as defined in http://resources.data.gov.au/water/def/water-quality/wq- quantity. The XML element om:observedProperty SHALL have an xlink:href property that is an instance of the http://qudt.org/schema/qudt#Quantity scheme as defined in http://resources.data.gov.au/water/def/water- quality/wq-quantity. The XML element om:featureOfInterest SHOULD have an xlink:href property that is an instance of a GroundWaterML 1 GroundWaterBody feature or sub-type of HydrologicUnit feature as specified in the XML schema at http://ngwd-bdnes.cits.nrcan.gc.ca/service/gwml/schemas/gwml.xsd OR The XML element om:featureOfInterest SHOULD have an xlink:href property that is an instance of an OGC HY_Features HY_HydroFeature or sub-type as specified at "HY_Features: a Common Hydrologic Feature Model Discussion Paper OGC 11-039r2" «Type» Measurement (TVP) Timeseries:: MeasurementTimeseriesTVP Timeseries «FeatureType» Interleav ed (TVP) Timeseries:: TimeseriesTVP TimeValuePair «Type» Measurement (TVP) Timeseries:: MeasureTimeValuePair + value :Measure «FeatureType» WQ_Measurement:: WQ_Measurement «FeatureType» WQ_MeasurementTimeSeriesTVPObserv ation:: WQ_MeasurementTimeSeriesTVPObserv ation «FeatureType» WQ_MeasurementTimeseriesTVP:: WQ_MeasurementTimeSeriesTVP «metaclass» WQ_Observ ation:: WQ_PropertyType O&M Classes WaterML 2 Classes Water Quality Classes Legend +generatedObservation 0..* ProcessUsed +procedure 1 Phenomenon +observedProperty 1 Metadata +metadata 0..1 +result Range +collection 0..*CoverageFunction +element 0..* +result 0..* +relatedObservation 0..* +propertyValueProvider 0..* Domain +featureOfInterest 1 +carrierOfCharacteristics 0..* +theGF_FeatureType 1 «instanceOf» • Subject is a groundwater or geofabric feature • observed property is water-quality property • units of measure must match
  • 28.
    Observables vocabulary AGU Fall2013 | IN42A-01 | Cox | Leptoukh dissolved nitrogen concentration nitrogen elemental nitrogen (CHEBI_33267) Concentration MolePercent MilliGramsPerLitre AmountOfSubstancePerUnitVolume nitrogen concentration +qudt:generalization +objectOfInterest +exactMatch +qudt:quantityKind +qudt:quantityKind +qudt:generalization +qudt:unit +qudt:unit +qudt:generalization 28 | QUDT WQOP ChEBI
  • 29.
    AGU Fall 2013| IN42A-01 | Cox | Leptoukh29 | Environmental Information Standards | Simon Cox29 | SamplingFeature Spatial SamplingFeature Specimen Geometry SoilSample SoilSpecimen Domain Feature Observation ANZSoilML extends O&M Sampling Features SoilThematic Object Soil SoilLandscape SoilProfile Element SoilSurface Stream Channels Climate Vegetation LandSurface Anthropogenic Activity SoilFeature AbstractSoil Landform Landscape Feature ... also GeoSciML
  • 30.
    Motivation From instrument toUser Existing Activity 30
  • 31.
    Existing Activity SWE Adoption SeismicProfile O&M Diviacco, P. et al. 2011 Marine Seismic Metadata for an Integrated European Scale Data Infrastructure 31
  • 32.
    Integration with existingSeaDataNet Standards Existing Activity SWE Adoption Seismic 32
  • 33.
    EU Air QualityDirective 33
  • 34.
  • 35.
  • 36.
    36 12Z 7-May 9-May8-May6-May5-May 00Z00Z12Z00Z12Z12Z00Z12Z 00Z result forecast: OM_Observation parameter.name = “analysisTime” parameter.value = 2010-05-06T00:00Z phenomenonTime.begin = 2010-05-06T00:00Z phenomenonTime.end = 2010-05-09T12:00Z resultTime = 2010-05-06T04:30Z validTime [optional – not specified] resultQuality [optional – not specified] ISO19156 Observations and measurements: also suitable for numerical simulations – including forecasts
  • 37.
    37 Common constraints applicableto all WMO METCE Observation types WMO METCE «FeatureType» OM_ComplexObservation (ISO 19156) «Type» Record (ISO 19103) +result «FeatureType» ComplexSamplingMeasurement «FeatureType» Process +procedure «FeatureType» OM_Observation (ISO 19156) «FeatureType» Process +procedure (ISO 19156) «FeatureType» GFI_FeatureType +featureOfInterest (ISO 19156) «FeatureType» SF_SpatialSamplingFeature (ISO 19156) +featureOfInterest All specialisations of OM_Observation defined in WMO METCE require: • association role ‘featureOfInterest’ shall be of type SF_SpatialSamplingFeature • association role ‘procedure’ shall be of type Process (from WMO METCE)
  • 38.
    SamplingFeature as OM_Observation.featureOfInterest 38 Inmeteorology, we define a sampling regime that enables us to observe, measure or simulate the real-world. Sampling Features (from ISO 19156 ‘Observations and measurements’) provide a way to characterise this sampling regime and the relationship to the real-world. :ComplexSamplingMeasurement03839:SF_SamplingPoint +featureOfInterest :Point @srsName “EPSG:4326” pos = “50.737 -3.405” +shape 84579:NamedPlace name = “Exeter Airport” +sampledFeature
  • 39.
    National Environmental Information Infrastructure 39|AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
  • 40.
  • 41.
    Metadata Catalog Observing Methods Register Monitoring Sites Register Vocabulary Service Data Service (observations) DataService (geographies) Dataset Site Geography Parameter Instrument / Method Service … … … … … … … … … … … … … … … Integration through linking table Observation observed property feature of interest procedure result sampling feature 41 |AGU Fall 2013 | IN42A-01 | Cox | Leptoukh
  • 42.
    ODM2 Procedure Observed property Feature of Interest result Observation OM_Observation +phenomenonTime + resultTime + validTime [0..1] + resultQuality [0..*] + parameter [0..*] GF_PropertyType GFI_Feature OM_Process Any +observedProperty 1 0..* +featureOfInterest 1 0..* +procedure1 +result 42
  • 43.
    - IGSN - ResourceURI -Registrant ID - timeStamp - status IGSN Registry - IGSN - Registrant - MetadataTimeStamp - Title - Description - SamplingLocation - SamplingTime - Distributor - Originator - SpecimenType - MaterialClass - SamplingMethod SESAR Catalog - IGSN - SampleEvent - SamplePhysicalSize - RelatedResource - SamplingMethodDetails - ProcessingHistory - CurationHistory - More local detail… Allocating Agent CatalogISO & OGC O&M compliant AGU Fall 2013 | IN42A-01 | Cox | Leptoukh 43
  • 44.
    <spec:SF_Specimen gml:id="abc123"> <gml:identifier codeSpace="http://igsn.org">IGSN.SIOabc123</gml:identifier> <sa:sampledFeaturexlink:href="http://example.org/midAtlanticRidge"/> <sa:relatedSamplingFeature> <sa:SamplingFeatureComplex> <sa:role xlink:href="http://example.org/parent"/> <sa:relatedSamplingFeature xlink:href="http://handle.net/10273/IGSN.SIOxyz456"/> </sa:SamplingFeatureComplex> </sa:relatedSamplingFeature> <spec:materialClass xlink:href="http://example.org/rock"/> <spec:samplingTime> <gml:TimeInstant gml:id="tim123"> <gml:timePosition>2009-06-30T08:00:00.00-02:00</gml:timePosition> </gml:TimeInstant> </spec:samplingTime> <spec:samplingMethod xlink:href="http://example.org/ghostbuster"/> <spec:samplingLocation> <gml:Point gml:id="loc123" srsName="http://www.opengis.net/def/crs/EPSG/0/4326"> <gml:pos>45.12 -81.78</gml:pos> </gml:Point> </spec:samplingLocation> <spec:size uom="kg">0.46</spec:size> <spec:currentLocation xlink:href="http://example.org/Warehouse3/shelf9/box67"/> <spec:specimenType xlink:href="http://example.org/splitCore"/> </spec:SF_Specimen> AGU Fall 2013 | IN42A-01 | Cox | Leptoukh IGSN Sample in OMXML
  • 45.
    AGU Fall 2013| IN42A-01 | Cox | Leptoukh <http://handle.net/10273/IGSN.SIOabc123> a sam:Specimen ; rdfs:label "SIO specimen abc123"^^xsd:string ; sam:currentLocation <http://example.org/various/Warehouse3/shelf9/box67> ; sam:materialClass p1:rock ; sam:preparationStep [ sam:processOperator p1:JohnDoe ; sam:processingDetails <http://example.org/various/sf-process/jkl987> ; sam:time <http://handle.net/10273/IGSN.SIOabc123/tim2> ] ; sam:sampledFeature p1:midAtlanticRidge ; sam:samplingFeatureComplex [ sam:relatedSamplingFeature <igsn:SIOxyz456> ; sam:role p1:parent ] ; sam:samplingLocation p1:loc123 ; sam:samplingMethod <http://ldeo.columbia.edu/sampling/ghostbuster> ; sam:samplingTime "2013-06-12T09:25:00.00+11:00"^^xsd:dateTime sam:size [ basic:uom unit:kg ; basic:value "0.46"^^basic:Number ] ; sam:specimenType p1:splitCore . IGSN Sample in RDF
  • 46.
    AGU Fall 2013| IN42A-01 | Cox | Leptoukh class Specimen SF_Specimen + currentLocation :Location [0..1] + materialClass :GenericName + samplingMethod :SF_Process [0..1] + samplingTime :TM_Object + specimenType :GenericName [0..1] «estimatedProperty» + samplingLocation :GM_Object [0..1] + size :Measure [0..1] SF_SamplingFeature + lineage :LI_Lineage [0..1] + parameter :NamedValue [0..*] GFI_Feature SamplingFeatureComplex + role :GenericName PreparationStep + processOperator :CI_ResponsibleParty [0..1] + time :TM_Object SF_Process OM_Observation SF_SamplingFeatureCollection Design +relatedObservation 0..* +processingDetails 0..* Intention +sampledFeature 1..* 0..* +relatedSamplingFeature 0..* +member 1..* Single model underlies different implementations
  • 47.
    Application of O&M •Direct • SOS + OMXML – [Metadata for values from] data services • Specializations – WaterML2, WaterML-WQ, SoilML, SeaDataNet, INSPIRE, WMO ... • Bridging vocabulary for integration • TERENO (GFZ), NEII, ODM2, IGSN/SESAR, OBOE ... • Checklist ... • Slots for implicit  explicit for x-domain use AGU Fall 2013 | IN42A-01 | Cox | Leptoukh47 |
  • 48.
    Modernizing AGU Fall 2013| IN42A-01 | Cox | Leptoukh48 |
  • 49.
    XSD OWL fromUML AGU Fall 2013 | IN42A-01 | Cox | Leptoukh UML Model Output in OWL 49 |
  • 50.
    om:Observation a :Class ; rdfs:label"Observation"@en ; rdfs:subClassOf gf:AnyFeature , h2o:FeatureType ; :disjointWith om:Process ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:result ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:observedProperty ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:featureOfInterest ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:phenomenonTime ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:procedure ] ; rdfs:subClassOf [ a :Restriction ; :cardinality "1"; :onProperty om:resultTime ] . http://def.seegrid.csiro.au/isotc211/iso19156/2011/observation AGU Fall 2013 | IN42A-01 | Cox | Leptoukh50 | O&M in OWL2
  • 51.
    sam:Specimen AGU Fall 2013| IN42A-01 | Cox | Leptoukh51 |
  • 52.
    AGU Fall 2013| IN42A-01 | Cox | Leptoukh OM_Observation + phenomenonTime + resultTime + validTime [0..1] + resultQuality [0..*] + parameter [0..*] GF_PropertyType GFI_Feature OM_Process Any +observedProperty 1 0..* +featureOfInterest 1 0..* +procedure1 +result 52 | SSN ontology
  • 53.
    Alignment AGU Fall 2013| IN42A-01 | Cox | Leptoukh Is Observation a Social Object, or an Event? 53 |
  • 54.
    Lessons AGU Fall 2013| IN42A-01 | Cox | Leptoukh54 |
  • 55.
    Lessons • Ontology forobservations, models and forecasts • X-domain interoperability • Extensible, specialize with vocabularies, link to other ontologies • Standards • A generic model can provide a checklist for design, and a basis for harmonization and incremental design • Consensus introduces an overhead • Change • Common conceptual model supports multiple implementations • Widely used AGU Fall 2013 | IN42A-01 | Cox | Leptoukh55 |
  • 56.
    Additional credits O&M: Fowler& Odell; GeoSciML team; OGC; ISO/TC 211; Rob Atkinson, Rob Woodcock(CSIRO) WaterML: CUAHSI; Gavin Walker, Pete Taylor, Laurent Lefort (CSIRO); Paul Sheahan (BOM) Soil, WQ: Bruce Simons, Peter Wilson, Jonathan Yu (CSIRO); Alistair Ritchie (Landcare NZ) SeaDataNet: Jordi Sorribas, Paolo Diviacco AQD: Michael Lutz, Ale Sarretta (JRC); Kathi Schleidt (Env. Agency Austria) WMO: Jeremy Tandy (UK Met Office); Aaron Braeckel (UCAR) NEII: Andrew Woolf, Dom Lowe (BOM) ODM2: Jeff Horsburgh IGSN: Kerstin Lehnert (LDEO), Jens Klump (GFZ) Vocabulary services: Stuart Williams (Epimorphics), Roy Lowry, Adam Leadbetter (BODC) AGU Fall 2013 | IN42A-01 | Cox | Leptoukh56 |
  • 57.
    LAND AND WATER ThankyouCSIRO Land and Water Simon Cox Research Scientist t +61 3 9252 6342 e simon.cox@csiro.au w www.csiro.au/people/simon.cox