Presentation at the SSN workshop at ISWC 2012 (best paper award)

1. A Linked Sensor Data Cube
for a 100 year homogenised daily temperature dataset
Laurent Lefort
5th Semantic Sensor Network Workshop, 12 November 2012
CSIRO ICT CENTRE

2. Outline
• ACORN-SAT dataset
• Role of SSN ontology
• Role of RDF Data Cube vocabulary
• Integration of SSN and RDF Data Cube
• Lessons learned
• Conclusions
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 2 | dataset | Laurent Lefort

3. The ACORN-SAT dataset
• Released by Aus. Bureau of Meteorology (23 March 2012)
• Available at http://www.bom.gov.au/climate/change/acorn-sat/
• 112 stations in total - 60 from 1910 to 2011
• Homogenised (adjusted) daily temperatures
• Tabular format (1 file per time series/station)
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 3 | dataset | Laurent Lefort

4. The Linked Data version of ACORN-SAT
• Experimental version of ACORN-SAT data
• Available at http://lab.environment.data.gov.au/
• Developed for the Australian Bureau of Meteorology (BOM) by CSIRO in
cooperation with the Australian Government Information Management Office
(AGIMO)
• Temperature (homogenised) plus Rainfall (not homogenised)
• First version presented at Australian GovHack Day
• Alternative to tabular data
• Last version, uploaded to LOD cloud
• http://thedatahub.org/dataset/acorn-sat
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 4 | dataset | Laurent Lefort

5. Motivation: linked gov. agencies data in Australia
• Linked data (and well managed URIs) to build the bridges between
the different agencies
• Current linked data pilot is one agency (BoM) and one server but
applies solutions and schemes already in place in multi-agencies
and multi-service providers context (e.g. UK)
• Thanks to AGIMO for helping us to set up
http://lab.environment.data.gov.au/

6. SSN Ontology
• SSN-XG report http://www.w3.org/2005/Incubator/ssn/XGR-ssn/
• SSN Ontology http://purl.oclc.org/NET/ssnx/ssn
• Navigable documentation on wiki auto derived
http://www.w3.org/2005/Incubator/ssn/wiki/SSN
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 6 | dataset | Laurent Lefort

7. SSN: deployed systems and observations
Device
Skeleton
Deployment
PlatformSite
System
ssn:System
onPlatform
hasSubsystem
deploymentProcesPart
ssn:DeploymentRelatedProcess
hasDeployment
ssn:Deployment
deployedSystem
deployedOnPlatform
ssn:Platform
attachedSystem
ssn:Device
ssn:Sensor
ssn:SensingDevice
observes
inDeployment
observedBy
ssn:Property
observedProperty
ssn:Observation

8. Specific challenges for the SSN Ontology
• ACORN-SAT data derived from multiple stations with complex history
• Uses homogenisation algorithm to make adjustments to raw data
• “Metadata” used by the algorithm to identify “breakpoints” in time series
– Site changes (moves, building or vegetation having an impact on the quality of
observation), sensor (and sensor screens) changes, procedure changes (hours
of observations)
• BoM station numbering system “somewhat confusing over time”
• Desire to retain a single site number for upper-air observations at obs sites
• Several numbering conventions have been used at one or more locations where
an overlap occurs between an old (comparison) and new site:
– Old site retains old number, new site opens with new number.
– Old site switches to new number for the duration of the comparison, new site
takes over old number from the start of its observations.
– New site opens under new number then switches to old number after end of
comparison.
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 8 | dataset | Laurent Lefort

9. Linked ACORN SAT deployment data with SSN
• Data describing the deployment history
• Available in ACORN-SAT station catalogue (pdf)
• Not available in tabular format distribution
• ACORN-SAT composite stations
– System composed of one or several BoM stations
• BoM (Bureau of Meteorology) stations
– System composed of one or several station sharing the same codes
• Textual description of significant events
• Data describing the detailed conditions of observations
• Sensors
• Screens
• Automatic Weather stations
• Procedures e.g. hours of observation
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 9 | dataset | Laurent Lefort

10. Example (Darwin)
Time series – Weather stations – Sites – (Sensors)
Darwin Post Office
014016 (1910-1942)
Darwin Airport
014015 (1941-2007 & 2001-now)
2 sites – 1km apart – same code used
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 10 | dataset | Laurent Lefort

11. Deployment phases in Darwin
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 11 | dataset | Laurent Lefort

12. RDF Data cube http://purl.org/linked-data/cube
• RDF Data Cube: a method to organise linked data in slices
• A vocabulary published by the W3C Government Linked Data (GLD) Working
Group (Working Draft)
• Also the method used to publish statistics data and environmental data in
Europe e.g. for Bathing Water Quality in UK
http://www.epimorphics.com/web/projects/bathing-water-quality
• Advantages
• Allows multiple views on the same data (similar to OLAP)
• Generic approach which supports the links to domain-specific definitions
• Useable:
• In any browser via Linked Data API (HTML output)
• In JavaScript via Linked Data API (JSON output)
• In R via SPARQL
12 | A Linked Sensor Data Cube for a 100 year homogenised daily temperature dataset | Laurent Lefort

13. From: The RDF Data Cube Vocabulary
W3C Working Draft 05 April 2012
http://www.w3.org/TR/vocab-data-cube/
13 | A Linked Sensor Data Cube for a 100 year homogenised daily temperature dataset | Laurent Lefort

14. Data cube, slice and observation
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 14 | dataset | Laurent Lefort
Dimension d7
Dimension d6
Dimension d1
Dimension d2
Dimension d3
Dimension d4
Dimension d5
Measure m1, m2, …
Attribute a1, a2, …
Cube
Slice
Observation

15. QB: Dataset, Slice, Observation
Cube and Slice
qb:Slice
qb:Dataset
slice
Cube observation
observation
qb:Observation
subslice

16. Data Cube Structure:
dimensions, measure, attributes
Current Data Cube structure (and URI/API logic)
Observation
- MinTemperature
- MaxTemperature
- Rainfall
- Booleans for missing data
(2) Year
(3) Month
Day
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 16 | dataset | Laurent Lefort
(1) ACORN-SAT Series/System (station)
• Stations/time series
• Year
• Month
• All linking to observations

17. Slices and URI scheme
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 17 | dataset | Laurent Lefort

18. Coupling SSN and RDF Data Cube
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 18 | dataset | Laurent Lefort

19. Device
Skeleton
acorn-system
Cube observation
observation
structure qb:DataStructureDefinition
component
qb:ComponentSpecification
deployedSystem deploymentProcesPart
bom-station acorn-deploy
hasSubsystem
bom-station:System
acorn-site
raindist
locatedIn
parentADM1
parentFeature, parentCountry
parentADM1, parentADM2
bom-station:Station deployedOnPlatform
acorn-site:Site
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 19 | dataset | Laurent Lefort
acorn-deploy:Deployment
deployment-
ProcessPart
currentSite
acorn-system:System
concept
etcddi
etcddi:xxx
acorn-series:xxx
acorn-series
acorn-series:TimeSeries
acorn-deploy:StandaloneOperation
acorn-sat:Observation
acorn-sat observation
acorn-deploy:PreDeployment
acorn-deploy:PostDeployment
observedBy
acorn-sat:xxx
time:Interval (Instant)
Intervals OWL Time
interval:CalendarInterval (Instant)
raindist:RainfallDistrict
rainstate
rainstate:RainfallState
gn:Feature
geonames
Deployment
System PlatformSite
ssn:System
onPlatform
hasSubsystem
hasDeployment
ssn:DeploymentRelatedProcess
ssn:Deployment
ssn:Platform
deployedOnPlatform
attachedSystem
ssn:Device
ssn:Sensor
ssn:SensingDevice
observes
inDeployment
observedBy
ssn:Property
observedProperty
Cube and Slice
qb:Slice
qb:Dataset
slice
qb:Observation
qb:ComponentProperty
DSD
componentProperty
Dataset
void:Dataset
Concepts
skos:Concept
ssn:Observation

20. Access to data with Elda via
http://lab.environment.data.gov.au/
ssn:hasSubSystem
ssn:hasDeployment
ssn:observedBy ssn:deploymentProcessPart
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 20 | dataset | Laurent Lefort

21. Mashups
• Display the station locations and their average temperature
readings on a map
• http://lab.environment.data.gov.au/mashup/drilldown
• Select a Date range for climate readings for a given location
• http://lab.environment.data.gov.au/mashup
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 21 | dataset | Laurent Lefort

22. Lessons learned
• Flexible URI scheme
• ELDA-friendly, UK-style: using nested list endpoints and item endpoints
– http://lab.environment.data.gov.au/data/acorn/climate/slice/station
– http://lab.environment.data.gov.au/data/acorn/climate/slice/station/014015
• Extra slice(s) easy to add to allow multiple access to the same observations
• RDF Data Cube vocabulary (QB)
• Some clarifications needed for qb:structure, qb:sliceKey, qb:sliceStructure,
qb:component and qb:componentAttachment properties e.g. through the
publication of validation rules
• Coupling of SSN ontology and RDF Data Cube vocabulary
• Different ecosystems (OWL vs. RDF/RDFS)
– OK for RDF Data Cube, not OK for other reused vocabularies e.g. UK Intervals
(Jena Eyeball used for validation)
• Observed properties are classes in the SSN ontology and properties in the RDF
Data Cube
– Possibility to reuse/extend the qb:concept properties defined to manage
references to skos:Concept in QB
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 22 | dataset | Laurent Lefort

23. Conclusions
• Approach is applicable to all climate time series
• Several climate-specific issues not addressed
• Transparency/reproducibility of homogenisation process
– Require raw data plus extra (meta)data (sensors, screen types, sensors
exposure, “qualified” observed properties during a specific observation
interval), plus data used/generated during homogenisation algorithm (ACORN-SAT
uses different values for different value distribution percentiles)
– More ontology work needed (compared to SSN) on homogenisation algorithms
parameters, types of breakpoints and types of adjustment lookup table
• Opportunities to link to other datasets (Australia, World)
• Geo-features (e.g. GeoNames - done) for weather station sites, districts
• Other climate data e.g. regional and world climate data archives, cyclone tracks
(not yet available as linked data)
• Other environmental data (not yet available as linked data)
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 23 | dataset | Laurent Lefort

24. Thank you
Division/Unit Name
Laurent Lefort
Ontologist
t +61 2 9123 4567
e laurent.lefort@csiro.au
w ict.csiro.au
CSIRO ICT CENTRE

25. Images credits
• Blair Trewin The ACORN-SAT station at Butlers Gorge in central
Tasmania (surfacetemperatures.blogspot.com.au )
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 25 | dataset | Laurent Lefort

26. Reused ontologies
Ontology Short Description URL
DOLCE Ultra
Lite (DUL)
A lightweight foundational ontology for
modeling either physical or social contexts
http://www.loa-cnr.
it/ontologies/DUL.owl
Semantic
Sensor Network
An ontology for the description of sensors
and observations, and related concepts.
http://purl.oclc.org/NET/ssnx/ssn
RDF Data Cube
A vocabulary for the publication of multi-dimensional
data as linked data
http://purl.org/linked-data/cube
OWL Time An ontology of temporal concepts http://www.w3.org/2006/time
Intervals
A vocabulary (and URI scheme) for the
definition of instants and intervals.
http://reference.data.gov.uk/def/in
tervals
WGS84_Pos
A vocabulary for representing latitude,
longitude and altitude information in the
WGS84 geodetic reference datum
http://www.w3.org/2003/01/geo/w
gs84_pos
GeoNames
An ontology for the description of
geographical features, their characteristics
and relationships
http://www.geonames.org/ontolog
y/ontology_v3.01.rdf
VoID (Vocabula-ry
of Interlinked
Datasets)
A vocabulary for expressing metadata
about RDF datasets
http://vocab.deri.ie/void
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 26 | dataset | Laurent Lefort

27. Developed ontologies
Ontology Short Description URL
ETCCDI
Indicators defined by the joint
CCl/CLIVAR/JCOMM Expert Team on
Climate Change Detection and Indices
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 27 | dataset | Laurent Lefort
http://purl.oclc.org/NET/ssnx/etccdi
Rainfall
districts and
states
Geographical areas defined as part of the
Bureau's numbering system for observation
sites
http://lab.environment.data.gov.au/
def/stations/raindist
…/rainstate
BoM Station
Definition for the weather stations
registered in the Bureau’s Weather Station
Directory
http://lab.environment.data.gov.au/
def/stations/station
Surface Air
Temperature
ACORN-SAT observation (temperature,
rainfall) for one day
http://lab.environment.data.gov.au/
def/acorn/sat
Time Series
Time series data defined as data cube
slices (aggregated at different levels)
http://lab.environment.data.gov.au/
def/acorn/time-series
ACORN-SAT
deployment
Phases and sub-phases recorded in the
ACORN-SAT documentation pack
http://lab.environment.data.gov.au/
def/acorn/deployment
ACORN-SAT
system
The sensing asset used for a deployment
phases (or sub-phase)
http://lab.environment.data.gov.au/
def/acorn/system
ACORN-SAT
site
The site used for a deployment phase (or
sub-phase)
http://lab.environment.data.gov.au/
def/acorn/site

28. RDF Data Cube (qb:ComponentAttachement)
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 28 | dataset | Laurent Lefort

29. Reference to skos:Concept
A Linked Sensor Data Cube for a 100 year homogenised daily temperature 29 | dataset | Laurent Lefort