The document introduces the eReefs Data Brokering Layer, which uses semantics to support data discovery and access for reef monitoring. It describes challenges around discovering and accessing heterogeneous data from different data provider nodes. The Data Brokering Layer addresses this through a common ontology, semantic descriptions of data provider nodes and their services, and domain vocabularies. This allows flexible querying and access to data from various sources. Prototypes demonstrate enhanced data discovery and the potential to facilitate automated data processing. Future work includes expanding its use and demonstrating applications beyond reef monitoring.

1. eReefs Data Brokering Layer
LAND AND WATER FLAGSHIP | OCEANS AND ATMOSPHERE FLAGSHIP
Jonathan Yu | Research Software Engineer
Environmental Information Infrastructures / IBMI / WRM / LWF
12 February 2014
Using semantics to support data discovery & access

2. Outline
1. Background/Use cases - eReefs
2. Data discovery/access challenges
3. Data Provider Node (DPN) concept and semantics
4. Data Brokering Layer (DBL) concept and API
5. Applications

3. eReefs
What is the
health of the
reef?
I want to know
the Chlorophyll-a
concentration
levels in the reef
Where are good
diving sites to
bring people?
Remote sensing
In-situ sensors
(Weather stations
and monitoring sites)
Modelled
data
3 | eReefs Data Brokering Layer | Jonathan Yu

4. Data provider nodes (DPN)
eReefs Data Brokering Layer | Jonathan Yu4 |
Feature
Service
Coverage
Service
Map
service
Geoserver MapInfo THREDDS 52north
Obs
service
?
API
Multiple combinations of services, interfaces, datasets for
each DPN - relies on some cataloguing solutions

5. Complex systems - inefficient, chaos
eReefs Data Brokering Layer | Jonathan Yu5 |
Portal X
Portal Y

6. Current cataloguing solutions and challenges
Strengths
• Harvest capability and keyword search
• Integration with open data catalogs
• Metadata standards (ISO19XXX and
others)
Limitations
• Doesn’t quite capture DPN governance
• Doesn’t resolve semantic heterogeneity
• Doesn’t allow semantics-rich search
and discovery - limited to keywords
eReefs Data Brokering Layer | Jonathan Yu6 |

7. Semantic Heterogeneity
eReefs Data Brokering Layer | Jonathan Yu7 |
Enviro
Application
#1
Data
DB
Chl_MIM
Enviro
Application
#2
Data
DB
mass_conc_
chlorophyll_
In_sea_water
Enviro
Application
#3
Data
DB
mass_conc_
chlorophyll_a_
In_sea_water

8. Semantic Heterogeneity… leads to data silos
eReefs Data Brokering Layer | Jonathan Yu8 |
Enviro
Application
#1
Enviro
Application
#2
Enviro
Application
#3
Data Data Data
DB DBDB
Chl_MIM
mass_conc_
chlorophyll_
In_sea_water
mass_conc_
chlorophyll_a_
In_sea_water
X X
Meetings

9. Harmonised Publish, Discovery, Access and Use
eReefs Data Brokering Layer | Jonathan Yu9 |
Relies on community
agreed vocabularies
Describe/Publish
the data
Query/Use data
Enviro
Application
#1
Enviro
Application
#2
Enviro
Application
#3
Data Data Data
DB DBDB
substanceOrTaxon=
http://environment.data.gov.au/def/object/chlorophyll
scaledQuantityKind =
http://environment.data.gov.au/def/property/chlorophyll_
concentration
Need to communicate more consistently
Requires shared, precise, agreed semantics

10. eReefs Data Brokering Layer (DBL)
eReefs Data Brokering Layer | Jonathan Yu10 |
Data
Brokering
Layer
End users
eReefs
Portal
Remote sensing DPN
DPN
Ontology
Domain
Vocabs
e.g. Water
Quality
Hydrodynamic DPN
In-situ monitoring DPN
Catalogue
APIS
CKAN,
pyCSW,
ERDDAP
Flexible, ‘pluggable’ DPNs, data
services and datasets
Flexible discovery and delivery of data

11. DPN Semantic Description Example
eReefs Data Brokering Layer | Jonathan Yu11 |
dpn:Serviceorg:Organization dpn:Node
dpn:THREDDS
OC THREDDS
Service
Ocean Colour
DPN
CSIRO Ocean
Colour Group
rdfs:subClassOf
rdf:type rdf:typerdf:type
DPN ontology
http://purl.org/dpn
DPN description
instance

12. Data provider nodes (DPN)
eReefs Data Brokering Layer | Jonathan Yu12 |
Feature
Service
Coverage
Service
Map
service
Geoserver MapInfo THREDDS 52north
Obs
service
DPN
Service
Description
Lightweight RDF
description

13. DPN Semantic Description Example
eReefs Data Brokering Layer | Jonathan Yu13 |
dpn:Serviceorg:Organization dpn:Node
dpn:THREDDS
OC THREDDS
Service
Ocean Colour
DPN
CSIRO Ocean
Colour Group
rdfs:subClassOf
rdf:type rdf:typerdf:type
THREDDS
NCI deployment
Data Brokering
Layer
DPN
registered
with DBL

14. Supporting domain vocabularies and def’s
eReefs Data Brokering Layer | Jonathan Yu14 |
http://environment.data.gov.au/water/quality/def/op
Observable Property ontology

15. Example: Domain vocab term
eReefs Data Brokering Layer | Jonathan Yu15 |

16. Example: Domain vocab term
eReefs Data Brokering Layer | Jonathan Yu16 |
...
wqp:chlorophyll_a_concentration
a skos:Concept, op:ScaledQuantityKind,
qudt:ChemistryQuantityKind ;
skos:broader wqp:chlorophyll_concentration ;
skos:prefLabel "chlorophyll a concentration"@en ;

17. Searches over vocabularies
eReefs Data Brokering Layer | Jonathan Yu17 |
http://sissvoc.ereefs.info/search/?q=chlo
rophyll&endpoint=http://sissvoc.ereefs.i
nfo/sissvoc/ereefs

18. Data annotated with bindings to vocab URIs
eReefs Data Brokering Layer | Jonathan Yu18 |
THREDDS
THREDDS
Catalog
Domain Vocabs
(Water Quality at
environment.data.gov.au)
Quantities/ Units ontology
(QUDT)
substanceOrTaxon=
http://environment.data.gov.au
/def/object/chlorophyll
scaledQuantityKind
= http://environment.data.gov.au
/def/property/chlorophyll_concentra
tion
unit
= http://qudt.org/vocab/unit#Unitless
medium
= http://environment.data.gov.au
/def/feature/ocean

19. DBL Harvesting and End Use
eReefs Data Brokering Layer | Jonathan Yu19 |
Data Brokering
Layer
THREDDS
THREDDS
Catalog
Domain Vocabs
(Water Quality at
environment.data.gov.au)
Quantities/ Units ontology
(QUDT)
substanceOrTaxon=
http://environment.data.gov.au
/def/object/chlorophyll
scaledQuantityKind
= http://environment.data.gov.au
/def/property/chlorophyll_concentra
tion
unit
= http://qudt.org/vocab/unit#Unitless
medium
= http://environment.data.gov.au
/def/feature/ocean
DPN
Service
Description
End users
Client
application
chlorophyll

20. eReefs Visualisation Portal Prototype
 http://vizportal.meteor.com
eReefs Data Brokering Layer | Jonathan Yu20 |

21. eReefs Data Brokering Layer | Jonathan Yu21 |

22. eReefs Data Brokering Layer | Jonathan Yu22 |

23. eReefs Data Brokering Layer | Jonathan Yu23 |

24. eReefs Data Brokering Layer | Jonathan Yu27 |

25. Benefits
eReefs Data Brokering Layer | Jonathan Yu35 |
Data Provider X
Data
Brokering
Layer
DPN
Ontology
Domain
Vocabs
e.g. Water
Quality
Catalogue
APIS
CKAN,
pyCSW,
ERDDAP
End users
eReefs
Portal
Data Provider Y
Flexible, ‘pluggable’ DPNs, data
services and datasets
Allows flexible discovery and delivery of data

26. Benefits
 Decoupled system allows for expanding components in isolation:
• DPN set
• Vocabulary definitions
 Enhanced data discovery
 Aid operational monitoring of DPN services
 Potential to facilitate data assimilation and automated data
processing
eReefs Data Brokering Layer | Jonathan Yu36 |

27. Future/Current Work
1. Test whether this the best approach for data services and
dataset discovery
- Compare/integrate with CKAN, Geonetwork, pyCSW, ERDDAP
2. Explore richer queries using semantics of the DPN metadata,
vocabularies and services
3. Use DBL for data assimilation – hydrodynamic models with
remote sensing
4. Demonstrate DBL applications beyond eReefs
eReefs Data Brokering Layer | Jonathan Yu37 |

28. Summary
• The ‘Data Provider Node’ concept provides a way of thinking
about data ownership, publication, implementation and
governance
• Semantic web technologies allow for machine-readable DPN
descriptions precisely and richly
• The ‘Data Brokering Layer’ concept uses the domain and DPN
semantics to provide:
1. Flexible and ‘pluggable’ DPNs - data services and datasets
2. Allows flexible discovery and use of data
• Prototypes and demos via client applications
• Future work
eReefs Data Brokering Layer | Jonathan Yu38 |

29. LAND AND WATER
Thank you
Land and Water
Jonathan Yu
Research Software Engineer
t +61 3 9252 6440
e jonathan.yu@csiro.au
Land and Water
Ben Leighton
Software Engineer
t +61 3 9252 6516
e ben.leighton@csiro.au
Land and Water
Shane Seaton
Software Engineer
t +61 2 6246 5825
e shane.seaton@csiro.au
More info:
https://wiki.csiro.au/display/ER/Data+Brokering+Layer
https://wiki.csiro.au/display/ER/Semantic+service+descriptions
https://wiki.csiro.au/display/ER/Semantic+descriptions+of+datasets