Sensor metadata management with SWM (SMWCon fall 2013)


Published on

Towards a collaborative metadata management system for sensor networks using SMW. Steps taken and lessons learned

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sensor metadata management with SWM (SMWCon fall 2013)

  1. 1. Effective metadata management of sensor networks using SMW Jan Willem Noteboom 29 October 2013 SWMCon Fall 2013 | 29 October 2
  2. 2. Outline Need for effective metadata management Towards a collaborative environment Experiences with Semantic MediaWiki (SMW) 2
  3. 3. Need for effective metadata management Situation • sensor networks more popular, bigger and complex • access to many metadata sources needed for interpretation • consistency of metadata sources is demanding Needs • integrated access to all sensor network metadata • linked sources of sensor network metadata • collaborative management of metadata • interfaces for scientists, field inspectors and applications • using international standards and open technology 3
  4. 4. Need for effective metadata management What metadata Collection & Discovery metadata Measurement Data info Metadata in scope Interpretation metadata Measurement System info Provenance & Lineage metadata 4 Measurement Process info Platform & instrument metadata Phenomena properties (dynamic nature) Measuring characteristics (“static” nature)
  5. 5. Need for effective metadata management What services Discovery, View and Retrieval of sensor network information Discovery Annotation of sensor network information Notification about sensor network information updates Update of sensor network information Administration of users, information and components Users & Applications Content managers Service managers Update Administration Services Discovery View Retrieval Annotation Notification (linked) Sensor Network Metadata (processes, systems) 5
  6. 6. Towards a collaborative environment Phase I: Inventory & Analysis collaborative data model for sensor network metadata collaborative platform/architectuur Phase II: Definition & PoC & Migration definition of requirements, architecture and data model proof of concept (PoC) using SMW data migration toolkit Phase III: Realization, Cleansing & Operationalization Realization of version 1.0 Migration and cleansing of migrated data (automated and manually) Implementation in the operational environment 6
  7. 7. Towards a collaborative environment (meta)Data model Property-oriented (linked data) Linked data with time validation and status KNMI terminology and taxonomy “Based on” international standards (ISO 19115, SensorML) Systems and processes Annotations (commentary) Events of several types Links to media (docs, pictures) and people/organisations 7 Example: Element T(temperature) is measured according to measurement process P using instrument I that is deployed at location L in field F of station S (managed by person P) and part of network N
  8. 8. Towards a collaborative environment Collaborative platform Discovery View Notification Retrieval Update Annotation Administration Repository Linked Sensor Data (systems, processes) Migration of metadata Metadata sources 8 Internal sources External sources Existing Services Other collaborative environments MetaData layer Services External MetaData (Geo)Portal Service layer Appl layer Applications, Users Data managers
  9. 9. Experiences with Semantic MediaWiki Context: MW 1.21.2, SMW 1.8, 30 extensions, no SPARQL endpoint, 23410 pages, 25(50) namespaces, 190 properties (611,171 property values), 150 templates, 25 forms, 12 categories (classes)…. Experiences: In general Data modelling or structuring Data discovery (searching/browsing) Data view & retrieval (displaying/encoding/querying) Data update (creating/editing/transferring) Data annotation (commenting) Data notification (watching) Administration (user management, access control) 9
  10. 10. Experiences with Semantic MediaWiki General Well-known environment to most users Powerful collaborative environment Steep learning curve Many useful extensions Documentation is getting better (hint: “Working with Mediawiki” Yaron Korn) Note Differences in the quality of extensions (error messaging, documentation, governance) 10
  11. 11. Experiences with Semantic MediaWiki Data structure Easy implementation of the data model using properties, templates, categories and pages Usage of several namespaces Usage of semantic internal objects to add time validation to links (property qualifier) Note Import of ontologies (foaf, skos, ssn, dc, etc) considered but not applied. 11
  12. 12. Experiences with Semantic MediaWiki Data discovery Searching/Browsing: Query forms via site-bar menu very flexible filtering and preloadable tabs for standard vs advance filters no linkage of filters… Standard search/browse Semantic maps & timelines Note No use of drilldown for performance reasons 12
  13. 13. Experiences with Semantic MediaWiki Data View & Retrieval Displaying/encoding/query: Pages with Infoboxes Several Result Formats and custom templates Collapsible sections/infoboxes Custom templates to construct “complex” ask queries (increased $smwgQMaxDepth, $maxRecursionDepth) Concepts to ease retrieval Notes Restricting “view” access of page parts with Userfunctions (weak). 13
  14. 14. Experiences with Semantic MediaWiki Data update & edit Creating/editing/transferring: Namespaces and Usergroups to restrict (create/edit) access Forms with Headertabs Formlink buttons for creation Several form input types (e.g. combobox, two listboxes) • Data transfer for data migration (e.g. xml import) Note Forms not well suited to handle of many instances (>50) of a multiple instance template (e.g. internal objects/subobjects) 14
  15. 15. Experiences with Semantic MediaWiki Data Annotation Page commenting Notes Events permanent temporary (period) planned/unplanned Attachments Talk (discussion) Note Clear policy needed for event registration 15
  16. 16. Experiences with Semantic MediaWiki Data Notification Watchlist (pages) Notification (Echo) Note Need for summary reports of changes for dedicated user groups…. 16
  17. 17. Experiences with Semantic MediaWiki Data Administration Special page utilities User management Data housekeeping Statistics Maintenance scripts Daily backup Rebuilding MW Refreshing SMW 17
  18. 18. To summarize Semantic Mediawiki (SWM) is a powerful collaborative environment to manage sensor network metadata effectively. Notes/Needs Extensions differs in quality (error handling, documentation, governance). Documentation has improved Need for forms that can handle many instances (>50) of a multiple instance template Need for summary reporting of changes (dedicated for user groups) 18
  19. 19. Many thanks for your attention! 19