Download

1,160 views
1,061 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,160
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Download

  1. 1.
  2. 2. Semantic Sensor Web ARC Research Network on Intelligent Sensors, Sensor Networks and Information Processing – ISSNIP talk Melbourne, August 1, 2008 Amit Sheth LexisNexis Ohio Eminent Scholar Kno.e.sis Center, Wright State University Thanks: Cory Henson and Kno.e.sis Semantic Sensor Web team
  3. 3. <ul><li>Motivating scenario </li></ul><ul><li>Sensor Web Enablement </li></ul><ul><li>Metadata in the domain of Sensors </li></ul><ul><li>Semantic Sensor Web </li></ul><ul><li>Prototyping the Semantic Sensor Web </li></ul>Presentation Outline
  4. 4. High-level Sensor Low-level Sensor <ul><li>How do we determine if the three images depict … </li></ul><ul><ul><li>the same time and same place ? </li></ul></ul><ul><ul><li>same entity ? </li></ul></ul><ul><ul><li>a serious threat ? </li></ul></ul>Motivating Scenario
  5. 5. Collection and analysis of information from heterogeneous multi-layer sensor nodes The Challenge
  6. 6. <ul><li>There is a lack of uniform operations and standard representation for sensor data. </li></ul><ul><li>There exists no means for resource reallocation and resource sharing. </li></ul><ul><li>Deployment and usage of resources is usually tightly coupled with the specific location, application, and devices employed. </li></ul><ul><li>Resulting in a lack of interoperability. </li></ul>Why is this a Challenge?
  7. 7. Interoperability <ul><li>The ability of two or more autonomous, heterogeneous, distributed digital entities to communicate and cooperate among themselves despite differences in language, context, format or content. </li></ul><ul><li>These entities should be able to interact with one another in meaningful ways without special effort by the user – the data producer or consumer – be it human or machine. </li></ul>
  8. 8. Survey <ul><li>GSN (Global Sensor Network, Digital Enterprise Research Institute (DERI), http:// gsn.sourceforge.net / </li></ul><ul><ul><ul><li>Hourglass (Harvard, http:// www.eecs.harvard.edu /~syrah/hourglass/ ) </li></ul></ul></ul><ul><ul><ul><li>An Infrastructure for Connecting Sensor Networks and Applications </li></ul></ul></ul><ul><ul><ul><li>IrisNet (Intel & Carnegie Mellon University, http://www.intel-iris.net / ) </li></ul></ul></ul><ul><ul><ul><li>Internet-Scale Resource-Intensive Sensor Network Service </li></ul></ul></ul>Many diverse sensor data management application frameworks were compared, such as: These application frameworks provided only localized interoperability and that a standards-based framework was necessary. <ul><li>Recent work that does follow key standard (SWE/OGC framework/standards) </li></ul><ul><li>SensorWeb project at University of Melbourne ( http:// www.gridbus.org/sensorweb / ) </li></ul><ul><li>52°North's Sensor Web Community </li></ul><ul><li>NASA JPL/GSFC SersorWeb, Northrop Grumman's PULSENet </li></ul>
  9. 9. The Open Geospatial Consortium Sensor Web Enablement Framework
  10. 10. <ul><li>Consortium of 330+ companies, government agencies, and academic institutes </li></ul><ul><li>Open Standards development by consensus process </li></ul><ul><li>Interoperability Programs provide end-to-end implementation and testing before spec approval </li></ul><ul><li>Develop standard encodings and Web service interfaces </li></ul><ul><li>Sensor Web Enablement </li></ul>OGC Mission To lead in the development, promotion and harmonization of open spatial standards Open Geospatial Consortium
  11. 11. What is Sensor Web Enablement? http://www.opengeospatial.org/projects/groups/sensorweb
  12. 12. <ul><li>An interoperability framework for accessing and utilizing sensors and sensor systems in a space-time context via Internet and Web protocols </li></ul><ul><li>A set of web-based services may be used to maintain a registry of available sensors and observation queries </li></ul><ul><li>The same web technology standard for describing the sensors’ outputs, platforms, locations, and control parameters should be used across applications </li></ul><ul><li>This standard encompasses specifications for interfaces, protocols, and encodings that enable the use of sensor data and services </li></ul>What is Sensor Web Enablement? http://www.opengeospatial.org/projects/groups/sensorweb
  13. 13. <ul><li>Quickly discover sensors (secure or public) that can meet my needs – location, observables, quality, ability to task </li></ul><ul><li>Obtain sensor information in a standard encoding that is understandable by me and my software </li></ul><ul><li>Readily access sensor observations in a common manner, and in a form specific to my needs </li></ul><ul><li>Subscribe to and receive alerts when a sensor measures a particular phenomenon </li></ul>Sensor Web Enablement Desires
  14. 14. Network Services Vast set of users and applications Constellations of heterogeneous sensors Weather Chemical Detectors Biological Detectors Sea State Surveillance Airborne Satellite <ul><li>Distributed self-describing sensors and related services </li></ul><ul><li>Link sensors to network and network-centric services </li></ul><ul><li>Common XML encodings, information models, and metadata for sensors and observations </li></ul><ul><li>Access observation data for value added processing and decision support applications </li></ul>Sensor Web Enablement OGC Sensor Web Enablement http://www.opengeospatial.org/projects/groups/sensorweb
  15. 15. GeographyML (GML) Observations & Measurements (O&M) Information Model for Observations and Sensing Sensor and Processing Description Language Multiplexed, Real Time Streaming Protocol Common Model for Geographical Information SensorML (SML) Sam Bacharach, “GML by OGC to AIXM 5 UGM,” OGC, Feb. 27, 2007. SWE Components - Languages TransducerML (TML)
  16. 16. <ul><li>Sensor Model Language (SensorML) – Standard models and XML Schema for describing sensors systems and processes; provides information needed for discovery of sensors, location of sensor observations, processing of low-level sensor observations, and listing of taskable properties </li></ul><ul><li>Transducer Model Language (TransducerML) – The conceptual model and XML Schema for describing transducers and supporting real-time streaming of data to and from sensor systems </li></ul><ul><li>Observations and Measurements (O&M) – Standard models and XML Schema for encoding observations and measurements from a sensor, both archived and real-time </li></ul>SWE Components - Languages
  17. 17. Access Sensor Description and Data Command and Task Sensor Systems Dispatch Sensor Alerts to registered Users Sam Bacharach, “GML by OGC to AIXM 5 UGM,” OGC, Feb. 27, 2007. SWE Components – Web Services Catalog Service SOS SAS SPS Clients Discover Services, Sensors, Providers, Data Accessible from various types of clients from PDAs and Cell Phones to high end Workstations
  18. 18. <ul><li>Sensor Observation Service (SOS) – Standard Web service interface for requesting, filtering, and retrieving observations and sensor system information. This is the intermediary between a client and an observation repository or near real-time sensor channel </li></ul><ul><li>Sensor Alert Service (SAS) – Standard Web service interface for publishing and subscribing to alerts from sensors </li></ul><ul><li>Sensor Planning Service (SPS) – Standard Web service interface for requesting user-driven acquisitions and observations. This is the intermediary between a client and a sensor collection management environment </li></ul><ul><li>Web Notification Service (WNS) – Standard Web service interface for asynchronous delivery of messages or alerts from SAS and SPS web services and other elements of service workflows </li></ul>SWE Components – Web Services
  19. 19. OGC Catalog Service for the Web (CSW) SWE Components - Dictionaries Sam Bacharach, “GML by OGC to AIXM 5 UGM,” OGC, Feb. 27, 2007. Applications Sensor Types Registry Service Units of Measure Phenomena
  20. 20. Sensor Model Language ( SensorML )
  21. 21. <ul><li>SensorML is an XML schema for defining the geometric, dynamic, and observational characteristics of a sensor </li></ul><ul><li>The purpose of the sensor description: </li></ul><ul><ul><li>provide general sensor information in support of data discovery </li></ul></ul><ul><ul><li>support the processing and analysis of the sensor measurements </li></ul></ul><ul><ul><li>support the geolocation of the measured data. </li></ul></ul><ul><ul><li>provide performance characteristics (e.g. accuracy, threshold, etc.) </li></ul></ul><ul><ul><li>archive fundamental properties and assumptions regarding sensor </li></ul></ul><ul><li>SensorML provides functional model for sensor, not detail description of hardware </li></ul><ul><li>SensorML separates the sensor from its associated platform(s) and target(s) </li></ul>SensorML Overview
  22. 22. <ul><li>Designed to support a wide range of sensors </li></ul><ul><ul><li>Including both dynamic and stationary platforms </li></ul></ul><ul><ul><li>Including both in-situ and remote sensors </li></ul></ul><ul><li>Examples: </li></ul><ul><ul><li>Stationary, in-situ – chemical “ sniffer ” , thermometer, gravity meter </li></ul></ul><ul><ul><li>Stationary, remote – stream velocity profiler, atmospheric profiler, Doppler radar </li></ul></ul><ul><ul><li>Dynamic, in-situ – aircraft mounted ozone “ sniffer ” , GPS unit, dropsonde </li></ul></ul><ul><ul><li>Dynamic, remote – satellite radiometer, airborne camera, soldier-mounted video </li></ul></ul>Scope of SensorML Support
  23. 23. <ul><li>Observation characteristics </li></ul><ul><ul><li>Physical properties measured (e.g. radiometry, temperature, concentration, etc.) </li></ul></ul><ul><ul><li>Quality characteristics (e.g. accuracy, precision) </li></ul></ul><ul><ul><li>Response characteristics (e.g. spectral curve, temporal response, etc.) </li></ul></ul><ul><li>Geometry Characteristics </li></ul><ul><ul><li>Size, shape, spatial weight function (e.g. point spread function) of individual samples </li></ul></ul><ul><ul><li>Geometric and temporal characteristics of sample collections (e.g. scans or arrays) </li></ul></ul><ul><li>Description and Documentation </li></ul><ul><ul><li>Overall information about the sensor </li></ul></ul><ul><ul><li>History and reference information supporting the SensorML document </li></ul></ul>Information provided by SensorML
  24. 24. Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville SML Concepts – Sensor
  25. 25. Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville SML Concepts – Sensor Description
  26. 26. SML Concepts – Accuracy and Range Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville
  27. 27. Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville SML Concepts – Platform
  28. 28. <ul><li>In SensorML, everything is modeled as a Process </li></ul><ul><li>ProcessModel </li></ul><ul><ul><li>defines atomic process modules (detector being one) </li></ul></ul><ul><ul><li>has five sections </li></ul></ul><ul><ul><ul><li>metadata </li></ul></ul></ul><ul><ul><ul><li>inputs, outputs, parameters </li></ul></ul></ul><ul><ul><ul><li>method </li></ul></ul></ul><ul><ul><li>Inputs, outputs, and parameters defined using SWE Common data definitions </li></ul></ul>SML Concepts – Process Model Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville
  29. 29. <ul><li>Process </li></ul><ul><ul><li>defines a process chain </li></ul></ul><ul><ul><li>includes: </li></ul></ul><ul><ul><ul><li>metadata </li></ul></ul></ul><ul><ul><ul><li>inputs, outputs, and parameters </li></ul></ul></ul><ul><ul><ul><li>processes (ProcessModel, Process) </li></ul></ul></ul><ul><ul><ul><li>data sources </li></ul></ul></ul><ul><ul><ul><li>connections between processes and between processes and data </li></ul></ul></ul><ul><li>System </li></ul><ul><ul><li>defines a collection of related processes along with positional information </li></ul></ul>SML Concepts – Process Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville
  30. 30. <ul><li>Metadata is primarily for discovery and assistance, and not typically used within process execution </li></ul><ul><li>Includes </li></ul><ul><ul><li>Identification, classification, description </li></ul></ul><ul><ul><li>Security, legal, and time constraints </li></ul></ul><ul><ul><li>Capabilities and characteristics </li></ul></ul><ul><ul><li>Contacts and documentation </li></ul></ul><ul><ul><li>History </li></ul></ul>SML Concepts – Metadata Group Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville
  31. 31. SML Concepts – Event Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville
  32. 32. An Observation is an Event whose result is an estimate of the value of some Property of the Feature-of-interest , obtained using a specified Procedure The Feature-of-interest concept reconciles remote and in-situ observations Example: Observation Mike Botts, &quot;SensorML and Sensor Web Enablement,&quot; Earth System Science Center, UAB Huntsville
  33. 33. <ul><li>Motivating scenario </li></ul><ul><li>Sensor Web Enablement </li></ul><ul><li>Metadata in the domain of Sensors </li></ul><ul><li>Semantic Sensor Web </li></ul><ul><li>Prototyping the Semantic Sensor Web </li></ul>Presentation Outline
  34. 34. Data Pyramid
  35. 35. Sensor Data Pyramid Raw Sensor (Phenomenological) Data Feature Metadata Entity Metadata Ontology Metadata Expressiveness Data Information Knowledge Data Pyramid
  36. 36. Sensor Data Pyramid Raw Sensor Data Ontology Metadata Entity Metadata Feature Metadata <ul><li>Avalanche of data </li></ul><ul><li>Streaming data </li></ul><ul><li>Multi-modal/level data fusion </li></ul><ul><li>Lack of interoperability </li></ul>(e.g., binary images, streaming video, etc.)
  37. 37. Sensor Data Pyramid Raw Sensor Data Ontology Metadata Entity Metadata Feature Metadata <ul><li>Extract features from data </li></ul><ul><li>Annotate data with feature metadata </li></ul><ul><li>Store and query feature metadata </li></ul>(e.g., lines, color, texture, etc.)
  38. 38. Sensor Data Pyramid Raw Sensor Data Ontology Metadata Entity Metadata Feature Metadata <ul><li>Detect objects-events from features </li></ul><ul><li>Annotate data with objects-event metadata </li></ul><ul><li>Store and query objects-events </li></ul>(e.g., objects and events such as cars driving)
  39. 39. Sensor Data Pyramid Raw Sensor Data Ontology Metadata Entity Metadata Feature Metadata <ul><li>Discover and reason over associations: </li></ul><ul><li>objects and events </li></ul><ul><li>space and time </li></ul><ul><li>provenance/context </li></ul>(e.g., situations such as cars speeding dangerously)
  40. 40. <ul><li>Motivating scenario </li></ul><ul><li>Sensor Web Enablement </li></ul><ul><li>Metadata in the domain of Sensors </li></ul><ul><li>Semantic Sensor Web </li></ul><ul><li>Prototyping the Semantic Sensor Web </li></ul>Presentation Outline
  41. 41. Semantic Sensor Web <ul><li>What is the Semantic Sensor Web? </li></ul><ul><li>Adding semantic annotations to existing standard Sensor Web languages in order to provide semantic descriptions and enhanced access to sensor data </li></ul><ul><li>This is accomplished with model-references to ontology concepts that provide more expressive concept descriptions </li></ul>
  42. 42. Semantic Sensor Web <ul><li>What is the Semantic Sensor Web? </li></ul><ul><li>For example, </li></ul><ul><ul><li>using model-references to link O&M annotated sensor data with concepts within an OWL-Time ontology allows one to provide temporal semantics of sensor data </li></ul></ul><ul><ul><li>using a model reference to annotate sensor device ontology enables uniform/interoperable characterization/descriptions of sensor parameters regardless of different manufactures of the same type of sensor and their respective proprietary data representations/formats </li></ul></ul>
  43. 43. Standards Organizations OGC Sensor Web Enablement <ul><li>SensorML </li></ul><ul><li>O&M </li></ul><ul><li>TransducerML </li></ul><ul><li>GeographyML </li></ul>Web Services <ul><li>Web Services Description Language </li></ul><ul><li>REST </li></ul>National Institute for Standards and Technology <ul><li>Semantic Interoperability Community of Practice </li></ul><ul><li>Sensor Standards Harmonization </li></ul>W3C Semantic Web <ul><li>Resource Description Framework </li></ul><ul><li>RDF Schema </li></ul><ul><li>Web Ontology Language </li></ul><ul><li>Semantic Web Rule Language </li></ul><ul><li>SAWSDL * </li></ul><ul><li>SA-REST </li></ul><ul><li>SML-S </li></ul><ul><li>O&M-S </li></ul><ul><li>TML-S </li></ul>Sensor Ontology Sensor Ontology * SAWSDL - now a W3C Recommendation is based on our work.
  44. 44. Semantic Sensor Web
  45. 45. Semantic Annotation <ul><li>RDFa </li></ul><ul><li>Used for semantically annotating XML documents.  </li></ul><ul><li>Several  important attributes within RDFa include: </li></ul><ul><ul><li>about : describes subject of the RDF triple </li></ul></ul><ul><ul><li>rel : describes the predicate of the RDF triple </li></ul></ul><ul><ul><li>resource : describes the object of the RDF triple </li></ul></ul><ul><ul><li>instanceof : describes the object of the RDF triple with the predicate as “rdf:type” </li></ul></ul><ul><li>Other used Model Reference in Semantic Annotations </li></ul><ul><li>SAWSDL : Defines mechanisms to add semantic annotations to WSDL and XML-Schema components ( W3C Recommendation ) </li></ul><ul><li>SA-REST : Defines mechanisms to add semantic annotations to REST-based Web services. </li></ul>W3C, RDFa, http://www.w3.org/TR/rdfa-syntax/
  46. 46. Semantically Annotated O&M <swe:component name=&quot;time&quot;> <swe:Time definition=&quot;urn:ogc:def:phenomenon:time&quot; uom=&quot;urn:ogc:def:unit:date-time&quot;> <sa:swe rdfa:about=&quot;?time&quot; rdfa:instanceof=&quot;time:Instant&quot;> <sa:sml rdfa:property=&quot;xs:date-time&quot;/> </sa:swe> </swe:Time> </swe:component> <swe:component name=&quot;measured_air_temperature&quot;> <swe:Quantity definition=&quot;urn:ogc:def:phenomenon:temperature“ uom=&quot;urn:ogc:def:unit:fahrenheit&quot;> <sa:swe rdfa:about=&quot;?measured_air_temperature“ rdfa:instanceof=“senso:TemperatureObservation&quot;> <sa:swe rdfa:property=&quot;weather:fahrenheit&quot;/> <sa:swe rdfa:rel=&quot;senso:occurred_when&quot; resource=&quot;?time&quot;/> <sa:swe rdfa:rel=&quot;senso:observed_by&quot; resource=&quot;senso:buckeye_sensor&quot;/> </sa:sml> </swe:Quantity> </swe:component> <swe:value name=“weather-data&quot;> 2008-03-08T05:00:00,29.1 </swe:value>
  47. 47. Semantically Annotated O&M <swe:component name=&quot;time&quot;> <swe:Time definition=&quot;urn:ogc:def:phenomenon:time&quot; uom=&quot;urn:ogc:def:unit:date-time&quot;> <sa:swe rdfa:about=&quot;?time&quot; rdfa:instanceof=&quot;time:Instant&quot;> <sa:sml rdfa:property=&quot;xs:date-time&quot;/> </sa:swe> </swe:Time> </swe:component> <swe:component name=&quot;measured_air_temperature&quot;> <swe:Quantity definition=&quot;urn:ogc:def:phenomenon:temperature“ uom=&quot;urn:ogc:def:unit:fahrenheit&quot;> <sa:swe rdfa:about=&quot;?measured_air_temperature“ rdfa:instanceof=“senso:TemperatureObservation&quot;> <sa:swe rdfa:property=&quot;weather:fahrenheit&quot;/> <sa:swe rdfa:rel=&quot;senso:occurred_when&quot; resource=&quot;?time&quot;/> <sa:swe rdfa:rel=&quot;senso:observed_by&quot; resource=&quot;senso:buckeye_sensor&quot;/> </sa:sml> </swe:Quantity> </swe:component> <swe:value name=“weather-data&quot;> 2008-03-08T05:00:00,29.1 </swe:value>
  48. 48. Semantically Annotated O&M <swe:component name=&quot;time&quot;> <swe:Time definition=&quot;urn:ogc:def:phenomenon:time&quot; uom=&quot;urn:ogc:def:unit:date-time&quot;> <sa:swe rdfa:about=&quot;?time&quot; rdfa:instanceof=&quot;time:Instant&quot;> <sa:sml rdfa:property=&quot;xs:date-time&quot;/> </sa:swe> </swe:Time> </swe:component> <swe:component name=&quot;measured_air_temperature&quot;> <swe:Quantity definition=&quot;urn:ogc:def:phenomenon:temperature“ uom=&quot;urn:ogc:def:unit:fahrenheit&quot;> <sa:swe rdfa:about=&quot;?measured_air_temperature“ rdfa:instanceof=“senso:TemperatureObservation&quot;> <sa:swe rdfa:property=&quot;weather:fahrenheit&quot;/> <sa:swe rdfa:rel=&quot;senso:occurred_when&quot; resource=&quot;?time&quot;/> <sa:swe rdfa:rel=&quot;senso:observed_by&quot; resource=&quot;senso:buckeye_sensor&quot;/> </sa:sml> </swe:Quantity> </swe:component> <swe:value name=“weather-data&quot;> 2008-03-08T05:00:00,29.1 </swe:value> ?time rdf:type time:Instant ?time xs:date-time &quot;2008-03-08T05:00:00&quot; ?measured_air_temperature rdf:type senso:TemperatureObservation ?measured_air_temperature weather:fahrenheit &quot;29.1&quot; ?measured_air_temperature senso:occurred_when ?time ?measured_air_temperature senso:observed_by senso:buckeye_sensor
  49. 49. Semantic Query <ul><li>Semantic Temporal Query </li></ul><ul><li>Model-references from SML to OWL-Time ontology concepts provides the ability to perform semantic temporal queries </li></ul><ul><li>Supported semantic query operators include: </li></ul><ul><ul><li>contains : user-specified interval falls wholly within a sensor reading interval (also called inside ) </li></ul></ul><ul><ul><li>within : sensor reading interval falls wholly within the user-specified interval (inverse of contains or inside ) </li></ul></ul><ul><ul><li>overlaps : user-specified interval overlaps the sensor reading interval </li></ul></ul><ul><li>Example SPARQL query defining the temporal operator ‘within’ </li></ul>
  50. 50. <ul><li>Data </li></ul><ul><li>Raw Phenomenological Data </li></ul>Semantic Sensor Data-to-Knowledge Architecture <ul><li>Information </li></ul><ul><li>Entity Metadata </li></ul><ul><li>Feature Metadata </li></ul><ul><li>Knowledge </li></ul><ul><li>Object-Event Relations </li></ul><ul><li>Spatiotemporal Associations </li></ul><ul><li>Provenance/Context </li></ul>Feature Extraction and Entity Detection Data Storage (Raw Data, XML, RDF) Semantic Analysis and Query Sensor Data Collection Ontologies <ul><li>Space Ontology </li></ul><ul><li>Time Ontology </li></ul><ul><li>Situation Theory Ontology </li></ul><ul><li>Domain Ontology </li></ul>Semantic Annotation
  51. 51. <ul><li>Motivating scenario </li></ul><ul><li>Sensor Web Enablement </li></ul><ul><li>Metadata in the domain of Sensors </li></ul><ul><li>Semantic Sensor Web </li></ul><ul><li>Prototyping the Semantic Sensor Web </li></ul>Presentation Outline
  52. 52. <ul><li>Application 1: Temporal Semantics for Video Sensor Data </li></ul><ul><li>Semantically annotated police cruiser videos collected from YouTube with model references to an OWL-Time ontology </li></ul><ul><li>Enables time-interval based queries, such as contains, within, overlaps </li></ul>Prototyping the Semantic Sensor Web
  53. 53. Storage Query Extraction & Metadata Creation Video Conversion Filtering & OCR SML Annotation Generation Time & Date information SML (XML-DB) Ontology (OWL/RDF-DB) UI SML Interface Ontology Interface AVI OWL-Time Annotation Generation Google Maps GWT (Java to Ajax) Temporal Semantics for Video Sensor Data Data Collection Data Source (e.g., YouTube) Converted Videos
  54. 54. Temporal Semantics for Video Sensor Data <ul><li>Optical Character Recognition (OCR) </li></ul><ul><ul><li>Feature Extraction </li></ul></ul><ul><ul><li>Temporal Entity Recognition </li></ul></ul><ul><ul><li>Metadata Generation & Semantic annotation </li></ul></ul>
  55. 55. Demo: Temporal Semantics for Video Sensor Data Demo: http:// knoesis.wright.edu/library/demos/ssw/prototype.htm
  56. 56. <ul><li>Application 2: Semantic Sensor Observation Service </li></ul><ul><li>Semantically annotated weather data collected from BuckeyeTraffic.org with model references to an OWL-Time ontology, geospatial ontology, and weather ontology </li></ul><ul><li>Capable of multi-level weather queries and inferences on a network of multi-modal sensors </li></ul>Prototyping the Semantic Sensor Web
  57. 57. <ul><li>Ontology & Rules </li></ul><ul><li>Weather </li></ul><ul><li>Time </li></ul><ul><li>Space </li></ul>Oracle SensorDB Get Observation Describe Sensor Semantic Sensor Observation Service Collect Sensor Data BuckeyeTraffic.org Get Capabilities SA-SML Annotation Service S-SOS Client SWE Annotated SWE HTTP-GET Request O&M-S or SML-S Response SOS-S Architecture
  58. 58. SOS-S D ata Collection BuckeyeTraffic, http:// www.buckeyetraffic.org /
  59. 59. Observation Sensor Phenomena Time Location Weather_Condition Temperature Precipitation observed_by measured occurred_when occurred_where described subClassOf subClassOf <ul><li>Key </li></ul><ul><li>Sensor Ontology </li></ul><ul><li>Weather Ontology </li></ul><ul><li>Temporal Ontology </li></ul><ul><li>Geospatial Ontology </li></ul>S-SOS Ontology Concepts …
  60. 60. Icy Blizzard Weather_Condition Wet S-SOS Ontology Concepts Freezing Potentially Icy subClassOf Instances of simple weather conditions created directly from BuckeyeTraffic data Instances of complex weather conditions inferred through rules
  61. 61. <ul><li>Rules allow inferred knowledge from the sensor data </li></ul><ul><li>For example: Based on temperature, wind speed, precipitation, etc., we can infer the “ potential ” road condition the type of storm being observed </li></ul>S-SOS Rules for Weather Conditions Example Potential_Ice_with_Rain_and_Celcius_Temp Observation(?obs) ^ measured(?obs, ?precip) ^ Rain(?precip) ^ measured(?obs, ?temp) ^ Temperature(?temp) ^ temperature_value(?temp, ?tval) ^ lessThanOrEqual(?tval, 0) ^ unit_of_measurement(?temp, “celcius&quot;) -> described(?obs, Potential_Ice)‏ <ul><li>Blizzard </li></ul><ul><li>Potential Ice </li></ul><ul><li>Freezing </li></ul><ul><li>etc. </li></ul>
  62. 62. SOS-S Client Get Observation Describe Sensor Semantic Sensor Observation Service Get Capabilities HTTP-GET Request http://knoesis1.wright.edu/we ather/weather ?service=SOS &version=1.0 &request=GetObservation &offering=WEATHER_DATA &format=application/com-xml &time=2008-03-08T05:00:00Z/2008-03-08T06:00:00Z &interval_type=within &we ather _condition=potentially_icy O&M-S Response <swe:Time definition=&quot;urn:ogc:def:phenomenon:time&quot; uom=&quot;urn:ogc:def:unit:date-time&quot;> <sa:swe rdfa:about=&quot;?time“rdfa:instanceof=&quot;time:Instant&quot;> <sa:sml rdfa:property=&quot;xs:date-time&quot;/> </sa:swe> </swe:Time> <swe:value name=“weather-data&quot;> 2008-03-08T05:00:00,29.1 </swe:value>
  63. 63. SOS-S Client Get Observation Describe Sensor Semantic Sensor Observation Service Get Capabilities HTTP-GET Request http://knoesis1.wright.edu/we ather/weather ?service=SOS &version=1.0 &request=GetObservation &offering=WEATHER_DATA &format=application/com-xml &time=2008-03-08T05:00:00Z/2008-03-08T06:00:00Z &interval_type=within &we ather _condition=potentially_icy O&M-S Response <swe:Time definition=&quot;urn:ogc:def:phenomenon:time&quot; uom=&quot;urn:ogc:def:unit:date-time&quot;> <sa:swe rdfa:about=&quot;?time“rdfa:instanceof=&quot;time:Instant&quot;> <sa:sml rdfa:property=&quot;xs:date-time&quot;/> </sa:swe> </swe:Time> <swe:value name=“weather-data&quot;> 2008-03-08T05:00:00,29.1 </swe:value>
  64. 64. Demo: Semantic Sensor Observation Service Demo: http://knoesis.wright.edu/research/semsci/application_domain/sem_sensor/afrl/demo/ssw.html
  65. 65. Spatial, Temporal, Thematic Analytics within the Semantic Sensor Web
  66. 66. Value to Sensor Networks <ul><li>Simple (Analyze Infrastructure): </li></ul><ul><ul><li>What types of sensors are available? </li></ul></ul><ul><ul><li>What sensors can observe a particular phenomenon at a given geolocation? </li></ul></ul><ul><ul><li>Get all observations for a particular geolocation during a given time interval. </li></ul></ul><ul><li>Complex (More background thematic information): </li></ul><ul><ul><li>How do I detect weather events from observation data? </li></ul></ul><ul><ul><li>What do I know about the buildings (georeferenced) in this image? </li></ul></ul><ul><ul><li>Which sensors cover an area which intersects with a planned event? </li></ul></ul>
  67. 67. Challenges <ul><li>Data Modeling and Querying: </li></ul><ul><ul><li>Thematic relationships can be directly stated but many spatial and temporal relationships (e.g. distance) are implicit and require additional computation </li></ul></ul><ul><ul><li>Temporal properties of paths aren’t known until query execution time … hard to index </li></ul></ul><ul><li>RDFS Inferencing: </li></ul><ul><ul><li>If statements have an associated valid time this must be taken into account when performing inferencing </li></ul></ul><ul><ul><li>(x, rdfs:subClassOf, y) : [1, 4] AND (y, rdfs:subClassOf, z) : [3, 5]  (x, rdfs:subClassOf, z) : [3, 4] </li></ul></ul>
  68. 68. Work to Date <ul><li>Ontology-based model for spatiotemporal data using temporal RDF 1 </li></ul><ul><ul><li>Illustrated benefits in flexibility, extensibility and expressiveness as compared with existing spatiotemporal models used in GIS </li></ul></ul><ul><li>Definition, implementation and evaluation of corresponding query operators using an extensible DBMS (Oracle) 2 </li></ul><ul><ul><li>Created SQL Table Functions which allow SPARQL graph patterns in combination with Spatial and Temporal predicates over Temporal RDF graphs </li></ul></ul><ul><li>Matthew Perry, Farshad Hakimpour, Amit Sheth. &quot;Analyzing Theme, Space and Time: An Ontology-based Approach&quot; , Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006 </li></ul><ul><li>Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. &quot;What, Where and When: Supporting Semantic, Spatial and Temporal Queries in a DBMS&quot; , Kno.e.sis Center Technical Report. KNOESIS-TR-2007-01, April 22, 2007 </li></ul>
  69. 69. Sample STT Query <ul><li>Scenario (Blizzard Detection): Find all sensors that have observed a Blizzard within a 100 mile radius of a given location. </li></ul><ul><li>Query specifies </li></ul><ul><li>a relationship between a sensor, observation, blizzard, and location </li></ul><ul><li>a spatial filtering condition based on the proximity of the sensor and the defined point </li></ul>select * from table (spatial_find( ‘ (?sensor :location ?loc) (?sensor :generatedObservation ?obs) (?obs :featureOfInterest :Blizzard)', ‘loc', 'POINT(-149.40572 61.29302)', 'GEO_DISTANCE(distance=100 unit=mile)‘);
  70. 70. Current Work & Future Demo <ul><li>MesoWest Dataset </li></ul><ul><ul><li>20,000 Sensor Systems predominately within United States </li></ul></ul><ul><ul><li>Archive observation data since April 2002 </li></ul></ul><ul><ul><li>Building dataset of ~1 billion triples </li></ul></ul><ul><li>Trusted Sensors </li></ul><ul><ul><li>Reputation based framework to detect trustworthiness of sensors </li></ul></ul><ul><ul><li>Model-based diagnosis to detect abnormal and/or malicious sensor behavior </li></ul></ul><ul><li>Abductive Perception </li></ul><ul><ul><li>Generating explanations for sensor observations through abductive inference and ranking </li></ul></ul><ul><ul><li>Validating explanations through deductive inference prediction and comparison with subsequent observation data </li></ul></ul>
  71. 71. Future Work <ul><li>Incorporation of spatial ontology in order to include spatial analytics and query (perhaps with OGC GML Ontology or ontology developed by W3C Geospatial Incubator Group - GeoXG) </li></ul><ul><li>Extension with enhanced datasets including MesoWest (Univ. of Utah) and OOSTethys (OGC Oceans IE) </li></ul><ul><li>Trust calculation and analysis over multi-layer sensor networks </li></ul><ul><li>Integration of framework with emergent applications, including video on mobile devices running Android OS </li></ul>
  72. 72. <ul><li>Cory Henson, Amit Sheth, Prateek Jain, Josh Pschorr, Terry Rapoch, “ Video on the Semantic Sensor Web ,” W3C Video on the Web Workshop , December 12-13, 2007, San Jose, CA, and Brussels, Belgium </li></ul><ul><li>Matthew Perry, Amit Sheth, Farshad Hakimpour, Prateek Jain. “ Supporting Complex Thematic, Spatial and Temporal Queries over Semantic Web Data ,” Second International Conference on Geospatial Semantics (GEOS ’07), Mexico City, MX, November 29-30, 2007 </li></ul><ul><li>Matthew Perry, Farshad Hakimpour, Amit Sheth. “ Analyzing Theme, Space and Time: An Ontology-based Approach ,” Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS ’06), Arlington, VA, November 10-11, 2006 </li></ul><ul><li>Farshad Hakimpour, Boanerges Aleman-Meza, Matthew Perry, Amit Sheth. “ Data Processing in Space, Time, and Semantic Dimensions ,” Terra Cognita 2006 – Directions to Geospatial Semantic Web, in conjunction with the Fifth International Semantic Web Conference (ISWC ’06), Athens, GA, November 6, 2006 </li></ul><ul><li>Mike Botts, George Percivall, Carl Reed, John Davidson, “OGC Sensor Web Enablement: Overview and High Level Architecture (OGC 07-165),” Open Geospatial Consortium White Paper , December 28, 2007. </li></ul><ul><li>Open Geospatial Consortium, Sensor Web Enablement WG, http:// www.opengeospatial.org/projects/groups/sensorweb </li></ul>References
  73. 73. Kno.e.sis Labs (3rd floor, Joshi) Bioinformatics Lab (Dr Raymer) Semantic Sciences Lab (Dr Sheth ) Metadata and Languages Lab (Dr Prasad) Semantic Web Lab (Dr Sheth + Dr. S.Wang) Joint Proposals With Each Data Mining Lab (Dr Dong) Service Research Lab (Dr Sheth) Sensor Networking Bin Wang
  74. 74. Kno.e.sis Members – a subset
  75. 75. References <ul><li>Semantic Sensor Web projects: http://knoesis.org/research/semsci/application_domain/sem_sensor/ </li></ul><ul><li>Spatio-temporal-thematic Query Processing & Reasoning: http://knoesis.org/research/semweb/projects/stt/ </li></ul><ul><li>Demos at: http:// knoesis.wright.edu /library/demos/ </li></ul><ul><li>Publications: http:// knoesis.wright.edu /library </li></ul><ul><li>Rest: http:// knoesis.org </li></ul>

×