Data Processing and Semantics for AdvancedInternet of Things (IoT) Applications:modeling, annotation, integration, and per...
2Part 1: Introductionto Internet of “Things”Image source: CISCO
Internet of Things “sensors and actuators embedded in physicalobjects — from containers to pacemakers —are linked through...
“Thing” connected to the internet“In 2013 cumulative shipments of Bluetooth-enabled devices will surpass 10 billionand Wi-...
5Network connected Things andDevicesImage courtesy: CISCO
6Sensor devices are becoming widelyavailable- Programmable devices- Off-the-shelf gadgets/tools
7More “Things” are being connectedHome/daily-life devicesBusiness andPublic infrastructureHealth-care…
8People Connecting to ThingsMotion sensorMotion sensorMotion sensorECG sensorInternet
9Things Connecting to Things- Complex and heterogeneousresources and networks
10Wireless Sensor Networks (WSN)Sinknode GatewayCore networke.g. InternetGatewayEnd-userComputer services- The networks ty...
11Key characteristics of IoT devices Often inexpensive sensors (actuators) equipped with a radiotransceiver for various a...
12Beyond conventional sensors Human as a sensor (citizen sensors) e.g. tweeting real world data and/or events Virtual (...
13Cyber, Physical and Social Data
14Citizen SensorsSource: How Crisis Mapping Saved Lives in Haiti, Ushahidi Haiti Project (UHP).
15Cosm- Air Quality Egg
16Cosm- data readingsTagsData formatsLocation
17Making Sense of DataIn the next few years, sensor networks will produce10-20 time the amount of data generated by social...
18Things, Data, and lots of itimage courtesy: Smarter Data - I.03_C by Gwen Vanhee
19Big Data and IoT "Big data" is a term applied to data sets whose size is beyond the ability ofcommonly used software to...
20The seduction of data Turn 12 terabytes of Tweets created each day into sentiment analysisrelated to different events/o...
21Do we need all these data?
22“Raw data is both an oxymoron and baddata”Geoff Bowker, 2005ource: Kate Crawford, "Algorithmic Illusions: Hidden Biases ...
23IoT Data in the CloudImage courtesy: http://images.mathrubhumi.comhttp://www.anacostiaws.org/userfiles/image/Blog-Photos...
24Perceptions and IntelligenceDataInformationKnowledgeWisdomRaw sensory dataStructured data (withsemantics)Abstraction and...
25Change in communication paradigmSinknode GatewayCore networke.g. Internet End-userDataSenderDataReceiverA sample data co...
26 Collaboration and in-network processing In some applications a single sensor node is not able to handle thegiven task...
“People want answers, not numbers”(Steven Glaser, UC Berkley)Sinknode GatewayCore networke.g. InternetWhat is the temperat...
28IoT Data alone is not enough Domain knowledge Machine interpretable meta data Delivery, sharing and representation se...
29Storing, Handling and Processingthe DataImage courtesy: IEEE Spectrum
30IoT Data Challenges Discovery: finding appropriate device and data sources Access: Availability and (open) access to I...
Energy consumption of the nodes Batteries have small capacity and rechargingcould be complex (if not impossible) in somec...
ActuatorsStepper Motor [1]Image credits:[1] http://directory.ac/telco-motion.html[2] http://bruce.pennypacker.org/category...
Wireless Sensor Networks (WSN)-gateway connection
Distributed WSN
What are the main issues? Heterogeneity Interoperability Mobility Energy efficiency Scalability Security
What is important? Robustness Quality of Service Scalability Seamless integration Security, privacy, Trust
In-network processing Mobile Ad-hoc Networks are supposed to deliverbits from one end to the other WSNs, on the other en...
In-network processing- exampleApplying Symbolic Aggregate Approximation (SAX)SAX Pattern (blue) with word length of 20 and...
Data-centric networking In typical networks (including ad hoc networks),network transactions are addressed to the identit...
Implementation options fordata-centric networking Overlay networks & distributed hash tables (DHT) Hash table: content-a...
IoT and Semantic technologies The sensors (and in general “Things”) are increasinglybeing integrated into the Internet/We...
Semantics and IoT resources anddata Semantics are machine-interpretable metadata (for mark-up),logical inference mechanis...
43A Few WordsonSemantic Web
SSW Introductionlivesinhaspetis ahas petPersonPersonAnimalAnimalConcrete FactsResource Description FrameworkConcrete Facts...
Semantic Web Stack
Linked Open Data
Linked Open Data~ 50 Billion Statements~ 50 Billion Statements
SW is moving from academiato industry
In the last few years, we haveseen many successes …Knowledge GraphWatsonAppleSiri
Google Knowledge Graph
51Sensors and the Web
Sensors are ubiquitous
Enabling the Internet of ThingsSituational awarenessenables: Devices/things to functionand adapt within theirenvironment...
Sensor systems aretoo often stovepiped.Closed centralizedmanagement of sensingresourcesClosed inaccessibledata and sensors
We want to set this data freeWith freedom comesresponsibilityDiscovery, access, and searchIntegration and interpretation...
Drowning in DataA cross-country flight from New York to Los Angeles on aBoeing 737 plane generates a massive 240 terabytes...
Drowning in Data
ChallengesTo fulfill this vision, there are difficult challenges to overcome suchas the discovery, access, search, integra...
SolutionSemantic Sensor WebInternet Computing, July/Aug.2008Uses the Web as platform formanaging sensor resources anddat...
SolutionDiscovery, access, and search Using standard Web services OGC Sensor Web Enablement
SolutionIntegration Using shared domain models / datarepresentation OGC Sensor Web Enablement W3C Semantic Sensor Netwo...
SolutionInterpretation Abstraction – converting low-level data to high-level knowledge Machine Perception – w/ prior kno...
SolutionScalability Data overload – sensors produce too much data Computational complexity of semantic interpretation “...
SSW Adoption and Applications
65Part 2: Data and knowledgemodelling requirements,semantic annotationImage source: CISCO
Recall of the Internet of Things A primary goal of interconnecting devices andcollecting/processing data from them is toc...
IoT challenges Numbers of devices and different users and interactions required. Challenge: Scalability Heterogeneity o...
IoT: one paradigm, many visionsDiagram adapted from L. Atzori et al., 2010, “the Internet of Things: a Survey”
Semantic oriented vision “The object unique addressing and the representationand storing of the exchanged information bec...
What is expected? Unified access to data: unified descriptions Deriving additional knowledge (data mining) Reasoning su...
Semantic technologies and IoT There are already Sensor Web Enablement(SWE) standards developed by the OpenGeospatial Cons...
Identify IoT domain concepts Users Physical entities Virtual entities Devices Resource Services …Diagram adapted fr...
IoT domain concepts - Entity Physical entities (or entity ofinterests): objects in the physical world,features of interes...
IoT domain concepts –Device, Resource and Service A Device mediates the interactions between usersand entities. The soft...
Other concepts need to considered Gateways Directories Platforms Systems Subsystems … Relationships among them And...
Don’t forget the IoT data Sensors and devices provide observation andmeasurement data about the physical world objectswhi...
Semantics for IoT resources and data Semantics are machine-interpretable metadata, logical inferencemechanisms, query and...
Characteristics of IoT resources Extraordinarily large number Limited computing capabilities Limited memory Resource c...
Characteristics of IoT data Stream data (depends on time) Transient nature Almost always related to a phenomenon orqual...
Utilise semantics Find all available resources (which can providedata) and data related to “Room A” (which is anobject in...
81Part 3: Semantics and data modellingfor IoTImage source: CISCO
Semantic modelling Lightweight: experiences show that a lightweightontology model that well balances expressiveness andin...
Existing models for resources and data W3C Semantic Sensor Network IncubatorGroup’s SSN ontology (mainly for sensorsand s...
Existing models for services OWL-S and WSMO are heavy weight models: practicaluse? Minimal service model Deprecated Pr...
W3C’S SSN ontologyDiagram adapted from SSN report
Some existing IoT models andontologies FP7 IoT-A project’s Entity-Resource-Serviceontology A set of ontologies for entit...
IoT-A resource modelDiagram adapted from IoT-A project D2.1
IoT-A resource descriptionDiagram adapted from IoT-A project D2.1
IoT-A service modelDiagram adapted from IoT-A project D2.1
IoT-A service descriptionDiagram adapted from IoT-A project D2.1
Service modelling in IoT.estDiagrams adapted from Iot.est D3.1
IoT.est service profile highlight ServiceType class represents the servicetechnologies: RESTful and SOAP/WSDL services. ...
93Part 4: Linked-data and semanticenabled systemsImage source: CISCO
Linked data principles using URI’s as names for things: Everything isaddressed using unique URI’s. using HTTP URI’s to e...
Linked data in IoT Using URI’s as names for things;- URI’s for naming IoT resources and data (and also streamingdata); U...
Linked data layer for not only IoT…Images from Stefan Decker, http://fi-ghent.fi-week.eu/files/2010/10/Linked-Data-scheme1...
Creating and using linked sensor datahttp://ccsriottb3.ee.surrey.ac.uk:8080/IOTA/
Sensor discovery using linked sensordata
Semantics in IoT - reality If we create an Ontology our data is interoperable Reality: there are/could be a number of on...
100Part 5: Semantic Sensor WebandPerceptionImage source: semanticweb.com; CISCO
What is the Sensor Web? Sensor Web is an additional layer connecting sensornetworks to the World Wide Web. Enables an in...
Why is the Sensor Web important? In general Enable tight coupling of the cyber andphysical world In relation to IoT En...
Bridging the Cyber-Physical DividePsyleron’s Mind-Lamp (Princeton U),connections between the mind and thephysical world.Ne...
Bridging the Cyber-Physical DivideFoursquare is an online application whichintegrates a persons physical location andsocia...
Bridging the Cyber-Physical DivideTweeting Sensorssensors are becoming social
How do we design the Sensor Web? Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Se...
OGC Sensor Web Enablement
Role of OGC SWE
Vision of Sensor Web Quickly discover sensors (secure or public) that can meetmy needs – location, observables, quality, ...
Principles of Sensor Web Sensors will be web accessible Sensors and sensor data will be discoverable Sensors will be se...
OGC SWE Services Sensor Observation Service (SOS) access sensor information (SensorML) and sensorobservations (O&M Sens...
OGC SWE Services
OGC SWE Languages Sensor Model Language (SensorML) Models and schema for describing sensorcharacteristics Observation &...
OCG SWE Observation
Semantic Sensor WebRDF OWLOGC Sensor WebEnablement
Sensor Web + Semantic WebSemantic WebThe web of data where web content is processedby machines, with human actors at the ...
So, what is a Semantic Sensor Web? Reduce the difficulty and open up sensor networks by: Allowing high-level specificati...
W3C SSN Incubator Group SSN-XG commenced: 1 March 2009 Chairs: Amit Sheth, Kno.e.sis Center, Wright State University K...
W3C SSN Incubator GroupTwo main objectives:The development of an ontology for describingsensing resources and data, andT...
Sensor Standards Landscape
SSN Ontology OWL 2 DL ontology Authored by the XGparticipants Edited by MichaelCompton Driven by Use Cases Terminolog...
SSN Use Cases
SSN Use Cases
SSN Ontology
Stimulus-Sensor-Observation The SSO Ontology Design Pattern is developed following the principle ofminimal ontological co...
SSN Ontology Modules
SSN Ontology Modules
SSN Sensor A sensor can do (implements) sensing: that is, a sensor is any entity that canfollow a sensing method and thus...
SSN Measurement Capability Collects together measurement properties (accuracy, range, precision, etc) andthe environmenta...
SSN Observation An Observation is a Situation in which a Sensing method has been used to estimate orcalculate a value of ...
Alignment with DOLCE
What SSN does not model Sensor types and models Networks: communication, topology Representation of data and units of m...
Semantic Annotation of SWERecommendedtechnique via Xlinkattributes requires nochange to SWExlink:href - link toontology i...
How do we design the Sensor Web? Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Se...
136Interpretation of data A primary goal of interconnecting devices andcollecting/processing data from them is tocreate s...
137Observation and measurement dataSource: W3C Semantic Sensor Networks, SSN Ontology presentation, Laurent Lefort et al.
138How to say what a sensor is andwhat it measures?SinknodeGateway
139Data/Service description frameworks There are standards such as Sensor Web Enablement (SWE) setdeveloped by the Open G...
140Sensor Markup Language (SensorML)Source: http://www.mitre.org/
141W3C SSN Ontologymakes observationsof this typeWhere it isWhat itmeasuresunitsSSN-XG ontologiesSSN-XG annotationsSSN-XG ...
142Semantics and IoT data Creating ontologies and defining data models is not enough tools to create and annotate data ...
143Semantics and sensor dataSource: W. Wang, P. Barnaghi, "Semantic Annotation and Reasoning for Sensor Data", In proceedi...
144Semantics and Linked-data The principles in designing the linked data aredefined as: using URI’s as names for things;...
145Linked Sensor data
146Myth and reality #1: If we create an Ontology our data is interoperable Reality: there are/could be a number of ontol...
147Processing Streaming Sensor Data
148148Symbolic Aggregate Approximation(SAX)Variable String Length and Vocabulary size.Length: 10, VocSize: 10 Length: 10, ...
149SAX representationSAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbolsover the original sensor time...
150Data Processing Frameworkfggfffhfffffgjhghfff dddfffffffffffddd cccddddccccdddccc aaaacccaaaaaaaaccccdddcdcdcdcddasdddP...
151SensorSAXF. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 20...
152Evaluation results of abstractioncreationF. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Re...
153Data size reductionF. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data...
154Enabling the Internet of Things- Diversity range of applications- Interacting with large number ofdevices with various ...
155Challenges and opportunitiesProviding infrastructure Publishing, sharing, and access solutions on a global scale Ind...
156Part 6: Cognitive aspects ofknowledge representation, reasoningand perceptionImage source: CISCO
Abstraction provides the ability to interpret and synthesize informationin a way that affords effective understanding and ...
 People are excellent at abstraction;of sensing and interpreting stimulito understand and interact with theworld. The pr...
observe perceiveconceptualizationof “real-world”“real-world”Abstraction
Semantic Perception/AbstractionFundamental QuestionsWhat is perception, and how canwe design machines to perceive?What c...
What is Perception?Perception is the act of Abstracting Explaining Discriminating Choosing
What can we learn fromCognitive Models ofPerception? A-priori background knowledge is a keyenabler Perception is a cycli...
Is Semantic Web up to the taskof modeling perception?RepresentationHeterogeneous sensors, sensing, and observationrecords...
Both people and machines are capable of observingqualities, such as redness.* Formally described in a sensor/ontology (SSN...
The ability to perceive is afforded through the use ofbackground knowledge, relating observable qualities toentities in th...
With the help of sophisticated inference, both people andmachines are also capable of perceiving entities, such asapples....
Perceptual Inferenceminimizeexplanationsdegrade gracefullytractableAbductive Logic (e.g.,PCT)high complexityDeductive Logi...
The ability to perceive efficiently is afforded through thecyclical exchange of information between observers andperceiver...
Neisser’s Perceptual Cycle
 1970’s – Perception is an active, cyclical process ofexploration and interpretation.- Nessier’s Perception Cycle 1980’s...
Key InsightsBackground knowledge plays a crucial role in perception; what weknow (or think we know/believe) influences ou...
observesinheres inIntegrated together, we have an general model – capable ofabstraction – relating observers, perceivers, ...
 Ontology of Perception – as an extension of SSN Provides abstraction of sensor data through perceptualinference of sema...
Prior KnowledgeW3C SSN Ontology Bi-partite Graph Prior knowledge conformant to SSN ontology (left),structured as a bipart...
Explanation is the act of accounting for sensory observations (i.e.,abstraction); often referred to as hypothesis building...
ExampleAssume the properties elevated blood pressure andpalpitations have been observed, and encoded in RDF(conformant wi...
Discrimination is the act of deciding how to narrow down the multitude ofexplanatory features through further observation....
ExampleGiven the explanatory features from the previousexample, Hypertension and Hyperthyroidism, thefollowing classes ar...
How do we design the Sensor Web? Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Se...
Efficient Algorithms for IntellegO Use of OWL-DL reasoner too resource-intensive for use inresource constrained devices (...
Efficient Algorithms for IntellegOSemantic (RDF)EncodingBit Vector EncodingLowerLift First, developed lifting andlowering...
Efficient Algorithms for IntellegOExplanation AlgorithmDiscrimination AlgorithmUtilize bit vector operators to efficiently...
Efficient Algorithms for IntellegOEvaluation: The bit vector encodings and algorithms yield significant andnecessary compu...
Adoption of SSN
185Part 7: Physical-Cyber-SocialComputingImage source: CISCO
186J. McCarthyJ. McCarthy M. WeiserM. WeiserD.EngelbartD.Engelbart J. C. R. LickliderJ. C. R. LickliderSimilar Visions of ...
187If people do not believe that mathematics is simple, it is onlybecause they do not realize how complicated life is.- Jo...
What has changed now?Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, S...
189What has not changed?What has not changed?We need computational paradigms to tapinto the rich pulse of the human popula...
PCS Computing: Asthma Scenario190Sensordrone – for monitoringenvironmental air qualityWheezometer – for monitoringwheezing...
Personal, Public Health, andPopulation Level Signals for MonitoringAsthmaData Processing and Semantics for Advanced Intern...
Asthma Early Warning ModelData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madr...
Health Signal Extraction toUnderstandingData Processing and Semantics for Advanced Internet of Things (IoT) Applications, ...
194J. McCarthyJ. McCarthy M. WeiserM. WeiserD.EngelbartD.Engelbart J. C. R. LickliderJ. C. R. Licklider
195Part 8: IoT/PCS Systems:ApplicationsImage source: CISCO
SSN ApplicationsApplications ofSSN HealthcareWeather RescueTraffic Fire Fighting Logistics
SSN Application: Weather50% savings in sensingresource requirements duringthe detection of a blizzardOrder of magnitude ...
Linked Sensor DataLinked Sensor Data(~2 Billion Statements)
Sensor Discovery ApplicationQuery w/ location name to find nearby sensors
Real-Time Feature StreamsDemo: http://www.youtube.com/watch?v=_ews4w_eCpgData Processing and Semantics for Advanced Intern...
SSN Application: Fire DetectionWeather ApplicationSECURE: Semantics-empowered RescueEnvironment(detect different types of ...
SSN Application: Health CareMOBILEMD: Mobile app to help reduce re-admission of patients with Chronic Heart Failure
SSN Application: Health CarePassive Monitoring PhasePassive Monitoring Phase• Abnormal heart rate• Clammy skin• Panic Diso...
SSN Application: Health CareActive Monitoring PhaseActive Monitoring PhaseAre you feeling lightheaded?Are you feeling ligh...
Domain ExpertsDomain ExpertsColdWeatherColdWeatherPoorVisibilityPoorVisibilitySlowTrafficSlowTrafficIcyRoadIcyRoadDeclarat...
Traffic jamTraffic jamLinkDescriptionLinkDescriptionScheduledEventScheduledEventtraffic jamtraffic jambaseball gamebasebal...
207ScheduledEventScheduledEventActive EventActive EventDay of weekDay of weekTime of dayTime of daydelaydelayTraveltimeTra...
SemMOBData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainFirst respo...
SemMOBData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainPlatformPla...
LOKILAS(LOst Key Identification, Localization and Alert System)Data Processing and Semantics for Advanced Internet of Thin...
LOKILASData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain
LOKILASData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain
Future work Creating ontologies and defining data models are notenough tools to create and annotate data Tools for publ...
Some of the open issues Efficient real-time IoT resource/servicequery/discovery Directory Indexing Abstraction of IoT ...
Selected references Payam Barnaghi, Wei Wang, Cory Henson, Kerry Taylor, "Semantics for the Internet of Things: early pro...
Some useful links related to IoT Internet of Things, ITU http://www.itu.int/osg/spu/publications/internetofthings/Intern...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and p...
Upcoming SlideShare
Loading in...5
×

Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception

3,661

Published on

This tutorial presents tools and techniques for effectively utilizing the Internet of Things (IoT) for building advanced applications, including the Physical-Cyber-Social (PCS) systems. The issues and challenges related to IoT, semantic data modelling, annotation, knowledge representation (e.g. modelling for constrained environments, complexity issues and time/location dependency of data), integration, analy- sis, and reasoning will be discussed. The tutorial will de- scribe recent developments on creating annotation models and semantic description frameworks for IoT data (e.g. such as W3C Semantic Sensor Network ontology). A review of enabling technologies and common scenarios for IoT applications from the data and knowledge engineering point of view will be discussed. Information processing, reasoning, and knowledge extraction, along with existing solutions re- lated to these topics will be presented. The tutorial summarizes state-of-the-art research and developments on PCS systems, IoT related ontology development, linked data, do- main knowledge integration and management, querying large- scale IoT data, and AI applications for automated knowledge extraction from real world data.

Related: Semantic Sensor Web: http://knoesis.org/projects/ssw
Physical-Cyber-Social Computing: http://wiki.knoesis.org/index.php/PCS

Published in: Technology, Education
1 Comment
6 Likes
Statistics
Notes
  • Excellent presentation. Thank you.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
3,661
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
449
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide
  • Scalability and interoperability problems
  • Intelligence at the edge or hub; still no good answer; could be app or design dependent
  • Traditional networking: host to host Now: data-oriented communication; looking for data, not the host providing the data unless you want to manage the node
  • IoT data different from traditional content; transient, small. The more important thing is how to find the service that can provide the data; could be huge stream of data
  • Take about something on the web of data
  • Since then bring in the semantic web technologies
  • Images: http://www.google.com/imgres?q=abstract+earth+puzzle&um=1&hl=en&safe=off&biw=1548&bih=829&tbm=isch&tbnid=fCWWmELEgLspwM:&imgrefurl=http://depositphotos.com/4946293/stock-photo-Abstract-earth-puzzle.html&docid=ObBXLAbfdYscyM&imgurl=http://static5.depositphotos.com/1021974/494/i/450/dep_4946293-Abstract-earth-puzzle.jpg&w=450&h=397&ei=QEKXTrSIFLCrsALi0LnqBA&zoom=1&iact=hc&vpx=206&vpy=160&dur=463&hovh=166&hovw=201&tx=86&ty=75&sig=102505865583293696354&page=1&tbnh=160&tbnw=196&start=0&ndsp=30&ved=1t:429,r:0,s:0
  • Images: http://massthink.wordpress.com/2007/06/10/husserl-in-indubitable-response-to-descartes-and-kant/
  • Four characteristics of perceptual inference
  • Images: http://www.ida.liu.se/~eriho/COCOM_M.htm http://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/ucd%20lecture-3/sld019.htm
  • Images: http://www.ida.liu.se/~eriho/COCOM_M.htm http://www.idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/ucd%20lecture-3/sld019.htm
  • Declarative knowledge + statistical correlation This slide illustrates the three operations to enrich the correlation structure extracted using statistical methods These operations utilize declarative knowledge form ConceptNet5 as shown in each step
  • Statistical correlation structure shown above The enriched structure is shown below The enrichment of the graphical model will potentially allow us to capture the domain precisely and also improve our prediction as the model would get closer to the underlying probabilistic distribution in the real-world Log-Likelihood score is one way of quantifying how good a structure is based on the observed data There may be many candidate structures extracted from data which result in the log likelihood score Declarative knowledge will help us ground statistical models to reality which will allow us to pick one structure over the other Pramod Anantharam, Krishnaprasad Thirunarayan and Amit Sheth, 'Traffic Analytics using Probabilistic Graphical Models Enhanced with Knowledge Bases,' 2nd International Workshop on Analytics for Cyber-Physical Systems (ACS-2013) at SIAM International Conference on Data Mining (SDM13), pp. 13--20, Texas, USA, May 2-4, 2013. We stopped at structure extraction for our workshop paper (SIAM ACS workshop) since the declarative knowledge we used (ConceptNet5) and statistical model (nodes and edges) are at the same level of abstraction
  • Data Processing and Semantics for Advanced Internet of Things (IoT) Applications: modeling, annotation, integration, and perception

    1. 1. Data Processing and Semantics for AdvancedInternet of Things (IoT) Applications:modeling, annotation, integration, and perceptionPramod Anantharam1, Payam Barnaghi2, Amit Sheth11Kno.e.sis Center, Wright State University2Centre for Communication Systems Research, University of SurreyMadrid, Spain, June 12-14, 2013http://aida.ii.uam.es/wims13/keynotes.php#tutorialsSpecial Thanks to:Cory Henson, Kno.e.sis Research Center, Wright State UniversityWei Wang, Centre for Communication Systems Research, University of Surrey
    2. 2. 2Part 1: Introductionto Internet of “Things”Image source: CISCO
    3. 3. Internet of Things “sensors and actuators embedded in physicalobjects — from containers to pacemakers —are linked through both wired and wirelessnetworks to the Internet.” “When objects in the IoT can sense theenvironment, interpret the data, andcommunicate with each other, they becometools for understanding complexity and forresponding to events and irregularities swiftly”source: http://www.iot2012.org/
    4. 4. “Thing” connected to the internet“In 2013 cumulative shipments of Bluetooth-enabled devices will surpass 10 billionand Wi-Fi enabled devices will surpass 10 billion cumulative shipments in 2015,”- Peter Cooney, wireless analyst with ABI Research“73% of Countries with 4G Services Have Dropped Their 4G Tariffs by anAverage of 30% in the Past 6 Months”1- ABI Research (07 Feb 2013)1http://www.abiresearch.com/
    5. 5. 5Network connected Things andDevicesImage courtesy: CISCO
    6. 6. 6Sensor devices are becoming widelyavailable- Programmable devices- Off-the-shelf gadgets/tools
    7. 7. 7More “Things” are being connectedHome/daily-life devicesBusiness andPublic infrastructureHealth-care…
    8. 8. 8People Connecting to ThingsMotion sensorMotion sensorMotion sensorECG sensorInternet
    9. 9. 9Things Connecting to Things- Complex and heterogeneousresources and networks
    10. 10. 10Wireless Sensor Networks (WSN)Sinknode GatewayCore networke.g. InternetGatewayEnd-userComputer services- The networks typically run Low Power Devices- Consist of one or more sensors, could be different type of sensors (or actuators)
    11. 11. 11Key characteristics of IoT devices Often inexpensive sensors (actuators) equipped with a radiotransceiver for various applications, typically low data rate ~ 10-250kbps. Deployed in large numbers The sensors should coordinate to perform the desired task. The acquired information (periodic or event-based) is reported backto the information processing centre (or sometimes in-networkprocessing is required) Solutions are application-dependent.11
    12. 12. 12Beyond conventional sensors Human as a sensor (citizen sensors) e.g. tweeting real world data and/or events Virtual (software) sensors e.g. Software agents/servicesgenerating/representing dataRoad block, A3Road block, A3Suggest a different route
    13. 13. 13Cyber, Physical and Social Data
    14. 14. 14Citizen SensorsSource: How Crisis Mapping Saved Lives in Haiti, Ushahidi Haiti Project (UHP).
    15. 15. 15Cosm- Air Quality Egg
    16. 16. 16Cosm- data readingsTagsData formatsLocation
    17. 17. 17Making Sense of DataIn the next few years, sensor networks will produce10-20 time the amount of data generated by socialmedia. (source: GigaOmni Media)
    18. 18. 18Things, Data, and lots of itimage courtesy: Smarter Data - I.03_C by Gwen Vanhee
    19. 19. 19Big Data and IoT "Big data" is a term applied to data sets whose size is beyond the ability ofcommonly used software tools to capture, manage, and process the datawithin a tolerable elapsed time. Big data sizes are a constantly movingtarget, as of 2012 ranging from a few dozen terabytes to many petabytes ofdata in a single data set.” (wikipedia) Every day, we create 2.5 quintillion bytes of data — so much that 90% ofthe data in the world today has been created in the last two years alone.(source IBM)
    20. 20. 20The seduction of data Turn 12 terabytes of Tweets created each day into sentiment analysisrelated to different events/occurrences or relate them to products andservices. Convert (billions of) smart meter readings to better predict and balancepower consumption. Analyze thousands of traffic, pollution, weather, congestion, public transportand event sensory data to provide better traffic management. Monitor patients, elderly care and much more…Adapted from: What is Bog Data?, IBM
    21. 21. 21Do we need all these data?
    22. 22. 22“Raw data is both an oxymoron and baddata”Geoff Bowker, 2005ource: Kate Crawford, "Algorithmic Illusions: Hidden Biases of Big Data", Strata 2013.
    23. 23. 23IoT Data in the CloudImage courtesy: http://images.mathrubhumi.comhttp://www.anacostiaws.org/userfiles/image/Blog-Photos/river2.jpg
    24. 24. 24Perceptions and IntelligenceDataInformationKnowledgeWisdomRaw sensory dataStructured data (withsemantics)Abstraction and perceptionsActionable intelligence
    25. 25. 25Change in communication paradigmSinknode GatewayCore networke.g. Internet End-userDataSenderDataReceiverA sample data communication in conventional networksA sample data communication in WSNFire! Some bits01100011100
    26. 26. 26 Collaboration and in-network processing In some applications a single sensor node is not able to handle thegiven task or provide the requested information. Instead of sending the information form various source to an externalnetwork/node, the information can be processed in the network itself. e.g. data aggregation, summarisation and then propagating the processeddata with reduced size (hence improving energy efficiency by reducing theamount of data to be transmitted). Data-centric Conventional networks often focus on sending data between twospecific nodes each equipped with an address. Here what is important is data and the observations and measurementsnot the node that provides it.Required mechanisms
    27. 27. “People want answers, not numbers”(Steven Glaser, UC Berkley)Sinknode GatewayCore networke.g. InternetWhat is the temperature at home?Freezing!
    28. 28. 28IoT Data alone is not enough Domain knowledge Machine interpretable meta data Delivery, sharing and representation services Query, discovery, aggregation services Publish, subscribe, notification, and access interfaces/services
    29. 29. 29Storing, Handling and Processingthe DataImage courtesy: IEEE Spectrum
    30. 30. 30IoT Data Challenges Discovery: finding appropriate device and data sources Access: Availability and (open) access to IoT resources and data Search: querying for data Integration: dealing with heterogeneous device, networks and data Interpretation: translating data to knowledge usable by people andapplications Scalability: dealing with large number of devices and myriad of dataand computational complexity of interpreting the data.
    31. 31. Energy consumption of the nodes Batteries have small capacity and rechargingcould be complex (if not impossible) in somecases. The main consumers of the energy are: thecontroller, radio, to some extent memory anddepending on the type, the sensor(s). A controller can go to: “active”, “idle” and “sleep” A radio modem could turn transmitter,receiver, or both on or off, sensors and memory can be also turned onand off.
    32. 32. ActuatorsStepper Motor [1]Image credits:[1] http://directory.ac/telco-motion.html[2] http://bruce.pennypacker.org/category/theater/[3] http://www.busytrade.com/products/1195641/TG-100-Linear-Actuator.html[4] http://www.arbworx.com/services/fencing-garden-fencing/[2][3][4]
    33. 33. Wireless Sensor Networks (WSN)-gateway connection
    34. 34. Distributed WSN
    35. 35. What are the main issues? Heterogeneity Interoperability Mobility Energy efficiency Scalability Security
    36. 36. What is important? Robustness Quality of Service Scalability Seamless integration Security, privacy, Trust
    37. 37. In-network processing Mobile Ad-hoc Networks are supposed to deliverbits from one end to the other WSNs, on the other end, are expected to provideinformation, not necessarily original bits Gives addition options E.g., manipulate or process the data in the network Main example: aggregation Applying aggregation functions to a obtain an averagevalue of measurement data Typical functions: minimum, maximum, average, sum, … Not amenable functions: mediansource: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor NetworksHolger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
    38. 38. In-network processing- exampleApplying Symbolic Aggregate Approximation (SAX)SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbolsover the original sensor time-series data (green)
    39. 39. Data-centric networking In typical networks (including ad hoc networks),network transactions are addressed to the identitiesof specific nodes A “node-centric” or “address-centric” networking paradigm In a redundantly deployed sensor networks, specificsource of an event, alarm, etc. might not be important Redundancy: e.g., several nodes can observe the samearea Thus: focus networking transactions on the datadirectly instead of their senders and transmitters !data-centric networking Principal design changesource: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor NetworksHolger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
    40. 40. Implementation options fordata-centric networking Overlay networks & distributed hash tables (DHT) Hash table: content-addressable memory Retrieve data from an unknown source, like in peer-to-peer networking – withefficient implementation Some disparities remain Static key in DHT, dynamic changes in WSN DHTs typically ignore issues like hop count or distance between nodes whenperforming a lookup operation Publish/subscribe Different interaction paradigm Nodes can publish data, can subscribe to any particular kind of data Once data of a certain type has been published, it is delivered to all subscribes Subscription and publication are decoupled in time; subscriber and published areagnostic of each other (decoupled in identity); There is concepts of Semantic Sensor Networks- to annotate sensor resourcesand observation and measurement data!Adapted from: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor NetworksHolger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
    41. 41. IoT and Semantic technologies The sensors (and in general “Things”) are increasinglybeing integrated into the Internet/Web. This can be supported by embedded devices thatdirectly support IP and web-based connection (e.g.6LowPAN and CoAp) or devices that are connectedvia gateway components. Broadening the IoT to the concept of “Web of Things” There are already Sensor Web Enablement (SWE)standards developed by the Open GeospatialConsortium that are widely being adopted in industry,government and academia. While such frameworks provide some interoperability,semantic technologies are increasingly seen as keyenabler for integration of IoT data and broader Webinformation systems.
    42. 42. Semantics and IoT resources anddata Semantics are machine-interpretable metadata (for mark-up),logical inference mechanisms, query mechanism, linked datasolutions For IoT this means: ontologies for: resource (e.g. sensors), observation andmeasurement data (e.g. sensor readings), domain concepts (e.g.unit of measurement, location), services (e.g. IoT services) andother data sources (e.g. those available on linked open data) Semantic annotation should also supports data representedusing existing forms Reasoning /processing to infer relationships and hierarchiesbetween different resources, data Semantics (/ontologies) as meta-data (to describe the IoTresources/data) / knowledge bases (domain knowledge).
    43. 43. 43A Few WordsonSemantic Web
    44. 44. SSW Introductionlivesinhaspetis ahas petPersonPersonAnimalAnimalConcrete FactsResource Description FrameworkConcrete FactsResource Description FrameworkSemantic Web(according to Farside)General KnowledgeWeb Ontology LanguageGeneral KnowledgeWeb Ontology Language“Now! – That should clear up a few things aroundis a
    45. 45. Semantic Web Stack
    46. 46. Linked Open Data
    47. 47. Linked Open Data~ 50 Billion Statements~ 50 Billion Statements
    48. 48. SW is moving from academiato industry
    49. 49. In the last few years, we haveseen many successes …Knowledge GraphWatsonAppleSiri
    50. 50. Google Knowledge Graph
    51. 51. 51Sensors and the Web
    52. 52. Sensors are ubiquitous
    53. 53. Enabling the Internet of ThingsSituational awarenessenables: Devices/things to functionand adapt within theirenvironment Devices/things to worktogether
    54. 54. Sensor systems aretoo often stovepiped.Closed centralizedmanagement of sensingresourcesClosed inaccessibledata and sensors
    55. 55. We want to set this data freeWith freedom comesresponsibilityDiscovery, access, and searchIntegration and interpretationScalability
    56. 56. Drowning in DataA cross-country flight from New York to Los Angeles on aBoeing 737 plane generates a massive 240 terabytes ofdata- GigaOmni Media
    57. 57. Drowning in Data
    58. 58. ChallengesTo fulfill this vision, there are difficult challenges to overcome suchas the discovery, access, search, integration, and interpretation ofsensors and sensor data at scaleDiscovery finding appropriate sensing resources and data sourcesAccess sensing resources and data are open and availableSearch querying for sensor dataIntegration dealing with heterogeneous sensors and sensor dataInterpretation translating sensor data to knowledge usable by peopleandapplicationsScalability dealing with data overload and computational complexityof interpreting the data
    59. 59. SolutionSemantic Sensor WebInternet Computing, July/Aug.2008Uses the Web as platform formanaging sensor resources anddataUses semantic technologies forrepresenting data and knowledge,integration, and interpretation
    60. 60. SolutionDiscovery, access, and search Using standard Web services OGC Sensor Web Enablement
    61. 61. SolutionIntegration Using shared domain models / datarepresentation OGC Sensor Web Enablement W3C Semantic Sensor Networks
    62. 62. SolutionInterpretation Abstraction – converting low-level data to high-level knowledge Machine Perception – w/ prior knowledge and abductivereasoning IntellegO – Ontology of Perception
    63. 63. SolutionScalability Data overload – sensors produce too much data Computational complexity of semantic interpretation “Intelligence at the edge” – local and distributed integration andinterpretation of sensor data
    64. 64. SSW Adoption and Applications
    65. 65. 65Part 2: Data and knowledgemodelling requirements,semantic annotationImage source: CISCO
    66. 66. Recall of the Internet of Things A primary goal of interconnecting devices andcollecting/processing data from them is tocreate situation awareness and enableapplications, machines, and human users tobetter understand their surroundingenvironments. The understanding of a situation, or context,potentially enables services and applicationsto make intelligent decisions and to respond tothe dynamics of their environments.Barnaghi et al 2012, “Semantics for the Internet of Things: early progress and back to the future”
    67. 67. IoT challenges Numbers of devices and different users and interactions required. Challenge: Scalability Heterogeneity of enabling devices and platforms Challenge: Interoperability Low power sensors, wireless transceivers, communication, and networkingfor M2M Challenge: Efficiency in communications Huge volumes of data emerging from the physical world, M2M and newcommunications Challenge: Processing and mining the data, Providing secure access andpreserving and controlling privacy. Timeliness of data Challenge: Freshness of the data and supporting temporal requirements inaccessing the data Ubiquity Challenge: addressing mobility, ad-hoc access and service continuity Global access and discovery Challenge: Naming, Resolution and discovery
    68. 68. IoT: one paradigm, many visionsDiagram adapted from L. Atzori et al., 2010, “the Internet of Things: a Survey”
    69. 69. Semantic oriented vision “The object unique addressing and the representationand storing of the exchanged information become themost challenging issues, bringing directly to a ‘‘Semanticoriented”, perspective of IoT”, [Atzori et al., 2010] Data collected by different sensors and devices isusually multi-modal (temperature, light, sound, video,etc.) and diverse in nature (quality of data can vary withdifferent devices through time and it is mostly locationand time dependent [Barnaghi et al, 2012] some of challenging issues: representation, storage, andsearch/discovery/query/addressing, and processing IoTresources and data.
    70. 70. What is expected? Unified access to data: unified descriptions Deriving additional knowledge (data mining) Reasoning support and association to other entities andresources Self-descriptive data an re-usable knowledge In general: Large-scale platforms to support discoveryand access to the resources, to enable autonomousinteractions with the resources, to provide self-descriptive data and association mechanisms to reasonthe emerging data and to integrate it into the existingapplications and services.
    71. 71. Semantic technologies and IoT There are already Sensor Web Enablement(SWE) standards developed by the OpenGeospatial Consortium that are widely adopted. While such frameworks provide certain levels ofinteroperability, semantic technologies are seenas key enabler for integration of IoT data andand existing business information systems. Semantic technologies provide potential supportfor: Interoperability and machine automation IoT resource and data annotation, logical inference, queryand discovery, linked IoT data
    72. 72. Identify IoT domain concepts Users Physical entities Virtual entities Devices Resource Services …Diagram adapted from IoT-A project D2.1
    73. 73. IoT domain concepts - Entity Physical entities (or entity ofinterests): objects in the physical world,features of interest that are of interests tousers (human users or any digital artifacts). Virtual entities: virtual representation of thephysical entities. Entities are the main focus of interactionsbetween humans and/or software agents. This interaction is made possible by a hardwarecomponent called Device.Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
    74. 74. IoT domain concepts –Device, Resource and Service A Device mediates the interactions between usersand entities. The software component that provides informationon the entity or enables controlling of the device, iscalled a Resource. A Service provides well-defined and standardisedinterfaces, offering all necessary functionalities forinteracting with entities and related processes.Definition adapted from De et al, 2012, “Service modeling for the Internet of Things”
    75. 75. Other concepts need to considered Gateways Directories Platforms Systems Subsystems … Relationships among them And links to existing knowledge base and linked data
    76. 76. Don’t forget the IoT data Sensors and devices provide observation andmeasurement data about the physical world objectswhich also need to be semantically described and canbe related to an event, situation in the physical world. The processing of data into knowledge/ perception andusing it for decision making, automated control, etc. Huge amount of data from our physical world that needto be Annotated Published Stored (temporary or for longer term) Discovered Accessed Proceeded Utilised in different applications
    77. 77. Semantics for IoT resources and data Semantics are machine-interpretable metadata, logical inferencemechanisms, query and search mechanism, linked data… For IoT this means: ontologies for: resource (e.g. sensors), observation andmeasurement data (e.g. sensor readings), services (e.g. IoTservices), domain concepts (e.g. unit of measurement, location)and other data sources (e.g. those available on linked opendata) Semantic annotation should also supports data represented usingexisting forms Reasoning/processing to infer relationships between differentresources and services, detecting patterns from IoT data
    78. 78. Characteristics of IoT resources Extraordinarily large number Limited computing capabilities Limited memory Resource constrained environments (e.g.,battery life, signal coverage) Location is important Dynamism in the physical environments Unexpected disruption of services …
    79. 79. Characteristics of IoT data Stream data (depends on time) Transient nature Almost always related to a phenomenon orquality in our physical environments Large amount Quality in many situations cannot be assured(e.g., accuracy and precision) Abstraction levels (e.g., raw, inferred orderived) …
    80. 80. Utilise semantics Find all available resources (which can providedata) and data related to “Room A” (which is anobject in the linked data)? What is “Room A”? What is its location? returns “location” data What type of data is available for “Room A” or that “location”?(sensor category types) Predefined Rules can be applied based onavailable data (TempRoom_A > 80°C) AND (SmokeDetectedRoom_A position==TRUE) FireEventRoom_A Learning these rules needs data mining or pattern recognitiontechniques
    81. 81. 81Part 3: Semantics and data modellingfor IoTImage source: CISCO
    82. 82. Semantic modelling Lightweight: experiences show that a lightweightontology model that well balances expressiveness andinference complexity is more likely to be widely adoptedand reused; also large number of IoT resources andhuge amount of data need efficient processing Compatibility: an ontology needs to be consistent withthose well designed, existing ontologies to ensurecompatibility wherever possible. Modularity: modular approach to facilitate ontologyevolution, extension and integration with externalontologies.
    83. 83. Existing models for resources and data W3C Semantic Sensor Network IncubatorGroup’s SSN ontology (mainly for sensorsand sensor networks, observation andmeasurement, and platforms and systems) Quantity Kinds and Units Used together with the SSN ontology based on QUDV model OMG SysML(TM) Working group of the SysML 1.2 Revision TaskForce (RTF) and W3C Semantic Sensor NetworkIncubator Group
    84. 84. Existing models for services OWL-S and WSMO are heavy weight models: practicaluse? Minimal service model Deprecated Procedure-Oriented Service Model (POSM) and Resource-Oriented Service Model (ROSM): two different models fordifferent service technologies Defines Operations and Messages No profile, no grounding SAWSDL: mixture of XML, XML schema, RDF and OWL hRESTS and SA-REST: mixture of HTML and referenceto a semantic model; sensor services are not anticipatedto have HTML
    85. 85. W3C’S SSN ontologyDiagram adapted from SSN report
    86. 86. Some existing IoT models andontologies FP7 IoT-A project’s Entity-Resource-Serviceontology A set of ontologies for entities, resources, devicesand services Based on the SSN and OWL-S ontology FP7 IoT.est project’s service descriptionframework A modular approach for designing a descriptionframework A set of ontologies for IoT services, testing andQoS/QoI
    87. 87. IoT-A resource modelDiagram adapted from IoT-A project D2.1
    88. 88. IoT-A resource descriptionDiagram adapted from IoT-A project D2.1
    89. 89. IoT-A service modelDiagram adapted from IoT-A project D2.1
    90. 90. IoT-A service descriptionDiagram adapted from IoT-A project D2.1
    91. 91. Service modelling in IoT.estDiagrams adapted from Iot.est D3.1
    92. 92. IoT.est service profile highlight ServiceType class represents the servicetechnologies: RESTful and SOAP/WSDL services. serviceQos and serviceQoI are defined assubproperty of serviceParameter; they link to conceptsin the QoS/QoI ontology. serviceArea: the area where the service is provided;different from the sensor observation area Links to the IoT resources through “exposedBy”property Future extension: serviceNetwork, servicePlatform andserviceDeployment Service lifecycle, SLA…
    93. 93. 93Part 4: Linked-data and semanticenabled systemsImage source: CISCO
    94. 94. Linked data principles using URI’s as names for things: Everything isaddressed using unique URI’s. using HTTP URI’s to enable people to look upthose names: All the URI’s are accessible viaHTTP interfaces. provide useful RDF information related toURI’s that are looked up by machine orpeople; including RDF statements that link to otherURI’s to enable discovery of other relatedconcepts of the Web of Data: The URI’s arelinked to other URI’s.
    95. 95. Linked data in IoT Using URI’s as names for things;- URI’s for naming IoT resources and data (and also streamingdata); Using HTTP URI’s to enable people to look up thosenames;- Web-level access to low level sensor data and real worldresource descriptions (gateway and middleware solutions); Providing useful RDF information related to URI’s that are lookedup by machine or people;- publishing semantically enriched resource and data descriptionsin the form of linked RDF data; Including RDF statements that link to other URI’s to enablediscovery of other related things of the web of data;- linking and associating the real world data to the existing data onthe Web;
    96. 96. Linked data layer for not only IoT…Images from Stefan Decker, http://fi-ghent.fi-week.eu/files/2010/10/Linked-Data-scheme1.png; linked data diagram: http://richard.cyganiak.de/2007/10/lod/
    97. 97. Creating and using linked sensor datahttp://ccsriottb3.ee.surrey.ac.uk:8080/IOTA/
    98. 98. Sensor discovery using linked sensordata
    99. 99. Semantics in IoT - reality If we create an Ontology our data is interoperable Reality: there are/could be a number of ontologies for a domain Ontology mapping Reference ontologies Standardisation efforts Semantic data will make my data machine-understandable and my systemwill be intelligent. Reality: it is still meta-data, machines don’t understand it but can interpret it. It stilldoes need intelligent processing, reasoning mechanism to process and interpret thedata. It’s a Hype! Ontologies and semantic data are too much overhead; we dealwith tiny devices in IoT. Reality: Ontologies are a way to share and agree on a common vocabulary andknowledge; at the same time there are machine-interpretable and represented ininteroperable and re-usable forms; You don’t necessarily need to add semantic metadata in the source- it could be addedto the data at a later stage (e.g. in a gateway);
    100. 100. 100Part 5: Semantic Sensor WebandPerceptionImage source: semanticweb.com; CISCO
    101. 101. What is the Sensor Web? Sensor Web is an additional layer connecting sensornetworks to the World Wide Web. Enables an interoperable usage of sensor resources byenabling web based discovery, access, tasking, andalerting. Enables the advancement ofcyber-physical applications throughimproved situation awareness.
    102. 102. Why is the Sensor Web important? In general Enable tight coupling of the cyber andphysical world In relation to IoT Enable shared situation awareness (orcontext) between devices/things
    103. 103. Bridging the Cyber-Physical DividePsyleron’s Mind-Lamp (Princeton U),connections between the mind and thephysical world.Neuro Skys mind-controlled headset toplay a video game.MIT’s Fluid Interface Group: wearabledevice with a projector for deepinteractions with the environment
    104. 104. Bridging the Cyber-Physical DivideFoursquare is an online application whichintegrates a persons physical location andsocial network.Community of enthusiasts that share experiences ofself-tracking and measurement.FitBit Community allows theautomated collection andsharing of health-related data,goals, and achievements
    105. 105. Bridging the Cyber-Physical DivideTweeting Sensorssensors are becoming social
    106. 106. How do we design the Sensor Web? Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Semantic Annotation Interpretation through integration ofheterogeneous data and reasoning with priorknowledge Semantic Perception/Abstraction Linked Open Data as prior knowledge Scale through distributed local interpretation “intelligence at the edge”
    107. 107. OGC Sensor Web Enablement
    108. 108. Role of OGC SWE
    109. 109. Vision of Sensor Web Quickly discover sensors (secure or public) that can meetmy needs – location, observables, quality, ability to task Obtain sensor information in a standard encoding that isunderstandable by me and my software Readily access sensor observations in a common manner,and in a form specific to my needs Task sensors, when possible, to meet my specific needs Subscribe to and receive alerts when a sensor measures aparticular phenomenon
    110. 110. Principles of Sensor Web Sensors will be web accessible Sensors and sensor data will be discoverable Sensors will be self-describing to humans and software(using a standard encoding) Most sensor observations will be easily accessible in realtime over the web
    111. 111. OGC SWE Services Sensor Observation Service (SOS) access sensor information (SensorML) and sensorobservations (O&M Sensor Planning Service (SPS) task sensors or sensor systems Sensor Alert Service (SAS) asynchronous notification of sensor events (tasks,observation of phenomena) Sensor Registries discovery of sensors and sensor data
    112. 112. OGC SWE Services
    113. 113. OGC SWE Languages Sensor Model Language (SensorML) Models and schema for describing sensorcharacteristics Observation & Measurement (O&M) Models and schema for encoding sensorobservations
    114. 114. OCG SWE Observation
    115. 115. Semantic Sensor WebRDF OWLOGC Sensor WebEnablement
    116. 116. Sensor Web + Semantic WebSemantic WebThe web of data where web content is processedby machines, with human actors at the end of thechain.The web as a huge, dynamic, evolving databaseof facts, rather than pages, that can be interpretedand presented in many ways (mashups).Fundamental importance of ontologies to describethe fact that represents the data. RDF(S)emphasises labelled links as the source of meaning:essentially a graph model . A label (URI) uniquelyidentifies a concept.OWL emphasises inference as the source ofmeaning: a label also refers to a package of logicalaxioms with a proof theory.Usually, the two notions of meaning fit.Goal to combine information and servicesfor targeted purpose and new knowledgeSensor WebThe internet of things made up of Wireless SensorNetworks, RFID, stream gauges, orbiting satellites,weather stations, GPS, traffic sensors, ocean buoys,animal and fish tags, cameras, habitat monitors,recording data from the physical world.Today there are 4 billion mobile sensing devicesplus even more fixed sensors. The US NationalResearch Council predicts that this may grow totrillions by 2020, and they are increasingly connectedby internet and Web protocols.Record observations of a wide variety ofmodalities: but a big part is time-series‟ of numericmeasurements.The Open Geospatial Consortium has some web-service standards for shared data access (SensorWeb Enablement).Goal is to open up access to real-time andarchival data, and to combine in applications.
    117. 117. So, what is a Semantic Sensor Web? Reduce the difficulty and open up sensor networks by: Allowing high-level specification of the data collection process; Across separately deployed sensor networks; Across heterogeneous sensor types; and Across heterogeneous sensor network platforms; Using high-level descriptions of sensor network capability; and Interfacing to data integration methods using similar query andcapability descriptions. To create a Web of Real Time Meaning!
    118. 118. W3C SSN Incubator Group SSN-XG commenced: 1 March 2009 Chairs: Amit Sheth, Kno.e.sis Center, Wright State University Kerry Taylor, CSIRO Amit Parashar  Holger Neuhaus  Laurent Lefort, CSIRO Participants: 39 people from 20 organizations, including: Universities in: US, Germany, Finland, Spain, Britain, Ireland Multinationals: Boeing, Ericsson Small companies in semantics, communications, software Research institutes: DERI (Ireland), Fraunhofer (Germany),ETRI (Korea), MBARI (US), SRI International (US), MITRE(US), US Defense, CTIC (Spain), CSIRO (Australia), CESI(China)
    119. 119. W3C SSN Incubator GroupTwo main objectives:The development of an ontology for describingsensing resources and data, andThe extension of the SWE languages to supportsemantic annotations.
    120. 120. Sensor Standards Landscape
    121. 121. SSN Ontology OWL 2 DL ontology Authored by the XGparticipants Edited by MichaelCompton Driven by Use Cases Terminology carefullytracked to sources throughannotation properties Metrics Classes: 117 Properties: 148 DL Expressivity:SIQ(D)SSN Ontology –
    122. 122. SSN Use Cases
    123. 123. SSN Use Cases
    124. 124. SSN Ontology
    125. 125. Stimulus-Sensor-Observation The SSO Ontology Design Pattern is developed following the principle ofminimal ontological commitments to make it reusable for a variety of applicationareas. Introduces a minimal set of classes and relations centered around the notionsof stimuli, sensor, and observations. Defines stimuli as the (only) link to thephysical environment. Empirical science observes these stimuli using sensors to infer informationabout environmental properties and construct features of interest.
    126. 126. SSN Ontology Modules
    127. 127. SSN Ontology Modules
    128. 128. SSN Sensor A sensor can do (implements) sensing: that is, a sensor is any entity that canfollow a sensing method and thus observe some Property of aFeatureOfInterest. Sensors may be physical devices, computational methods, a laboratory setupwith a person following a method, or any other thing that can follow a Sensing
    129. 129. SSN Measurement Capability Collects together measurement properties (accuracy, range, precision, etc) andthe environmental conditions in which those properties hold, representing aspecification of a sensors capability in those conditions.
    130. 130. SSN Observation An Observation is a Situation in which a Sensing method has been used to estimate orcalculate a value of a Property. Links to Sensing and Sensor describe what made the Observation and how; links toProperty and Feature detail what was sensed; the result is the output of a Sensor; othermetadata gives the time(s) and the quality. Different from OGC’s O&M, in which an “observation” is an act or event, although it alsoprovides the record of the event.
    131. 131. Alignment with DOLCE
    132. 132. What SSN does not model Sensor types and models Networks: communication, topology Representation of data and units of measurement Location, mobility or other dynamic behaviours Animate sensors Control and actuation ….
    133. 133. Semantic Annotation of SWERecommendedtechnique via Xlinkattributes requires nochange to SWExlink:href - link toontology individualxlink:role - link toontology classxlink:arcrole - link toontology object property
    134. 134. How do we design the Sensor Web? Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Semantic Annotation Interpretation through integration ofheterogeneous data and reasoning with priorknowledge Semantic Perception/Abstraction Linked Open Data as prior knowledge Scale through distributed local interpretation “intelligence at the edge”
    135. 135. 136Interpretation of data A primary goal of interconnecting devices andcollecting/processing data from them is tocreate situation awareness and enableapplications, machines, and human users tobetter understand their surroundingenvironments. The understanding of a situation, or context,potentially enables services and applicationsto make intelligent decisions and to respond tothe dynamics of their environments.
    136. 136. 137Observation and measurement dataSource: W3C Semantic Sensor Networks, SSN Ontology presentation, Laurent Lefort et al.
    137. 137. 138How to say what a sensor is andwhat it measures?SinknodeGateway
    138. 138. 139Data/Service description frameworks There are standards such as Sensor Web Enablement (SWE) setdeveloped by the Open Geospatial Consortium that are widely beingadopted in industry, government and academia. While such frameworks provide some interoperability, semantictechnologies are increasingly seen as key enabler for integration ofIoT data and broader Web information systems.
    139. 139. 140Sensor Markup Language (SensorML)Source: http://www.mitre.org/
    140. 140. 141W3C SSN Ontologymakes observationsof this typeWhere it isWhat itmeasuresunitsSSN-XG ontologiesSSN-XG annotationsSSN-XG Ontology Scope
    141. 141. 142Semantics and IoT data Creating ontologies and defining data models is not enough tools to create and annotate data data handling components Complex models and ontologies look good, but design lightweight versions for constrained environments think of practical issues make it as compatible as possible and/or link it to the other existingontologies Domain knowledge and instances Common terms and vocabularies Location, unit of measurement, type, theme, … Link it to other resources Linked-data URIs and naming
    142. 142. 143Semantics and sensor dataSource: W. Wang, P. Barnaghi, "Semantic Annotation and Reasoning for Sensor Data", In proceedings of the 4th European Conference on SmartSensing and Context (EuroSSC2009), 2009.
    143. 143. 144Semantics and Linked-data The principles in designing the linked data aredefined as: using URI’s as names for things; using HTTP URI’s to enable people to look upthose names; provide useful RDF information related to URI’sthat are looked up by machine or people; including RDF statements that link to other URI’sto enable discovery of other related concepts ofthe Web of Data;
    144. 144. 145Linked Sensor data
    145. 145. 146Myth and reality #1: If we create an Ontology our data is interoperable Reality: there are/could be a number of ontologies for a domain Ontology mapping Reference ontologies Standardisation efforts #2: Semantic data will make my data machine-understandable and mysystem will be intelligent. Reality: it is still meta-data, machines don’t understand it but can interpret it. It still doesneed intelligent processing, reasoning mechanism to process and interpret the data. #3: It’s a Hype! Ontologies and semantic data are too much overhead; wedeal with tiny devices in IoT. Reality: Ontologies are a way to share and agree on a common vocabulary and knowledge;at the same time there are machine-interpretable and represented in interoperable and re-usable forms; You don’t necessarily need to add semantic metadata in the source- it could be added to thedata at a later stage (e.g. in a gateway); Legacy applications can ignore it or to be extended to work with it.
    146. 146. 147Processing Streaming Sensor Data
    147. 147. 148148Symbolic Aggregate Approximation(SAX)Variable String Length and Vocabulary size.Length: 10, VocSize: 10 Length: 10, VocSize: 4“gijigdbabd” “cdddcbaaab”Green Curve: consists of 100 Samples, Blue Curve: SAX
    148. 148. 149SAX representationSAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbolsover the original sensor time-series data (green)P. Barnaghi, F. Ganz, C. Henson, A. Sheth, "Computing Perception from Sensor Data",in Proc. of the IEEE Sensors 2012, Oct. 2012.fggfffhfffffgjhghfffjfhiggfffhfffffgjhgifggfffhfffffgjhghfff
    149. 149. 150Data Processing Frameworkfggfffhfffffgjhghfff dddfffffffffffddd cccddddccccdddccc aaaacccaaaaaaaaccccdddcdcdcdcddasdddPIR Sensor Light SensorTemperatureSensorRaw sensor datastreamRaw sensor datastreamRaw sensor datastreamAttendance PhoneHotTemperatureColdTemperatureBrightDay-timeNight-timeOffice roomBA0121On goingmeetingWindow hasbeen left open….Temporal data(extracted fromSSN descriptions)Spatial data(extracted fromSSN descriptions)Thematicdata(low levelabstractions)ParsimoniousCovering TheoryObservationsPerceptionsDomain knowledgeSAX PatternsRaw Sensor Data(Annotated with SSNOntology)…….PerceptionComputationHigh-levelPerceptions
    150. 150. 151SensorSAXF. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
    151. 151. 152Evaluation results of abstractioncreationF. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
    152. 152. 153Data size reductionF. Ganz, P. Barnaghi, F. Carrez, “Information Abstraction for Heterogeneous Real World Internet Data”, Feb. 2013.
    153. 153. 154Enabling the Internet of Things- Diversity range of applications- Interacting with large number ofdevices with various types-Multiple heterogeneous networks-Deluge of data-Processing and interpretation of theIoT data
    154. 154. 155Challenges and opportunitiesProviding infrastructure Publishing, sharing, and access solutions on a global scale Indexing and discovery (data and resources) Aggregation and fusion Trust, privacy and security Data mining and creating actionable knowledge Integration into services and applications in e-health, the public sector,retail, manufacturing and personalized apps. Mobile apps, location-based services, monitoring control etc. New business models
    155. 155. 156Part 6: Cognitive aspects ofknowledge representation, reasoningand perceptionImage source: CISCO
    156. 156. Abstraction provides the ability to interpret and synthesize informationin a way that affords effective understanding and communication ofideas, feelings, perceptions, etc. between machines and people.Abstraction
    157. 157.  People are excellent at abstraction;of sensing and interpreting stimulito understand and interact with theworld. The process of interpreting stimuliis called perception; and studyingthis extraordinary human capabilitycan lead to insights for developingeffective machine perception.Abstraction
    158. 158. observe perceiveconceptualizationof “real-world”“real-world”Abstraction
    159. 159. Semantic Perception/AbstractionFundamental QuestionsWhat is perception, and how canwe design machines to perceive?What can we learn from cognitivemodels of perception?Is the Semantic Web up to the taskof modeling perception?
    160. 160. What is Perception?Perception is the act of Abstracting Explaining Discriminating Choosing
    161. 161. What can we learn fromCognitive Models ofPerception? A-priori background knowledge is a keyenabler Perception is a cyclical, active processUlric Neisser (1976)Ulric Neisser (1976) Richard Gregory (1997)Richard Gregory (1997)
    162. 162. Is Semantic Web up to the taskof modeling perception?RepresentationHeterogeneous sensors, sensing, and observationrecordsBackground knowledge (observable properties,objects/events, etc.)InferenceExplain observations (hypothesis building)Focus attention by seeking additional stimuli (thatdiscriminate between explanations)Difficult Issues to OvercomePerception is an inference to the best explanationHandle streaming dataReal-time processing (or nearly)
    163. 163. Both people and machines are capable of observingqualities, such as redness.* Formally described in a sensor/ontology (SSN ontology)observesObserver Quality
    164. 164. The ability to perceive is afforded through the use ofbackground knowledge, relating observable qualities toentities in the world.* Formally described indomain ontologies(and knowledge bases)inheres inQualityEntity
    165. 165. With the help of sophisticated inference, both people andmachines are also capable of perceiving entities, such asapples. the ability to degrade gracefully with incomplete information the ability to minimize explanations based on newinformation the ability to reason over data on the Web fast (tractable)perceivesEntityPerceiver
    166. 166. Perceptual Inferenceminimizeexplanationsdegrade gracefullytractableAbductive Logic (e.g.,PCT)high complexityDeductive Logic (e.g.,OWL)(relatively) low complexityWebreasoningPerceptual Inference(i.e., abstraction)
    167. 167. The ability to perceive efficiently is afforded through thecyclical exchange of information between observers andperceivers.Traditionally called thePerceptual Cycle(or Active Perception)sendsfocussendsobservationObserverPerceiver
    168. 168. Neisser’s Perceptual Cycle
    169. 169.  1970’s – Perception is an active, cyclical process ofexploration and interpretation.- Nessier’s Perception Cycle 1980’s – The perception cycle is driven bybackground knowledge in order to generate and testhypotheses.- Richard Gregory (optical illusions) 1990’s – In order to effectively test hypotheses,some observations are more informative than others.- Norwich’s Entropy Theory of PerceptionCognitive Theories of Perception
    170. 170. Key InsightsBackground knowledge plays a crucial role in perception; what weknow (or think we know/believe) influences our perception of theworld.Semantics will allow us to realize computational models ofperception based on background knowledge.Contemporary IssuesInternet/Web expands our background knowledge to a globalscope; thus our perception is global in scopeSocial networks influence our knowledge and beliefs, thusinfluencing our perception
    171. 171. observesinheres inIntegrated together, we have an general model – capable ofabstraction – relating observers, perceivers, and backgroundknowledge.perceivessendsfocussendsobservationObserver QualityEntityPerceiver
    172. 172.  Ontology of Perception – as an extension of SSN Provides abstraction of sensor data through perceptualinference of semantically annotated data
    173. 173. Prior KnowledgeW3C SSN Ontology Bi-partite Graph Prior knowledge conformant to SSN ontology (left),structured as a bipartite graph (right)
    174. 174. Explanation is the act of accounting for sensory observations (i.e.,abstraction); often referred to as hypothesis building.Observed Property: A property that has been observed.  ObservedProperty ≡ ∃ssn:observedProperty—.{o1} ⊔ … ⊔∃ssn:observedProperty—.{on} Explanatory Feature: A feature that explains the set of observedproperties. ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{p1} ⊓ … ⊓∃ssn:isPropertyOf—.{pn} Semantics of Explanation
    175. 175. ExampleAssume the properties elevated blood pressure andpalpitations have been observed, and encoded in RDF(conformant with SSN): ssn:Observation(o1), ssn:observedProperty(o1, elevated bloodpressure)ssn:Observation(o2), ssn:observedProperty(o2, palpitations) Given these observations, the followingExplanatoryFeature class is constructed:ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{elevated bloodpressure} ⊓ ∃ssn:isPropertyOf—.{palpitations}Given the KB, executing the queryExplanatoryFeature(?y) can infer the features,Hypertension and Hyperthyroidism, as explanations:ExplanatoryFeature(Hypertension)ExplanatoryFeature(Hyperthyroidism)Semantics of Explanation
    176. 176. Discrimination is the act of deciding how to narrow down the multitude ofexplanatory features through further observation.Expected Property: A property is expected with respect to (w.r.t.) a setof features if it is a property of every feature in the set.ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}  NotApplicable Property: A property is not-applicable w.r.t. a set offeatures if it is not a property of any feature in the set.NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓¬∃ssn:isPropertyOf.{fn}Discriminating Property: A property is discriminating w.r.t. a set offeatures if it is neither expected nor not-applicable.DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicablePropertySemantics of Discrimination
    177. 177. ExampleGiven the explanatory features from the previousexample, Hypertension and Hyperthyroidism, thefollowing classes are constructed:ExpectedProperty ≡ ∃ssn:isPropertyOf.{Hypertension} ⊓∃ssn:isPropertyOf.{Hyperthyroidism} NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{Hypertension} ⊓¬∃ssn:isPropertyOf.{Hyperthyroidism} Given the KB, executing the queryDiscriminatingProperty(?x) can infer the property clammyskin as discriminating: DiscriminatingProperty(clammy skin)Semantics of Discrimination
    178. 178. How do we design the Sensor Web? Integration through shared semantics OGC Sensor Web Enablement W3C SSN ontology and Semantic Annotation Interpretation through integration ofheterogeneous data and reasoning with priorknowledge Semantic Perception/Abstraction Linked Open Data as prior knowledge Scale through distributed local interpretation “intelligence at the edge”
    179. 179. Efficient Algorithms for IntellegO Use of OWL-DL reasoner too resource-intensive for use inresource constrained devices (such as sensor nodes, mobilephones, IoT devices) Runs out of resources for problem size (prior knowledge) > 20concepts Asymptotic complexity: O(n3) [Experimentally determined] To enable their use on resource-constrained devices, we nowdescribe algorithms for efficient inference of explanation anddiscrimination. These algorithms use bit vector encodings and operations,leveraging a-priori knowledge of the environment.
    180. 180. Efficient Algorithms for IntellegOSemantic (RDF)EncodingBit Vector EncodingLowerLift First, developed lifting andlowering algorithms to translatebetween RDF and bit vectorencodings of observations.
    181. 181. Efficient Algorithms for IntellegOExplanation AlgorithmDiscrimination AlgorithmUtilize bit vector operators to efficientlycompute explanation anddiscriminationExplanation: Use of the bit vector ANDoperation to discover and dismiss thosefeatures that cannot explain the set ofobserved propertiesDiscrimination: Use of the bit vector ANDoperation to discover and indirectly assemblethose properties that discriminate between aset of explanatory features. The discriminatingproperties are those that are determined to beneither expected nor not-applicable
    182. 182. Efficient Algorithms for IntellegOEvaluation: The bit vector encodings and algorithms yield significant andnecessary computational enhancements – including asymptotic order ofmagnitude improvement, with running times reduced from minutes tomilliseconds, and problem size increased from 10’s to 1000’s.
    183. 183. Adoption of SSN
    184. 184. 185Part 7: Physical-Cyber-SocialComputingImage source: CISCO
    185. 185. 186J. McCarthyJ. McCarthy M. WeiserM. WeiserD.EngelbartD.Engelbart J. C. R. LickliderJ. C. R. LickliderSimilar Visions of Computing
    186. 186. 187If people do not believe that mathematics is simple, it is onlybecause they do not realize how complicated life is.- John von NeumannIf people do not believe that mathematics is simple, it is onlybecause they do not realize how complicated life is.- John von NeumannComputational paradigms have always dealt with asimplified representations of the real-world…Computational paradigms have always dealt with asimplified representations of the real-world…Algorithms work on these simplifiedrepresentationsAlgorithms work on these simplifiedrepresentationsSolutions from these algorithms are transcended backto the real-world by humans as actionsSolutions from these algorithms are transcended backto the real-world by humans as actionshttp://ngs.ics.uci.edu/blog/?p=1501Grand challenges in the real-world arecomplex
    187. 187. What has changed now?Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainThere are more devices connected to the internet thanthe entire human population.There are more devices connected to the internet thanthe entire human population.Today we have around 10 billion devices connectedto the internet making it an era of IoT (Internet of Things)Today we have around 10 billion devices connectedto the internet making it an era of IoT (Internet of Things)http://www.cisco.com/web/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf
    188. 188. 189What has not changed?What has not changed?We need computational paradigms to tapinto the rich pulse of the human populaceWe need computational paradigms to tapinto the rich pulse of the human populaceWe are still working on the simpler representations of the real-world!We are still working on the simpler representations of the real-world!Represent, capture, and compute with richer andfine-grained representations of real-worldproblemsRepresent, capture, and compute with richer andfine-grained representations of real-worldproblemsWhat should change?What should change?
    189. 189. PCS Computing: Asthma Scenario190Sensordrone – for monitoringenvironmental air qualityWheezometer – for monitoringwheezing soundsCan I reduce my asthma attacks at night?What are the triggers?What is the wheezing level?What is the propensity toward asthma?What is the exposure level over a day?What is the air quality indoors?Commute to WorkPersonalPublic HealthPopulation LevelClosing the window at homein the morning and taking analternate route to office maylead to reduced asthma attacksActionableInformationActionableInformation
    190. 190. Personal, Public Health, andPopulation Level Signals for MonitoringAsthmaData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ;*consider referral to specialistAsthma Controland Actionable InformationSensors and their observationsfor understanding asthma
    191. 191. Asthma Early Warning ModelData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain192PersonalLevelSignalsPersonalLevelSignalsSocietal LevelSignalsSocietal LevelSignals(Personal Level Signals)(PersonalizedSocietal Level Signal)(Societal Level Signals)Societal LevelSignals Relevant tothe Personal LevelSocietal LevelSignals Relevant tothe Personal LevelPersonal Level Sensors(mHealth**)QualifyQualify QuantifyQuantifyActionRecommendationActionRecommendationWhat are the features influencing my asthma?What is the contribution of each of these features?How controlled is my asthma? (risk score)What will be my action plan to manage asthma?StorageStorageSocietal Level SensorsAsthma Early Warning Model (AEWM)Query AEWMVerify & augmentdomain knowledgeRecommendedActionActionJustification*http://www.slideshare.net/jain49/eventshop-120721, ** http://www.youtube.com/watch?v=btnRi64hJp4(EventShop*)
    192. 192. Health Signal Extraction toUnderstandingData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainPopulation LevelPersonalWheeze – YesDo you have tightness of chest? –YesObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding<Wheezing=Yes, time, location><ChectTightness=Yes, time, location><PollenLevel=Medium, time, location><Pollution=Yes, time, location><Activity=High, time, location>WheezingChectTightnessPollenLevelPollutionActivityWheezingChectTightnessPollenLevelPollutionActivityRiskCategory<PollenLevel, ChectTightness, Pollution,Activity, Wheezing, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory><2, 1, 1,3, 1, RiskCategory>...ExpertKnowledgeBackgroundKnowledgetweet reporting pollution leveland asthma attacksAcceleration readings fromon-phone sensorsSensor and personalobservationsSignals from personal, personalspaces, and community spacesRisk Category assigned by doctorsQualifyQuantifyEnrichOutdoor pollen and pollutionPublic HealthWell Controlled - continueNot Well Controlled – contactnursePoor Controlled – contact doctor
    193. 193. 194J. McCarthyJ. McCarthy M. WeiserM. WeiserD.EngelbartD.Engelbart J. C. R. LickliderJ. C. R. Licklider
    194. 194. 195Part 8: IoT/PCS Systems:ApplicationsImage source: CISCO
    195. 195. SSN ApplicationsApplications ofSSN HealthcareWeather RescueTraffic Fire Fighting Logistics
    196. 196. SSN Application: Weather50% savings in sensingresource requirements duringthe detection of a blizzardOrder of magnitude resourcesavings between storing observationsvs. relevant abstractions
    197. 197. Linked Sensor DataLinked Sensor Data(~2 Billion Statements)
    198. 198. Sensor Discovery ApplicationQuery w/ location name to find nearby sensors
    199. 199. Real-Time Feature StreamsDemo: http://www.youtube.com/watch?v=_ews4w_eCpgData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain
    200. 200. SSN Application: Fire DetectionWeather ApplicationSECURE: Semantics-empowered RescueEnvironment(detect different types of fires)DEMO: http://www.youtube.com/watch?v=in2KMkD_uqg
    201. 201. SSN Application: Health CareMOBILEMD: Mobile app to help reduce re-admission of patients with Chronic Heart Failure
    202. 202. SSN Application: Health CarePassive Monitoring PhasePassive Monitoring Phase• Abnormal heart rate• Clammy skin• Panic Disorder• Hypoglycemia• Hyperthyroidism• Heart Attack• Septic ShockObserved Symptoms Possible ExplanationsPassive Sensors – heart rate, galvanic skinresponse
    203. 203. SSN Application: Health CareActive Monitoring PhaseActive Monitoring PhaseAre you feeling lightheaded?Are you feeling lightheaded?Are you have trouble taking deepbreaths?Are you have trouble taking deepbreaths?yesyesyesyesHave you taken your Methimazolemedication?Have you taken your Methimazolemedication?Do you have low blood pressure?Do you have low blood pressure?yesyes• Abnormal heart rate• Clammy skin• Lightheaded• Trouble breathing• Low blood pressure• Panic Disorder• Hypoglycemia• Hyperthyroidism• Heart Attack• Septic ShockObserved Symptoms Possible ExplanationsnonoActive Sensors – blood pressure, weight scale, pulseoxymeterDemo: http://www.youtube.com/watch?v=btnRi64hJp4
    204. 204. Domain ExpertsDomain ExpertsColdWeatherColdWeatherPoorVisibilityPoorVisibilitySlowTrafficSlowTrafficIcyRoadIcyRoadDeclarative domain knowledgeCausalknowledgeLinked Open DataColdWeather(YES/NO)IcyRoad (ON/OFF) PoorVisibility (YES/NO)SlowTraffic (YES/NO)1 0 1 11 1 1 01 1 1 11 0 1 0Domain ObservationsDomain ObservationsDomainKnowledgeDomainKnowledgeStructure and parameters205WinterSeasonWinterSeason OtherknowledgeCorrelations to causations usingDeclarative knowledge on theSemantic WebkTraffic
    205. 205. Traffic jamTraffic jamLinkDescriptionLinkDescriptionScheduledEventScheduledEventtraffic jamtraffic jambaseball gamebaseball gameAdd missing random variablesTime of dayTime of daybad weather CapableOf slow trafficbad weatherbad weatherTraffic data from sensors deployed on roadnetwork in San Francisco Bay Areatime of daytime of daytraffic jamtraffic jambaseball gamebaseball gametime of daytime of dayslow trafficslow trafficThree Operations: Complementing graphical model structure extractionAdd missing links bad weatherbad weathertraffic jamtraffic jambaseball gamebaseball gametime of daytime of dayslow trafficslow trafficAdd link directionbad weatherbad weathertraffic jamtraffic jambaseball gamebaseball gametime of daytime of dayslow trafficslow trafficgo to baseball game Causes traffic jamKnowledge from ConceptNet5traffic jam CapableOfoccur twice each daytraffic jam CapableOf slow traffic206
    206. 206. 207ScheduledEventScheduledEventActive EventActive EventDay of weekDay of weekTime of dayTime of daydelaydelayTraveltimeTraveltimespeedspeedvolumevolumeStructure extracted formtraffic observations(sensors + textual) usingstatistical techniquesScheduledEventScheduledEventActive EventActive EventDay of weekDay of weekTime of dayTime of daydelaydelayTraveltimeTraveltimespeedspeedvolumevolumeBad WeatherBad WeatherEnriched structure which haslink directions and newnodes such as “Bad Weather”potentially leading to betterdelay predictionskTraffic
    207. 207. SemMOBData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainFirst responders have limited time to analyzesensor (on and around them) observations.First responders have limited time to analyzesensor (on and around them) observations.Fire fighters need to combine prior knowledge offires and its behavior, extinguishers, and floorplan of the building for rescue strategy and operations.Fire fighters need to combine prior knowledge offires and its behavior, extinguishers, and floorplan of the building for rescue strategy and operations.There are a variety of sensors used to monitorvitals of firefighters, location, and poisonous gases.There are a variety of sensors used to monitorvitals of firefighters, location, and poisonous gases.O2O2Heart rateHeart rateCOCOCO2CO2GPSGPSAccelerometerAccelerometer CompassCompassFootstepsFootstepsThere is a team of them makingit further difficult for analysisThere is a team of them makingit further difficult for analysisSemantic Web allow us to describe the domain, sensors, andfirst responders -- apply reasoning techniques to derive actionableinsightsSemantic Web allow us to describe the domain, sensors, andfirst responders -- apply reasoning techniques to derive actionableinsightsSensors on each responder providesdifferent view of the event and they needto register dynamically as the firefightersarrive.Sensors on each responder providesdifferent view of the event and they needto register dynamically as the firefightersarrive.
    208. 208. SemMOBData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, SpainPlatformPlatform SensingDeviceSensingDeviceSystem_AGTC1System_AGTC1Sensor_LSM11Sensor_LSM11SensorOutputSensorOutputO2Concentraction_outputO2Concentraction_outputMeasurementCapabilityMeasurementCapabilityMeasurementCapability_LSM11MeasurementCapability_LSM11PropertyPropertyconcentrationconcentrationonPlatformhasOutputhasMeasurementCapabilityObservesObservationObservationO2ConcentrationO2ConcentrationObservedPropertyO2Obs_15052012O2Obs_15052012observationResultTimehttp://geonames.org/6298640http://geonames.org/6298640hasLocationnamexsd:floatxsd:floatlatlongDayton, Dayton-Wright Brothers Airport^^xsd:stringFeatureFeatureDemo: http://www.youtube.com/watch?v=nX11OqgtQlw
    209. 209. LOKILAS(LOst Key Identification, Localization and Alert System)Data Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain
    210. 210. LOKILASData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain
    211. 211. LOKILASData Processing and Semantics for Advanced Internet of Things (IoT) Applications, WIMS2013, Madrid, Spain
    212. 212. Future work Creating ontologies and defining data models are notenough tools to create and annotate data Tools for publishing linked IoT data Designing lightweight versions for constrainedenvironments think of practical issues make it as much as possible compatible and/or link it to the otherexisting ontologies Linking to domain knowledge and other resources Location, unit of measurement, type, theme, … Linked-data URIs and naming
    213. 213. Some of the open issues Efficient real-time IoT resource/servicequery/discovery Directory Indexing Abstraction of IoT data Pattern extraction Perception creation IoT service composition and compensation Integration with existing Web services Service adaptation
    214. 214. Selected references Payam Barnaghi, Wei Wang, Cory Henson, Kerry Taylor, "Semantics for the Internet of Things: early progress and back to the future",(to appear) International Journal on Semantic Web and Information Systems (special issue on sensor networks, Internet of Things andsmart devices), 2012. Atzori, L., Iera, A. & Morabito, G. , “The Internet of Things: A survey”, Computer Networks, Volume 54, Issue 15, 28 October 2010, 2787-2805. Suparna De, Tarek Elsaleh, Payam Barnaghi , Stefan Meissner, "An Internet of Things Platform for Real-World and Digital Objects",Journal of Scalable Computing: Practice and Experience, vol 13, no.1, 2012. Suparna De, Payam Barnaghi, Martin Bauer, Stefan Meissner, "Service modelling for the Internet of Things", in Proceedings of theConference on Computer Science and Information Systems (FedCSIS), pp.949-955, Sept. 2011. Cory Henson, Amit Sheth, and Krishnaprasad Thirunarayan, “Semantic Perception: Converting Sensory Observations to Abstractions”,IEEE Internet Computing, Special Issue on Context-Aware Computing, March/April 2012. Payam Barnaghi, Frieder Ganz, Cory Henson, Amit Sheth, “Computing Perception from Sensor Data”, In proceedings of the 2012 IEEESensors Conference, Taipei, Taiwan, October 28-31, 2012. Michael Compton et al, “The SSN Ontology of the W3C Semantic Sensor Network Incubator Group”, Journal of Web Semantics, 2012. Harshal Patni, Cory Henson, and Amit Sheth , “Linked Sensor Data”, in Proceedings of 2010 International Symposium on CollaborativeTechnologies and Systems (CTS 2010), Chicago, IL, May 17-21, 2010. Amit Sheth, Cory Henson, and Satya Sahoo , “Semantic Sensor Web IEEE Internet Computing”, vol. 12, no. 4, July/August 2008, pp. 78-83. Wei Wang, Payam Barnaghi, Gilbert Cassar, Frieder Ganz, Pirabakaran Navaratnam, "Semantic Sensor Service Networks", (to appear)in Proceedings of the IEEE Sensors 2012 Conference, Taipei, Taiwan, October 2012. Wang W, De S, Toenjes R, Reetz E, Moessner K, "A Comprehensive Ontology for Knowledge Representation in the Internet of Things",International Workshop on Knowledge Acquisition and Management in the Internet of Things (KAMIoT 2012) in conjunction with IEEIUCC-2012, Liverpool, UK. Liverpool. 25-27 June, 2012.
    215. 215. Some useful links related to IoT Internet of Things, ITU http://www.itu.int/osg/spu/publications/internetofthings/InternetofThings_summary.pdf IoT Comic Book http://www.theinternetofthings.eu/content/mirko-presser-iot-comic-book Internet of Things Europe, http://www.internet-of-things.eu/ Internet of Things Architecture (IOT-A) http://www.iot-a.eu/public/public-documents W3C Semantic Sensor Networks http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×