Semantic Metadata to Support Device Interaction in Smart Environments

  • 679 views
Uploaded on

Slides for a talk held at the Fourth International Workshop on the Web of Things (WoT 2013) on Sep 9th, 2013. For the full paper, see http://www.webofthings.org/wot/2013/program.php

Slides for a talk held at the Fourth International Workshop on the Web of Things (WoT 2013) on Sep 9th, 2013. For the full paper, see http://www.webofthings.org/wot/2013/program.php

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
679
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
24
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Semantic Metadata to Support Device Interaction in Smart Environments 1| Simon Mayer http://people.inf.ethz.ch/mayersi Simon Mayer and Gianin Basler, ETH Zurich Fourth International Workshop on the Web of Things (WoT 2013) September 9th 2013, Zurich, Switzerland Semantic Metadata to Support Device Interaction in Smart Environments
  • 2. Semantic Metadata to Support Device Interaction in Smart Environments 2| Simon Mayer http://people.inf.ethz.ch/mayersi Myself Simon Mayer, PhD Student at ETH Zurich, WoT since 2009
  • 3. Realize the full potential of smart environments by integrating functionality across services I know the temperature! What’s the temperature? Is it going to rain today? Do I need to be watered?
  • 4. I can find song files for you! I want to listen to “Yellow Submarine” I can play song files! Realize the full potential of smart environments by integrating functionality across services
  • 5.  Enabling services to talk to each other and form mashups  Enabling people to talk to service mashups
  • 6. Semantic Metadata to Support Device Interaction in Smart Environments 6| Simon Mayer http://people.inf.ethz.ch/mayersi  Heterogeneous services from different providers should be dynamically composable  Services don’t know anything about how to interact with other services when they are created (i.e., no “standards”)  Interaction metadata discoverable on the fly!  Flexibility  System should work in the user’s home, office, hotels, etc. Challenges
  • 7. Simon’s Office - Contains 2 services Think about a smart environment… I have a song title, “Yellow Submarine”. I’d like to play that song. Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple, Inc. Media Player Method: PUT Input: MP3 URL Output: Plays song Not possible! No Link! I can do that! :-(
  • 8. The songs search machine doesn’t provide any hypermedia controls that link to the media player Nor any information that helps the client find out that it could play a song using the services I can find that out, actually YES. I GOT IT! Our goal is to enable it for this guy!
  • 9. Semantic Metadata to Support Device Interaction in Smart Environments 9| Simon Mayer http://people.inf.ethz.ch/mayersi  Crowd-sourced Service Integration  Web Linking, “Globalized” HATEOAS  Semantics-based Service Integration  Web Linking + Functional metadata Service Integration: “Self-aware” Things? Techniques for Semantic Service Integration
  • 10. Semantic Metadata to Support Device Interaction in Smart Environments 10| Simon Mayer http://people.inf.ethz.ch/mayersi  Crowd-sourced Service Integration  Web Linking, “Globalized” HATEOAS  Semantics-based Service Integration  Web Linking + Functional metadata Service Integration: “Self-aware” Things? Techniques for Semantic Service Integration
  • 11. Semantic Metadata to Support Device Interaction in Smart Environments 11| Simon Mayer http://people.inf.ethz.ch/mayersi  Rely on developers or users to publish links Semantic Service Integration Crowd-sourced Service Integration Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music I know that you can connect these services! Let me show you!
  • 12. Semantic Metadata to Support Device Interaction in Smart Environments 12| Simon Mayer http://people.inf.ethz.ch/mayersi Crowd-sourced Service Integration Approach: Service Marketplaces S. Mayer, D. Karam. A Computational Marketplace for the Web of Things, 2012  Mashup developers find links manually, but publish their mashups in a “WoT Marketplace”  Clients can ask the marketplace for linking information  These links make sense, because they are used in a mashup!  Clients only need to choose which link to follow  We called these «Computational» and «WoT» marketplaces…
  • 13. Semantic Metadata to Support Device Interaction in Smart Environments 13| Simon Mayer http://people.inf.ethz.ch/mayersi Crowd-sourced Service Integration Service Marketplaces Marketplace Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music Node A Node B Node C Node D Node E
  • 14. Marketplace Node A Node B Node C Node D Node E I know that you can connect the services at A and E! Let me show you!
  • 15. POST wot_marketplace.org/nodes/A?link=nodes/E&name=play_song 202 Created
  • 16. Marketplace Node A Node B Node C Node D Node E Now, others can use that information! play_song
  • 17. GET wot_marketplace.org/nodes/A { “service” : “www.songsearch.org” “forward_paths” : { “play_song” : “nodes/E” } } I can follow that link now, too!
  • 18. Issue 1: Availability and reliability of the crowd? Issue 2: Clients are compiled against the link names (e.g., “play_song”)! This didn’t enable fully automatic service usage! Marketplace Node A Node B Node C Node D Node E play_song
  • 19. Can we automate the publishing of links and the traversal itself? Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music
  • 20. Semantic Metadata to Support Device Interaction in Smart Environments 20| Simon Mayer http://people.inf.ethz.ch/mayersi Idea: Semantic metadata to find potential links! Semantic Service Integration Functional Semantic Metadata Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music R.Verborgh et al. RESTdesc - A Functionality-Centered Approach to Semantic Service Description and Composition, 2012 S. Mayer and G. Basler. Embedded Semantic Metadata to Support Device Interaction in Smart Environments, 2013
  • 21. Semantic Metadata to Support Device Interaction in Smart Environments 21| Simon Mayer http://people.inf.ethz.ch/mayersi Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music Based on what I know about these services, there could be a link here! Semantic Service Integration Functional Semantic Metadata
  • 22. Embedding of semantic functional service descriptions How? Connect that information to its API What? Describe what a service does
  • 23. Based on what I know about these services, … Use of a reasoning engine to process these descriptions
  • 24. Semantic Metadata to Support Device Interaction in Smart Environments 24| Simon Mayer http://people.inf.ethz.ch/mayersi  Usability as one major constraint  It should not be too hard to create service descriptions  It should be easy to create user goals  Pragmatism as another  Make use of previous research in semantic technologies  Use existing ontologies. Avoid creating new ones!  Make it work for real use cases  Format: Notation3 (user-friendly RDF with inference rules) Semantic Service Integration Functional Semantic Metadata
  • 25. The title of the resource people.org/person1/, is “Person 1”, and it knows another resource, people.org/person2/ @prefix dc: <http://purl.org/dc/elements/1.1/>. @prefix foaf: <http://xmlns.com/foaf/0.1/>. <people.org/person1> dc:title "Person 1"; foaf:knows "people.org/person2/". Notation3 Brief N3 Primer: Statements
  • 26. Brief N3 Primer: Rules 𝑹𝒖𝒍𝒆: 𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 “people.org/person1/ knows people.org/person2/ ” implies “people.org/person2/ knows people.org/person1/ ” If one person knows another person, this second person also knows the first. (“foaf:knows is symmetric”)
  • 27. Notation3 Brief N3 Primer: Rules @prefix dc: <http://purl.org/dc/elements/1.1/>. @prefix foaf: <http://xmlns.com/foaf/0.1/>. { Precondition. } => { Postcondition. } is shorthand for=> log:implies If one person knows another person, this second person also knows the first. (“foaf:knows is symmetric”)
  • 28. Notation3 Brief N3 Primer: Rules @prefix dc: <http://purl.org/dc/elements/1.1/>. @prefix foaf: <http://xmlns.com/foaf/0.1/>. { ?x a foaf:Person; foaf:knows ?y. } => { Postcondition. } If one person knows another person, this second person also knows the first. (“foaf:knows is symmetric”) is shorthand for=> log:implies Superb tutorial at n3.restdesc.org (by Ruben Verborgh)
  • 29. Notation3 Brief N3 Primer: Rules @prefix dc: <http://purl.org/dc/elements/1.1/>. @prefix foaf: <http://xmlns.com/foaf/0.1/>. { ?x a foaf:Person; foaf:knows ?y. } => { ?y foaf:knows ?x. } If one person knows another person, this second person also knows the first. (“foaf:knows is symmetric”) is shorthand for=> log:implies Superb tutorial at n3.restdesc.org (by Ruben Verborgh)
  • 30. Semantic Metadata to Support Device Interaction in Smart Environments 30| Simon Mayer http://people.inf.ethz.ch/mayersi Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music Now: Semantic metadata for the Songs Search Machine Semantic Service Integration Functional Semantic Metadata
  • 31. Semantic Metadata to Support Device Interaction in Smart Environments 31| Simon Mayer http://people.inf.ethz.ch/mayersi Precondition: “What facts does a meaningful execution of the service require?” We have a Song Title Postcondition: “What additional facts hold after an execution of the service?” We have the URL of a song with that title Soodle S.A Songs Search Machine Method: GET Input: Song Title Output: MP3 URL Semantic Service Integration Functional Semantic Metadata
  • 32. Notation3 The Songs Search Machine @prefix dc: <http://purl.org/dc/elements/1.1/>. { Precondition. } => { Postcondition. } If we know a song title, we also know the URL of a song with that title.
  • 33. Notation3 The Songs Search Machine @prefix dc: <http://purl.org/dc/elements/1.1/>. { ?songTitle a dc:title. } => { Postcondition. } If we know a song title, we also know the URL of a song with that title.
  • 34. If we know a song title, we also know the URL of a song with that title. Notation3 The Songs Search Machine @prefix dc: <http://purl.org/dc/elements/1.1/>. { ?songTitle a dc:title. } => { ?songURL a dc:Song; dc:title ?songTitle. } Store this as ssm_rule.n3
  • 35. “Yellow Submarine” a dc:title. “Yellow Submarine” a dc:Title. ?songURL a dc:Song; dc:title “Yellow Submarine”. [+ proof] ssm_rule.n3 But I still don’t know how to call that service!!
  • 36. How to include the request? Basically two options… 𝑹𝒖𝒍𝒆: 𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 ∧ 𝑅𝑒𝑞𝑢𝑒𝑠𝑡 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑹𝒖𝒍𝒆: 𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 ∧ 𝑅𝑒𝑞𝑢𝑒𝑠𝑡 This works, but is impractical (knowledge about all requests required from the start…) It also messes up the quantification! “Given the Preconditions, all GET requests give the Postcondition” This works too, and it is sound! “Given the Preconditions, there exists a GET request that gives the Postcondition” R.Verborgh et al. RESTdesc - A Functionality-Centered Approach to Semantic Service Description and Composition, 2012 The second option sounds better to me!
  • 37. Notation3 The Songs Search Machine @prefix dc: <http://purl.org/dc/elements/1.1/>. { ?songTitle a dc:title. } => { ?songURL a dc:Song; dc:title ?songTitle. Request. } If we know a song title, we also know the URL of a song with that title. Plus, we know about an HTTP request.
  • 38. Notation3 The Songs Search Machine @prefix dc: <http://purl.org/dc/elements/1.1/>. @prefix http: <http://www.w3.org/2011/http#>. { ?songTitle a dc:title. } => { ?songURL a dc:Song; dc:title ?songTitle. Request. } If we know a song title, we also know the URL of a song with that title. Plus, we know about an HTTP request.
  • 39. Notation3 The Songs Search Machine @prefix dc: <http://purl.org/dc/elements/1.1/>. @prefix http: <http://www.w3.org/2011/http#>. { ?songTitle a dc:title. } => { ?songURL a dc:Song; dc:title ?songTitle. _:request http:methodName “GET”; http:requestURI (<www.ssm.org?title=>?songTitle); http:resp [ http:body ?songURL ]. } Store this as ssm_rule.n3 We’ll refer to it as a “RESTdesc” description If we know a song title, we also know the URL of a song with that title. Plus, we know about an HTTP request.
  • 40. “Yellow Submarine” a dc:title. “Yellow Submarine” a dc:Title. ?songURL a dc:Song; dc:title “Yellow Submarine”. _:request http:methodName “GET”; http:requestURI (<www.ssm.org?title=>?songTitle); http:resp [ http:body ?songURL ]. [+ proof] ssm_rule.n3
  • 41. Semantic Metadata to Support Device Interaction in Smart Environments 41| Simon Mayer http://people.inf.ethz.ch/mayersi Now, we only need to link that to the service! Multiple ways to do that… We use HTTP OPTIONS “This method allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval.” (RFC 2616) OPTIONS www.ssm.org Link: <ssm_rule.n3>; rel=“describedBy“ And the Link Header (RFC 5988) to return the description Soodle S.A Songs Search Machine ssm_rule.n3
  • 42. How do I say that I want a song URL?
  • 43. “Yellow Submarine” a dc:title. Goal: ?song a dc:Song; hasTitle “Yellow Submarine”. A proof that is parsed to an internal HTTP request description format Send a GET request to www.ssm.org?title={songTitle} and substitute “songTitle” with “Yellow Submarine”. Then, unpack the response. Its body contains the song. GET ?title=YellowSubmarine www.songs.org/yellowsub.mp3 ssm_rule.n3 Soodle S.A Songs Search Machine ssm_rule.n3
  • 44. And how do I play “Yellow Submarine” now??
  • 45. Semantic Metadata to Support Device Interaction in Smart Environments 45| Simon Mayer http://people.inf.ethz.ch/mayersi Missing: Semantic metadata for the Media Player! Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine ssm_rule.n3 Sapple Inc. Media Player Method: PUT Input: MP3 URL Output: Will play music Semantic Service Integration Functional Semantic Metadata
  • 46. Semantic Metadata to Support Device Interaction in Smart Environments 46| Simon Mayer http://people.inf.ethz.ch/mayersi Perfect! Simon’s Office - Contains 2 services Soodle S.A Songs Search Machine ssm_rule.n3 Sapple Inc. Media Player mp_rule.n3 Semantic Service Integration Functional Semantic Metadata
  • 47. I want to play “Yellow Submarine” now!! Then formulate your goal. But be precise! ?state a st:State; log:includes { :SimonsOffice myOnt:musicPlaying [ dc:title “Yellow Submarine” ] } I want Simon’s office to be in a state where “Yellow Submarine” is playing.
  • 48. “Yellow Submarine” a dc:title. Client Goal A proof that is parsed to an internal HTTP request description format Send a GET request to www.ssm.org?title={songTitle} and substitute “songTitle” with “Yellow Submarine”. Then, unpack the response. Its body contains the songURL. Take this variable and send a PUT to www.mp.org?song={songURL} ssm_rule.n3 Execute the requests! mp_rule.n3
  • 49. Send a GET request to www.ssm.org?title={songTitle} and substitute “songTitle” with “Yellow Submarine”. Then, unpack the response. Its body contains the songURL. Take this variable and send a PUT to www.mp.org?song={songURL} Sapple Inc. Media Player mp_rule.n3 GET ?title=YellowSubmarine www.songs.org/yellowsub.mp3 Soodle S.A Songs Search Machine ssm_rule.n3 PUT ?song=www.songs.org/yellowsub.mp3 www.songs.org/yellowsub.mp3 Yay!
  • 50. Sounds complicated. Could you sum up what I have to do? 1. Create your goal and inputs 2. Find a reasoner 3. Ask the reasoner how the goal is reachable 4. Parse the proof to extract the requests 5. Execute the requests
  • 51. I don’t believe this! Let’s see it!
  • 52. http://192.168.0.21:8000/proof? q=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals /user-goal-music.n3& r=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals /user-input.n3 user-input.n3 contains “Yellow Submarine” a dc:title.” Reasoner has access to ssm_rule.n3, mp_rule.n3, and a states ontology. Do Proof!
  • 53. Oh, it actually works… But I can’t understand a thing!
  • 54. Do Proof! http://192.168.0.21:8000/proofHumanReadable? q=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals /user-goal-music.n3& r=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals /user-input.n3 user-input.n3 contains “Yellow Submarine” a dc:title.” Reasoner has access to ssm_rule.n3, mp_rule.n3, and a states ontology.
  • 55. But isn’t it too hard to create the goal, for a client like me? That’s tricky. But we’re working on it and have come up with some good ideas… one of those ideas would even allow for integration with Clickscript ! And you know how to use that, right? Now I get it!
  • 56. Hm. So I should just state my goals. And I can find and use services, too!
  • 57. Semantic Metadata to Support Device Interaction in Smart Environments 57| Simon Mayer http://people.inf.ethz.ch/mayersi  Localization + stream transmitter+receivers in all rooms  First version took 10s to switch songs (mostly loading .n3 files… described in the paper)  Cut that by actively discovering services and pre-loading descriptions This is in the paper… Prototype: A music stream that follows you!
  • 58. Semantic Metadata to Support Device Interaction in Smart Environments 58| Simon Mayer http://people.inf.ethz.ch/mayersi Our Goal
  • 59. Semantic Metadata to Support Device Interaction in Smart Environments 59| Simon Mayer http://people.inf.ethz.ch/mayersi  User defines preferences (temperature, preferred song, etc.)  User agent negotiates with the user’s environment to set these preferences  User gets feedback regarding success/failure Our Goal
  • 60. Thank you! Simon Mayer and Gianin Basler: Semantic Metadata to Support Device Interaction in Smart Environments. Proceedings of the 4th International Workshop on the Web of Things (WoT 2013). Zurich, Switzerland, September 2013
  • 61. Semantic Metadata to Support Device Interaction in Smart Environments 61| Simon Mayer http://people.inf.ethz.ch/mayersi Credits  Nadine Inhelder, Gianin Basler  Ruben Verborgh
  • 62.  http://andrewbleakley.com  http://www.webmarchand.com  http://www.patentspostgrant.com/  http://www.orkin.com  http://sweetclipart.com  http://la-matrice.org  http://learn.randolph.k12.ma.us  http://theparisreview.org  http://bestclipartblog.com  http://freepik.com  http://http://www.rugdots.com  http://www.productdose.com  http://westwoodblinds.co.uk  http://www.ambientdevices.com  http://www.pats.ua.ac.be  http://www.vesternet.com  http://www.rovingnetworks.com  http://www.qualitymechanicalservices.com  http://chantalteakettle.org  http://blog.eeye.com  http://cartographersguild.com  http://haverford.edu  http://clker.com  http://stickers-addict.fr  http://en.wikipedia.org Image Sources