Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
On-the-fly Integration of Static and Dynamic Linked
Data
Andreas Harth (KIT), Craig Knoblock (USC), Steffen Stadtmüller (K...
Outline
Motivation
Scenario and Overview
Modelling Sources: Karma
Accessing and Integrating Sources: Data-Fu
Demo
Conclusi...
Motivation
The relevance of many types of data perishes or degrades over time
(e.g., weather information, moving objects)
...
Static vs. Dynamic Sources
Various sources have different update intervals (from minutes to
weeks)
We treat the access to ...
Scenario

POIs
(Crunchbase, OS
M, Wikimapia)
5

Venues/Events

Buses/Stops

(Eventful, LastFM)

(LA Metro)

On-the-fly Int...
Architecture

6

On-the-fly Integration of Static and Dynamic Linked Data
Karma
Interactive tool for rapidly
extracting, cleaning, transforming, integrating, and publishing data

Tabular
Sources

...
Modelling Sources with Karma
Karma is a data integration tool

Linked API

Map data onto an ontology to generate Linked Da...
Linked Data Access to Event APIs
Venues/Events

LastFM API

(Eventful, LastFM)

Given a lat/lon of a location, return a li...
LastFM Data-Fu Program (I)
Program at http://km.aifb.kit.edu/services/data-fu/lastfm
with input lat/lon (in RDF via HTTP P...
LastFM Data-Fu Program (II)
Rule for retrieving information about the found events, including
geolocation of event:
{ ?e r...
Data Source Characteristics

12

On-the-fly Integration of Static and Dynamic Linked Data
Demo
Load http://people.aifb.kit.edu/aha/2013/d3/index.kml into Google Earth
Location of buses and ships are updated

13

...
Conclusion
System interoperation in distributed environments with Linked Data as
interface
Rapid integration of new source...
Challenges
Data is provided at different places, by different owners, often over the
web (decentralised data publishing)
D...
Upcoming SlideShare
Loading in …5
×

On-the-fly Integration of Static and Dynamic Linked Data

657 views

Published on

Slides of COLD 2013 Paper "On-the-fly Integration of Static and Dynamic Linked Data", Andreas Harth (KIT), Craig Knoblock (USC), Steffen Stadtmüller (KIT), Rudi Studer (KIT), Pedro Szekely (USC)

Published in: Technology
  • Be the first to comment

  • Be the first to like this

On-the-fly Integration of Static and Dynamic Linked Data

  1. 1. On-the-fly Integration of Static and Dynamic Linked Data Andreas Harth (KIT), Craig Knoblock (USC), Steffen Stadtmüller (KIT), Rudi Studer (KIT), Pedro Szekely (USC) INSTITUTE OF APPLIED INFORMATICS AND FORMAL DESCRIPTION METHODS (AIFB) KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association www.kit.edu
  2. 2. Outline Motivation Scenario and Overview Modelling Sources: Karma Accessing and Integrating Sources: Data-Fu Demo Conclusion 2 On-the-fly Integration of Static and Dynamic Linked Data
  3. 3. Motivation The relevance of many types of data perishes or degrades over time (e.g., weather information, moving objects) Timely decision making requires access to live data and inclusion of new sources in a flexible manner. Our goals (Near) real-time access to a variety of data sources in a range of data formats and access modalities Rapidly integrate sources via modeling and to generate a Linked Data interface to live sources 3 On-the-fly Integration of Static and Dynamic Linked Data
  4. 4. Static vs. Dynamic Sources Various sources have different update intervals (from minutes to weeks) We treat the access to all sources in the same way via polling (HTTP GETs) Thus, the only distinction between „static“ and „dynamic“ sources is how fast we refresh the query results for each source 4 On-the-fly Integration of Static and Dynamic Linked Data
  5. 5. Scenario POIs (Crunchbase, OS M, Wikimapia) 5 Venues/Events Buses/Stops (Eventful, LastFM) (LA Metro) On-the-fly Integration of Static and Dynamic Linked Data Vehicles (Campus Cruisers) Marine Vessels (AIS)
  6. 6. Architecture 6 On-the-fly Integration of Static and Dynamic Linked Data
  7. 7. Karma Interactive tool for rapidly extracting, cleaning, transforming, integrating, and publishing data Tabular Sources Karma Hierarchical Sources Database Services Model … See http://isi.edu/integration/karma/ for more info and download 7 On-the-fly Integration of Static and Dynamic Linked Data
  8. 8. Modelling Sources with Karma Karma is a data integration tool Linked API Map data onto an ontology to generate Linked Data Karma extension to enable the on-the-fly lifting of API I/O data according to a pre-defined mapping model Web API Vehicles (Campus Cruisers) 8 On-the-fly Integration of Static and Dynamic Linked Data
  9. 9. Linked Data Access to Event APIs Venues/Events LastFM API (Eventful, LastFM) Given a lat/lon of a location, return a list of event identifiers http://km.aifb.kit.edu/services/lastfmwrap/geo.getevents? lat={?lat}&long={?lon} Given an event identifier, return details about the event http://lastfm.rdfize.com/events/{event-id} Eventful API List events given a keyword search term and a date range http://km.aifb.kit.edu/services/eventfulwrap/search?locat ion={?loc}&date={?date} 9 On-the-fly Integration of Static and Dynamic Linked Data
  10. 10. LastFM Data-Fu Program (I) Program at http://km.aifb.kit.edu/services/data-fu/lastfm with input lat/lon (in RDF via HTTP POST) Rule to search for events at given location: { ?p geo:long ?lon . ?p geo:lat ?lat . } => { [] http:mthd http:GET ; http:requestURI <http://km.aifb.kit.edu/services/lastfm wrap/geo.getevents?lat={?lat}&long={?lo n}> . } . 10 On-the-fly Integration of Static and Dynamic Linked Data “For the input point with lat/long perform an HTTP GET at the KIT LastFM Wrapper URI constructed with the lat/long”
  11. 11. LastFM Data-Fu Program (II) Rule for retrieving information about the found events, including geolocation of event: { ?e rdf:type lode:Event. } => { [] http:mthd http:GET ; http:requestURI ?e . } . “For every resource of type event perform an HTTP GET at the resource URI” Query to return a table with lat/lon and label to transform to KML/Google Earth: :q1 qrl:select ( ?event ?place ?label ?lat ?lon ) ; qrl:where { ?event <http://purl.org/NET/c4dm/event.owl#place> ?place . ?event rdfs:label ?label . “Output is every entity with ?place geo:lat ?lat . latitude, longitude and associated ?place geo:long ?lon . label” } . 11 On-the-fly Integration of Static and Dynamic Linked Data
  12. 12. Data Source Characteristics 12 On-the-fly Integration of Static and Dynamic Linked Data
  13. 13. Demo Load http://people.aifb.kit.edu/aha/2013/d3/index.kml into Google Earth Location of buses and ships are updated 13 On-the-fly Integration of Static and Dynamic Linked Data
  14. 14. Conclusion System interoperation in distributed environments with Linked Data as interface Rapid integration of new sources (via Karma models and Data-Fu scripts) Realtime access to networked data via Data-Fu scripts/programs http://code.google.com/p/data-fu/ Ability to rapidy integrate new sources via Karma models http://www.isi.edu/integration/karma/ Future work Modular organisation of programs Manipulating resource state (Read-Write Linked Data) Optimisations for limited bandwidth environments 14 On-the-fly Integration of Static and Dynamic Linked Data
  15. 15. Challenges Data is provided at different places, by different owners, often over the web (decentralised data publishing) Data and links are provided in a many different formats/protocols Developers have to gain a deep understanding of every API by reading textual descriptions Applications (user agents) are supposed to follow links as found during runtime of the application Developers have to define their desired interaction at design time Developers have to write individually tailored code to consume services in applications 15 On-the-fly Integration of Static and Dynamic Linked Data

×