Triplewave: a step towards RDF Stream Processing on the Web

Department of Informatics
TripleWave: a step towards
RDF Stream Processing on the
Web
Daniele Dell’Aglio
dellaglio@ifi.uzh.ch http://dellaglio.org @dandellaglio
Galway, 16.12.2016

An (incomplete) overview of the RSP/SR research
IMaRS
STARQL
DynamiTE
TROWL
EuropeMapfromWikipedia
StreamRule
StarCITY
SPARKWAVE
LARS
Key
Daniele Dell'Aglio - TripleWave 2/28

Connecting RSPs on the Web
Morph
Streams
CSPARQL
Etalis
TrOWL
Stream
Rule
CQELS
How far are we?
• Working prototypes/systems
• Formal models for RSPs and reasoning
• Minimal agreements: standards, serialization, interfaces

Looking for minimal agreements: the RSP
Community group
Research work
Many Papers
PhD Theses
Datasets
Prototypes
Benchmarks
RDF Streams
Stream Reasoning
Complex Event Processing
Stream Query Processing
Stream Compression
Semantic Sensor Web
Manytopics
Tonsofwork
http://www.w3.org/community/rsp
W3C RSP Community Group
Effort to our work on RDF stream processing
discuss
standardize
combine
formalize
evangelize

W3C RSP Documents
https://www.w3.org/community/rsp/
http://streamreasoning.github.io/RSP-QL/RSP_Requirements_Design_Document/
Use cases
Implementations
State of the Art
Challenges & issues
Requirements
Design principles
• RDF Stream model
• RDF Stream query language
Abstract syntax RDF Streams

But...
W3C RSP set some foundations and requirements, but:
• Standard protocols and exchanging mechanisms for
RDF stream are missing.
• We need generic and flexible solutions for making RDF
streams available and exchangeable on the Web.

TripleWave
TripleWave is an open-source
framework for creating and publishing
RDF streams over the Web.
Triple
Wave
how?input?
RDF Stream
what is it?

TripleWave’s RDF streams
TripleWave should exploit and be compatible to
existing standards and recommendations
• The data model should be compatible with the abstract
model defined by the W3C RSP CG
• The output format compatible with RDF

TripleWave serialization format
In TripleWave, an RDF stream is an (infinite) ordered
sequence of time-annotated data items (RDF graphs)…
... serialized in JSON-LD
[{ "@graph": {
"@id": "http://.../G1",
{"@id": "http://.../a",
"http://.../isIn": {"@id":"http://.../rRoom"}}
},{ "@id": "http://.../G1",
"generatedAt":"2016-16-12T00:01:00"
}
},
{ "@graph": {
"@id": "http://.../G2",
{"@id": "http://.../b",
"http://.../isIn": {"@id":"http://.../rRoom"}}
},{ "@id": "http://.../G2",
"generatedAt":" 2016-16-12T00:03:00"
}
},…
G1
G2
G3
{:a :isIn :rRoom}
{:b :isIn :bRoom}
{:c :talksIn :rRoom,
:d :talksIn :bRoom}
S
3
5
1
t

Building TripleWave
Triple
Wave
how?input?
RDF Streams

Spreading the RDF streams
TripleWave must be able to provide the stream to
RDF Stream Processing engines (query
processors and reasoners) through the Web.
• HTTP
• HTTP chunk
• Web sockets
• MQTT (upcoming)

TripleWave Stream Descriptor
TripleWave must provide information about how
to access the stream
• TripleWave exposes an RDF description of the RDF
stream
• RDF Stream Descriptor (sGraph)
• It contains:
• The identifier of the stream
• Data item samples (see next slide)
• A description of the schema
• The location of the stream endpoint (e.g. WebSocket URL)

TripleWave Stream Descriptor - Example

Building TripleWave
Triple
Wave
input?
RDF Streams
(Web socket |
HTTP-chunk |
etc.)
RDF Stream
Descriptor

Feeding TripleWave
TripleWave should support a variety of data
sources.
• RDF dumps with temporal information
• RDF with temporal information exposed through
SPARQL endpoints
• Streams available on the Web

From RDF to RDF streams
Converts RDF stored in files/SPARQL endpoints
• Containing some time information
… into an RDF stream
• continuous flow of RDF data
• ordered according the original timestamps
• the time between two items is preserved
Use Cases
• Evaluation, testing and benchmarking
• Simulation systems

From Web stream to RDF stream
Consumes an existing Web stream…
• through connectors
… and converts it into an RDF Stream
• Each data item is lifted to RDF
Use Cases
• Querying and reasoning
• Data integration
Web
Service Connector TW Core
Web Service API

Convertion is made through R2RML
• Mappings to convert each data item in RDF
Example: map a field
{
“userUrl”:”foo”
}
rr:predicateObjectMap [
rr:predicate schema:agent;
rr:objectMap [ rr:column "userUrl"] ];
{
"https://schema.org/agent": {"@id": ”foo"},
}

Example: map a field with template
{
“time”:”value”
}
rr:subjectMap [
rr:template ”something {time}”
{
“@id”:”something value”
}

Example: add a new constant field
rr:predicateObjectMap
[ rr:predicate rdf:type; rr:objectMap
[ rr:constant schema:UpdateAction]];
{
"http://www.w3.org/1999/02/22-rdf-syntax-ns#type":
{"@id": "https://schema.org/UpdateAction"}
}

Building TripleWave
Replay
Conversion to
RDF Stream
R2RML
Mappings
Live Non-RDF Streams
JSON
RDF
Time-annotated
RDF datasets
Finite RDF
substreams
Replay Loop
Triple
Wave RDF Streams
(Web socket |
HTTP-chunk |
etc.)
RDF Stream
Descriptor

Implementing TripleWave
TripleWave is a NodeJS Web Application
• NodeJS is a JavaScript runtime built on Chrome's V8 JavaScript
engine.
TripleWave is open source
• Released with a Apache 2.0 licence
• Source code available at:
https://github.com/streamreasoning/TripleWave

TripleWave Architecture
Web API
Transform
Stream
Graph
Stream
Connector
Stream
Datagen
Stream
Scheduler
Stream
Web
Service
SPARQL
Endpoint
File
R2RML
Mapping
Conversion

TripleWave Architecture
Web API
Transform
Stream
Graph
Stream
Connector
Stream
Datagen
Stream
Scheduler
Stream
Web
Service
SPARQL
Endpoint
File
R2RML
Mapping
Replay
Replay loop

Consuming TripleWave RDF Stream - Push
The TripleWave stream can be consumed via push by
extending the RSP service framework1
1https://github.com/streamreasoning/rsp-services
TripleWave
RSP-
Service
C-SPARQL
Register the stream, the query
and the observers
Connect to the
RDF stream
descriptor
Connect to the
RDF stream
endpoint Declare the stream, the query
and the observers
Inject the stream

Show cases
Three demos have been deployed to show the capabilities
of the system.
Wikipedia changes stream conversion.
http://131.175.141.249/TripleWave-transform/sgraph
Endlessly replay as a stream the Linked Sensor Data
dataset.
http://131.175.141.249/TripleWave-endless/sgraph
Endlessly replay as a stream the LDBC social graph dataset.
http://131.175.141.249/TripleWave-ldbc/sgraph

Find more...
• Andrea Mauri, Jean-Paul Calbimonte, Daniele Dell’Aglio, Marco
Balduini, Marco Brambilla, Emanuele Della Valle, Karl Aberer:
TripleWave: Spreading RDF Streams on the Web. Resource Paper
at International Semantic Web Conference 2016.
• Andrea Mauri, Jean-Paul Calbimonte, Daniele Dell’Aglio, Marco
Balduini, Emanuele Della Valle, Karl Aberer: Where Are the RDF
Streams?: On Deploying RDF Streams on the Web of Data with
TripleWave. Poster at International Semantic Web Conference
2015.
• A special thanks to Jean-Paul Calbimonte and Andrea Mauri for
supplying me parts of the today slides

Conclusions
RDF streams are getting a momentum
• Several active research groups
• Prototypes, methods and applications
TripleWave shows that it is possible to exchange RDF
streams over the Web
• It uses standard technologies
• It feeds C-SPARQL (and soon CQELS)
There is a potential huge value in putting together the
results we are obtaining

Thank you! Questions?
TripleWave: a step towards RDF Stream
Processing on the Web
http://streamreasoning.github.io/TripleWave
Daniele Dell’Aglio
dellaglio@ifi.uzh.ch
http://dellaglio.org
@dandellaglio

Triplewave: a step towards RDF Stream Processing on the Web

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Triplewave: a step towards RDF Stream Processing on the Web

Similar to Triplewave: a step towards RDF Stream Processing on the Web (20)

More from Daniele Dell'Aglio

More from Daniele Dell'Aglio (20)

Recently uploaded

Recently uploaded (20)

Triplewave: a step towards RDF Stream Processing on the Web