EP-SPARQL: A Unified Language for Event Processing and Stream Reasoning
1. EP-SPARQL: A Unified Language for Event
Processing and Stream Reasoning
Darko Anicic1 Paul Fodor2 Sebastian Rudolph3
Nenad Stojanovic1
1 FZI Research Center for Information Technology, Karlsruhe, Germany
2 State University of New York at Stony Brook, USA
3 Karlsruhe Institute of Technology, Karlsruhe, Germany
WWWW 2011, Hyderabad, India
2. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Introduction
Real-time data appear increasingly today everywhere.
How to effectively process this data?
Financial sector
to find dealing opportunities across available assets
to detect fraud and enable real-time surveillance
to monitor operational risks
Traffic control systems
to observe traffic-update events and (re)plan the traffic
to route the traffic and optimise paths
Sensor networks
to process sensor data and detect real-time observations
e.g., weather observations like tsunamis, hurricanes etc.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
3. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Introduction
Real-time data appear increasingly today everywhere.
How to effectively process this data?
Financial sector
to find dealing opportunities across available assets
to detect fraud and enable real-time surveillance
to monitor operational risks
Traffic control systems
to observe traffic-update events and (re)plan the traffic
to route the traffic and optimise paths
Sensor networks
to process sensor data and detect real-time observations
e.g., weather observations like tsunamis, hurricanes etc.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
4. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Introduction
Real-time data appear increasingly today everywhere.
How to effectively process this data?
Financial sector
to find dealing opportunities across available assets
to detect fraud and enable real-time surveillance
to monitor operational risks
Traffic control systems
to observe traffic-update events and (re)plan the traffic
to route the traffic and optimise paths
Sensor networks
to process sensor data and detect real-time observations
e.g., weather observations like tsunamis, hurricanes etc.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
5. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Introduction
Real-time data appear increasingly today everywhere.
How to effectively process this data?
Financial sector
to find dealing opportunities across available assets
to detect fraud and enable real-time surveillance
to monitor operational risks
Traffic control systems
to observe traffic-update events and (re)plan the traffic
to route the traffic and optimise paths
Sensor networks
to process sensor data and detect real-time observations
e.g., weather observations like tsunamis, hurricanes etc.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
6. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Supporting Real-Time Information Processing
An event is something that occurs, happens or changes
the current state of affairs.
To detect more complex dynamic matters, (simpler) events
are combined into complex events.
Event Processing
deals with the task of processing events with the goal of
identifying meaningful situations, using event operators as well
as temporal and semantic relationships.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
7. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Supporting Real-Time Information Processing
An event is something that occurs, happens or changes
the current state of affairs.
To detect more complex dynamic matters, (simpler) events
are combined into complex events.
Event Processing
deals with the task of processing events with the goal of
identifying meaningful situations, using event operators as well
as temporal and semantic relationships.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
8. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Supporting Real-Time Knowledge Processing
Current EP systems provide on-the-fly analysis of data
streams, but fall short of combining events with higher-level
background knowledge.
Background knowledge describes the context or domain in
which events are interpreted.
Reasoning techniques are necessary for handling
background knowledge as events occur!
Stream Reasoning
deals with the task of conjunctively reasoning over streaming
data and background knowledge.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
9. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Supporting Real-Time Knowledge Processing
Current EP systems provide on-the-fly analysis of data
streams, but fall short of combining events with higher-level
background knowledge.
Background knowledge describes the context or domain in
which events are interpreted.
Reasoning techniques are necessary for handling
background knowledge as events occur!
Stream Reasoning
deals with the task of conjunctively reasoning over streaming
data and background knowledge.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
10. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Toward Real-Time Semantic Web
Rapidly changing data
represented as events
handles
Event Processing
(EP)
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
11. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Toward Real-Time Semantic Web
Rapidly changing data Static or slowly evolving
represented as events background knowledge
handles
handles
Event Processing Semantic Web technologies
(EP) including
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
12. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Toward Real-Time Semantic Web
Rapidly changing data Static or slowly evolving
represented as events background knowledge
handles
handles
Event Processing Semantic Web technologies
(EP) including
EP SPARQL EP-SPARQL
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
13. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Toward Real-Time Semantic Web
Rapidly changing data Static or slowly evolving
represented as events background knowledge
handles
handles
Event Processing Semantic Web technologies
(EP) including
EP SPARQL EP-SPARQL
• Temporal relatedness
• Semantic relatedness
• Stream reasoning
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
14. Introduction
EP-SPARQL Motivation
Experimental Results Related Work
Conclusion
Related Work
Streaming Databases
SASE [1]
ZStream [10]
CEDR [6]
TelegraphCQ [5]
Temporal (and Spatial) RDF
Introducing time into RDF [8]
SPARQL-ST [11], Temporal SPARQL [12], stSPARQL [9],
and T-SPARQL [7]
Stream Reasoning
C-SPARQL [3]
Streaming Knowledge Bases [13]
Streaming SPARQL [4]
Incremental reasoning [2]
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
15. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Syntax of the Language
EP-SPARQL
– extends SPARQL to enable event-based processing that
takes into account temporal situatedness of triple assertions.
– syntactical and semantic downward-compatibility to plain
SPARQL.
1 Operators: FILTER , AND, UNION , OPTIONAL , SEQ , EQUALS ,
OPTIONALSEQ , and EQUALSOPTIONAL ;
2 getDURATION() yields a literal of type xsd:duration
giving the time interval associated to the graph pattern;
3 getSTARTTIME() and getENDTIME() retrieve the time
stamps of type xsd:dateTime of the start and end of the
interval;
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
16. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Syntax of the Language
EP-SPARQL
– extends SPARQL to enable event-based processing that
takes into account temporal situatedness of triple assertions.
– syntactical and semantic downward-compatibility to plain
SPARQL.
1 Operators: FILTER , AND, UNION , OPTIONAL , SEQ , EQUALS ,
OPTIONALSEQ , and EQUALSOPTIONAL ;
2 getDURATION() yields a literal of type xsd:duration
giving the time interval associated to the graph pattern;
3 getSTARTTIME() and getENDTIME() retrieve the time
stamps of type xsd:dateTime of the start and end of the
interval;
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
17. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Informal Semantics
RDF stream – a set of triple occurrences s, p, o , tα , tω where
s, p, o is an RDF triple and tα , tω are the start and end of the
interval.
FILTER – restricts variable bindings to those µ, tα , tω for
which the filter expression evaluates to true;
AND – joins µ, tα , tω and µ , tα , tω . The joined tuple has
timestamp tα = min(tα , tα ), tω = max(tω , tω );
UNION – forms the disjunction of µ, tα , tω and µ , tα , tω ;
OPTIONAL – matches µ, tα , tω optionally with µ , tα , tω
when the filter expression evaluates to true;
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
18. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Informal Semantics (cont’d)
SEQ – joins µ, tα , tω and µ , tα , tω only if µ , tα , tω
occurs strictly after µ, tα , tω ;
EQUALS – joins µ, tα , tω and µ , tα , tω if they occur
simultaneously;
OPTIONALSEQ and EQUALSOPTIONAL are
temporal-sensitive variants of OPTIONAL;
CONSTRUCT – generates the stream enriched by triples
from possibly iterative CONSTRUCT rules. SELECT-queries
get evaluated not against the pure input stream but against
the enriched generated stream.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
19. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Example I
Continuously search for companies having a larger than 20%
stock price increase in less than 15 days without having
acquired another company during that period.
SELECT ?company WHERE
{ ?company hasStockprice ?price1 }
SEQ { { ?company hasAcquired ?othercompany }
OPTIONALSEQ
{ ?company hasStockPrice ?price2 } }
FILTER ( ?price2 > ?price1 * 1.2 &&
!BOUND(?othercompany) &&
getDURATION() < "P15D"ˆˆxsd:duration)
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
20. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Example II
Detect slow traffic and its cause which could be used to
automatically modify a speed limit on a certain roads.
SELECT ?road ?speed WHERE
{ ?road tr: slowTrafficDue ?observ }
SEQ {{ ?road tr: slowTrafficDue ?observ }
{ ?observ rdfs:subClassOf tr:SlowTraffic }
{ ?observ wt:speed ?speed }}
FILTER ( getDURATION() < "P1H"ˆˆxsd:duration)
Observ_1 Observ_2
rdf:type tr:GhostDriver ; rdf:type tr:IceConditions ;
wt:speed "50"ˆˆxsd:int . wt:speed "40"ˆˆxsd:int .
tr:GhostDriver rdfs:subClassOf tr:SlowTraffic.
tr:IceConditions rdfs:subClassOf tr:BadWeather.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
24. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Other Operators
SELECT ?company WHERE
...
FILTER ( ?price2 < ?price1 * 0.7 &&
?price3 > ?price1 * 1.05)
FILTER – Rule transformation
condition(Price1, Price2, Price3) : −
P1 is (Price1 ∗ 0.7), P1 >Price2,
P2 is (Price1 ∗ 0.5), Price3>P2 .
EQUALS – Rule transformation
equals(TI1 , TI2 ) : −
TI1 = [TI1 S, TI1 E], validTimeInterval(TI1 ),
TI2 = [TI2 S, TI2 E], validTimeInterval(TI2 ),
TI1 S = TI2 S, TI1 E = TI2 E.
validTimeInterval(TI) ←
TI = [TI S, TI E], TI S@ < TI E.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
25. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Other Operators
SELECT ?company WHERE
...
FILTER ( ?price2 < ?price1 * 0.7 &&
?price3 > ?price1 * 1.05)
FILTER – Rule transformation
condition(Price1, Price2, Price3) : −
P1 is (Price1 ∗ 0.7), P1 >Price2,
P2 is (Price1 ∗ 0.5), Price3>P2 .
EQUALS – Rule transformation
equals(TI1 , TI2 ) : −
TI1 = [TI1 S, TI1 E], validTimeInterval(TI1 ),
TI2 = [TI2 S, TI2 E], validTimeInterval(TI2 ),
TI1 S = TI2 S, TI1 E = TI2 E.
validTimeInterval(TI) ←
TI = [TI S, TI E], TI S@ < TI E.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
26. Introduction
Syntax
EP-SPARQL
Semantics
Experimental Results
Execution Model
Conclusion
Other Operators
SELECT ?company WHERE
...
FILTER ( ?price2 < ?price1 * 0.7 &&
?price3 > ?price1 * 1.05)
FILTER – Rule transformation
condition(Price1, Price2, Price3) : −
P1 is (Price1 ∗ 0.7), P1 >Price2,
P2 is (Price1 ∗ 0.5), Price3>P2 .
EQUALS – Rule transformation
equals(TI1 , TI2 ) : −
TI1 = [TI1 S, TI1 E], validTimeInterval(TI1 ),
TI2 = [TI2 S, TI2 E], validTimeInterval(TI2 ),
TI1 S = TI2 S, TI1 E = TI2 E.
validTimeInterval(TI) ←
TI = [TI S, TI E], TI S@ < TI E.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
27. Introduction
Test 1: Event Processing
EP-SPARQL
Test 2: Stream Reasoning
Experimental Results
Test 3: Example applications
Conclusion
Event Processing
Test pattern: monitoring
Throughput vs. Window size Throughput vs. Window size (Sum over Sequence)
Esper 3.3.0 EALIS-YAP
Throughput (1000 x Events/Sec)
ETALIS-YAP Esper
10000
the average stock price 30
of a company X with
Throughput (Event/Sec)
25
9000
8000 CONSTRUCT queries 20
15
7000
6000
Intel Core Quad CPU 10
5000 Q9400 2,66GHz, 8GB of 5
0
4000 RAM, Ubuntu 9.10
1 3 10
100 500 1000 50000
Count window size
Time window size (Sec)
ETALIS on SWI Prolog
5.6.64 and YAP Prolog Figure: Aggregation over count
5.1.3 vs. Esper 3.3.0 sliding window
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
28. Introduction
Test 1: Event Processing
EP-SPARQL
Test 2: Stream Reasoning
Experimental Results
Test 3: Example applications
Conclusion
Stream Reasoning
!"#$"%$&'()*+,$-,#$.+/,"0(01$2+*/3
%&'()*+,'-
$!!!
Test pattern: infer over
.+/,"0(01$2+*/3$(0$5,
#"!!
streaming triples
whether the subject of a #!!!
triple is an instance of "!!
the class of concern,
!
"!!! #!!!! #"!!! $!!!!
or any of its 40,080 !456+'$"%$&'()*+,
subclasses.
Figure: Delay caused by stream
reasoning
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
29. Introduction
Test 1: Event Processing
EP-SPARQL
Test 2: Stream Reasoning
Experimental Results
Test 3: Example applications
Conclusion
Example application I
No. of Locations vs. Consumed Memory No. of Locations vs. Consumed Time
1 Visitor 10 Visitors 1 Visitor 10 Visitors
2000
Goods Delivery System 1500
1295
Consumed time in ms
in the city of Milan;
Consumed Memory in kB
1503 1076
1500
1000 889
RDF knowledge base to 1039
1000 810
represent locations and 500
500
traffic links;
204
86 119 165 31 94 109 130
35 3
0
0
The system “listens” to
5 10 15 20 5 10 15 20
Number of locations Number of locations
traffic-update events and
reschedule paths. Figure: Delay caused by
processing
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
30. Introduction
Test 1: Event Processing
EP-SPARQL
Test 2: Stream Reasoning
Experimental Results
Test 3: Example applications
Conclusion
Example application II
Pressure difference
Threshold
0.08
A tsunami detection 0.06
system; 0.04
Real buoy sensor data
Pressure
0.02
continuously processed; 0
GeoNames to provide -0.02
geographical places -0.04
Time
within a certain radius
from the sensor location. Figure: Tsunami detection
histogram
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
31. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
Conclusion
1 Addressing dynamics and notification on the Web has
become an important area of research;
2 The challenge is to get advantage of real-time data, and
recognise important situations in a timely fashion;
3 EP-SPARQL – a new language for Event Processing and
Stream Reasoning;
4 Future work: complete implementation, more expressive
formalisms, and adaptive optimizations.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
32. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
Thank you! Questions...
ETALIS is open source:
http://code.google.com/p/etalis
On-line demo:
http://etalis.fzi.de
Contact:
Darko.Anicic@fzi.de
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
33. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
Jagrati Agrawal, Yanlei Diao, Daniel Gyllstrom, and Neil
Immerman.
Efficient pattern matching over event streams.
In Proceedings of the 28th ACM SIGMOD Conference,
pages 147–160, 2008.
Davide Francesco Barbieri, Daniele Braga, Stefano Ceri,
Emanuele Della Valle, and Michael Grossniklaus.
Incremental reasoning on streams and rich background
knowledge.
In Proceedings of the 7th Extended Semantic Web
Conference (ESWC’10), pages 1–15, 2010.
Davide Francesco Barbieri, Daniele Braga, Stefano Ceri,
and Michael Grossniklaus.
An execution environment for C-SPARQL queries.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
34. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
In Proceedings of the 13th International Conference on
Extending Database Technology (EDBT’10), pages
441–452, 2010.
Andre Bolles, Marco Grawunder, and Jonas Jacobi.
Streaming SPARQL - extending SPARQL to process data
streams.
In Proceedings of the 5th European Semantic Web
Conference (ESWC’08), pages 448–462, 2008.
Sirish Chandrasekaran, Owen Cooper, Amol Deshpande,
Michael J. Franklin, Joseph M. Hellerstein, Wei Hong,
Sailesh Krishnamurthy, Samuel Madden, Vijayshankar
Raman, Frederick Reiss, and Mehul A. Shah.
Telegraphcq: Continuous dataflow processing for an
uncertain world.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
35. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
In Proceedings of the 1st Biennial Conference on
Innovative Data Systems Research (CIDR’03), 2003.
Mitch Cherniack, Hari Balakrishnan, Magdalena
Balazinska, Donald Carney, Ugur Cetintemel, Ying Xing,
¸
and Stanley B. Zdonik.
Scalable distributed stream processing.
In Proceedings of the 1st Biennial Conference on
Innovative Data Systems Research (CIDR’03), 2003.
Fabio Grandi.
T-SPARQL: a TSQL2-like temporal query language for
RDF.
In International Workshop on on Querying Graph
Structured Data, pages 21–30, 2010.
Claudio Gutierrez, Carlos A. Hurtado, and Alejandro A.
Vaisman.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
36. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
Introducing time into rdf.
The IEEE Transactions on Knowledge and Data
Engineering, 19(2):207–218, 2007.
Manolis Koubarakis and Kostis Kyzirakos.
Modeling and querying metadata in the semantic sensor
web: The model stRDF and the query language stSPARQL.
In Proceedings of the 7th Extended Semantic Web
Conference (ESWC’10), pages 425–439, 2010.
Yuan Mei and Samuel Madden.
Zstream: a cost-based query processor for adaptively
detecting composite events.
In Proceedings of the 29th ACM SIGMOD Conference,
pages 193–206, 2009.
Matthew Perry, Amit P. Sheth, and Prateek Jain.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011
37. Introduction
Conclusion
EP-SPARQL
Example
Experimental Results
Semantics
Conclusion
SPARQLST: Extending SPARQL to support spatiotemporal
queries.
In Technical Report. KNOESIS-TR-2009-01, 2008.
Jonas Tappolet and Abraham Bernstein.
Applied temporal RDF: Efficient temporal querying of RDF
data with SPARQL.
In Proceedings of the 6th European Semantic Web
Conference (ESWC’09), pages 308–322, 2009.
Onkar Walavalkar, Anupam Joshi, Tim Finin, and Yelena
Yesha.
Streaming knowledge bases.
In International Workshop on Scalable Semantic Web
Knowledge Base Systems, 2008.
D. Anicic, P. Fodor, S. Rudolph, N. Stojanovic EP-SPARQL – ETALIS – WWW 2011