The document discusses stream reasoning, which involves performing logical reasoning over data streams in real-time to support decision making. It presents background on data streams and stream processing, introduces the concept of stream reasoning, discusses past achievements including defining RDF streams and a continuous query language (C-SPARQL), and presents the authors' approach for incremental reasoning over streams using background knowledge.

1. Incremental Reasoning on Streams and Rich Background Knowledge http://streamreasoning.org Emanuele Della Valle DEI - Politecnico di Milano [email_address] http://emanueledellavalle.org Joint work with: Davide Francesco Barbieri, Daniele Braga, Stefano Ceri, and Michael Grossniklaus

2. Agenda Motivation Background Stream Reasoning Concept Past Achievements Main Contribution Retrospective and Conclusions ESWC 2010, Heraklion, Greece, June 1st, 2010

3. Motivation It‘s a streaming World! [IEEE-IS2009] Sensor networks, … traffic engineering, … social networking, … … and many others … generate streams! ESWC 2010, Heraklion, Greece, June 1st, 2010

4. Motivation Questions People are Asking Given this brand of turbine, what is the expected time to failure when the barring starts to vibrate as now detected? Is a traffic jam going to happen in this highway? And is then convenient to reallocate travelers based upon the forecast? Who are the opinion makers? i.e., the users who are likely to influence the behavior of other users who follow them ESWC 2010, Heraklion, Greece, June 1st, 2010

5. Motivation Problem Statement Making sense in real time of gigantic and inevitably noisy data streams in order to support the decision process of extremely large numbers of concurrent users ESWC 2010, Heraklion, Greece, June 1st, 2010

6. Background What are data streams anyway? Formally: Data streams are unbounded sequences of time-varying data elements Less formally: an (almost) “continuous” flow of information with the recent information being more relevant as it describes the current state of a dynamic system time ESWC 2010, Heraklion, Greece, June 1st, 2010

7. Background Can the Semantic Web Process Data Stream? The Semantic Web, the Web of Data is doing fine RDF, RDF Schema, SPARQL, OWL, DL well understood theory, rapid increase in scalability BUT it pretends that the world is static or at best a low change rate both in change-volume and change-frequency ontology versioning belief revision time stamps on named graphs It sticks to the traditional one-time semantics ESWC 2010, Heraklion, Greece, June 1st, 2010

8. Background Continuous Semantics Processing data streams in the space of one-time semantics is difficult because of the very nature of the underlying data Innovative * assumption: continuous semantics! streams can be consumed on the fly rather than being stored forever and queries are registered and continuously produce answers * This innovation arose in DB community in ’90s ESWC 2010, Heraklion, Greece, June 1st, 2010

9. Background Stream Processing Continuous queries registered over streams that are observed trough windows ESWC 2010, Heraklion, Greece, June 1st, 2010 window input stream stream of answer Registered Continuous Query

10. Background Key Optimization in Stream Processing When a continuous query is registered, generate a query execution plan New plan merged with existing plans Global scheduler for plan execution maximizing experience gathered with previous queries. ESWC 2010, Heraklion, Greece, June 1st, 2010

11. Background Data Stream Management Systems (DSMS) Research Prototypes Amazon/Cougar (Cornell) – sensors Aurora (Brown/MIT) – sensor monitoring, dataflow Gigascope: AT&T Labs – Network Monitoring Hancock (AT&T) – Telecom streams Niagara (OGI/Wisconsin) – Internet DBs & XML OpenCQ (Georgia) – triggers, view maintenance Stream (Stanford) – general-purpose DSMS Stream Mill (UCLA) - power & extensibility Tapestry (Xerox) – publish/subscribe filtering Telegraph (Berkeley) – adaptive engine for sensors Tribeca (Bellcore) – network monitoring High-tech startups Streambase, Coral8, Apama, Truviso Major DBMS vendors are all adding stream extensions as well Oracle http://www.oracle.com/technology/products/dataint/htdocs/streams_fo.html DB2 http://www.eweek.com/c/a/Database/IBM-DB2-Turns-25-and-Prepares-for-New-Life/ ESWC 2010, Heraklion, Greece, June 1st, 2010

12. Concept Stream Reasoning [IEEE-IS2009,Dagstuhl2010] Idea origination Can continuous semantics be ported to reasoning? This is an unexplored yet high impact research area! Stream Reasoning Logical reasoning in real time on gigantic and inevitably noisy data streams in order to support the decision process of extremely large numbers of concurrent users. -- S. Ceri , E. Della Valle , F. van Harmelen and H. Stuckenschmidt , 2010 Note: making sense of streams necessarily requires processing them against rich background knowledge ESWC 2010, Heraklion, Greece, June 1st, 2010

13. Concept Research Challenges (selection) [IEEE-IS2009] Relation with data-stream systems Just as RDF relates to data-base systems? Query languages for semantic streams Just as SPARQL for RDF but with continuous semantics? Reasoning on Streams Efficient incremental updates of deductive closures? How to combine streams and background knowledge? Distributed and parallel processing Streams are parallel in nature Real time constrains A reasoning task must be completed before the answer become useless ESWC 2010, Heraklion, Greece, June 1st, 2010

14. Past Achievements Explored Continuous Semantics for SeWeb We gave up one-time semantics in Semantic Web and explored the benefits provided by continuous semantics when dealing with streams We investigated RDF streams [WWW2009] the natural extension of the RDF data model to the new continuous scenario and Continuous SPARQL (or simply C-SPARQL ) [WWW2009, EDBT2010] A syntactic and semantic extension of SPARQL for querying RDF streams ESWC 2010, Heraklion, Greece, June 1st, 2010

15. Past Achievements RDF Stream RDF Stream Data Type Ordered sequence of pairs, where each pair is made of an RDF triple and its timestamp t (< triple >, t) E.g., (<:ourmaninsa :isIn :Munich>, 2010-05-31T18:34:41) (<:MadamMichelle :isIn :SouthAfrica >, 2010-05-31T18:24:28) (<:Ayngelina :isIn :Nicaragua >, 2010-05-31T18:19:21) ESWC 2010, Heraklion, Greece, June 1st, 2010 “ just arrived in”

16. Past Achievements An Example of C-SPARQL Query Who has landed in USA in the last hour? REGISTER QUERY WhoHasLandedInUSAinTheLastHour AS PREFIX gno: <http://www.geonames.org/ontology#> PREFIX c: < http://www.geonames.org/countries/#> PREFIX : <http://example> SELECT ?traveller ?place ?type FROM <http://sws.geonames.org/nonExistingUSfeatureGraph> FROM STREAM <http://someStreamGeneratedFromTwitter> [ RANGE 60m STEP 5m ] WHERE { ?traveller :isIn ?place . ?place gno:inCountry c:US . ?place gno:featureCode ?type . } ESWC 2010, Heraklion, Greece, June 1st, 2010

17. Past Achievements An Example of C-SPARQL Query Explained Who has landed in USA in the last hour? REGISTER QUERY WhoHasLandedInUSAinTheLastHour AS PREFIX gno: <http://www.geonames.org/ontology#> PREFIX c: < http://www.geonames.org/countries/#> PREFIX : <http://example> SELECT ?traveller ?place ?type FROM <http://sws.geonames.org/nonExistingUSfeatureGraph> FROM STREAM <http://someStreamGeneratedFromTwitter> [ RANGE 60m STEP 5m ] WHERE { ?traveller :isIn ?place . ?place gno:inCountry c:US . ?place gno:featureCode ?type . } ESWC 2010, Heraklion, Greece, June 1st, 2010 Combined with triples a RDF graph triples from a stream Query registration (for continuous execution) FROM STREAM clause WINDOW

18. Past Achievements C-SPARQL Engine Architecture We implemented a C-SPARQL engine based on LarKC conceptual framework ESWC 2010, Heraklion, Greece, June 1st, 2010 Performed by a DSMS Select Abstract Reason Streamed Input Window Content RDF Streams Answers Streams Window RDF Graphs

19. Main Contribution Achievements vs. Research Challenges Relation with data-stream systems Notion of RDF stream [WWW2009] Query languages for semantic streams C-SPARQL [WWW2009,EDBT2010] Reasoning on Streams Efficient incremental updates of deductive closures How to combine streams and background knowledge Distributed and parallel processing Streams are parallel in nature Real time constrains A reasoning task must be completed before the answer become useless Contribution of this work ESWC 2010, Heraklion, Greece, June 1st, 2010

20. Main Contribution State-of-the-Art Approach [Ceri1994,Volz2005] Overestimation of deletion : Overestimates deletions by computing all direct consequences of a deletion. Rederivation : Prunes those estimated deletions for which alternative derivations (via some other facts in the program) exist. Insertion : Adds the new derivations that are consequences of insertions to extensional predicates. ESWC 2010, Heraklion, Greece, June 1st, 2010

21. Main Contribution Our approach 1/2 Assumption Insertions and deletions are triples respectively entering and exiting the window The window size is known Therefore The time when each triple will expire is known and determined by the window size E.g. if the window is 10s long a triple entering at time t then it will exit at time t+10s Note: all knowledge can be annotated with an expiration time i.e., background knowledge is annotated with +  ESWC 2010, Heraklion, Greece, June 1st, 2010

22. Main Contribution Our approach 2/2 The algorithm computes the entailments derived by the inserts, annotates each entailed triple with a expiration time, and eliminates from the current state all copies of derived triples except the one with the highest timestamp. ESWC 2010, Heraklion, Greece, June 1st, 2010

23. Main Contribution Our Approach at Work ESWC 2010, Heraklion, Greece, June 1st, 2010 12 Jan 2009 A B A B C 1 2 TS Triples in the Window Entailments in the Window A C [11] [11] [11] [12] A B C 3 A C [11] [11] [12] D [13] D B [12] [11] A B C 4 A C [11] [11] [12] D [13] D B [12] [11] E [14] [14] [14] x A B C 12 A C [12] D [13] D B [12] E [14] [14] [14] A C 13 A D [13] D E [14] [14] [14] [11] [11] 11

24. Main Contribution Comparative Evaluation Hypothesis Background knowledge do not change and it is materialized Changes only take place in the window An experiment comparing the time required to compute a new materialization using Re-computing from scratch (i.e.,1250 ms in our setting) State of the art incremental approach [Volz, 2005] Our approach Results at increasing % of the triples updated . ESWC 2010, Heraklion, Greece, June 1st, 2010

25. Retrospective and Conclusions Achievements vs. Research Challenges Relation with data-stream systems Notion of RDF stream :-| Query languages for semantic streams C-SPARQL :-D Reasoning on Streams Efficient incremental updates of deductive closures This paper :-) ... but much more work is needed! How to combine streams and background knowledge This paper :-| ... but a lot needs to be studied ... Distributed and parallel processing Future work :-P Real time constrains Future work :-P ESWC 2010, Heraklion, Greece, June 1st, 2010

26. References (selection) Vision [IEEE-IS2009] Emanuele Della Valle, Stefano Ceri, Frank van Harmelen, Dieter Fensel It's a Streaming World! Reasoning upon Rapidly Changing Information . IEEE Intelligent Systems 24(6): 83-89 (2009) bibtex Continuous SPARQL (C-SPARQL) [EDBT2010] Davide Francesco Barbieri, Daniele Braga, Stefano Ceri and Michael Grossniklaus. An Execution Environment for C-SPARQL Queries . EDBT 2010 [WWW2009] Davide Francesco Barbieri, Daniele Braga, Stefano Ceri, Emanuele Della Valle, Michael Grossniklaus: C-SPARQL: SPARQL for continuous querying . WWW 2009: 1061-1062 bibtex Stream Reasoning [Dagstuhl2010] Heiner Stuckenschmidt, Stefano Ceri, Emanuele Della Valle and Frank van Harmelen. Towards Expressive Stream Reasoning. Proceedings of the Dagstuhl Seminar on Semantic Aspects of Sensor Networks, 2010. ESWC 2010, Heraklion, Greece, June 1st, 2010

27. Thank You! Questions? Much More to Come! Keep an eye on http://www.streamreasoning.org ESWC 2010, Heraklion, Greece, June 1st, 2010

28. Back-up Slides The Entailment Regime That We Used In the current implementation we support RDF-S++ rdf:type rdfs:subClassOf rdfs:domain and rdfs:range rdfs:subPropertyOf owl:sameAs owl:inverseOf owl:TransitiveProperty ESWC 2010, Heraklion, Greece, June 1st, 2010

29. Back-up Slides Volz 2005 rewriting rules ESWC 2010, Heraklion, Greece, June 1st, 2010

30. Back-up Slides Example of maintenance program Original Rule Maintenance Program ESWC 2010, Heraklion, Greece, June 1st, 2010

31. Back-up Slides Our rewriting rules ESWC 2010, Heraklion, Greece, June 1st, 2010

32. Back-up Slides Example of maintenance program for streams Original Rule Maintenance Program ESWC 2010, Heraklion, Greece, June 1st, 2010

33. Back-up Slides Simple Stream Reasoner Architecture ESWC 2010, Heraklion, Greece, June 1st, 2010

34. Achievements Incremental Reasoning: State-of-the-Art Incremental Maintenance of Materialized Views Stefano Ceri, Jennifer Widom: Deriving Incremental Production Rules for Deductive Data. Inf. Syst. 19(6): 467-490 (1994) HA Kuno, EA Rundensteiner: Incremental Maintenance of Materialized Object-Oriented Views in MultiView: Strategies and Performance Evaluation. TDKE 1998 Raphael Volz, Steffen Staab, Boris Motik: Incrementally Maintaining Materializations of Ontologies Stored in Logic Databases. J. Data Semantics 2: 1-34 (2005) Incremental Rule-based Reasoning F Fabret, M Regnier, E Simon: An Adaptive Algorithm for Incremental Evaluation of Production Rules in Databases. VLDB 1993 B. Berster: Extending the RETE Algorithm for Event Management.TIME’02 Incremental DL Reasoning Cuenca-Grau et al : History Matters: Incremental Ontology Reasoning Using Modules. ISWC 2007. Parsia et al: Towards incremental reasoning through updates in OWL-DL. - Reasoning on the Web-Workshop at WWW-2006 ESWC 2010, Heraklion, Greece, June 1st, 2010