This document discusses incremental materialization for RDF streams (IMaRS). IMaRS is an approach for incremental reasoning over sliding windows of RDF streams. It avoids recomputing the entire materialization when the window slides by tracking expiration times and computing only the changes (additions and removals) needed for the new materialization. The maintenance is done through execution of a logic program that uses contexts to build the delta sets for updating the materialization incrementally as new data enters the window.

1. Stream Reasoning
For Linked Data
M. Balduini, J-P Calbimonte, O. Corcho,
D. Dell'Aglio, E. Della Valle, and J.Z. Pan
http://streamreasoning.org/sr4ld2013
IMaRS: Incremental
Materialization for RDF Streams
Daniele Dell'Aglio – daniele.dellaglio@polimi.it
Emanuele Della Valle – emanuele.dellavalle@polimi.it

2. Share, Remix, Reuse — Legally
 This work is licensed under the Creative Commons
Attribution 3.0 Unported License.
 Your are free:
to Share — to copy, distribute and transmit the work
to Remix — to adapt the work
 Under the following conditions
Attribution — You must attribute the work by inserting
– “[source http://streamreasoning.org/sr4ld2013]” at the end of
each reused slide
– a credits slide stating
-
These slides are partially based on “Streaming Reasoning for Linked
Data 2013” by M. Balduini, J-P Calbimonte, O. Corcho, D. Dell'Aglio,
E. Della Valle, and J.Z. Pan http://streamreasoning.org/sr4ld2013
 To view a copy of this license, visit
http://creativecommons.org/licenses/by/3.0/
http://streamreasoning.org/sr4ld2013
2

3. Running Example – Data Model
Streaming
information
Background
information
where
Observation
isIn
observes
subClassOf
subPropOf
Post
posts
Room
isConnectedTo
Sensor
subClassOf
discusses
who
Person
isWith
http://streamreasoning.org/sr4ld2013
3

4. Running Example – Data Model
Streaming
information Instances
Background
information
where
1
2
3
Observation
isIn
observes
subClassOf
subPropOf
Post
posts
Room
isConnectedTo
Sensor
subClassOf
discusses
who
Person
isWith
http://streamreasoning.org/sr4ld2013
4

5. What
 Add reasoning in window-based RSPs
 Naïve solution: materialize everything, every time
 But windows slide:
• The materialisation is executed every time the
window updates
• Only part of data changes at each window update
• Materialisation is (usually) an expensive task
http://streamreasoning.org/sr4ld2013
4

6. Naïve solution: an example
TBOX
dom(
)⊑
rng(
)⊑
1
∈
2
∈
S
∈
1
http://streamreasoning.org/sr4ld2013
∈
2
5

7. Naïve solution: an example
TBOX
dom(
)⊑
rng(
)⊑
S
1
∈
2
∈
1
http://streamreasoning.org/sr4ld2013
∈
∈
2
3
5

8. Naïve solution: an example
TBOX
dom(
)⊑
rng(
)⊑
S
1
http://streamreasoning.org/sr4ld2013
2
3
5

9. Naïve solution: an example
TBOX
dom(
)⊑
rng(
)⊑
S
2
3
∈
∈
1
http://streamreasoning.org/sr4ld2013
2
3
5

10. Incremental maintenance
 Adopt an incremental approach
 Compute only the differences that should be removed and
added from the materialization
Inferred statements
To be added
To be removed
∆
−
∆
+
Explicit statements
http://streamreasoning.org/sr4ld2013
6

11. Incremental maintenance: an example
TBOX
dom(
)⊑
rng(
)⊑
1
∈
2
∈
S
∈
1
http://streamreasoning.org/sr4ld2013
∈
2
7

12. Incremental maintenance: an example
TBOX
dom(
)⊑
rng(
)⊑
S
1
∈
2
∈
1
http://streamreasoning.org/sr4ld2013
∈
∈
2
3
7

13. Incremental maintenance: an example
TBOX
dom(
)⊑
rng(
)⊑
S
To be
deleted
1
∈
2
∈
1
http://streamreasoning.org/sr4ld2013
∈
∈
2
3
7

14. Incremental maintenance: an example
TBOX
dom(
)⊑
rng(
)⊑
S
3
To be
deleted
1
∈
2
∈
1
http://streamreasoning.org/sr4ld2013
∈
To be
added
∈
∈
2
3
7

15. Incremental maintenance: an example
TBOX
dom(
)⊑
rng(
To be
renewed
)⊑
S
3
To be
deleted
1
∈
2
∈
1
http://streamreasoning.org/sr4ld2013
∈
To be
added
∈
∈
2
3
7

16. Which technique?
 The common problem in designing incremental
maintenance techniques is in the management of
deletions
 In general it is not possible to foresee the statement
deletions
• DRed works with random insertions and deletions
 In our setting it is possible
• The window operator allow us to determine when
statements will be removed
http://streamreasoning.org/sr4ld2013
8

17. IMaRS
 Variation of DRed for RDF streams
 It pushes the maintenance algorithm in the window
operator
 An IMaRS window is a sliding window with four
parameters:
• ω: the size of the window
• β: the slide of the window
• T: the TBox that describes the data model
• M: the mantinance program
 One of the central IMaRS concepts is the expiration
time
http://streamreasoning.org/sr4ld2013
9

18. Expiration time
 Every time a statement is added to the window, it is
annotated with an expiration time
 The expiration time indicates when the statement should
be removed from the materialization
TBOX
dom(
)⊑
rng(
Window (2,1)
)⊑
Current time
10
1
S
7
8
http://streamreasoning.org/sr4ld2013
9
t
10
10

19. Expiration time
 Every time a statement is added to the window, it is
annotated with an expiration time
 The expiration time indicates when the statement should
be removed from the materialization
TBOX
dom(
)⊑
rng(
Window (2,1)
)⊑
Current time
11
The statement
will exit at 13
1
S
8
9
http://streamreasoning.org/sr4ld2013
10
t
11
10

20. Expiration time
 Every time a statement is added to the window, it is
annotated with an expiration time
 The expiration time indicates when the statement should
be removed from the materialization
TBOX
Window (2,1)
)⊑
dom(
1
)⊑
rng(
∈
The inferred
statements will
exit at 13
∈
Current time
11
The statement
will exit at 13
1
S
8
9
http://streamreasoning.org/sr4ld2013
10
t
11
10

21. Expiration time
 Every time a statement is added to the window, it is
annotated with an expiration time
 The expiration time indicates when the statement should
be removed from the materialization
TBOX
dom(
)⊑
rng(
Window (2,1)
)⊑
1
∈
∈
Current time
12
1
S
9
10
http://streamreasoning.org/sr4ld2013
11
t
12
10

22. Expiration time
 Every time a statement is added to the window, it is
annotated with an expiration time
 The expiration time indicates when the statement should
be removed from the materialization
The inferred
TBOX
statements
expire
Window (2,1)
dom(
)⊑
rng(
)⊑
1
∈
∈
Current time
13
1
S
10
11
http://streamreasoning.org/sr4ld2013
12
t
13
10

23. Expiration time
 Every time a statement is added to the window, it is
annotated with an expiration time
 The expiration time indicates when the statement should
be removed from the materialization
The inferred
TBOX
statements
expire
Window (2,1)
dom(
)⊑
rng(
)⊑
Current time
13
1
S
10
11
http://streamreasoning.org/sr4ld2013
12
t
13
10

24. Expiration time generation
 At each window update
Expiration time
update
Statement
deletion
Inferred
statements
Explicit
statements
 The computation is done through the
execution of a maintenance program
http://streamreasoning.org/sr4ld2013
To be
added
To be
removed
To be renewed
Expiration time
assignment
Expiration time
assignment
11

25. IMaRS at a glance
DRed
IMaRS
Delete
Lookup
Rederive
Insert +
Renew
Insert
http://streamreasoning.org/sr4ld2013
25

26. IMaRS maintenance program
 The maintenance program computes the delta sets ∆−
and ∆+
• It is a logic program
 The program is executed every time the content
changes
• In our context, the program is executed every time
the window slides
 The program is composed by maintenance rules
• A maintenance rule adds a statement in a set
(context) if the preconditions are satisfied
http://streamreasoning.org/sr4ld2013
13

27. IMaRS contexts
 The maintenance program uses four contexts to
build the delta sets ∆− and ∆+
• Mat: the current materialization
• Ins: the input that enters the stream and the
related inferred statements
 Additionally, two support sets are used:
• New: statements to be added to the
materialization
• Ren: renewed statements
 The new materialization is computed as
Mat U
http://streamreasoning.org/sr4ld2013
∆+ ∆−
14

28. IMaRS maintenance rules
 Two examples of maintenance rules:
Δ-(?s,?p,?o)[e] Mat(?s,?p,?o)[e] .
e < now
A triple is removed by the materialization when its
expiration time expires
Ins(?x,?p,?z)[e] New(?x,?p,?y)[e1].
Ins(?x,?p,?y)[e2] .
Ins(?p,isA,TransitiveProperty)[e3].
e=min{e1,e2,e3}
When a triple <s,p,o> enters the window, p is transitive
and there is a triple <o,p,k> in the Ins context, then the
triple <s,p,k> is a candidate for the addition in the
materialization
http://streamreasoning.org/sr4ld2013
15

29. IMaRS maintenance program
 The maintenance program is composed by two sets of
maintenance rules:
• One set of fixed maintenance rules
• One dependent on the ontological language
 The ontological language should be expressed as a
set of inference rules, e.g.
T(?x, ?p, ?z) :T(?p, rdf:type, owl:TransitiveProperty),
T(?x, ?p, ?y), T(?y, ?p, ?z)
 It does not depend on the TBox!
http://streamreasoning.org/sr4ld2013
16

30. Generation of the maintenance program
Rewriting
functions
Maintenance
program
generator
Ontological
language
Maintenance
program
TBox
IMaRS Window (ω, β)
http://streamreasoning.org/sr4ld2013
30

31. Example: DRed
2
1
TBOX
tr(
Window (3,1)
)
Current time
11
2
S
7
8
http://streamreasoning.org/sr4ld2013
9
1
t
10
18

32. Example: DRed
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
12
3
1
3
4
2
S
8
9
http://streamreasoning.org/sr4ld2013
2
1
1
10
t
11
18

33. Example: DRed
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
13
3
3
2
S
1
2
1
3
4
9
10
http://streamreasoning.org/sr4ld2013
4
1
11
t
12
18

34. Example: DRed
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
14
3
3
3
S 2
4
2
1
4
10
1
1
11
http://streamreasoning.org/sr4ld2013
12
t
13
18

35. Example: DRed
Delete
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
14
3
3
3
S 2
4
2
1
4
10
1
1
11
http://streamreasoning.org/sr4ld2013
12
t
13
18

36. Example: DRed
Delete
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
14
3
3
3
S 2
4
2
1
4
10
1
1
11
http://streamreasoning.org/sr4ld2013
12
t
13
18

37. Example: DRed
Delete
2
1
3
Rederive
4
TBOX
tr(
Window (3,1)
)
Current time
14
3
3
3
S 2
4
2
1
4
10
1
1
11
http://streamreasoning.org/sr4ld2013
12
t
13
18

38. Example: IMaRS
2
1
TBOX
tr(
Window (3,1)
)
Current time
11
2
S
7
8
http://streamreasoning.org/sr4ld2013
9
1
t
10
19

39. Example: IMaRS
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
12
3
1
3
4
2
S
8
9
http://streamreasoning.org/sr4ld2013
2
1
1
10
t
11
19

40. Example: IMaRS
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
13
3
3
2
S
1
2
1
3
4
9
10
http://streamreasoning.org/sr4ld2013
4
1
11
t
12
19

41. Example: IMaRS
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
13
3
3
2
S
1
2
1
3
4
9
10
http://streamreasoning.org/sr4ld2013
4
1
11
t
12
19

42. Example: IMaRS
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
13
3
3
2
S
1
2
1
3
4
9
10
http://streamreasoning.org/sr4ld2013
4
1
11
t
12
19

43. Example: IMaRS
2
1
3
4
TBOX
tr(
Window (3,1)
)
Current time
14
3
3
3
S 2
4
2
1
4
10
1
1
11
http://streamreasoning.org/sr4ld2013
12
t
13
19

44. Stream Reasoning
For Linked Data
M. Balduini, J-P Calbimonte, O. Corcho,
D. Dell'Aglio, E. Della Valle, and J.Z. Pan
http://streamreasoning.org/sr4ld2013
IMaRS: Incremental
Materialization for RDF Streams
Daniele Dell'Aglio – daniele.dellaglio@polimi.it
Emanuele Della Valle – emanuele.dellavalle@polimi.it