The document discusses stream processing and querying of process logs. It defines a process log as a mapping of case identifiers to traces, which are sequences of events. Queries can check properties of individual traces or perform aggregations across multiple traces. Hyper-queries generalize this to operate on the full log. Various methods of incrementally evaluating queries and hyper-queries are discussed as the log is produced, such as processing one case or event at a time. The state of the art in constraint and query languages is reviewed.

1. Stream Processing
Stream Processing
Chukri Soueidi
Yliès Falcone
Université Grenoble-Alpes
France
Monitoring
Multiple Executions
Process
with
across
Business
Compliance
Monitoring
Multiple Executions
Process
with
across
Business
Compliance
Université du Québec à Chicoutimi
Canada
Sylvain Hallé

2. C. Soueidi, Y. Falcone, S. Hallé
A school admission process
a. Assign
Officer
b. Review
Application
c. Request
Documents
d. Schedule
Interview
f. Evaluate
Interview
e. Conduct
Interview
g. Accep-
tance Letter
h. Rejection
Letter
Admission
Process Ends
Application
Submitted

3. C. Soueidi, Y. Falcone, S. Hallé
A school admission process
a
b
c
d
f g
e h

4. C. Soueidi, Y. Falcone, S. Hallé
A school admission process
a
b
c
d
f g
e h
Executions of this process are called cases
or instances

5. C. Soueidi, Y. Falcone, S. Hallé
A school admission process
a
b
c
d
f g
e h
Executions of this process are called cases
or instances
a
b d
f g
e
c
h

6. C. Soueidi, Y. Falcone, S. Hallé
A school admission process
a
b
c
d
f g
e h
Executions of this process are called cases
or instances
,

7. C. Soueidi, Y. Falcone, S. Hallé
A school admission process
a
b
c
d
f g
e h
Executions of this process are called cases
or instances
, , , ...
,

8. C. Soueidi, Y. Falcone, S. Hallé
Process log
These executions can be recorded in the form of
a log

9. C. Soueidi, Y. Falcone, S. Hallé
Process log
These executions can be recorded in the form of
a log
a g
f
b d e

10. C. Soueidi, Y. Falcone, S. Hallé
Process log
These executions can be recorded in the form of
a log
a g
f
b d e
a f
b d e
c h

11. C. Soueidi, Y. Falcone, S. Hallé
Process log
These executions can be recorded in the form of
a log
a g
f
b d e
a f
b d e
c h
a g
f
b d e
c

12. C. Soueidi, Y. Falcone, S. Hallé
Process log
These executions can be recorded in the form of
a log
a g
f
b d e
a f
b d e
c h
a g
f
b d e
c
a g
f
b d e

13. C. Soueidi, Y. Falcone, S. Hallé
Process log
Formally, let...

14. C. Soueidi, Y. Falcone, S. Hallé
Process log
Formally, let...
Σ = {σ1, ..., σn} set of events
σ ∈ Σ* is a trace
a f b e
( (
d

15. C. Soueidi, Y. Falcone, S. Hallé
Process log
Formally, let...
Σ = {σ1, ..., σn} set of events
σ ∈ Σ* is a trace
a f b e
( (
d
C = {c1, ..., cm} set of case
identifiers ( (

16. C. Soueidi, Y. Falcone, S. Hallé
Process log
Formally, let...
Σ = {σ1, ..., σn} set of events
σ ∈ Σ* is a trace
a f b e
( (
d
C = {c1, ..., cm} set of case
identifiers ( (
A log is a function λ : C → Σ*, mapping a trace to
each case identifier.
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦

17. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Many questions can be asked about this log:

18. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Many questions can be asked about this log:
Does the process
start with ?
a
Does come
after ?
a
b
Does the case end
after fewer than n
events?

19. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Many questions can be asked about this log:
Does the process
start with ?
a
Does come
after ?
a
b
Does the case end
after fewer than n
events?
What is the number
of events in a trace?
What are the events
in the trace?
What is the average
delay between two
events?

20. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Many questions can be asked about this log:
Does the process
start with ?
a
Does come
after ?
a
b
Does the case end
after fewer than n
events?
What is the number
of events in a trace?
What are the events
in the trace?
What is the average
delay between two
events?
"Constraints"
answer by yes or no
"Queries"
answer by something else

21. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Given a trace σ ∈ Σ*, a query is a function
q : Σ* → D
for some arbitrary image D. A constraint is the
particular case where D = {⊤,⊥}.

22. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Given a trace σ ∈ Σ*, a query is a function
q : Σ* → D
for some arbitrary image D. A constraint is the
particular case where D = {⊤,⊥}.
A trace σ is compliant with respect to
a constraint q iff q(σ) = ⊤.

23. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
Given a trace σ ∈ Σ*, a query is a function
q : Σ* → D
for some arbitrary image D. A constraint is the
particular case where D = {⊤,⊥}.
A log λ is compliant with respect to a constraint
q iff q(λ(c)) = ⊤ for every case identifier c ∈ C.
A trace σ is compliant with respect to
a constraint q iff q(σ) = ⊤.

24. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
A query (and compliance) is concerned with each
execution taken in isolation. Generalize queries to
calculations involving multiple executions:

25. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
A query (and compliance) is concerned with each
execution taken in isolation. Generalize queries to
calculations involving multiple executions:
Are there at least
50% of cases
ending in ?
g
Is the average case
duration below k?

26. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
A query (and compliance) is concerned with each
execution taken in isolation. Generalize queries to
calculations involving multiple executions:
Are there at least
50% of cases
ending in ?
g
Is the average case
duration below k?
What is the maximum
number of concurrent
active cases?
What event occurs in
the fewest cases?

27. C. Soueidi, Y. Falcone, S. Hallé
Querying the log
A query (and compliance) is concerned with each
execution taken in isolation. Generalize queries to
calculations involving multiple executions:
Are there at least
50% of cases
ending in ?
g
Is the average case
duration below k?
What is the maximum
number of concurrent
active cases?
What event occurs in
the fewest cases?
"Hyper-Constraints"
answer by yes or no
"Hyper-Queries"
answer by something else

28. C. Soueidi, Y. Falcone, S. Hallé
Hyper-querying the log
Given a log λ ∈ L, a hyper-query is a function
q : L → D
for some arbitrary image D. A hyper-constraint is
the particular case where D = {⊤,⊥}.

29. C. Soueidi, Y. Falcone, S. Hallé
Evaluating hyper-queries
Two goals:

30. C. Soueidi, Y. Falcone, S. Hallé
Evaluating hyper-queries
Two goals:
A
provide formal means of defining
expressive (hyper) queries
stateful, aggregations, unconventional types

31. C. Soueidi, Y. Falcone, S. Hallé
Evaluating hyper-queries
Two goals:
A
provide formal means of defining
expressive (hyper) queries
stateful, aggregations, unconventional types
evaluate these queries incrementally
as the log is produced
P

32. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 1
The complete log is available at once; the
hyperquery is evaluated on the log
a g
f
b d e
a f
b d e
c h
a g
f
b d e
c
a g
f
b d e

33. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 2
One complete case is processed at a time

34. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 2
One complete case is processed at a time
a g
f
b d e

35. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 2
One complete case is processed at a time
a g
f
b d e
a f
b d e
c h

36. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 2
One complete case is processed at a time
a g
f
b d e
a f
b d e
c h
a g
f
b d e
c

37. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 2
One complete case is processed at a time
a g
f
b d e
a f
b d e
c h
a g
f
b d e
c
a g
f
b d e

38. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 3
One event from each case is processed at a time

39. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 3
One event from each case is processed at a time
a
a
a
a

40. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 3
One event from each case is processed at a time
a
a
a
a
b
b
b
b

41. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 3
One event from each case is processed at a time
a
a
a
a
b
b
b
b
d
c
c
d

42. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 3
One event from each case is processed at a time
a
a
a
a
b
b
b
b
d
c
c
d
. . .
. . .
. . .
. . .

43. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved

44. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a

45. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a
a

46. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a
a
b

47. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a
a
b
a

48. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a
a
b
a
d

49. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a
a
b
a
d
a

50. C. Soueidi, Y. Falcone, S. Hallé
Incremental evaluation
Case 4
Events from each case are arbitrarily
interleaved
a
a
b
a
d
a
b

51. C. Soueidi, Y. Falcone, S. Hallé
State of the art

52. C. Soueidi, Y. Falcone, S. Hallé
State of the art
Constraints Queries
Classical
Hyper

53. C. Soueidi, Y. Falcone, S. Hallé
State of the art
Constraints Queries
Classical
Hyper
Temporal Logic(s)
Petri nets
Finite-state machines
A
P
✓
✓
✓

54. C. Soueidi, Y. Falcone, S. Hallé
State of the art
Constraints Queries
Classical
Hyper
Temporal Logic(s)
Petri nets
Finite-state machines
A
P
✓
✓
✓
Stream equations
Stream pipelines
A
P
✓
✓

55. C. Soueidi, Y. Falcone, S. Hallé
State of the art
Constraints Queries
Classical
Hyper
Temporal Logic(s)
Petri nets
Finite-state machines
A
P
✓
✓
✓
Stream equations
Stream pipelines
A
P
✓
✓
Hyper-LTL
HyperLDLf
Event calculus
A
P
±
?
✓

56. C. Soueidi, Y. Falcone, S. Hallé
State of the art
Constraints Queries
Classical
Hyper
Temporal Logic(s)
Petri nets
Finite-state machines
A
P
✓
✓
✓
Stream equations
Stream pipelines
A
P
✓
✓
Hyper-LTL
HyperLDLf
Event calculus
A
P
±
?
✓
SQL (incremental)
APQL
A
P
✓
?

57. C. Soueidi, Y. Falcone, S. Hallé
Log updates

58. C. Soueidi, Y. Falcone, S. Hallé
Log updates
Log update: a special type of log λ with a single
event for a single case
L ⊆ L is the set of updates
^
{ }
a
↦
^

59. C. Soueidi, Y. Falcone, S. Hallé
Log updates
Log update: a special type of log λ with a single
event for a single case
L ⊆ L is the set of updates
^
{ }
a
↦
^
Update
operation:
an operator ∘ : L × L → L that appends
an event to an existing log
^

60. C. Soueidi, Y. Falcone, S. Hallé
C. Soueidi, Y. Falcone, S. Hallé
C. Soueidi, Y. Falcone, S. Hallé
Log updates
Log updates
Log updates
Log update: a special type of log λ with a single
event for a single case
L ⊆ L is the set of updates
^
{ }
a
↦
^
Log update: a special type of log λ with a single
event for a single case
L ⊆ L is the set of updates
^
{ }
a
↦
^
Log update: a special type of log λ with a single
event for a single case
L ⊆ L is the set of updates
^
{ }
a
↦
^
Update
operation:
an operator ∘ : L × L → L that appends
an event to an existing log
^
Update
operation:
an operator ∘ : L × L → L that appends
an event to an existing log
^
Update
operation:
an operator ∘ : L × L → L that appends
an event to an existing log
^
{ a
↦
a b
↦ }
∘ { }
b
↦ = { a
↦
a b
↦ }
b
λ ∘ λ
^
=

61. C. Soueidi, Y. Falcone, S. Hallé
Log updates
A log is progressvely built through a sequence
(i.e. a stream) of log updates from an empty log

62. C. Soueidi, Y. Falcone, S. Hallé
Log updates
A log is progressvely built through a sequence
(i.e. a stream) of log updates from an empty log
∅

63. C. Soueidi, Y. Falcone, S. Hallé
Log updates
A log is progressvely built through a sequence
(i.e. a stream) of log updates from an empty log
∅ { }
a
↦
∘

64. C. Soueidi, Y. Falcone, S. Hallé
Log updates
A log is progressvely built through a sequence
(i.e. a stream) of log updates from an empty log
∅ { }
a
↦
∘ ∘ { }
a
↦

65. C. Soueidi, Y. Falcone, S. Hallé
Log updates
A log is progressvely built through a sequence
(i.e. a stream) of log updates from an empty log
∅ { }
a
↦
∘ ∘ { }
a
↦ ∘ { }
b
↦ ∘ ...

66. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
πq

67. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
q( )
{ }
a
↦ { }
a
↦
πq

68. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
πq

69. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
πq
πq
{ }
a
↦
}
a
↦
{ a
↦
q(

70. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
πq

71. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
πq
πq
}
a
↦
{ a
↦
q(
)
{ }
b
↦ b

72. C. Soueidi, Y. Falcone, S. Hallé
Problem
Problem Given a hyperquery q : L → D, define a
processor πq : L* → D* such that
πq(λ1 ⋅ ... ⋅ λn)[n] = q(λ1 ∘ ... ∘ λn)
Hyper-processors
πq
πq
}
a
↦
{ a
↦
q(
)
{ }
b
↦ b
In other words, πq incrementally evaluates q on
each update
Avoid re-evaluating q from scratch every time!

73. C. Soueidi, Y. Falcone, S. Hallé
Solution
Solution Follow a compositional approach
Hyper-processors

74. C. Soueidi, Y. Falcone, S. Hallé
Solution
Solution Follow a compositional approach
Hyper-processors
elementary (incremental)
hyperquery processors

75. C. Soueidi, Y. Falcone, S. Hallé
Solution
Solution Follow a compositional approach
Hyper-processors
elementary (incremental)
hyperquery processors
allow composition to express
complex hyperqueries

76. C. Soueidi, Y. Falcone, S. Hallé
Solution
Solution Follow a compositional approach
Hyper-processors
elementary (incremental)
hyperquery processors
allow composition to express
complex hyperqueries

77. C. Soueidi, Y. Falcone, S. Hallé
Solution
Solution Follow a compositional approach
Hyper-processors
elementary (incremental)
hyperquery processors
allow composition to express
complex hyperqueries

78. C. Soueidi, Y. Falcone, S. Hallé
Solution
Solution Follow a compositional approach
Hyper-processors
Solution
Solution Follow a compositional approach
Operators on log updates
Log combinations
Qualified conditions

79. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.

80. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦

81. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.
1
2
3
4
5
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦

82. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.
1
2
3
4
5
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦
{ }
a
↦

83. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.
1
2
3
4
5
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦
{ }
a
↦

84. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.
1
2
3
4
5
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦
{ }
a
↦

85. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
XES Source
Reads a log file in XES format, and outputs it as an
interleaved stream of log updates.
1
2
3
4
5
a g
f
b d e
a f
b d e
c h
↦
↦
a g
f
b d e
c
a g
f
b d e
↦
↦
{ }
b
↦

86. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
f

87. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a

88. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a
{ }
a
↦

89. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a
{ }
a
↦ { }
a
↦

90. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a

91. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a
{ }
b
↦

92. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a

93. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a
{ }
b
↦

94. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a
{ }
b
↦ { }
b
↦

95. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a

96. C. Soueidi, Y. Falcone, S. Hallé
Operators on log updates
Sample
Evaluate condition f on first event σ of each case;
retain case in log if f(σ) = ⊤.
=?
a
{ }
a
↦

97. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
P

98. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
there are more
than 2 events

99. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
a
↦

100. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2

101. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
b
↦

102. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2

103. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
b
↦

104. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
b
↦ { }
a
↦ b

105. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
b
↦ { }
a
↦ b
{ }
a
↦ b
when the condition becomes ⊤,
all the updates from the start of
the case are output

106. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2

107. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
c
↦

108. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
c
↦ { }
c
↦

109. C. Soueidi, Y. Falcone, S. Hallé
Filter
Run processor P on each case; only output updates
for cases where P returns ⊤.
Operators on log updates
1
Σ
0
+
f
≥
2
{ }
c
↦ { }
c
↦
subsequent updates are let
through

110. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
P

111. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f

112. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
put two successive
events in a tuple

113. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f

114. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
a
↦

115. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
a
↦

116. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f

117. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
b
↦

118. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
b
↦

119. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f

120. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
b
↦

121. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
b
↦
a b

122. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
a b
{ }
↦
{ }
b
↦
a b

123. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
a b

124. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
a
↦
a b

125. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
a
↦
a b ba

126. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
ba
↦
{ }
a
↦
a b ba

127. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
a b ba

128. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
c
↦
a b ba

129. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
c
↦
a b
bc ba

130. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
bc
↦
{ }
c
↦
a b
bc ba

131. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
bc ba

132. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
b
↦
bc ba

133. C. Soueidi, Y. Falcone, S. Hallé
Slice
Run an instance of processor P on each case; output
resulting log as a new stream of updates.
Operators on log updates
1
f
{ }
b
↦
bc ba

134. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀ ∃
...
P

135. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?

136. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=? for every case,
there exists another
one that starts with
the same event

137. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?

138. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
{ }
a
↦

139. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
{ }
a
↦ ⊥

140. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?

141. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
{ }
b
↦

142. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
⊥
{ }
b
↦

143. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?

144. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
{ }
b
↦

145. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
⊥
{ }
b
↦

146. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?

147. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
{ }
a
↦

148. C. Soueidi, Y. Falcone, S. Hallé
Quantification
Given quantifiers Q1σ1, ..., Qnσn on cases,
evaluate quantified expression P(σ1, ..., σn) for a
processor P.
Log combinations
Q
∀
∃
↑
↑ f
=?
⊤
{ }
a
↦

149. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
P P'

150. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
P P'
aggregate output
run on each case

151. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1

152. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
count distinct events running average

153. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1

154. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
{ }
a
↦

155. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
{ }
a
↦

156. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
{ }
a
↦
1

157. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
{ }
a
↦ 1
1

158. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1

159. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1
{ }
b
↦

160. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1
{ }
b
↦

161. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1 1
{ }
b
↦

162. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1
1 1
{ }
b
↦

163. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1 1

164. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1 1
{ }
b
↦

165. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1
2 1
{ }
b
↦

166. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1.5
1
2 1
{ }
b
↦

167. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
2 1

168. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
2 1
{ }
b
↦

169. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
2 1
{ }
b
↦

170. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
2 1 1
{ }
b
↦

171. C. Soueidi, Y. Falcone, S. Hallé
Aggregation
Run processor P on each case and aggregate their
last output with function processor P'.
Log combinations
σ Σ
}
{
| |
f
f
Σ
0
+
Σ
0
+
÷
1
1.3
2 1 1
{ }
b
↦

172. C. Soueidi, Y. Falcone, S. Hallé
Weakening
Consider the verdict of a hyper-constraint ψ only
for logs that satisfy another hyper-constraint φ.
Qualified conditions
φ ψ
P P'
↑

173. C. Soueidi, Y. Falcone, S. Hallé
Weakening
Consider the verdict of a hyper-constraint ψ only
for logs that satisfy another hyper-constraint φ.
Qualified conditions
φ ψ
↑
f
Σ
0
+
Σ
0
+
÷
1
↑
1
σ Σ
f
k
≤
1
σ Σ
Σ
0
+
f
5
≤
↑

174. C. Soueidi, Y. Falcone, S. Hallé
Weakening
Consider the verdict of a hyper-constraint ψ only
for logs that satisfy another hyper-constraint φ.
Qualified conditions
φ ψ
↑
f
Σ
0
+
Σ
0
+
÷
1
↑
1
σ Σ
f
k
≤
1
σ Σ
Σ
0
+
f
5
≤
↑
the average case
length does not
exceed k

175. C. Soueidi, Y. Falcone, S. Hallé
Weakening
Consider the verdict of a hyper-constraint ψ only
for logs that satisfy another hyper-constraint φ.
Qualified conditions
φ ψ
↑
f
Σ
0
+
Σ
0
+
÷
1
↑
1
σ Σ
f
k
≤
1
σ Σ
Σ
0
+
f
5
≤
↑
on logs having
at least 5 cases

176. C. Soueidi, Y. Falcone, S. Hallé
Is it "just" a logical implication?
Qualified conditions
↑
φ ψ

177. C. Soueidi, Y. Falcone, S. Hallé
Is it "just" a logical implication?
Qualified conditions
↑
φ ψ
1. Separates the policy (ψ) from the
condition under which it applies (φ)

178. C. Soueidi, Y. Falcone, S. Hallé
Is it "just" a logical implication?
Qualified conditions
↑
φ ψ
1. Separates the policy (ψ) from the
condition under which it applies (φ)
2. Three-valued implication: distinguishes
between a non-violation caused by
satisfaction (φ = ⊤, ψ = ⊤)
exemption (φ = ?, ψ = *)
⊤
?
⊤
↑
⊤
?
⊤
?
⊤
⊤ ⊤ ⊤
⊤
⊤
? ?
⊤

179. C. Soueidi, Y. Falcone, S. Hallé
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
n
m

180. C. Soueidi, Y. Falcone, S. Hallé
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
n
m
temporary violations in a case (e.g. at most m
out of n successive events)

181. C. Soueidi, Y. Falcone, S. Hallé
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
n
m
temporary violations in a case (e.g. at most m
out of n successive events)
⊤ ⊤
⊤ ⊤
⊤
⊤
⊤ ⊤ ⊤ ⊤
⊤
⊤ ⊤

182. C. Soueidi, Y. Falcone, S. Hallé
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
n
m
temporary violations in a case (e.g. at most m
out of n successive events)
⊤ ⊤
⊤ ⊤
⊤
⊤
⊤ ⊤ ⊤ ⊤
⊤
⊤ ⊤
m=3, n=4

183. C. Soueidi, Y. Falcone, S. Hallé
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
n
m
temporary violations in a case (e.g. at most m
out of n successive events)
⊤ ⊤
⊤ ⊤
⊤
⊤
⊤ ⊤ ⊤ ⊤
⊤
⊤ ⊤
m=3, n=4

184. C. Soueidi, Y. Falcone, S. Hallé
violations of a fraction of all cases (e.g. at
most m
/n of all cases)
n
m
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions

185. C. Soueidi, Y. Falcone, S. Hallé
violations of a fraction of all cases (e.g. at
most m
/n of all cases)
n
m
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions

186. C. Soueidi, Y. Falcone, S. Hallé
violations of a fraction of all cases (e.g. at
most m
/n of all cases)
n
m
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
m=3, n=4

187. C. Soueidi, Y. Falcone, S. Hallé
violations of a fraction of all cases (e.g. at
most m
/n of all cases)
n
m
Dampening
Tolerate violations of a hyper-constraint; either...
Qualified conditions
m=3, n=4
↦
↦
↦
↦
⊤
⊤
⊤
⊤
⊤

188. C. Soueidi, Y. Falcone, S. Hallé
These "building blocks" have been implemented
as an extension of the event stream
processing library
Implementation
https://github.com/liflab/hypercompliance
They can be freely mixed with other processors
from the library to form complex hyper-queries

189. C. Soueidi, Y. Falcone, S. Hallé
Example
"All cases that contain an a must end in the
same state" (consistency)
f
F
=?
↑
1 a
σ Σ
}
{
f
#
1 f
≤

190. C. Soueidi, Y. Falcone, S. Hallé
Example
"The number of tasks of an employee may not
exceed the global average by a factor k."
f
emp
f
*
#
*
}
{
}
{
f
Σ
max
f
Σ
0
+
Σ
0
+
÷
1
≤
×
2 k
1

191. C. Soueidi, Y. Falcone, S. Hallé
Experimental results
Scenario Events Cases Hyperquery Throughput (Hz) Max memory (B)
Hospital 151434 1143
Concurrent instances 901392 Hz 8059
Directly follows 277351 Hz 3660379
Mean time interval 1130104 Hz 5615
Average length 369351 Hz 522071
Same next 304084 Hz 5401821
CAP 275287 13087
Concurrent instances 920692 Hz 12863
Directly follows 79700 Hz 39075895
Mean time interval 1228959 Hz 5615
Average length 7995 Hz 5991691
Same next 87698 Hz 38400801
WABO 39881 937
Concurrent instances 1375206 Hz 8055
Directly follows 419800 Hz 1873161
Mean time interval 1172970 Hz 5615
Average length 162117 Hz 278085
Same next 302128 Hz 2669627
Throughput and memory consumption for a sample
of properties evaluated on real-life logs
(details in the paper)

192. C. Soueidi, Y. Falcone, S. Hallé
Conclusion
Hyper-contraints and hyper-queries are calculated
on a set of process executions called a log
Elementary computing units (processors) can
incrementally evaluate hyper-queries in an
incremental (i.e. "real-time") fashion
Expressive hyper-queries can be obtained by
composing these units
These concepts have been...
1. formally defined
2. concretely implemented in a stream
processing library

193. C. Soueidi, Y. Falcone, S. Hallé
What's next ?
Journal paper in preparation, considering a
generalization of a hyperquery:
q : L → D q : L* → D
vs.
^
i.e. q depends on the log and the precise
interleavings of its updates.
^
^

194. C. Soueidi, Y. Falcone, S. Hallé
What's next ?
Journal paper in preparation, considering a
generalization of a hyperquery:
q : L → D q : L* → D
vs.
^
i.e. q depends on the log and the precise
interleavings of its updates.
^
^
Thank you!