This presentation is an introduction to Complex Event Processing (CEP) intended for an practicioners of Runtime Verification. It first describes typical CEP problems, popular tools and their query languages. It then presents BeepBeep 3, an event stream processor that attempts to bridge the gap between RV and CEP. Thanks to BeepBeep’s generic architecture and flexible input language, queries and properties from both fields can be efficiently processed.
1. When RV Meets CEP...When RV Meets CEP...
A NEW TUTORIAL BY SYLVAIN HALLÉ
RUNTIME
VERIFICATION
COMPLEX EVENT
PROCESSING
CRSNG
NSERC
(C) 2016 LABORATOIRE D'INFORMATIQUE FORMELLE, UNIVERSITÉ DU QUÉBEC À CHICOUTIMI, CANADA
PRESENTED AT THE INTERNATIONAL CONFERENCE ON RUNTIME VERIFICATION, MADRID, SPAIN, SEPTEMBER 27TH, 2016
WORK FUNDED BY THE CANADA RESEARCH CHAIR IN SPECIFICATION, TESTING AND VERIFICATION OF SOFTWARE SYSTEMS
Can
these
two fields
benefit
from
each
other?
5. public static void
main(String[] args) {
Map m = new HashMap();
...
Iterator i = m.iterator();
while (i.hasNext()) {
...
}
}
HashMap.new = HashMap$1263
HashMap$1263.iterator() = Iterator$32045
HashMap$1263.contains(MyObject$26) = false
Iterator$32045
HashMap$1263.hasNext = true
...
Method
calls
6. Any call to next() must be
immediately preceded by a call to
hasNext() that returns true.
Two calls to remove() must be
separated by at least one call to next
().
Calls to an iterator's methods are
forbidden once its parent collection has
been modified by calls like put() or
remove().
16. The
Beep
Store
GO
Sign inor regi ster
What is this?
Login
Ask for account
Contact us
Fault parameters
Search: Your Cart
Fault parameters
Don’t check Results’s type
In the detailed search form, sends an ItemSearch message without
checking that the Results element is an integer.
"Add to cart" enabled if item present in cart
Makes the "Add to cart" button available for items that are already in the
user's cart.
Message schemas
Cart manipulations
Highlights
documentation
Disables the
verification
17. 1. The element must be an integer between 1 and 20.
2. The element is mandatory only if is present,
otherwise it is forbidden.
3. The request cannot be resent if its response is
successful.
4. must follow a successful Logi nResponse.
5. There can be at most one active cart ID per session key.
6. You cannot add the same item twice to the shopping cart.
Page
Page Resul t s
Logi n
Car t Cr eat e
21. The lock graph of a set of tasks
should not contain cycles
A communication window has three
phases: prep, active and cleanup
Every command must be responded
by either a success or a failure
38. A call to next must be followed by a call
to hasNext
No CartCreate request can occur
before a LoginResponse message
A received order must eventually
be shipped
Three successive login attempts should
trigger an alarm
G (next → X hasNext)
¬ CartCreate U hasNext
G (receive → F ship)
G ¬(fail ∧ (X (fail ∧ X fail)))
Option #2 Linear Temporal Logic
47. $$$$$$$$$$
$$$$$$$$$$$
$$$$$$$$$$
$$$$$$$$$$$
$$$$$$$$$$
What is the closing price of MSFT
for the first five trading days?
Select all days after the 100th
where MSFT closed over $50.
On every 5th day, calculate the
average closing price for MSFT for
the last 5 days.
What are the stocks that closed
higher than MSFT for a given day?
48. $$$$$$$$$$
$$$$$$$$$$$
$$$$$$$$$$
$$$$$$$$$$$
$$$$$$$$$$
What is the closing price of MSFT
for the first five trading days?
Select all days after the 100th
where MSFT closed over $50.
On every 5th day, calculate the
average closing price for MSFT for
the last 5 days.
What are the stocks that closed
higher than MSFT for a given day?
Snapshot query
Landmark query
Sliding query
Join query
55. Notify me when two out of three
successive data points lie more than
1 standard deviation from the mean
on the same side of the mean line.
56. Notify me when two out of three
successive data points lie more than
1 standard deviation from the mean
on the same side of the mean line.
Trend query
62. Select AVG(closingPrice)
From ClosingStockPrices
Where stockSymbol = ‘MSFT’
for (t = ST; t < ST+50, t+= 5) {
WindowIs(ClosingStockPrices, t - 4, t);
}
Select c2.*
FROM
ClosingStockPrices as c1,
ClosingStockPrices as c2
WHERE c1.stockSymbol = ‘MSFT’ and
c2.stockSymbol!= ‘MSFT’ and
c2.closingPrice > c1.closingPrice and
c2.timestamp = c1.timestamp
for (t = ST; t < ST +20 ; t++ ){
WindowIs(c1, t - 4, t);
WindowIs(c2, t - 4, t);
}
Two queries in
the language of
TelegraphCQ
63. SELECT * FROM
FILTER {cnt >= 10}(
(SELECT *, 1 AS cnt FROM
FILTER {contains(summary,’iPod’) = 1}(webfeeds))
FOLD {, $.cnt < 10 AND DUR < 1 DAY, $.cnt + 1 AS cnt}
(SELECT * FROM
FILTER {contains(summary,’iPod’) = 1}(webfeeds))
)
PUBLISH ipod_popularity
A query in the language of
Cayuga
64. from inputtrace#window.length(10)
select stockSymbol, timestamp, avg(closingPrice) as avg10
insert into avg10trace;
from inputtrace#window.length(20)
select stockSymbol, timestamp, avg(closingPrice) as avg20
insert into avg20trace;
@info(name = 'query1')
from avg10trace#window.length(10) join
avg20trace#window.length(20)
on avg10trace.timestamp == avg20trace.timestamp
select avg10trace.stockSymbol as stockSymbol,
avg10trace.timestamp, avg20trace.timestamp as ts2,
avg10trace.avg10 as avg10, avg20trace.avg20 as avg20
having avg10 > 50 or avg20 > 50
insert into output;
SiddhiQL...
66. CEPRV
outputs T/F
result cannot be reused
in another property
mostly variants of FSM
and logic
good for sequential
patterns
not quite
When
outputs event streams
result can be reused as
input of another query
mostly "extensions" of
SQL
not quite
good for data
manipulation
68. CEPRVBeyond &
Both RV and CEP assume events to be "tuples":
associative maps from strings to single scalar values
start(vase,3,15).
bid(vase,15).
start(ring,5,30).
3/10/29 MSFT 1059.2
3/10/29 APPL 365.4
3/10/29 GOGL 1120.1
HashMap.new =
HashMap$1263
HashMap$1263.hasNext =
true
TollId
Toll
License
State
Model
48392
$3.50
QF5010
KW
T
69. CEPRVBeyond &
Both RV and CEP assume events to be "tuples":
associative maps from strings to single scalar values
start(vase,3,15).
bid(vase,15).
start(ring,5,30).
3/10/29 MSFT 1059.2
3/10/29 APPL 365.4
3/10/29 GOGL 1120.1
HashMap.new =
HashMap$1263
HashMap$1263.hasNext =
true
TollId
Toll
License
State
Model
48392
$3.50
QF5010
KW
T
And what if events
are not tuples?
70. CEPRVBeyond &
Both RV and CEP assume events to be "tuples":
associative maps from strings to single scalar values
start(vase,3,15).
bid(vase,15).
start(ring,5,30).
3/10/29 MSFT 1059.2
3/10/29 APPL 365.4
3/10/29 GOGL 1120.1
HashMap.new =
HashMap$1263
HashMap$1263.hasNext =
true
TollId
Toll
License
State
Model
48392
$3.50
QF5010
KW
T
And what if events
are not tuples?
And what if events
are not tuples?
We make them
fit!
71. CEPRVBeyond &
Suppose each event is a set:
{6,5,3,9}
{5,4,10}
...
We can "fit" it into a tuple:
a
6
a=4 ∨ b=4 ∨ c=4 ∨ d=4
c
3
d
9
Imposes upper
bound on set size
But then 4 ∈ S becomes:
b
5
75. CEPRVBeyond &
Both RV and CEP rely on monolithic
specification/query languages
Everything must be written
in that one language (FO-LTL,
QEA, SiddhiQL, etc.)
The language must
accommodate every possible
use case
In general, no support for
extensions
77. Event stream query engine developed
based on the previous observations
Aims at borrowing strengths from both
RV and CEP (and beyond)
Key concepts: composability, modularity,
extensibility
Open source, developed in Java
http://liflab.github.io/beepbeep-3
peeB peeB 3
78. EventsEvents
An event is an element e taken from
some set E, called the event type.
No restriction on the type! In BeepBeep,
events can be any Object.
Booleans Numbers
2
3
4
π
Strings
abc
Functions Sets PlotsTuples
3 8 a
3 8 a
2 6 c
+
⊇?
XML
documents
<a><a><a>
. . .
?
79. TracesTraces
An event trace (or event stream) is a potentially
infinite sequence of events of a given type:
2 0 6 3
4 9 . . .
Traces are symbolically denoted by:
e = e0 e1 e2 e3 ...
The set of all traces of type T is denoted as:
T*
80. FunctionsFunctions
A function takes 0 or more events as its
input, and returns 1 or more events.
Functions are first-class objects; they descend
from the class Function
1 : 1
function
2 : 1
function
1 : 2
function
⊇?
3
4
2+5i
₹
2 5
6
0 : 1
function
6
81. ProcessorsProcessors
A processor takes 0 or more event traces as
its input, and returns 0 or more event traces as
its output
1 : 1 processor
2 : 1 processor
. . . . . .
82. ProcessorsProcessors
When a processor takes more than one input
trace, the set of events at matching positions
in each trace is called a front.
bacd
3601
b 3
a 6
c 0
d 1
1st event
2nd
3rd
4th
. . .
. . . . . .
97. Synchronous processingSynchronous processing
Makes a couple of things simpler
Don't care about what event arrived
first or upstream computation time
"Pen and paper" calculation is identical
to the real one
Otherwise, can do a lot with simple
timeouts ⇒ contained asynchrony
Motto: Don't use asychronous processing...
98. Synchronous processingSynchronous processing
Makes a couple of things simpler
Don't care about what event arrived
first or upstream computation time
"Pen and paper" calculation is identical
to the real one
Otherwise, can do a lot with simple
timeouts ⇒ contained asynchrony
Motto: Don't use asychronous processing...
...unless you really have to
99. Synchronous processingSynchronous processing
In BeepBeep, all synchronous processors
are descendents of the SingleProcessor
class
Takes care of handling input/output
buffers
Calls (abstract) method process() when
an input front is ready to be consumed
Processor only needs to produce an
output front from this input
Makes it easy to create your own
(more on that later)
100. A high-level event trace can be produced by
composing ("piping") together one or more
processors from lower-level traces
CompositionComposition
106. ArchitectureArchitecture
BeepBeep provides only a few built-in
processors and functions
Palette
Set of processors and functions,
centered around a particular
use case
Concretely, a JAR library
defining new Processor and
Function objects
107. <?
+
<? =?
−
÷ ×
f Σ f
n
{
n
n
Function Cumulative Trim
ForkDecimate Group
WindowSliceFilter
Built-in processors
Built-in
functions
.n
<
<
108. SemanticsSemantics
Let P be a processor and a
b
c
= a1,a2,...
= b1,b2,...
= c1,c2,... be traces
a,b,c : P[[
n
= e1,e2,...
denotes the n-th output trace of P, given
traces a, b, c as input.
110. f
FunctionFunction
Applies an n-ary function f to
every front of size n
"Lifts" any function into a
processor
a,b : f[[
n
= f n
(ai,bi)
The n-th output
111. f
FunctionFunction
Applies an n-ary function f to
every front of size n
"Lifts" any function into a
processor
a,b : f[[
n
= f n
(ai,bi)
The n-th output
i
The i-th event
112. f
FunctionFunction
Applies an n-ary function f to
every front of size n
"Lifts" any function into a
processor
a,b : f[[
n
= f n
(ai,bi)
The n-th output
i
The i-th event
f
+
f
<0?
Pairwise sum of
events
Is each event
negative?
113. CumulativeCumulative
Applies a 2 : 1 function f to
its previous value and the
current event
a : Σf[[ = f(x,a1), f(f(x,a1),a2), ...
+
<Sum of all
events
Have we seen
⊤ so far?
Σ f
x
Σ f Σ f
0 ⊥
115. TrimTrim
Returns the input trace,
trimmed of its first n events
a : [[ = an+1, an+2, ...
n
n
1
f
=?=?
<
Σ f
⊥
Does the same
number
repeat twice in
a row?
120. FilterFilter
A n : n-1 processor; outputs
the n-1 first components of a
front if its last component is ⊤;
otherwise, discards it
a , a , ..., a : =[[ 2
k1 n
F
i
{a if a =⊤
ε otherwise
k
i
n
i
Powerful mechanism: "anything can
be filtered" (don't care about condition)
Boolean trace does not need to come
from the same source as the inputs being
filtered
121. FilterFilter
A n : n-1 processor; outputs
the n-1 first components of a
front if its last component is ⊤;
otherwise, discards it
f
<0?
Get only the negative
events of the input
122. WindowWindow
Returns the output of a
processor P on a sliding
window of width n
a : Υ =[[ P i
Powerful mechanism: "anything can
be windowed" (don't care about function)
n
{
n
ai, ... ai+n : P[[ *
123. WindowWindow
Returns the output of a
processor P on a sliding
window of width n
a : Υ =[[ P i
Powerful mechanism: "anything can
be windowed" (don't care about function)
n
{
n
ai, ... ai+n : P[[ *
The last event
149. WindowWindow
Returns the output of a
processor P on a sliding
window of width n
n
{
3
{+
Σ f
0
31 8 9
The sum of all 3
successive events
150. SliceSlice
Dispatches an event e to a
distinct instance of processor
P according to the value of
some function f
{a if f(ai)=k
ε otherwise
i
πf
k[a : =[i
P
f[a : =[i U πf
k[a1,...ai : [i[ [: P
k *
+
151. SliceSlice
Dispatches an event e to a
distinct instance of processor
P according to the value of
some function f
{a if f(ai)=k
ε otherwise
i
πf
k[a : =[i
P
f[a : =[i U πf
k[a1,...ai : [i[ [: P
k *
+
Multiset union
153. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
154. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
155. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
156. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
157. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
U+
158. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
U+
{9}{5}
159. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
U+
{9}{5}
160. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
{9}{5}6
161. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
{9}{5}
6
162. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
{9}{5}
6
U+
163. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
{9}{6,5}
6
U+
{9}{5}
164. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
{9}{6,5}
6
{9}{5}1
165. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
5
{9}{6,5}
6
{9}{5}
1
166. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
1
{9}{6,5}
6
{9}{5}
5
U+
167. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
1
{9}{6,1}
6
{9}{6,5}
5
U+
{5}
168. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
1
{9}{6,1}
6
{9}{6,5}
5
U+
{5}
The last odd and even
numbers seen so far
169. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
+
Σ f
0
170. f(x) = x mod 2
πf
0[a : =[subtrace of even numbers
πf
1[a : =[subtrace of odd numbers
mod 2
+
Σ f
0
The sum of all odd
numbers and all even
numbers seen so far
{9}{6,6} {9}{6,5} {5}
171. Input/outputInput/output
0 : 1 processors can be used to produce an
event trace out of an external source (i.e.
standard input, a file, etc.)
Ditto for 1 : 0 processors
a . . .b
a . . .b
173. PalettesPalettes
BeepBeep provides only a few built-in
processors and functions
Palette
Set of processors and functions,
centered around a particular
use case
Concretely, a JAR library
defining new (reusable!)
Processor and Function
objects
174. XMLXML
Provides two new Functions
Ditto for the JSON library (using json-lif)
XML parser: converts String events
into XMLElement events (from
xml-lif library)
XPath: evaluates an XPath expression
on an XMLElement (result is a set of
XMLElements)
176. GnuplotGnuplot
New Functions:
New processor:
Scatterplot: converts a set of (x,y) pairs
into a Gnuplot string producing a plot
(ditto for Histogram)
→ →
Gnuplot Caller
Calls Gnuplot on a string and
returns its output (i.e. a binary
string)
177. TuplesTuples
New event type: tuple
New function: SELECT
Creates an output tuple by combining
attributes from input tuples
Grammar extension: allow eSQL to use a
SELECT statement, backward-compatible with
SQL
3 8 a
3 8 a
2 6 c
179. FSMFSM
New processor: finite-state machine
→
→
?
?
T
/a/b
//status/text()
=? Walker
*
/a/b
//status/text()
=? Blocker
*
*
Guards on transitions
are arbitrary functions
on events
States output values
of any type (Moore machine)
180. Linear Temporal LogicLinear Temporal Logic
New processors for LTL temporal operators
and first-order quantification (LTL-FO+)
Two semantics for a formula φ:
Boolean: the i-th output is the (2-valued)
verdict of evaluating φ, starting at the i-th
event
⇒ filter
Troolean: the i-th output is the (3-valued)
verdict of evaluating φ up to the i-th event
⇒ monitor
181. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
182. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
c
183. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
?
184. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
?c
185. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
? ?
186. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
? ?b
187. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
⊤ ? ?
188. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
⊤ ? ?a
189. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a monitoring context, we want the
output of processor F b to be:
ac c b
F b
"Tell me whether
eventually b"
⊤ ⊤ ? ?
190. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
191. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
c
192. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
193. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
c
194. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
195. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
b
196. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
⊤ ⊤ ⊤
197. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
a ⊤ ⊤ ⊤
198. Linear Temporal LogicLinear Temporal Logic
Consider the LTL formula F b on the
trace
In a filtering context, we want the
output of processor F b to be:
ac c b
F b
"Get me all the events
where eventually b"
⊤ ⊤ ⊤
199. Linear Temporal LogicLinear Temporal Logic
Boolean operators are easy
X
G F
U
1
<
Σ f
⊥
<
Σ f
⊥
F
G
=
= =
= Σ f
?
♣
200. Linear Temporal LogicLinear Temporal Logic
First-order quantifiers
Processor to run
on each slice
→
A
→ x →
E
→ x
Quantified
variable
(Arbitrary) domain
function
Variable is added to
the processor's context
Boolean and Troolean
versions
201. Let's put it all
toghether!
i.e. a few examples of
queries from past publications
203. →
Fork f = new Fork(2);
Use BeepBeep as
a library in your
program
204. →
→→
→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor( );
f
Use BeepBeep as
a library in your
program
205. →
→→
+
→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
f
Use BeepBeep as
a library in your
program
206. →
→→
+
→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
f
n
Use BeepBeep as
a library in your
program
207. →
→→
+
→→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
Connector.connect(fork, LEFT, sum, LEFT)
f
n
Use BeepBeep as
a library in your
program
208. →
→→
+
→ →→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
Connector.connect(fork, LEFT, sum, LEFT)
.connect(fork, RIGHT, decimate, INPUT)
f
n
Use BeepBeep as
a library in your
program
209. →
→→
+
→ →→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
Connector.connect(fork, LEFT, sum, LEFT)
.connect(fork, RIGHT, decimate, INPUT)
.connect(decimate, OUTPUT, sum, RIGHT);
f
n
Use BeepBeep as
a library in your
program
210. →
→→
+
→ →→
Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
Connector.connect(fork, LEFT, sum, LEFT)
.connect(fork, RIGHT, decimate, INPUT)
.connect(decimate, OUTPUT, sum, RIGHT);
Pullable p = sum.getOutputPullable(OUTPUT);
while (p.hasNext() != NextStatus.NO) {
Object o = p.next();
...
}
f
n
Use BeepBeep as
a library in your
program
221. →
→
*
*
*
*
Create Auction
=?
0
@
Last Price
Days
0:=
3@Max. Days :=Min. Price 2@:=
Days :=
Days 1
+
End of Day
=?
0
@
>
<?
Days
Last Price
2
@:=
Bid
=?
0
@
>
Min. Price
<?
2
@
Last Price
>?
2
@
Bid
=?
0
@
>
Last Price
>?
2
@
Days :=
Days 1
+
End of Day
=?
0
@
Max. Days
→
End of Day
*
1@
*
→
→→
→
*
Sold
=?
0
@
Days
Days
+ | |.÷
231. import ca.uqac.lif.cep.*;
public class MyProcessor extends SingleProcessor {
public Queue<Object[]> compute(Object[] inputs)
{
}
public void build(Stack<Object> s)
{
}
}
. . . Create output events from input . . .
. . . Instantiate processor from parse stack . . .
<processor> := . . .
<number> := . . .
<string> := . . .
Add new rules to any symbol from
the basic grammar
232. Example: let us create a new 1 : 1 Processor
that increments its input by a fixed value
We would like to use it in eSQL with this
syntax:
INCREMENT ( P ) BY value
Any expression
defining a processor
Number
233. <processor> := <my_processor> ;
<my_processor> := INCREMENT ( <processor> ) BY <number> ;
Symbols already defined in basic grammar
Adds a new case to an existing rule
234. import ca.uqac.lif.cep.*;
public class MyProcessor extends SingleProcessor {
private int increment;
public Queue<Object[]> compute(Object[] inputs)
{
Queue<Vector<Object>> out = new Queue<Vector<Object>>();
Object[] v = new Object[1];
Integer i = (Integer) inputs[0] + increment;
v[0] = i;
out.put(v);
return out;
}
. . .
235. . . .
public void build(Stack<Object> s)
{
Number n = (Number) s.pop();
s.pop();
s.pop();
Processor p = (Processor) s.pop();
s.pop();
s.pop();
increment = n.intValue();
Connector.connect(p, this);
s.push(this);
}
}
Read contents of
parse stack
<number>
BY
(
<processor>
)
INCREMENT
Set processor's state
Pipe it to its input
Put on parse stack
236. Total: 6+9=15 lines of code
. . .
public void build(Stack<Object> s)
{
Number n = (Number) s.pop();
s.pop();
s.pop();
Processor p = (Processor) s.pop();
s.pop();
s.pop();
increment = n.intValue();
Connector.connect(p, this);
s.push(this);
}
}
Read contents of
parse stack
<number>
BY
(
<processor>
)
INCREMENT
Set processor's state
Pipe it to its input
Put on parse stack
237. Some pre-packaged grammar extensions:
Manipulation of name-value tuples
Set theory
Formatted input (CSV, XML, JSON)
Graphing (histograms, scatterplots, ...)
Basic signal processing (smoothing,
peak detection, ...)
Create your own!
238. import ca.uqac.lif.cep.*;
import ca.uqac.lif.cep.eml.tuples.*;
public class MyExample {
public static void main(String[] args)
{
Interpreter my_int = new Interpreter();
my_int.extendGrammar(TupleGrammar.class);
Pullable p = my_int.executeQuery(
""HELLO WORLD"");
for (int i = 0; i < 10; i++)
{
EmlString s = (EmlString) p.pull();
System.out.println(s);
}
}
}
Create query interpreter
Load a grammar extension
Execute a query
Pull an output event
239. Example: let us create a new 1 : 1 Processor
that parses XML events (using an existing
XML library)
This can be done by first defining a Function
that converts Strings into XML objects
240. import ca.uqac.lif.cep.*;
import ca.uqac.lif.xml.*;
public static class XmlParsingFunction extends UnaryFunction<String,XmlElement> {
public static XmlParsingFunction instance = new XmlParsingFunction();
private XmlParsingFunction() {
super(String.class, XmlElement.class);
}
public XmlElement getValue(String x) {
try {
return XmlElement.parse(x);
}
catch (XmlParseException e) {
}
return null;
}
}
}
241. import ca.uqac.lif.cep.*;
import ca.uqac.lif.xml.*;
public static class XmlParsingFunction extends UnaryFunction<String,XmlElement> {
public static XmlParsingFunction instance = new XmlParsingFunction();
private XmlParsingFunction() {
super(String.class, XmlElement.class);
}
public XmlElement getValue(String x) {
try {
return XmlElement.parse(x);
}
catch (XmlParseException e) {
}
return null;
}
}
}
Total: 10 lines of code
251. BeepBeep needs you!BeepBeep needs you!
Licensed under LGPL: download and
use it
Create you own domain-specific palettes
Take your own monitor/script and wrap
it into a new Processor object
For free: benefit from all existing processors
and functions already available
https://liflab.github.io/beepbeep-3