This talk reviews the basic principles behind the BeepBeep 3 event stream processing engine, and the facilities it provides to help you design you own, domain-specific query language.

1. A " Do-It-Yourself "
Specification Language
with
Sylvain Hallé
by
CRSNG
NSERC

2. EventsEvents are arbitrary data
elements produced in a sequence
called a stream. Any Java object can
be used as an event.
2
3
4
π abc
3 8 a
3 8 a
2 6 c
+
⊇?
<a><a><a>
Numbers Strings XML
documents
Booleans
Tuples Functions Sets Plots

3. ⊇?
3
4
FunctionsFunctions
1:1
function
2:1
function
6
6 0:1
function
transform events into other events.

4. Function
Applies a function to
every input event
Cumulative
Computes the
progressive "sum" of
all input events
Trim
Removes the first n
input events
Decimate
Outputs every
n-th event
Group
Encloses a group of
connected processors
Fork
Duplicates a stream
into multiple copies
Slice
Splits a stram into multiple
sub-streams
Window
Applies a function to
a sliding window of
n events
ProcessorsProcessors are simple computing
units that transform input streams into output
streams. BeepBeep's core is made of a handful of
general purpose processors.
f
Σ f
n
Filter
A first, Boolean stream, decides if each
event of a second stream should be output
{
f

5. PipingPiping the output of processors to the
input of others creates a chain that produces
the desired computation.
Any output can be connected to any input,
as long as they have the same type. Many types
can occur in the same chain. The final output
of the chain can be of any type, too.

6. Fork f = new Fork(2);
FunctionProcessor sum =
new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
Connector.connect(fork, LEFT, sum, LEFT)
. connect(fork, RIGHT, decimate, INPUT)
. connect(decimate, OUTPUT, sum, RIGHT);
Pullable p = sum.getOutputPullable(OUTPUT);
while (p.hasNext() != NextStatus.NO) {
Object o = p.next();
. ..
}
n
→
→
→
+
→
→
→
→
→
f
1 LOC per vertex
1 LOC per edge

7. They can contain arbitrary Java code, member
fields, etc. "What happens in the box remains in
the box."
CustomCustom processors and functions can
be created by inheriting from the basic Processor
and Function objects provided by BeepBeep.
class Counter extends UniformProcessor {
int cnt = 0;
public Counter() {
super(1,1);
}
public void compute(Object[] in, Queue<> out) {
cnt++;
out.add(new Object[]{cnt});
}
}

8. PalettesPalettes are groups of functions and
processors centered around a particular use case.
They represent a simple, but powerful way of
extending BeepBeep to a user's needs. A few of
the palettes available so far:
XML events
Plotting
Finite-state machines
Temporal logic
Signal processing
Various log formats
Networking
...create your own

9. 1 2 3 4 5
Σ
0
+6
...2 7 1 8 2
f
1
Σ
0
+
Σ
0
+
÷

10. 1
x,
y
x
+

11. t
+
x
1
Σ
0
+
½ s
x
4
{
4
4
x̅

12. →
A
/a/b
//character[status=Walker]/id/text()
→ p1
→
A
→ p2
→ →
→
→
/a/b
→
→
→
3
→
→
→
<?
→
→
→
→
→
→
→
→
f1
f2
→
→ →
→
→
→
→
//character[status=Blocker]/id/text()

13. →
→
*
*
*
*
Create Auction
=?
0
@
Last Price
Days
0:=
3@Max. Days :=Min. Price 2@:=
Days :=
Days 1
+
End of Day
=?
0
@
>
<?
Days
Last Price
2
@:=
Bid
=?
0
@
>
Min. Price
<?
2
@
Last Price
>?
2
@
Bid
=?
0
@
>
Last Price
>?
2
@
Days :=
Days 1
+
End of Day
=?
0
@
Max. Days
→
End of Day
*
1@
*
→
→→
→
*
Sold
=?
0
@
Days
Days
+ | |.÷
→→
→
→
SELECT
@0 AS x
@1 AS y
→→
+
0
Σ→ →1
→
→→
∅ +=
Σ

14. →
→
x
4
f
Σ
0
+
GroupGroup processors encapsulate a chain
of processors; they can have arguments.

15. →
→
x
4
f
Σ
0
+
GroupGroup processors encapsulate a chain
of processors; they can have arguments.

16. →
→
x
4
Σ
0
+f
GroupGroup processors encapsulate a chain
of processors; they can have arguments.

17. ?
? ??
foo
GroupGroup processors encapsulate a chain
of processors; they can have arguments.

18. BeepBeep provides a runtime parser
called Bullwinkle.
It comes with no grammar!
You can define your own, along
with a parse tree visitor, to
create your own BeepBeep
domain-specific language.

19. <Processor> := <CountDecimate> | <CumulativeSum>
| <Mutator> | <QueueSource> ;
<CountDecimate> := Take one every <Number> from <Processor> ;
<CumulativeSum> := Accumulate <Processor> ;
<Mutator> := Turn <Processor> into <Number> ;
<QueueSource> := [ <Number> , <Number> , <Number> ] ;
<Number> := ^[d]+;
Step 1Step 1
Define a grammar in BNF.

20. class MyInterpreter extends ExpressionParser<Processor> {
public void doCountDecimate(Stack<Object> stack) {
Processor p = (Processor) stack.pop();
stack.pop(); // from
Constant n = (Constant) stack.pop();
stack.pop(); // every
stack.pop(); // one
stack.pop(); // Take
CountDecimate dec = new CountDecimate(
((Number) n.getValue()).intValue());
Connector.connect(p, dec);
m_builtObject.addProcessor(dec);
stack.push(dec);
}
...
}
Step 2Step 2
Create an ExpressionParser, with a method
doXXX for every non-terminal symbol xxx you
want to handle.

21. MyInterpreter my_int = new MyInterpreter();
Processor proc = my_int.parse(
"Take one every 2 from The sum of [3, 4, 6]");
SomeOtherProcessor op = ..
Connector.connect(proc, op);
...
Step 3Step 3
Enjoy!
3 4
Σ
0
+6
2

22. Can be used to pipe together any
processor and/or function
...including those you define yourself
...(that may contain arbitrary Java code)
...and only those you want to include
...from any existing palette
...using a syntax you choose
An easy way to create simple
languages that do complex things
https://liflab.github.io/beepbeep-3
.../beepbeep-3-examples