1. Mark
Proctor
Project Lead
The SkyNet funding bill is passed.
The system goes online on August 4th, 1997.
Human decisions are removed from strategic defense.
SkyNet begins to learn at a geometric rate.
It becomes self-aware at 2:14am Eastern time, August 29th
In a panic, they try to pull the plug.
And, Skynet fights back

2. 2
What is C omplex E vent Proces s ing?

3. 3
What is C omplex E vent Proces s ing?
My answer: the ability to make more money than your competitors. ;)

4. 4
What is C omplex E vent Proces s ing
1. Detect 2. Correlate
3. React

5. 5
Time is Money
Business Event
Value Loss
Business Value
Reaction
Time
Time Loss
Adapted from a presentation by James Taylor, Sep/2011

6. 6
Terminology: Event
F or th e s cop e of th is p re s e ntation:
“An e ve nt is a
s ignificant
change of s tate
at a p articu lar point in time”

7. 7
Terminology: Complex Event
“C omplex E vent, is an ab s traction of oth e r e ve nts
calle d its m e m b e rs .”
E xam p le s :
The 1929 s toc k market cras h – an ab s traction d e noting m any th ou s and s
of m e m b e r e ve nts , inclu d ing ind ivid u al s tock trad e s )
The 2004 Indones ian Ts unami – an ab s traction of m any natu ral e ve nts
A c ompleted s tock purc has e -an ab s traction of th e e ve nts in a
trans action to p u rch as e th e s tock
A s uc c es s ful on-line s hopping c art c hec kout – an ab s traction of
s h op p ing cart e ve nts on an on-line we b s ite
S ource : http :/com p l xe ve nts.com
/ e

8. 8
Terminology: CEP
“C omplex E vent Proces s ing, or C E P , is p rim arily
an e ve nt p roce s s ing conce p t th at d e als with th e
tas k of p roce s s ing m u ltip le e ve nts with th e goal
of identifying the meaningful events with in th e
e ve nt clou d .
C E P e m p loys te ch niqu e s s u ch as detection of
com p le x p atte rns of m any e ve nts , e ve nt
correlation and abs traction, e ve nt h ie rarch ie s ,
and re lations h ip s b e twe e n e ve nts s u ch as
cau s ality, m e m b e rs h ip , and tim ing, and e ve nt-
d rive n p roce s s e s .”
-- wikip e d ia

9. 9
Terminology: CEP
E xam p le s :
E m e rge ncy R e s p ons e S ys te m s
C re d it C ard F rau d D e te ction
Logis tics R e al-Tim e Aware ne s s s olu tion
N e onatal IC U : infant vital s igns m onitoring

10. 10
Terminology: CEP vs ESP
C omplex E vent Proces s ing, or C E P , and E vent
S tream Proces s ing, or E S P , are two
te ch nologie s th at we re b orn s e p arate , b u t
conve rge d .
An ove rsim p l
ification: In th e ir origins ...
E ve nt S tre am P roce s s ing focu s e d on th e ab ility to p roce s s h igh
volu m e s treams of e ve nts .
C om p le x E ve nt P roce s s ing focu s e d on d e fining, d e te cting and
p roce s s ing th e relations hips am ong e ve nts .

11. 11
Terminology: CEP and ESP
F or th e s cop e of th is p re s e ntation:
“C E P is u s e d as a com m on te rm
m e aning b oth C E P and E S P.”

12. 12
Terminology: EDA
“E vent Driven A rc hitecture (E DA ) is a s oftware arch ite ctu re
p atte rn p rom oting th e p rod u ction, d e te ction, cons u m p tion of,
and re action to e ve nts . An e ve nt can b e d e fine d as "a
s ignificant ch ange in s tate "[1 ]. F or e xam p le , wh e n a
cons u m e r p u rch as e s a car, th e car's s tate ch ange s from "for
s ale " to "s old ". A car d e ale r's s ys te m arch ite ctu re m ay tre at
th is s tate ch ange as an e ve nt to b e p rod u ce d , p u b lis h e d ,
d e te cte d and cons u m e d b y variou s ap p lications with in th e
arch ite ctu re .”
h ttp :/ e n.wikip e d ia.org/
/ wiki/ ve nt_ D rive n_ Arch ite ctu re
E

13. 13
EDA vs CEP
CEP is a component of the EDA
Source: http://elementallinks.typepad.com/.shared/image.html?/photos/uncategorized/simple_event_flow.gif

14. 14
E DA vs S OA
E D A is **not** S O A 2.0
C om p le m e ntary arch ite ctu re s
M e tap h or
In ou r b od y:
S O A is u s e d to b u ild ou r m u s cle s and organs
E D A is u s e d to b u ild ou r s e ns ory s ys te m

15. 15
E vent Driven A rchitectures
edBPM + E DM

16. 16
E DA vs S OA
Source: http://soa-eda.blogspot.com/2006/11/how-eda-extends-soa-and-why-it-is.html

17. 17
C omplex E vent Proces s ing
A few characteris tics of common C E P s cenarios :
H u ge volu m e of e ve nts , b u t only a fe w of re al inte re s t
U s u ally e ve nts are im m u tab le
U s u ally qu e rie s / le s h ave to ru n in re active m od e
ru
S trong te m p oral re lations h ip s b e twe e n e ve nts
Ind ivid u al e ve nts are u s u ally not im p ortant
Th e com p os ition and aggre gation of e ve nts is im p ortant

18. 18
JBOSS Enterprise BRMS v5.2

19. 19
S mart S olutions
CEP
Business Rules Business Events Processing
Seamlessly Integrated (Re)Active component
Platform
Business Processes Business Ontologies

20. 20

21. 21
Drools Fus ion: E nables …
E vent Detection:
F rom an e ve nt clou d or s e t of s tre am s , s e le ct all th e m e aningfu l
e ve nts , and only th e m .
[Temporal] E vent C orrelation:
Ab ility to corre late e ve nts and facts d e claring b oth te m p oral and
non-te m p oral cons traints b e twe e n th e m .
Ab ility to re as on ove r e ve nt aggre gation
E vent A bs traction:
Ab ility to com p os e com p le x e ve nts from atom ic e ve nts AN D re as on
ove r th e m

22. 22
Drools Fus ion
Features :
E ve nt S e m antics as F irs t C las s C itize ns
Allow D e te ction, C orre lation and C om p os ition
Te m p oral C ons traints
S e s s ion C lock
S tre am P roce s s ing
S lid ing Wind ows
C E P volu m e s (s calab ility)
(R e )Active R u le s
D ata Load e rs for Inp u t

23. 23
Demo
Twitter S tream C E P Demo:
Lis te n to th e Twitte r S tre am AP I
Twitte r4J AP I
Lis te ns to a rand om s am p le of twe e ts
D e te cts p atte rns and re acts
D rools F u s ion
S im p le one p roce s s (m u lti-th re ad ) d e m o
F ocu s on s p e cific fe atu re s

24. 24
E vent Declaration and S emantics
// declaring existing class
import some.package.VoiceCall E ve nt s e m antics :
declare VoiceCall
@role( event )
P oint-in-tim e and Inte rval
@timestamp( calltime )
@duration( duration )
end An e ve nt is a fact with a fe w s p e cial
// generating an event class
ch aracte ris tics :
declare StockTick
U s u ally im m u tab le , b u t not e nforce d
@role( event )
S trong te m p oral re lations h ip s
symbol : String
price : double Life cycle m ay b e m anage d
end
Allow u s e of s lid ing wind ows
“All e ve nts are facts , b u t not all facts
are e ve nts .”

25. 25
Temporal Reas oning
S e m antics for:
time: d is cre te
events : p oint-in-tim e and inte rval
Ab ility to e xp re s s te m p oral re lations h ip s :
Alle n’s 1 3 te m p oral op e rators
J ames F. A llen d e fine d th e 1 3 p os s ib le te m p oral re lations
b e twe e n two e ve nts .
E iko Yoneki and J ean B acon d e fine d a u nifie d s e m antics for
e ve nt corre lation ove r tim e and s p ace .

26. 26
Temporal Relations hips
rule “Shipment not picked up in time”
when
Shipment( $pickupTime : scheduledPickupTime )
not ShipmentPickup( this before $pickupTime )
then
// shipment not picked up... action required.
end

27. 27
Temporal Relations hips
rule “Shipment not picked up in time”
when
Shipment( $pickupTime : scheduledPickupTime )
not ShipmentPickup( this before $pickupTime )
then
// shipment not picked up... Action required.
end
Temporal
Relationship

28. 28
A llen’s 13 Temporal Operators
Point-Point Point-Interval Interval-Interval
A
A before B B
A
A meets B B
A
A overlaps B
B
A
A finishes B B
A
A includes B
B
A
A starts B B
A
A coincides B
B

29. 29
A llen’s 13 Temporal Operators
Point-Point Point-Interval Interval-Interval
A
A after B B
A
A metBy B B
A
A overlapedBy B
B
A
A finishedBy B B
A
A during B
B
A
A finishes B B

30. 30
S treams : S imple E xample S cenario

31. 31
S tream S upport (entry-points )
A s coping abs traction for s tream s upport
R u le com p ile r gath e r all e ntry-p oint d e clarations and e xp os e th e m
th rou gh th e s e s s ion AP I
E ngine m anage s all th e s cop ing and s ynch ronization b e h ind th e
s ce ne s .
rule “Stock Trade Correlation”
when
$c : Customer( type == “VIP” )
BuyOrderEvent( customer == $c, $id : id ) from entry-point “Home Broker Stream”
BuyAckEvent( sourceEvent == $id ) from entry-point “Stock Trader Stream”
then
// take some action
end

32. 32
Cloud Mode, Stream Mode, Session Clock
CLOUD STREAM
N o notion of “flow of tim e ”: th e N otion of “flow of tim e ”:
e ngine s e e s all facts with ou t conce p t of “now”
re gard to tim e S e s s ion C lock h as an active
N o attach e d S e s s ion C lock role s ynch ronizing th e
N o re qu ire m e nts on e ve nt re as oning
ord e ring E ve nt S tre am s m u s t b e
N o au tom atic e ve nt life cycle ord e re d
m anage m e nt Au tom atic e ve nt life cycle
N o s lid ing wind ow s u p p ort m anage m e nt
S lid ing wind ow s u p p ort
Au tom atic ru le d e laying on
ab s e nce of facts

33. 33
Reference C lock
R e fe re nce clock d e fine s th e flow of tim e
N am e d S es s ion C lock
is as s igne d to e ach s e s s ion cre ate d
S ynch ronize s tim e s e ns itive op e rations
d u ration ru le s
e ve nt s tre am s
p roce s s tim e rs
s lid ing wind ows

34. 34
S es s ion C lock
Uses the strategy pattern and multiple implementations:
Real-time operation
Tests
Simulations
etc

35. 35
S es s ion C lock
Selecting the session clock:
API:
KnowledgeSessionConfiguration conf = ...
conf.setOption( ClockTypeOption.get( “realtime” ) );
System Property or Configuration File:
drools.clockType = pseudo

36. 36
S liding Window S upport
Allows re as oning ove r a m oving wind ow of “inte re s t”
Tim e
Le ngth
Sliding window 1
Sliding window 2

37. 37
S liding Window S upport
Allows re as oning ove r a m oving wind ow of “inte re s t”
Tim e
Le ngth
Sliding window 1
Sliding window 2
Joined window

38. 38
S liding Window S upport
Allows re as oning ove r a m oving wind ow of “inte re s t”
Tim e
Le ngth
rule “Average Order Value over 12 hours”
when
$c : Customer()
$a : Number() from accumulate (
BuyOrder( customer == $c, $p : price )
over window:time( 12h ),
average( $p ) )
then
// do something
end

39. 39
Delaying Rules
Negative patterns may require rule firings to be delayed.
rule “Order timeout”
when
$bse : BuyShares ( $id : id )
not BuySharesAck( id == $id, this after[0s,30s] $bse )
then
// Buy order was not acknowledged. Cancel operation
// by timeout.
end

40. 40
Delaying Rules
Negative patterns may require rule firings to be delayed.
rule “Order timeout”
when
$bse : BuyShares ( $id : id )
not BuySharesAck( id == $id, this after[0s,30s] $bse )
then
// Buy order was not acknowledged. Cancel operation
// by timeout.
end
Forces the rule to wait for 30 seconds before firing, because the
acknowledgement may arrive at any time!

41. 41
Temporal Dimens ion
R e qu ire s th e s u p p ort to th e te m p oral d im e ns ion
A ru le / e ry m igh t m atch in a give n p oint in tim e , and not m atch in
qu
th e s u b s e qu e nt p oint in tim e
Th at is th e s ingle m os t d ifficu lt re qu ire m e nt to s u p p ort in a way
th at th e e ngine :
s tays d e te rm inis tic
s tays a h igh -p e rform ance e ngine
Ach ie ve d m os tly b y com p ile tim e op tim izations th at e nab le :
cons traint tigh te ning
m atch s p ace narrowing
m e m ory m anage m e nt

42. 42
Temporal Dimens ion S upport
C E P s ce narios are s tate fu l b y natu re .
E ve nts u s u ally are only inte re s ting d u ring a s h ort p e riod of
tim e .
H ard for ap p lications to know wh e n e ve nts are not ne ce s s ary
anym ore
Te m p oral cons traints and s lid ing wind ows d e s crib e s u ch “wind ow of
inte re s t”

43. 43
S imple E xample Rule
rule “Bag was lost”
when
$b : BagScannedEvent() from entry-point “check-in”
not BagScannedEvent( id == $b.id, this after[0s,5m] $b )
from entry-point “pre-load”
then
// Bag was lost, do something
end
Easy to “see” that the only temporal relationship between the
events defines a 5 minutes interest window.

44. 44
C alendars
rule "weekdays are high priority"
calendars "weekday"
timer (int:0 1h) Execute now and after
1 hour duration
when
Alarm()
then
send( "priority high - we have an alarm” );
end
rule "weekend are low priority"
calendars "weekend"
timer (int:0 4h) Execute now and after
4 hour duration
when
Alarm()
then
send( "priority low - we have an alarm” );
end

45. 45
Timers
Field Name Mandatory? Allowed Values Allowed Special Characters
Seconds YES 0-59 ,-*/
Minutes YES 0-59 ,-*/
Hours YES 0-23 ,-*/
Day of month YES 1-31 ,-*?/LW
Month YES 1-12 or JAN-DEC ,-*/
Day of week YES 1-7 or SUN-SAT ,-*?/L#
Year NO empty, 1970-2099 ,-*/
Send alert every quarter of an
hour
rule “name”
timer ( cron: 0 0/15 * * * * )
when
Alarm( )
then
sendEmail( ”Alert Alert Alert!!!” )

46. 46
A bs tract E xample Rule
rule “Abstract event relationship example”
when
$a : A()
$b : B( this after[-2, 2] $a )
$c : C( this after[-3, 4] $a )
$d : D( this after[ 1, 2] $b, this after[2,3] $c)
not E( this after[ 1,10] $d )
then
// Bag was lost, do something
end
How about now? What is the temporal relationship between
A and E?

47. 47
Temporal Dependency Matrix
[-2,2] B [1,2]
[1,10]
A D E
[-3,4] C [2,3]
Constraint tightening
A B C D E
A [ 0, 0 ] [ -2, 2 ] [ -3, 2 ] [ -1, 4 ] [ 0, 14 ]
B [ -2, 2 ] [ 0, 0 ] [ -2, 0 ] [ 1, 2 ] [ 2, 12 ]
C [ -2, 3 ] [ 0, 2 ] [ 0, 0 ] [ 2, 3 ] [ 3, 13 ]
D [ -4, 1 ] [ -2, -1 ] [ -3, -2 ] [ 0, 0 ] [ 1, 10 ]
E [ -14, 0 ] [ -12, -2 ] [ -13, -3 ] [-10,-1 ] [ 0, 0 ]

48. 48
C E P A pplied at FedE x C us tom C ritical
* Presented by Adam Mollemkopf at ORF 2009

49. 49
C E P A pplied at FedE x C us tom C ritical
* Presented by Adam Mollemkopf at ORF 2009

50. 50
C E P A pplied at FedE x C us tom C ritical
* Presented by Adam Mollemkopf at ORF 2009
At least 50% of Alerts can be reasoned automatically, promoting
staff savings and improved Customer and Driver experiences.
Risk Avoidance via pro-active monitoring
Reduction in insurance claims and shipment service failures
Minimum 30% efficiency gains in shipment monitoring , saving at
least 15% of Operations staff cost.

51. 51
C E P A pplied at FedE x C us tom C ritical
* Presented by Adam Mollemkopf at ORF 2009
Some numbers (from early 2010):
24 x 7 sessions, no downtime
Average of 500k+ facts/events concurrently in memory
Business hours: 1M+ facts/events concurrently
Response time for reasoning cycles:
Average: 150 ms
Peak: 1.2 sec
Several hundred rules

52. 52
Differential Update
Differential Update (a.k.a. “true modify”)
Implements a real “modify/update” operation, instead of
retract+assert.
Reuses tuples, reduces GC stress, improves performance

53. Q&A
Drools project s ite:
h ttp :/ www.d rools .org ( h ttp :/ www.j os s .org/ rools / )
/ / b d
Doc umentation:
h ttp :/ www.j os s .org/ rools / ocu m e ntation.h tm l
/ b d d
Edson Tirelli – etirelli@redhat.com