Event Processing in Java
A Financial Services Case Study
Thomas Bernhardt
Founder/Project Lead, Esper
EsperTech Inc.
tom@espertech.com

Learn the concepts of
Complex Event Processing (CEP)
How to do real-time computation and pattern
detection over event streams
Understand its value in a concrete case study

Material
Introduction to CEP
Esper
A Financial Services Case Study
Code
Challenges and Future Direction

Enterprise 2.0
Real-time Enterprise
More agile, connected and responsive
Trends
• More fine-grained data
• Frequent data changes
• Timely responses are more valuable
....your enterprise architecture?
• Not just Financial Services / Trading Floor!

Real-Time Enterprise Enablers
Service-Oriented Architecture (SOA)
• Today mostly request-response over HTTP
• Gaining in one-way MEP, publish-subscribe
Event-Driven Architecture (EDA)
• Today mostly via MOM and strong message models
Extreme Transaction Processing (XTP)
• Real time performance, horizontal scalability, commodity hardware
Σ Processing events
...and generating more events

Business Event, such as
- Order Instructions Received
- Credit Check Succeeded
Application Event, such as
- User clicked submit button
- Credit Check web service
indicated success
System Event, such as
- Order took 1 sec to persist
- Credit Check service
connection pool is low

Multiple Events Together
Can Make An Actionable Event
Large Quantity Illegal
Fake E-Mail
Purchase of a + Newsletter = Promotion
Detected
Penny-Stock

Multiple Events Together
Can Make An Actionable Event
Large Quantity Illegal
Fake E-Mail
Purchase of a + Newsletter = Promotion
Detected
Penny-Stock
Complex Event Processing (CEP) :
processing multiple events to identify meaningful events

A CEP Engine = Events
Query 1 Query 2
When When
3 errors within 10 seconds average latency over the last 10 sec.
then alert is 10% above
average latency over the last 60 sec.
then alert

CEP Engine Abstract
Input Filter Output
Aggregate Pattern
Match
Join/Gather
Time

Material
Introduction to CEP
Esper
A Financial Services Case Study
Code
Challenges and Future Direction

Esper - Java CEP engine
http://esper.codehaus.org
Mission
• CEP a natural and easy task of your application
• Open platform to create competitive differentiators and maximize reuse
...and therefore
• Lightweight and embeddable into any Java technology process
• Open source
• Low latency and high throughput
Project background
• 3 years of development, 100% Java
• NEsper for .NET/C#, same semantics
• Commercial liability, support and services available EsperTech Inc.

Esper Provides...
an EPL – Event Processing Language
• SQL look-alike to process streams of events
• Data windows
• Pattern matching
• ... think continuous query
an API
• Create EPL statements, start and stop
• Attach listeners or observers to receive results
• Send events into the engine
• Plus some extras: pull API, SODA, configuration, event representations

Esper API - Hello World in CEP
1) Create Event
public class HelloWorld {}
2) Define EPL Statement
EPStatement statement = admin.create(
“select * from HelloWorld”);
3) Attach observer
public class MyObserver {
update(HelloWorld event)
{ System.out.println (“Hello World”); }
}
statement.setObserver(new MyObserver());
4) Send Events
runtime.sendEvent(new HelloWorld());

EPL Samples (1)
- Data Window
select * from TradeEvent.win:time(30 seconds)
- Filtering
select * from TradeEvent(symbol='IBM').win:time(30 seconds)
- Aggregation
select sum(price)
from TradeEvent(symbol='IBM').win:time(30 seconds)
- Join
select * from TradeEvent(symbol='IBM').win:time(30 seconds) as te,
SettleEvent(symbol='IBM').win:time(30 seconds) as se
where te.orderId = se.orderId

EPL Samples (2)
- Insert-Into and Output-Every
insert into SMA /* Simple Moving Average */
select symbol, avg(price) as avgPrice
from Tick.win:time(5 days)
group by symbol
output all every 10 minutes
- Pattern Matching
// Detect an upwards trend ('->' reads as followed-by)
every e1=SMA(symbol='GOOG')
-> ( timer:interval(60 min)
and not SMA(symbol='GOOG', avgPrice < e1.avgPrice)
)

Material
Introduction to CEP
Esper
A Financial Services Case Study
Code
Challenges and Future Direction

Financial Services Case Study
Foreign Exchange Trading at a Major European
Investment Bank
Report suspicious activity and prevent fraud
Java event server application that handles settlement
flow
• Spring wiring to JMS channels
• CEP with Esper
• Tomcat / DWR front-end

Use Case #1
Event
• Trade direction, amount, user id, account, currencies traded
Use Case #1
• Looking for trades booked by a group of counterparties > X million
dollars within a moving time window
select sum(amount)
from FXTrade(acct in ('1001', '1002')).win:time(1 hour)
having sum(amount) > 1000000

Use Case #2
A series of 3 or more Buy (or Sell) trades within 8 hours followed by a
Sell (or Buy) trade within 1 hour after the last Buy (or Sell) trade
select e1, e2, e3, final from pattern [
every e1=FXTrade(userId in (100,101))
-> (
(
e2=FXTrade(userId in (100,101), dir = e1.dir)
-> e3=FXTrade(userId in (100,101), dir = e1.dir)
) where timer:within(8 hours)
and not FXTrade(userId in (100,101), dir != e1.dir)
)
-> final=FXTrade(userId in (100,101), dir != e1.dir)
where timer:within(1 hour) ]

(IDE with code here)

Financial Services Case Study -Wrap Up
Numbers
• 50 statements
• Triggering ratio 5%
• 100k to 200k events per day
Quotes
• \"We had no major issues, it was simple to setup, test and proves to be
reliable\"
• “We are pleased with the quality of the core engine and the ease of
development, and the timely responses to any technical questions we
submit\"
All open source yet professionally supported software

Challenges and Future Direction
CEP is stateful publish/subscribe
• Low latency and high throughput
• Horizontal scalability and fail-over requirements
• ... are often trade-offs
Data model integration
• Different systems, different primary keys and meanings
Standards
• No CEP or EPL language standard, and no API standard

Join us online
http://www.espertech.com
http://esper.codehaus.org

Questions