Event Processing In Java A Financial Services Case Study with Esper

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    3 Favorites & 1 Group

    Event Processing In Java A Financial Services Case Study with Esper - Presentation Transcript

    1. Event Processing in Java A Financial Services Case Study Thomas Bernhardt Founder/Project Lead, Esper EsperTech Inc. tom@espertech.com
    2. Learn the concepts of Complex Event Processing (CEP) How to do real-time computation and pattern detection over event streams Understand its value in a concrete case study
    3. Material Introduction to CEP Esper A Financial Services Case Study Code Challenges and Future Direction
    4. Enterprise 2.0 Real-time Enterprise More agile, connected and responsive Trends • More fine-grained data • Frequent data changes • Timely responses are more valuable ....your enterprise architecture? • Not just Financial Services / Trading Floor!
    5. Real-Time Enterprise Enablers Service-Oriented Architecture (SOA) • Today mostly request-response over HTTP • Gaining in one-way MEP, publish-subscribe Event-Driven Architecture (EDA) • Today mostly via MOM and strong message models Extreme Transaction Processing (XTP) • Real time performance, horizontal scalability, commodity hardware Σ Processing events ...and generating more events
    6. Business Event, such as - Order Instructions Received - Credit Check Succeeded Application Event, such as - User clicked submit button - Credit Check web service indicated success System Event, such as - Order took 1 sec to persist - Credit Check service connection pool is low
    7. Multiple Events Together Can Make An Actionable Event Large Quantity Illegal Fake E-Mail Purchase of a + Newsletter = Promotion Detected Penny-Stock
    8. Multiple Events Together Can Make An Actionable Event Large Quantity Illegal Fake E-Mail Purchase of a + Newsletter = Promotion Detected Penny-Stock Complex Event Processing (CEP) : processing multiple events to identify meaningful events
    9. A CEP Engine = Events Query 1 Query 2 When When 3 errors within 10 seconds average latency over the last 10 sec. then alert is 10% above average latency over the last 60 sec. then alert
    10. CEP Engine Abstract Input Filter Output Aggregate Pattern Match Join/Gather Time
    11. Material Introduction to CEP Esper A Financial Services Case Study Code Challenges and Future Direction
    12. Esper - Java CEP engine http://esper.codehaus.org Mission • CEP a natural and easy task of your application • Open platform to create competitive differentiators and maximize reuse ...and therefore • Lightweight and embeddable into any Java technology process • Open source • Low latency and high throughput Project background • 3 years of development, 100% Java • NEsper for .NET/C#, same semantics • Commercial liability, support and services available EsperTech Inc.
    13. Esper Provides... an EPL – Event Processing Language • SQL look-alike to process streams of events • Data windows • Pattern matching • ... think continuous query an API • Create EPL statements, start and stop • Attach listeners or observers to receive results • Send events into the engine • Plus some extras: pull API, SODA, configuration, event representations
    14. Esper API - Hello World in CEP 1) Create Event public class HelloWorld {} 2) Define EPL Statement EPStatement statement = admin.create( “select * from HelloWorld”); 3) Attach observer public class MyObserver { update(HelloWorld event) { System.out.println (“Hello World”); } } statement.setObserver(new MyObserver()); 4) Send Events runtime.sendEvent(new HelloWorld());
    15. EPL Samples (1) - Data Window select * from TradeEvent.win:time(30 seconds) - Filtering select * from TradeEvent(symbol='IBM').win:time(30 seconds) - Aggregation select sum(price) from TradeEvent(symbol='IBM').win:time(30 seconds) - Join select * from TradeEvent(symbol='IBM').win:time(30 seconds) as te, SettleEvent(symbol='IBM').win:time(30 seconds) as se where te.orderId = se.orderId
    16. EPL Samples (2) - Insert-Into and Output-Every insert into SMA /* Simple Moving Average */ select symbol, avg(price) as avgPrice from Tick.win:time(5 days) group by symbol output all every 10 minutes - Pattern Matching // Detect an upwards trend ('->' reads as followed-by) every e1=SMA(symbol='GOOG') -> ( timer:interval(60 min) and not SMA(symbol='GOOG', avgPrice < e1.avgPrice) )
    17. Material Introduction to CEP Esper A Financial Services Case Study Code Challenges and Future Direction
    18. Financial Services Case Study Foreign Exchange Trading at a Major European Investment Bank Report suspicious activity and prevent fraud Java event server application that handles settlement flow • Spring wiring to JMS channels • CEP with Esper • Tomcat / DWR front-end
    19. Use Case #1 Event • Trade direction, amount, user id, account, currencies traded Use Case #1 • Looking for trades booked by a group of counterparties > X million dollars within a moving time window select sum(amount) from FXTrade(acct in ('1001', '1002')).win:time(1 hour) having sum(amount) > 1000000
    20. Use Case #2 A series of 3 or more Buy (or Sell) trades within 8 hours followed by a Sell (or Buy) trade within 1 hour after the last Buy (or Sell) trade select e1, e2, e3, final from pattern [ every e1=FXTrade(userId in (100,101)) -> ( ( e2=FXTrade(userId in (100,101), dir = e1.dir) -> e3=FXTrade(userId in (100,101), dir = e1.dir) ) where timer:within(8 hours) and not FXTrade(userId in (100,101), dir != e1.dir) ) -> final=FXTrade(userId in (100,101), dir != e1.dir) where timer:within(1 hour) ]
    21. (IDE with code here)
    22. Financial Services Case Study -Wrap Up Numbers • 50 statements • Triggering ratio 5% • 100k to 200k events per day Quotes • \"We had no major issues, it was simple to setup, test and proves to be reliable\" • “We are pleased with the quality of the core engine and the ease of development, and the timely responses to any technical questions we submit\" All open source yet professionally supported software
    23. Challenges and Future Direction CEP is stateful publish/subscribe • Low latency and high throughput • Horizontal scalability and fail-over requirements • ... are often trade-offs Data model integration • Different systems, different primary keys and meanings Standards • No CEP or EPL language standard, and no API standard
    24. Join us online http://www.espertech.com http://esper.codehaus.org
    25. Questions

    + espertechespertech, 2 years ago

    custom

    1135 views, 3 favs, 0 embeds more stats

    TheServerSideSymposium, March 2008

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1135
      • 1135 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 3
    • Downloads 34
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    Groups / Events