Event Processing - The Next Generation; March 2009

IBM Haifa Research Lab – Event Processing ' ( I am wearing three different hats today The industrial hat: IBM (which pays my salary and in return occupies most of my time) Once a week I am wearing the Professor hat and teach an event processing course The EPTS chair hat – a catalyst in establishing this community IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing )* + ,-./ 0 12 - ) 3 4 5 6 5 4 6 '# # ) ! #) # # # 4 \" 4 # 7 7# ) 4 4 7 4 ! 8 ' 2 ! #& 4 ) !4 # # 4 # # 0 $ $ $# 3 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing 4 What is event processing ? What is it used for? A quick Introduction second generation What are the current Research Challenges ? Survey of recent projects Summary IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing # ) !4 Every business person reacts to events, from time to time Well… there are all sort of reactions… IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing 7 4 4 I closed the deal with the I closed Australians the deal with the Canadians We closed two huge deals in a single day, It is a good opportunity to send all the team to Las-Vegas IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing ' ( 9 0 ) 3 0 ) 3 4 0 ! 4$ 3 ) 4 # 0 4 ) 5 63 # ) 4 + :# # ! ; < 8 7 7 & # ! 9 # ! 9 )5!4 4 6 #!) ) 07 4 4 #!) 3$ 2 0 3$ # ) # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing 9 4 ? $ $ $ 4 $ $# Event Processing Event Event Producer Consumer Generate and Consume and may publish react to either perform operations on events raw or complex events events ' 4 # 8 &=4 2& #4 $ # !4 # # + 4 CEP # # )# > 8 7 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing 4 4 * 4 Getting the right information in the right granularity BAM to the right Information Detect Quick observation into exceptional business Dissemination person at Decide behavior and notification to the appropriate the right Respond people. time Diagnose Predictive Mitigate or Active Diagnostics Processing problems eliminate based on predicted events RTE symptoms and Reactions to events are done as part of business resolve them transactions – achieving low latency decisions, and quick reaction to threats and opportunities IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing ? ) & 4 ! !4 # 4 ? # ## #! 8 # ) 4 ! Solution: $ 2 8 # 8 0# 3 Configurable aggregation rules # 4 # ) 4 4 # @ •Thresholds by currency •Thresholds by time •Thresholds by # 4#! # 4# 7 counterparty/client # 4#! # 4# 4 Transactional Context Complex Event Processing Transactional Context INPUT Filter and Persist CEP Routing, Output Transactional Aggregation Database Transactional JMS Input Queues Transform Orders to Aggregation, Output Queues For Temporal Logic Logic Update Pub/Sub Logic Database execution Aggregated Order Notification to CEP Direct passthrough of selected messages IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing % + #% A + ! # 4 4 ) 7 , #) Detect .-B Decide ) Respond + 8 #4 4 0 .- ? 1$ 32 ) #! # 4 4 ! # 4 4 4 # 4 > )7 # ) #) # 7 + 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing & 9 + # #4 9 + ,C 7 4 # #4 , 4 4 # ! 8 1- #4 ! # .C 4 9 #4 # ! ! 4 4 \" 9 + 9; #4 7 ! 8 ) 4 9 ! 8 $ 4 ## ! 84 7 , 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing . C . C ? + ! # F ., # + 9 ) ., H # 4 # .. E .- , + , # ! ( F / ., 9 + H ? ; .G .. + .1 1 ;,? ./ ? ? 1 ? ? .C + G 9 4 E .F ,- .- + .E ; 44 ? .H / IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Research Engineering Oriented Challenges &4 # # # 9 4 + ! # 2 4 9 9 EPN EPN Event EPA Event Producer EPA EPA Consumer Pattern Event Event Producer Consumer EPA EPA Event Pattern Pattern Event Producer Consumer # © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing Business User View Point # ! 4 ## Why a decision is needed? • Event Occurred Islands of • Event Pattern has detected tools • Fact has changed • An explicit request • Related to other decision Implementation Which information is needed driven tools In order to make this decision? • Current state • Past events • Past states • Future states and events. Gaps In coverage How the decision should be Done? • Inference from facts and rules • Match patterns on event history Ability to manage • Apply optimization tools Large complex Decision network Dependent Decisions IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing # )# # 4 + + ) 0 3 EP ) 0 3$ Gaps ) # 0 3 +;4 0 & 3 + I 7 44 0 3 8 7 0 & 3 BRMS # # 09 ) 3 Analytics Most Important Connection ) 4 4 + J # Business User Orientation: J The user should be able to author, J modify, understand, and audit individual Computerized decisions, as well as the J # Relationships among them IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing 9) M? 74 7 8 N &! 4 Stratum_1 Stratum_2 Stratum_S Producer_1 Consumer_1 Agent_11 Agent_21 Agent_S1 Producer_2 Consumer_2 Agent_12 Agent_22 Agent_S2 . … . . . . . . . . . . . . . . Agent_1N1 Agent_2N2 Agent_SN2 Producer_P Consumer_C IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Research RFID Store Scenario - 3 Stratum Levels Stratum1 Stratum2 Stratum3 TagRead Filter ProductAtCashier AtLeast5ProductsOf Loc= “Cashier” SameKindReturnedToShelf ProductExitStore ProductLeftAtCashier TagRead Filter Tag Sequence Atleast Loc = “Exit” Tag Reader Tag Reader TagRead Tag Filter AtLeast10ProductsOf Reader ProductInCart Loc = “Cart” ProductReturnedToShelf Reader SameKindSold Sequence Atleast Filter TagRead Loc = “Shelf” ProductOnShelf Levels Average Average Implementation Input events Derived input events derived total events total throughput events rate throughput rate (event/s) (event/s) Centralized 1,399 38 First stratum – 4 agents on 21,419 21,418 single node 3 Stratum Levels – second 21,419*4+69 4,485 stratum has the slowest 3*107+ Second stratum – 2 agents on 693 277 performance. Maximal flow 29,904 = single node performance can be achieved 189,731 by combination of 1 node in the Third stratum – 2 agents on 29,904 4,485 (upper limit) third level, 107 nodes in the single node second layer, 4 nodes in the Centralized – all the patterns 1,399 38 first level are detected by multiple 112 nodes improvement 13562% 11802% agents on same node (8 agents) Additional node improvement 121% 105.37% © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing 2 9 2\" 4 + 9 # + + \" 4 ; 4 + 4 991 4 ; # 8 \" 4 2; 4 + 991 4 # # 9 @ \" 4 2 2 ; 4 + 9 9 # 2 4 + @ IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing ! \" # $ % % & # & ' 4 +P 4 ,- E - #!) # !4 0 7 3 4# 4# # 7 $ 4$ 9 # M ) ) ! # ) # # $ # M # ) # # 9 ) # 0!4 7 ! # # 4 3$ 8 4 # # ) % ! # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing ' # 7 # # \" & ! ' ) 4 # #( 05 4 &4 9 #6$ ! * % 3 4# )! # 7 # 4# # )07 @ 3$ ## # #4 7 #! 8 =4 ) # )$ ! $ # )$ 4 ) # # # 4 0 ) ## 3+ D\" 2 # # ! # ! ) 0 ) ## 3+ 9 &Q # 4 0 $4 ) = $ 4 ) 3@ # ! # 4 ! # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing # # 4 \" 8 # # # # ! 7 ) =4 # # #+ # # 7 ) # ! ) # # 0 ! !) ' 2 <' 3 2 4 # #0 4 # #3 # # # ' ! # !) 4 # # 0 )3 ## # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

IBM Haifa Research Lab – Event Processing ' ) + ) 4 ! 0 7 7 4# # R 3 0 ) $ 3 5 6 4 ) 8 ! 4 7 )@ #7 7 # ! 0 !4 4 # # 7 # 44 3 ' # S $ ! $ $ T #4 7 # ! 4 $ #7 # # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation

Event Processing - The Next Generation; March 2009

0 comments

Notes on slide 1

Favorites, Groups & Events