Your SlideShare is downloading. ×
Event Processing - The Next Generation; March 2009
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Event Processing - The Next Generation; March 2009

1,017
views

Published on

This is a presentation describing the challenges and current research projects in IBM Haifa Research Lab around the area of event processing

This is a presentation describing the challenges and current research projects in IBM Haifa Research Lab around the area of event processing

Published in: Technology, Business

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,017
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
99
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. IBM Haifa Research Lab – Event Processing ! quot; #$ % quot; & © 2008IBM Corporation
  • 2. IBM Haifa Research Lab – Event Processing ' ( I am wearing three different hats today The industrial hat: IBM (which pays my salary and in return occupies most of my time) Once a week I am wearing the Professor hat and teach an event processing course The EPTS chair hat – a catalyst in establishing this community IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 3. IBM Haifa Research Lab – Event Processing )* + ,-./ 0 12 - ) 3 4 5 6 5 4 6 '# # ) ! #) # # # 4 quot; 4 # 7 7# ) 4 4 7 4 ! 8 ' 2 ! #& 4 ) !4 # # 4 # # 0 $ $ $# 3 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 4. IBM Haifa Research Lab – Event Processing 4 What is event processing ? What is it used for? A quick Introduction second generation What are the current Research Challenges ? Survey of recent projects Summary IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 5. IBM Haifa Research Lab – Event Processing IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 6. IBM Haifa Research Lab – Event Processing # ) !4 Every business person reacts to events, from time to time Well… there are all sort of reactions… IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 7. IBM Haifa Research Lab – Event Processing We answer phone calls everywhere IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 8. IBM Haifa Research Lab – Event Processing We have to make arrangements when our flight is delayed IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 9. IBM Haifa Research Lab – Event Processing 7 4 4 I closed the deal with the I closed Australians the deal with the Canadians We closed two huge deals in a single day, It is a good opportunity to send all the team to Las-Vegas IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 10. IBM Haifa Research Lab – Event Processing ' ( 9 0 ) 3 0 ) 3 4 0 ! 4$ 3 ) 4 # 0 4 ) 5 63 # ) 4 + :# # ! ; < 8 7 7 & # ! 9 # ! 9 )5!4 4 6 #!) ) 07 4 4 #!) 3$ 2 0 3$ # ) # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 11. IBM Haifa Research Lab – Event Processing 9 4 ? $ $ $ 4 $ $# Event Processing Event Event Producer Consumer Generate and Consume and may publish react to either perform operations on events raw or complex events events ' 4 # 8 &=4 2& #4 $ # !4 # # + 4 CEP # # )# > 8 7 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 12. IBM Haifa Research Lab – Event Processing 4 4 * 4 Getting the right information in the right granularity BAM to the right Information Detect Quick observation into exceptional business Dissemination person at Decide behavior and notification to the appropriate the right Respond people. time Diagnose Predictive Mitigate or Active Diagnostics Processing problems eliminate based on predicted events RTE symptoms and Reactions to events are done as part of business resolve them transactions – achieving low latency decisions, and quick reaction to threats and opportunities IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 13. IBM Haifa Research Lab – Event Processing ? ) & 4 ! !4 # 4 ? # ## #! 8 # ) 4 ! Solution: $ 2 8 # 8 0# 3 Configurable aggregation rules # 4 # ) 4 4 # @ •Thresholds by currency •Thresholds by time •Thresholds by # 4#! # 4# 7 counterparty/client # 4#! # 4# 4 Transactional Context Complex Event Processing Transactional Context INPUT Filter and Persist CEP Routing, Output Transactional Aggregation Database Transactional JMS Input Queues Transform Orders to Aggregation, Output Queues For Temporal Logic Logic Update Pub/Sub Logic Database execution Aggregated Order Notification to CEP Direct passthrough of selected messages IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 14. IBM Haifa Research Lab – Event Processing ; & 4 8 2 9 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 15. IBM Haifa Research Lab – Event Processing % + #% A + ! # 4 4 ) 7 , #) Detect .-B Decide ) Respond + 8 #4 4 0 .- ? 1$ 32 ) #! # 4 4 ! # 4 4 4 # 4 > )7 # ) #) # 7 + 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 16. IBM Haifa Research Lab – Event Processing & 9 + # #4 9 + ,C 7 4 # #4 , 4 4 # ! 8 1- #4 ! # .C 4 9 #4 # ! ! 4 4 quot; 9 + 9; #4 7 ! 8 ) 4 9 ! 8 $ 4 ## ! 84 7 , 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 17. IBM Haifa Research Lab – Event Processing 9D 8 4 #4 , # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 18. IBM Haifa Research Lab – Event Processing . #, # . + $ #$ #2 7 ) 4 , # + ) )!4 7 ) ## $ ! # 4 $ !# # # $ ) # ! #05 63$ * 4 D &=4 $ IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 19. IBM Haifa Research Lab – Event Processing + ! 4 # 7 8 # # #4 &4 ; 9 ! ! 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 20. IBM Haifa Research Lab – Event Processing . C . C ? + ! # F ., # + 9 ) ., H # 4 # .. E .- , + , # ! ( F / ., 9 + H ? ; .G .. + .1 1 ;,? ./ ? ? 1 ? ? .C + G 9 4 E .F ,- .- + .E ; 44 ? .H / IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 21. IBM Haifa Research Lab – Event Processing ' 4 ( IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 22. IBM Haifa Research Lab – Event Processing 4 : ?4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 23. IBM Haifa Research Lab – Event Processing #; 7 5 4 6 7 # # $ @ 4 0 # 3 + # # < < 4 22 4 < < 9 9 ! ### # !# # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 24. IBM Research Engineering Oriented Challenges &4 # # # 9 4 + ! # 2 4 9 9 EPN EPN Event EPA Event Producer EPA EPA Consumer Pattern Event Event Producer Consumer EPA EPA Event Pattern Pattern Event Producer Consumer # © 2008 IBM Corporation
  • 25. IBM Haifa Research Lab – Event Processing ?4 #; + # 2 % # ? # # ?4 # 4 ; ! # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 26. IBM Haifa Research Lab – Event Processing : #; 4#2 + & 2 4 # $ + 4 $ $ #!4 4# 4 quot; 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 27. IBM Haifa Research Lab – Event Processing Business User View Point # ! 4 ## Why a decision is needed? • Event Occurred Islands of • Event Pattern has detected tools • Fact has changed • An explicit request • Related to other decision Implementation Which information is needed driven tools In order to make this decision? • Current state • Past events • Past states • Future states and events. Gaps In coverage How the decision should be Done? • Inference from facts and rules • Match patterns on event history Ability to manage • Apply optimization tools Large complex Decision network Dependent Decisions IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 28. IBM Haifa Research Lab – Event Processing # )# # 4 + + ) 0 3 EP ) 0 3$ Gaps ) # 0 3 +;4 0 & 3 + I 7 44 0 3 8 7 0 & 3 BRMS # # 09 ) 3 Analytics Most Important Connection ) 4 4 + J # Business User Orientation: J The user should be able to author, J modify, understand, and audit individual Computerized decisions, as well as the J # Relationships among them IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 29. IBM Haifa Research Lab – Event Processing ?4 ; & % # 94 # : IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 30. IBM Haifa Research Lab – Event Processing 9 4 ) & K IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 31. IBM Haifa Research Lab – Event Processing % quot; & & ?4 : ! ) 'L 4 ) 9 ) 4 2 : ) # quot; & #4 4# IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 32. IBM Haifa Research Lab – Event Processing 9) M? 74 7 8 N &! 4 Stratum_1 Stratum_2 Stratum_S Producer_1 Consumer_1 Agent_11 Agent_21 Agent_S1 Producer_2 Consumer_2 Agent_12 Agent_22 Agent_S2 . … . . . . . . . . . . . . . . Agent_1N1 Agent_2N2 Agent_SN2 Producer_P Consumer_C IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 33. IBM Research RFID Store Scenario - 3 Stratum Levels Stratum1 Stratum2 Stratum3 TagRead Filter ProductAtCashier AtLeast5ProductsOf Loc= “Cashier” SameKindReturnedToShelf ProductExitStore ProductLeftAtCashier TagRead Filter Tag Sequence Atleast Loc = “Exit” Tag Reader Tag Reader TagRead Tag Filter AtLeast10ProductsOf Reader ProductInCart Loc = “Cart” ProductReturnedToShelf Reader SameKindSold Sequence Atleast Filter TagRead Loc = “Shelf” ProductOnShelf Levels Average Average Implementation Input events Derived input events derived total events total throughput events rate throughput rate (event/s) (event/s) Centralized 1,399 38 First stratum – 4 agents on 21,419 21,418 single node 3 Stratum Levels – second 21,419*4+69 4,485 stratum has the slowest 3*107+ Second stratum – 2 agents on 693 277 performance. Maximal flow 29,904 = single node performance can be achieved 189,731 by combination of 1 node in the Third stratum – 2 agents on 29,904 4,485 (upper limit) third level, 107 nodes in the single node second layer, 4 nodes in the Centralized – all the patterns 1,399 38 first level are detected by multiple 112 nodes improvement 13562% 11802% agents on same node (8 agents) Additional node improvement 121% 105.37% © 2008 IBM Corporation
  • 34. IBM Haifa Research Lab – Event Processing 2 9 2quot; 4 + 9 # + + quot; 4 ; 4 + 4 991 4 ; # 8 quot; 4 2; 4 + 991 4 # # 9 @ quot; 4 2 2 ; 4 + 9 9 # 2 4 + @ IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 35. IBM Haifa Research Lab – Event Processing & 2O $ % quot; & 4 4 # ) 7 ! # 4 )# # ! & # ) 2 !4 9! 4 4 # 4 ) # ) 7 # 8 #7 # # 4 ) IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 36. IBM Haifa Research Lab – Event Processing & 9 % quot;0 & # ,- / -3 &! 9 ) +9 ) 7 8 # ! 0# $ $ 4 4 ! $ # $ #@3 ) ) +9 4 ) $ $ # $ 4 # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 37. IBM Haifa Research Lab – Event Processing + ' 84 0K 4 7 9 3 9 4 # 4 # 4 ) + & 4 ) # 4 4 9 0 3 4 ! ! )=4 ; 4 !! # #! 4 ) # 4 # # 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 38. IBM Haifa Research Lab – Event Processing 9 I quot; # # 4 4 quot; 4 # ! 7 8 7 7 8 # # ! quot; ! #) # 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 39. IBM Haifa Research Lab – Event Processing ? ?4 # 7 ?O I 4 # : K 4 2 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 40. IBM Haifa Research Lab – Event Processing ! quot; # $ % % & # & ' 4 +P 4 ,- E - #!) # !4 0 7 3 4# 4# # 7 $ 4$ 9 # M ) ) ! # ) # # $ # M # ) # # 9 ) # 0!4 7 ! # # 4 3$ 8 4 # # ) % ! # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 41. IBM Haifa Research Lab – Event Processing ; 4 IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 42. IBM Haifa Research Lab – Event Processing ' # 7 # # quot; & ! ' ) 4 # #( 05 4 &4 9 #6$ ! * % 3 4# )! # 7 # 4# # )07 @ 3$ ## # #4 7 #! 8 =4 ) # )$ ! $ # )$ 4 ) # # # 4 0 ) ## 3+ Dquot; 2 # # ! # ! ) 0 ) ## 3+ 9 &Q # 4 0 $4 ) = $ 4 ) 3@ # ! # 4 ! # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 43. IBM Haifa Research Lab – Event Processing # # 4 quot; 8 # # # # ! 7 ) =4 # # #+ # # 7 ) # ! ) # # 0 ! !) ' 2 <' 3 2 4 # #0 4 # #3 # # # ' ! # !) 4 # # 0 )3 ## # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
  • 44. IBM Haifa Research Lab – Event Processing ' ) + ) 4 ! 0 7 7 4# # R 3 0 ) $ 3 5 6 4 ) 8 ! 4 7 )@ #7 7 # ! 0 !4 4 # # 7 # 44 3 ' # S $ ! $ $ T #4 7 # ! 4 $ #7 # # # IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation