Adding Rules to Improve Flexibility and Effectively Manage Complex Events Tim Bass, CISSP Principal Global Architect TIBCO...
Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Comple...
How Would We Solve These Problems? <ul><li>Fraud Detection </li></ul><ul><ul><li>Millions of users logging in to access th...
A Deeper Look into Events … What Do We Really Have Here? <ul><li>Positive Events </li></ul><ul><ul><li>User  X  logs in to...
A Deeper Look into Events … What Do We Really Have Here? - Continued <ul><li>Time Sensitivity </li></ul><ul><ul><li>A user...
Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Comple...
Need for Complex Events Processing (CEP) <ul><li>“ The events we have access to are not always tailored to the problems we...
What Is CEP? <ul><li>“ recognizing relevant patterns of events…” </li></ul><ul><ul><li>User  X  unsuccessfully logs in 3 c...
What Does CEP Give You? <ul><li>Ability to observe & recognize event patterns </li></ul><ul><li>Ability to aggregate event...
More Specifically … React and Predict Business Situations in Real-time Alert your resources to be ready at the next stop T...
Why CEP? Can I solve these business problems with a database? Database Triggers CEP When Situation Then Reaction <ul><li>L...
Details of a CEP Engine <ul><li>Has Access to the Event Cloud </li></ul><ul><ul><li>JMS, RV, SmartSockets, TCP/IP, etc… </...
Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Comple...
What is TIBCO BusinessEvents? <ul><li>At its core, BE is a Rules Inference Engine: </li></ul><ul><ul><li>Receives events <...
What is TIBCO BusinessEvents? <ul><li>Introduces a  high performance, low latency rules and policy engine  for enterprise ...
Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Comple...
Case Study: Fraud Detection <ul><li>Identify user logins from various Internet locations within a given time window. </li>...
Fraud Detection: Technical Requirements <ul><li>Acquire and correlate real time data </li></ul><ul><li>Acquire and correla...
How BusinessEvents Helped <ul><li>Detection Use Cases =>  Temporal Rules  operating on Log On  Events  correlated to User ...
How BusinessEvents Helped Multiple Engines deployed and Embedded Object Database used for memory management Millions of Us...
Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Comple...
A TIBCO Customer Case Study:  Dynamic Resource Management <ul><li>Each train demands the 5 critical resources over time </...
Dynamic Resource Management: Functional Requirements <ul><li>Provide a Train Lineup with dynamic updating of train schedul...
Dynamic Resource Management:  Technical Requirements <ul><li>Model and Manage Hundreds of Data Values and Relationships Be...
How Did BusinessEvents Help? A State Machine to Model and Monitor the Train <ul><li>State Machine to model and monitor a t...
How Did BusinessEvents Help? A Concept Diagram to Model Relationships
How Did BusinessEvents Help? Custom Visualization with TIBCO General Interface AJAX BASED IDE FOR EVENT & SITUATION VISUAL...
How Did BusinessEvents Help? <ul><li>TIBCO BusinessEvents for Robust Rules Processing </li></ul><ul><ul><li>Managed spatia...
Wrap Up How Did BusinessEvents Help? Provides the Capability for  Our Customers to Solve and Manage Their Most Challenging...
Thank You! Tim Bass, CISSP Principal Global Architect [email_address] Complex Event Processing at TIBCO With BusinessEvents™
Upcoming SlideShare
Loading in...5
×

Adding Rules to Improve Flexibility and Effectively Manage Complex Events

1,925

Published on

Adding Rules to Improve Flexibility and Effectively Manage Complex Events. Tim Bass, CISSP, Principal Global Architect
TIBCO Software Inc.TUCON 2006

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,925
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Adding Rules to Improve Flexibility and Effectively Manage Complex Events

  1. 1. Adding Rules to Improve Flexibility and Effectively Manage Complex Events Tim Bass, CISSP Principal Global Architect TIBCO Software Inc. Acknowledgement: Emerging Technology Group at TIBCO Software
  2. 2. Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Complex Event Processing </li></ul><ul><li>Summarize TIBCO BusinessEvents™ </li></ul><ul><li>Illustrate Rules in a Simple Fraud Detection Use Case </li></ul><ul><li>Discuss a More Complex Dynamic Resource Allocation Problem – a Train Scheduling Use Case </li></ul><ul><li>Wrap Up </li></ul>
  3. 3. How Would We Solve These Problems? <ul><li>Fraud Detection </li></ul><ul><ul><li>Millions of users logging in to access their financial information </li></ul></ul><ul><ul><li>IP addresses change between logins, maybe even during login </li></ul></ul><ul><ul><li>How do you identify known patterns of “suspicious” behavior? </li></ul></ul><ul><li>Train Scheduling </li></ul><ul><ul><li>Trains require critical resources (crew, terminals, track networks) at critical times </li></ul></ul><ul><ul><li>Train schedules are only ETAs, which leads to misallocation of the above– How do we do this better? </li></ul></ul><ul><li>School Attendance </li></ul><ul><ul><li>Thousands of students, hundreds of schools and districts </li></ul></ul><ul><ul><li>How do you “call home” in case of emergency? </li></ul></ul><ul><ul><li>How do you do this elegantly and in a scaleable fashion? </li></ul></ul>
  4. 4. A Deeper Look into Events … What Do We Really Have Here? <ul><li>Positive Events </li></ul><ul><ul><li>User X logs in to a financial website from a given IP Address </li></ul></ul><ul><ul><li>Train Y arrives at a terminal </li></ul></ul><ul><ul><li>Student Z calls in sick </li></ul></ul><ul><li>Negative Events </li></ul><ul><ul><li>User X has been inactive for 5 minutes </li></ul></ul><ul><ul><li>Train Y hasn’t arrived at the terminal by the ETA </li></ul></ul><ul><ul><li>Student Z hasn’t shown up yet </li></ul></ul><ul><li>Sets of Events </li></ul><ul><ul><li>User X unsuccessfully logs in 3 consecutive times </li></ul></ul><ul><ul><li>Train Y was 5 mins late at one stop, but made it early to the next </li></ul></ul><ul><ul><li>Some students haven’t shown up, and their bus has reported a breakdown </li></ul></ul>
  5. 5. A Deeper Look into Events … What Do We Really Have Here? - Continued <ul><li>Time Sensitivity </li></ul><ul><ul><li>A user doesn’t usually log in from different Internet addresses with seconds between attempts – fraudsters do </li></ul></ul><ul><ul><li>A train should take 40 minutes to travel a given track segment </li></ul></ul><ul><li>Distributed Event Sources </li></ul><ul><ul><li>A student didn’t sign-in at school, and her bus reported a breakdown a mile away </li></ul></ul><ul><ul><li>Successful login attempts coming from different continents </li></ul></ul><ul><li>How do we interact with and process this Event Cloud? </li></ul>
  6. 6. Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Complex Event Processing </li></ul><ul><li>Summarize TIBCO BusinessEvents™ </li></ul><ul><li>Illustrate Rules in a Simple Fraud Detection Use Case </li></ul><ul><li>Discuss a More Complex Dynamic Resource Allocation Problem – a Train Scheduling Use Case </li></ul><ul><li>Wrap Up </li></ul>
  7. 7. Need for Complex Events Processing (CEP) <ul><li>“ The events we have access to are not always tailored to the problems we are trying to solve. Therefore, we need a technology that enables us to progress in stages. </li></ul><ul><ul><li>The first stage is recognizing relevant patterns of events in the sources of events we do have access to and can monitor. </li></ul></ul><ul><ul><li>The second stage is aggregating information in those events to build up information that is needed to solve our problems.” </li></ul></ul><ul><ul><li>Dr. David Luckham - Stanford University Author, The Power of Events </li></ul></ul>
  8. 8. What Is CEP? <ul><li>“ recognizing relevant patterns of events…” </li></ul><ul><ul><li>User X unsuccessfully logs in 3 consecutive times </li></ul></ul><ul><ul><li>Train Y was 5 minutes late at one stop, but made it early to the next </li></ul></ul><ul><ul><li>Some students haven’t shown up, and their bus has reported a breakdown </li></ul></ul><ul><li>“ aggregating information in those events to build up useful information…” </li></ul><ul><ul><li>User X is behaving suspiciously </li></ul></ul><ul><ul><li>Train Y will likely make the next stop earlier than planned </li></ul></ul><ul><ul><li>Expect these students to be late </li></ul></ul><ul><li>CEP Vision </li></ul><ul><ul><li>Provide a technology to detect various business conditions by monitoring a flow of events and recognizing patterns as they occur </li></ul></ul><ul><ul><li>Aggregate and correlate these events into higher level “business events” that can be used to trigger business process to handle the various detected conditions </li></ul></ul>
  9. 9. What Does CEP Give You? <ul><li>Ability to observe & recognize event patterns </li></ul><ul><li>Ability to aggregate event patterns into higher level event structures </li></ul><ul><li>Ability to correlate / match events to business objects </li></ul><ul><li>Ability to take action - process and drive business object state </li></ul><ul><li>Ability to model process / state based timing expectations (e.g. timeouts / lack of event support) </li></ul>
  10. 10. More Specifically … React and Predict Business Situations in Real-time Alert your resources to be ready at the next stop Train Y will likely make the next stop earlier than planned Alert the school and emergency contacts Expect these students to be late Issued an automated challenge-response to user User X is behaving suspiciously (medium likelihood of fraud) Investigate for fraud manually User X is behaving suspiciously (high likelihood of fraud) Resulting Situation-Decision Detected Business Situation
  11. 11. Why CEP? Can I solve these business problems with a database? Database Triggers CEP When Situation Then Reaction <ul><li>Localized Event Cloud. </li></ul><ul><li>Rigid Schemas </li></ul><ul><li>Non Intuitive and Static Relationships. </li></ul><ul><li>Lack of Temporal Aspects. </li></ul><ul><li>Point in Time. </li></ul><ul><li>Distributed Configuration Management </li></ul><ul><li>Global Event Cloud. </li></ul><ul><li>Flexible, Dynamic Schemas </li></ul><ul><li>Intuitive, Static and Dynamic Relationships. </li></ul><ul><li>Temporal Reasoning. </li></ul><ul><li>Points Across Time. </li></ul><ul><li>Centralized Configuration Management </li></ul>Integration Backbone
  12. 12. Details of a CEP Engine <ul><li>Has Access to the Event Cloud </li></ul><ul><ul><li>JMS, RV, SmartSockets, TCP/IP, etc… </li></ul></ul><ul><ul><li>Timers [Lack of Events] </li></ul></ul><ul><li>Applies Business Logic and Intelligence </li></ul><ul><ul><li>Rule Based Systems </li></ul></ul><ul><ul><ul><li>When {condition} => Then {action} [Rules] </li></ul></ul></ul><ul><ul><ul><li>Optimized Condition Checking [RETE algorithm] </li></ul></ul></ul><ul><ul><ul><li>Maintains State and Facts [Working Memory] </li></ul></ul></ul><ul><ul><ul><li>Executes Rules based on addition, removal, modification of Facts [Forward/Backward Chaining] </li></ul></ul></ul>
  13. 13. Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Complex Event Processing </li></ul><ul><li>Summarize TIBCO BusinessEvents™ </li></ul><ul><li>Illustrate Rules in a Simple Fraud Detection Use Case </li></ul><ul><li>Discuss a More Complex Dynamic Resource Allocation Problem – a Train Scheduling Use Case </li></ul><ul><li>Wrap Up </li></ul>
  14. 14. What is TIBCO BusinessEvents? <ul><li>At its core, BE is a Rules Inference Engine: </li></ul><ul><ul><li>Receives events </li></ul></ul><ul><ul><li>Correlates events </li></ul></ul><ul><ul><li>Applies rules </li></ul></ul><ul><ul><li>Can generate internal events (which could trigger more rules) </li></ul></ul><ul><ul><li>Can send events out </li></ul></ul><ul><li>Also includes </li></ul><ul><ul><li>State Machine </li></ul></ul><ul><ul><li>Conceptual Model </li></ul></ul>
  15. 15. What is TIBCO BusinessEvents? <ul><li>Introduces a high performance, low latency rules and policy engine for enterprise messaging customers. </li></ul><ul><li>Introduces real-time operational process performance and decision support in market leading TIBCO Enterprise Integration Platform. </li></ul><ul><li>Introduces cross-application decision platform for business users in large and medium scale SOA and EDA initiatives. </li></ul><ul><li>Introduces semantic models to compliment the high performance XML stack. </li></ul><ul><li>Introduces a model and an inference engine to model complex, dynamic processes such as “complex order brokering”, “dynamic resource management”, “fraud detection”. </li></ul><ul><li>Introduces a high performance “model to code” approach for building event driven applications. </li></ul>Enterprise Metadata (UML, XSD, XSLT, WSDL, Business Rules and Process Flow) UML Conceptual UML State Business Rules Business Users Event Analyzer
  16. 16. Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Complex Event Processing </li></ul><ul><li>Summarize TIBCO BusinessEvents™ </li></ul><ul><li>Illustrate Rules in a Simple Fraud Detection Use Case </li></ul><ul><li>Discuss a More Complex Dynamic Resource Allocation Problem – a Train Scheduling Use Case </li></ul><ul><li>Wrap Up </li></ul>
  17. 17. Case Study: Fraud Detection <ul><li>Identify user logins from various Internet locations within a given time window. </li></ul><ul><li>Identity user logins from different IP addresses within a given time window with delayed input (within the window.) </li></ul><ul><li>Identify user login attempts from different geographic locations in less than a feasible or computable travel time between the locations. </li></ul><ul><li>Identity multiple user login attempts and/or successes from a single Internet address within a given time window. </li></ul><ul><li>Classify user behavior based on a profile and detect anomalies. </li></ul>
  18. 18. Fraud Detection: Technical Requirements <ul><li>Acquire and correlate real time data </li></ul><ul><li>Acquire and correlate static data from DB2 and Oracle </li></ul><ul><li>TIBCO must be able handle 20,000 transaction per second </li></ul><ul><li>Detect and alert fraud outlined in use cases </li></ul><ul><li>Send fraud alert multi-distribution channels such as SMTP, e-mail and wireless </li></ul><ul><li>Demonstrate that data can be aggregated from multiple sites. </li></ul><ul><li>Millions of Users </li></ul>
  19. 19. How BusinessEvents Helped <ul><li>Detection Use Cases => Temporal Rules operating on Log On Events correlated to User Concept Instances </li></ul>
  20. 20. How BusinessEvents Helped Multiple Engines deployed and Embedded Object Database used for memory management Millions of Users Channels and Events Send fraud alert via multi-distribution channels such as SMTP and wireless Modeled using Rules, Events and Concepts Detect and alert fraud outlined in use cases Plugging into the “Event Cloud” in a non-intrusive manner Demonstrate that data can be aggregated from multiple sites Native product support for basic DB Access Acquire and correlate static data from DB2 and Oracle “ Noise filtering”, messaging, optimized RETE network, and model- to-code paradigm TIBCO solution must be able handle 20,000 transaction per second Used Provided RV and Custom TCP/IP Channels Acquire and correlate real time data How TIBCO Implemented What Our Customer Required
  21. 21. Our Agenda <ul><li>Discuss a Few Classes of Complex Business Problems </li></ul><ul><li>Provide a Brief Overview of Complex Event Processing </li></ul><ul><li>Summarize TIBCO BusinessEvents™ </li></ul><ul><li>Illustrate Rules in a Simple Fraud Detection Use Case </li></ul><ul><li>Discuss a More Complex Dynamic Resource Allocation Problem – a Train Scheduling Use Case </li></ul><ul><li>Wrap Up </li></ul>
  22. 22. A TIBCO Customer Case Study: Dynamic Resource Management <ul><li>Each train demands the 5 critical resources over time </li></ul><ul><li>For a train plan to be effective, all trains must have all resources allocated </li></ul><ul><li>As the train schedule moves, the resource demand moves </li></ul>Time  Train Schedule Network Capacity Allocation Car Assignment Terminal Resource Allocation Locomotive Assignment Crew Assignment  Train Demand Dimension   Resource Supply Dimension 
  23. 23. Dynamic Resource Management: Functional Requirements <ul><li>Provide a Train Lineup with dynamic updating of train schedules </li></ul><ul><li>Associate trains on Train Lineup with crews </li></ul><ul><li>Improve train movement projections. </li></ul><ul><li>Detect opportunities to improve quality of decisions. </li></ul><ul><li>Assess the quality of the Operating Train Plan. </li></ul>
  24. 24. Dynamic Resource Management: Technical Requirements <ul><li>Model and Manage Hundreds of Data Values and Relationships Between Individual Crews, Trains, Terminals, and Network Segments, Points, and other entities </li></ul><ul><li>Model and Manage Hundreds of Rules depending on above relationships and data values </li></ul><ul><li>Enable Dynamic Allocation of hundreds of Resources separated by hundreds of miles of track </li></ul>
  25. 25. How Did BusinessEvents Help? A State Machine to Model and Monitor the Train <ul><li>State Machine to model and monitor a train’s progress over a route </li></ul><ul><li>=> Automatically Converted to Rules by Deployment </li></ul>
  26. 26. How Did BusinessEvents Help? A Concept Diagram to Model Relationships
  27. 27. How Did BusinessEvents Help? Custom Visualization with TIBCO General Interface AJAX BASED IDE FOR EVENT & SITUATION VISUALIZATION (RICH CLIENT WEB APPLICATION)
  28. 28. How Did BusinessEvents Help? <ul><li>TIBCO BusinessEvents for Robust Rules Processing </li></ul><ul><ul><li>Managed spatially distributed resources </li></ul></ul><ul><ul><li>Spatially partition allocation management </li></ul></ul><ul><ul><li>One instance of BusinessEvents per train or track </li></ul></ul><ul><li>TIBCO EMS as Communications Backbone </li></ul><ul><li>TIBCO General Interface for Rich Web Applications </li></ul>
  29. 29. Wrap Up How Did BusinessEvents Help? Provides the Capability for Our Customers to Solve and Manage Their Most Challenging Business Problems
  30. 30. Thank You! Tim Bass, CISSP Principal Global Architect [email_address] Complex Event Processing at TIBCO With BusinessEvents™

×