Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optimizing Your SOA with Event Processing


Published on

Optimizing Your SOA with Event Processing, TIBCO, TUCON 2007, Tim Bass, Principal Global Architect, Director Emerging Technologies Group TIBCO Software Inc.

Published in: Technology, Business

Optimizing Your SOA with Event Processing

  1. 1. Optimizing Your SOA with Event Processing Tim Bass Principal Global Architect, Director Emerging Technologies Group TIBCO Software Inc.
  2. 2. TUCON Session Information <ul><li>Can your business sense, respond, analyze, interpret and respond to event-driven business situations in real time? </li></ul><ul><li>In this session we explore how to get the maximum value from your SOA by understanding event-based architectures, complex event processing models, and how TIBCO BusinessEvents™ fits into your SOA and event-driven architecture (EDA) strategies. </li></ul>
  3. 3. Our Agenda <ul><li>Events, Event Processing & EDA </li></ul><ul><li>EDA, SOA, CEP & TIBCO BusinessEvents™ </li></ul><ul><li>Wrap Up </li></ul><ul><li>Question & Answers </li></ul>
  4. 4. EDA Business Value Proposition <ul><li>Sense and respond to business events – both threats and opportunities to your enterprise – quickly. </li></ul>
  5. 5. What is an Event? <ul><li>An Event is a significant change in state. </li></ul>State 1 State 2 Event Can make your flight connection Cannot make your flight Connection
  6. 6. What is a Significant State Change? Source: Ranadiv é , V., The Power to Predict , 2006. <ul><li>Sense reality </li></ul><ul><li>Compare reality with expectation </li></ul><ul><li>Adapt model and response to new reality </li></ul>
  7. 7. Summary of Types of Events <ul><li>Normal event </li></ul><ul><ul><li>Arrival of your airline baggage on time </li></ul></ul><ul><ul><li>Login attempt from known fraudulent IP address </li></ul></ul><ul><li>Anticipated “abnormal” event </li></ul><ul><ul><li>Penalty likely because of delayed or lost baggage </li></ul></ul><ul><ul><li>We don’t know when baggage will be delayed, but we need to be prepared for that eventuality </li></ul></ul><ul><li>Unanticipated event </li></ul><ul><ul><li>Baggage conveyor system “frozen” due to unusually severe ice storm </li></ul></ul><ul><ul><li>Airport workers union strike at major hub </li></ul></ul>
  8. 8. Detecting Situations from Events <ul><li>Anticipated event (pattern matching/detection) </li></ul><ul><ul><li>Specify pattern of the anticipated event and the appropriate response </li></ul></ul><ul><li>Unanticipated event (anomaly detection) </li></ul><ul><ul><li>Specify patterns of normality; event is deviation from pattern </li></ul></ul><ul><ul><li>when reality doesn’t fit “normality” then alert business user. </li></ul></ul>
  9. 9. Event Processing Characteristics <ul><li>Asynchronous Timing : The timing of events are not controlled by the enterprise. </li></ul><ul><li>Noise: External event data is noisy. </li></ul><ul><li>Complex Event Processing: The significant state-change for the enterprise is detected by fusing data from multiple sources. </li></ul>
  10. 10. Managing Uncertainty <ul><li>Asynchronous Timing : </li></ul><ul><ul><li>Integrate request-response SOA with asynchronous EDA </li></ul></ul><ul><li>Noise: </li></ul><ul><ul><li>Manage uncertainty about errors; both false positives and false negatives. </li></ul></ul><ul><li>Multisensor Event Fusion: </li></ul><ul><ul><li>Extreme decoupling . </li></ul></ul>Houston Denver Edmonton London Sydney NY, NY Trader Dashboards Risk Manager Houston Corporate VP, Risk Risk Management Dashboards Scheduler Dashboards
  11. 11. Is Your Enterprise Event-Driven? <ul><li>Does your enterprise monitor its external environment? </li></ul><ul><li>Does your enterprise monitor its competitors? Government agencies? </li></ul><ul><li>Do people in your enterprise correlate information from multiple sources? e.g., correlate power outage at a supplier’s factory with delivery deadlines for critical customers. </li></ul>
  12. 12. Is Your Enterprise Event-Driven? <ul><li>Are you expected to respond to events? </li></ul><ul><li>An unexpected storm has just occurred in a region that will severely effect air traffic customers. </li></ul><ul><li>Which scenario represents your business? </li></ul><ul><li>The CEO does not expect the VP of Operations to report anything unless the CEO asks. </li></ul><ul><li>The CEO expects VP of Operations to inform the CEO. </li></ul>
  13. 13. Key Take-Aways on Events <ul><li>Event Processing Characteristics: </li></ul><ul><li>Sense and Respond : Respond quickly when reality deviates from expectation or plans. </li></ul><ul><li>Asynchrony : The timing of events are not controlled by the enterprise. </li></ul><ul><li>Global situational awareness: Can be achieved only by correlating multiple sources of data from outside the enterprise with enterprise data. </li></ul><ul><li>Errors : Events are “noisy.” </li></ul>
  14. 14. What is an Event Driven Architecture? <ul><li>EDA is an architectural style that manages and executes rules of the form: </li></ul><ul><li>WHEN reality deviates from expectations </li></ul><ul><li>THEN update expectations and initiate response. </li></ul>
  15. 15. EDA Characteristics Aggregate events across multiple sources; compare reality with expectations Analyze Detect events across extended environment in real-time Sense Update expectations; Invoke distributed services in real-time Respond
  17. 17. Situational Awareness via Event Processing <ul><li>Multi-level inference in a distributed event-decision architectures </li></ul><ul><ul><li>User Interface (Dashboards, BAM, Visualization, Portals) </li></ul></ul><ul><ul><ul><li>Human visualization, monitoring, interaction and situation management </li></ul></ul></ul><ul><ul><li>Level 4 – Process Refinement (Adaptive BPM) </li></ul></ul><ul><ul><ul><li>Decide on control feedback, for example resource allocation, sensor and state management, parametric and algorithm adjustment </li></ul></ul></ul><ul><ul><li>Level 3 – Impact Assessment (Predictive Analytics) </li></ul></ul><ul><ul><ul><li>Impact assessment, i.e. assess intent on the basis of situation development, recognition and prediction </li></ul></ul></ul><ul><ul><li>Level 2 – Situation Refinement (Situational Detection) </li></ul></ul><ul><ul><ul><li>Identify situations based on sets of complex events, state estimation, etc. </li></ul></ul></ul><ul><ul><li>Level 1 – Event Refinement (Event Tracking) </li></ul></ul><ul><ul><ul><li>Identify events & make initial decisions based on association and correlation </li></ul></ul></ul><ul><ul><li>Level 0 – Event Preprocessing </li></ul></ul><ul><ul><ul><li>Cleansing of event-stream to produce semantically understandable data </li></ul></ul></ul>Level of Inference Low Med High
  18. 18. Event Processing Characteristics <ul><li>Event Processing Agents, Sensors and ESB </li></ul><ul><li>“System” can “learn” expectations from positive and negative examples </li></ul><ul><li>Users can specify expectations using: </li></ul><ul><ul><li>SQL-like queries </li></ul></ul><ul><ul><li>Fuzzy matches </li></ul></ul><ul><ul><li>Statistical operators </li></ul></ul><ul><ul><li>Regular expressions and rules </li></ul></ul><ul><ul><li>CEP </li></ul></ul>
  19. 19. Our Agenda <ul><li>Events, Event Processing & EDA </li></ul><ul><li>EDA, SOA, CEP & TIBCO BusinessEvents™ </li></ul><ul><li>Wrap Up </li></ul><ul><li>Question & Answers </li></ul>
  20. 20. EDA and SOA <ul><li>SOA: Components are collaborators </li></ul><ul><ul><li>Accounting “client” calls a “sales pipeline expectation” method on a sales “service” which returns with a report </li></ul></ul><ul><li>SOA: Time of interaction determined by client </li></ul><ul><li>SOA: Service protocols and schemas are well defined. Often transactional semantics. </li></ul><ul><li>SOA: Units obtain global situational awareness by invoking multiple services. </li></ul>
  21. 21. EDA and SOA <ul><li>SOA: Request-response and transactions are often the appropriate pattern within the enterprise. </li></ul><ul><li>SOA: Encapsulating existing capabilities (e.g., mainframe) with WS as required. </li></ul><ul><li>EDA: Monitor and respond to events within and outside the enterprise by processing asynchronous events. </li></ul>
  22. 22. EDA Visualized Asynchronous, Strongly Decoupled, Not Orchestrated A A A A A A A A A A Message Message Message Message Message Message Message Message
  23. 23. “Traditional SOA” + EDA Visualized Synchronous & Asynchronous, Loosely Coupled & Strongly Decoupled, Managed, Orchestrated, Not Orchestrated, Consumer-Driven, Producer-Driven Not Orchestrated Messaging (EDA) Note: Request/Reply Implied in “ Orchestrated Messaging” Orchestrated Messaging (“Traditional” SOA) Message A A A A A A A A A A Message Message Message Message Message Message Message Message
  24. 24. Table of Characteristics SOA and EDA Asynchronous Event Triggers Synchronous Service Invocation Application Interaction Flow Control Faster Sense/Respond Service Component Reuse Primary Technical Goal Reduced Costs and Increased Visibility Reduce Costs and Time-to-Market Primary Business Goal One-to-One, One-to-Many, Many-to-Many One-to-One Process Communication Models Publish/Subscribe Orchestration Process Management Producer Consumer Process Trigger No Scheduler Scheduler Required Process Coordination Strongly Decoupled Loosely Coupled Application Interaction EDA SOA Architectural Characteristic
  25. 25. Table of Characteristics “New SOA” and EDA Asynchronous Event Triggers Synchronous & Asynchronous Service Invocation Application Interaction Flow Control Faster Sense/Respond Distributed Computing Primary Technical Goal Reduced Costs and Increased Visibility Reduced Costs, Time-to-Market, and Increased Visibility Primary Business Goal One-to-One, One-to-Many, Many-to-Many One-to-One, One-to-Many, Many-to-Many Process Communication Models Publish/Subscribe Orchestration & Pub/Sub Process Management Producer Consumer or Producer Process Trigger No Scheduler Scheduled & Unscheduled Process Coordination Strongly Decoupled Decoupled, Loosely Coupled Application Interaction EDA “ New SOA” Architectural Characteristic
  26. 26. FYI: Event Processing (EP) and CEP Visualized A A RE A A A A A A A Messages Messages Messages Messages Messages Message Messages Messages Many-to-One Asynchronous Events Processing with Rules Engine (RE)
  27. 27. SOA + EDA + CEP Visualized (with Rules Engine) A A A A A A A RE A A Message Message Message Message Message Messages Messages Message Synchronous & Asynchronous, Loosely Coupled & Decoupled, Managed, Orchestrated, Not Orchestrated, Consumer-Driven, Producer-Driven Orchestrated Messaging (“Traditional” SOA) Not Orchestrated Messaging (EDA) Messages Note: Request/Reply Implied in “ Orchestrated Messaging” Messages
  28. 28. Complex Event Processing &quot; Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 &quot; --- Gartner July 2003 Situation Detection
  29. 29. Bloor Report on Event Processing Automated Operational Decisions Automated Predictive Decisions Human Predictive Decisions Human Operational Decisions Decision Latency Event Complexity Process Complexity Pattern Matching and Inferencing Anti-Money Laundering Credit-Card Fraud Exchange Compliance Database Monitoring Algorithmic Trading Trade Desk Monitoring Customer Interaction Order Routing RFID Tariff Look-Up Rail Networks Search & Rescue Baggage Handling Liquidity Management
  30. 30. A Business Optimization Perspective What Classes of Rule-Based Problems Do Businesses Need to Solve? Rule-Based <ul><li>Pattern Recognition </li></ul><ul><li>Anomaly Detection </li></ul><ul><li>Track and Trace </li></ul><ul><li>Monitoring (BAM) </li></ul><ul><li>Dynamic Resource Allocation </li></ul><ul><li>Adaptive Resource Allocation </li></ul><ul><li>Constraint Satisfaction (CSP) </li></ul><ul><li>Dynamic CSP </li></ul><ul><li>Adaptive Marketing </li></ul><ul><li>Dynamic CRM </li></ul><ul><li>Fault Management </li></ul><ul><li>Impact Assessment </li></ul><ul><li>Fraud Detection </li></ul><ul><li>Intrusion Detection </li></ul><ul><li>Fault Detection </li></ul><ul><li>Rule-Based Access Control </li></ul><ul><li>Exception Management </li></ul><ul><li>Compliance Work Flow </li></ul><ul><li>Risk Management </li></ul><ul><li>Fault Analysis </li></ul><ul><li>Impact Assessment </li></ul>Example Event Processing Applications Detection Prediction Scheduling
  31. 31. TIBCO BusinessEvents™ Solutions Overview BusinessEvents™ Solutions Space Data: Events & Databases -Real-Time & Historical Data Models: Statistical Financial Optimization Comms: Pub/Sub Messaging Queues Topics UIs Knowledge: Facts & Rules
  32. 32. TIBCO BusinessEvents™ Overview <ul><li>High performance, low latency business rules engine. </li></ul><ul><li>Top down business process modeling. </li></ul><ul><li>Real-time event processing. </li></ul><ul><li>Cross-application and cross-process integration. </li></ul><ul><li>Analytical and predictive models . </li></ul>Modeling Tools, Statefulness, Business Rules and Process Integration UML Conceptual UML State Business Rules Business Users Event Analyzer
  33. 33. TIBCO BusinessEvents™ Overview Collection, Normalization Metric of Managed Objects, Normalized Non-Contextual Events Metadata Repository Semantic Model Events Rules Design Environment State Model Event Management, Correlation, Aggregation, Inference and Analysis Correlated, Analyzed, Contextual Dialogue Events Rules, Knowledge, Patterns, Models Visualization, Reporting, Alert Management Application Interface Feeds Visualization: Detection Metrics Agents Synthetic Warehouse Visualization: Process View Dialogue Manager Inference Engine FDS/IDS Logfiles Edge Devs Sensors
  34. 34. BusinessEvents™ Components Enterprise Metadata (Concepts, Properties, State Models, XML Schemas, Business Rules) BusinessEvents Workbench (Designtime) BusinessEvents Engine (Runtime) Business User Interface Business User Language Decision Tables Runtime Viewer Management Server
  35. 35. Runtime – BusinessEvents™ Engine Engine Inference Engine <ul><li>forward chaining – optimized Rete based rule inferencing </li></ul><ul><li>history of objects – calculation of real-time time-series </li></ul><ul><li>persistence – virtual memory, 100% failsafe </li></ul><ul><li>performance – 10s of 1000s of rules per second </li></ul>Models <ul><li>ontology - objects, events, inheritance, relationships, properties </li></ul><ul><li>state model – objects life cycle, event patterns, time, alerts, reports </li></ul><ul><li>kpi model - real-time calculation, thresholds / alerts </li></ul>Monitor and Management Channels Embedded DB
  36. 36. Standard TIBCO Design, Deploy, Run Model
  37. 37. CEP Use Case Illustrated
  38. 38. CEP Use Case Illustrated
  39. 39. Wrap Up <ul><li>Your enterprise is already event-driven </li></ul><ul><li>SOA, EDA and CEP are both critical and complimentary technologies </li></ul><ul><li>So, an event processing strategy is important to help you achieve maximum value our of your SOA design and implementation </li></ul><ul><li>TIBCO BusinessEvents™ helps you sense, analyze, interpret and respond to event-driven business situations in real time. </li></ul>
  40. 40. Q & A <ul><li>? </li></ul>
  41. 41. Thank You! Tim Bass, CISSP Principal Global Architect, Director TIBCO Software Inc.
  42. 42. Appendix: SOA (1.0) Defined** <ul><li>SOA is a application architecture in which all functions, or services, are defined using a description language and have invokable interfaces that are called to perform business processes. </li></ul><ul><li>Each interaction is independent of each and every other interaction and the interconnect protocols of the communicating devices (i.e., the infrastructure components that determine the communication system do not affect the interfaces). </li></ul><ul><li>Because interfaces are platform-independent, a client from any device using any operating system in any language can use the service. </li></ul><ul><li> </li></ul>**Please Note: Gartner Principal Analysts now say that SOA 2.0 ~ “distributed computing services w/interfaces”
  43. 43. Appendix: EDA Defined <ul><li>EDA is an enterprise software infrastructure model in which events trigger the real-time exchange of messages between independent software applications. </li></ul><ul><li>EDA relies on an event-processing agent that detects events across an enterprise and, using a push approach, notifies all of the other software applications that need to be notified of the change in data, all at the same time. </li></ul><ul><li>For example: the e-commerce Web site of an enterprise receives an order for a product, completing a business event. An event agent detects this transaction and simultaneously notifies all other applications in the enterprise that need to know about the order, which can include such aspects as an inventory database, accounts receivable software, customer service applications, marketing and advertising monitors, and shipping software. </li></ul><ul><li> </li></ul>