• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Event Driven Architecture (EDA), November 2, 2006
 

Event Driven Architecture (EDA), November 2, 2006

on

  • 6,389 views

Event Driven Architecture (EDA), SOA Seminar Crystal City, Virginia, November 2nd, 2006, Tim Bass, CISSP, Principal Global Architect, Director. Co-Chair, Event Processing Reference Architecture ...

Event Driven Architecture (EDA), SOA Seminar Crystal City, Virginia, November 2nd, 2006, Tim Bass, CISSP, Principal Global Architect, Director. Co-Chair, Event Processing Reference Architecture Working Group (EPRAWG)

Statistics

Views

Total Views
6,389
Views on SlideShare
6,369
Embed Views
20

Actions

Likes
16
Downloads
0
Comments
0

2 Embeds 20

http://www.slideshare.net 18
http://jisi.dreamblog.jp 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Event Driven Architecture (EDA), November 2, 2006 Event Driven Architecture (EDA), November 2, 2006 Presentation Transcript

  • Event Driven Architecture (EDA) SOA Seminar Crystal City, Virginia November 2nd, 2006 Tim Bass, CISSP Principal Global Architect, Director Co-Chair, Event Processing Reference Architecture Working Group (EPRAWG)
  • Our Agenda
    • Introduction
    • Event Driven Architecture
      • Overview of SOA + EDA + CEP = “SOA on Steroids!”
      • High Level Overview of Decision Making and BusinessEvents™
      • Event-Decision Reference Architecture
    • Event Driven CEP Scenarios, Use Cases and Application
  • Introduction
    • “ Advanced SOA” is Evolving Toward SOA + EDA + CEP ++
    • “ Advanced SOA” Requires a Number of Technologies:
      • Distributed Computing, Publish/Subscribe and SOA
      • Event Driven Architecture for Complex Situations and Event Processing
      • Hierarchical, Cooperative Inference and Rules Processing
      • High Speed, Real Time Processing with State Management
    • Event Processing Communities are Working On a Common Vocabulary and Functional Reference Architecture based on Mature, Industry-Standard Event Processing / Inference Models
    • TIBCO is one of the Leading EDA / SOA Focused Commercial Software Companies
  • Gartner: Extended Application Platform Suite (APS) for Advanced SOA Will Require Expertise in Many Technology Areas Integrated Security and Systems Management g Portal Product, Rich Client User Interaction Shared Middleware Infrastructure (RPC, MOM, WS) APS Core Extended APS Advance APS Core BPM Suite Business Process Management EII, MDM Data Integration Application Integration Integration Broker Event Server Complex Event Processing Application Server, TPM Transaction Processing Business Component Library Application Building Blocks Common Metadata Repository End-to-End Development Framework
  • PredictiveBusiness TM
  • PredictiveBusiness TM & Complex Event Processing (CEP) An Event Driven Architecture Enables Business Optimization
    • More CEP Scenarios:
    • Stock Trading
      • Automatic identification of buy/sell opportunities.
    • Compliance Checks
      • Sarbanes-Oxley detection.
    • Fraud Detection
      • Odd credit card purchases performed within a period.
    • CRM
      • Alert if three orders from the same platinum customer were rejected.
    • Insurance Underwriting
      • Identification of risk.
    " Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 " --- Gartner July 2003 Graphic Sources: TIBCO Software Inc & IBM CEP Situation Manager Event Streams Historical Data Real-time Detection and Prediction
  • Example Event Processing Scenarios SOA + EDA + CEP + BPM = PredictiveBusiness™
    • Finance
      • Program (Opportunistic) Trading and Execution
      • Risk Management and Compliance
      • Pricing and Consumer Relationship Management
      • Fraud and Intrusion Detection
    • Business Process Management
      • Process Monitoring
      • Exception Management and Outage Prediction
      • Dynamic and Adaptive Scheduling
    • Sensor Networks
      • Reliability of Complex, Distributed Systems
      • RFID Applications
      • Manufacturing Floor – “Sense and Respond”
      • Power Grid Monitoring
      • Military Scenarios
  • Our Agenda
    • Introduction
    • Event Driven Architecture
      • Overview of SOA + EDA + CEP = “SOA on Steroids!”
      • High Level Overview of Decision Making and BusinessEvents™
      • Event-Decision Reference Architecture
    • Event Driven CEP Scenarios, Use Cases and Application
  • EDA Defined
    • EDA is an enterprise software infrastructure model in which events trigger the real-time exchange of messages between independent software applications.
    • EDA relies on an event-processing agent that detects events across an enterprise and, using a push approach, notifies all of the other software applications that need to be notified of the change in data, all at the same time.
    • Example: The e-commerce Web site of an enterprise receives an order for a product, completing a business event. An event agent detects this transaction and simultaneously notifies all other applications in the enterprise that need to know about the order, which can include such aspects as an inventory database, accounts receivable software, customer service applications, marketing and advertising monitors, and shipping software.
    • Ref: http://www.webopedia.com/TERM/E/EDA.html
  • EDA Visualized Asynchronous, Strongly Decoupled, Not Orchestrated Processes Event 9 P 11 P 33 P 12 P 31 P 32 P 13 P 34 P 21 P 14 P 22 Event 1 Event 5 Event 2 Event 6 Event 3 Event 4 Event 8 Event 7
  • “Traditional SOA” + EDA Visualized Orchestrated Process 1 in an SOA Orchestrated Process 2 in an SOA Processing Not Orchestrated in an EDA Note: Request/Reply Implied in “ Orchestrated Processes” Synchronous & Asynchronous, Loosely Coupled & Strongly Decoupled, Managed, Orchestrated, Not Orchestrated, Consumer-Driven, Producer-Driven Event 9 P 11 P 33 P 12 P 31 P 32 P 13 P 34 P 21 P 14 P 22 Event 1 Event 5 Event 2 Event 6 Event 3 Event 4 Event 8 Event 7
  • “ Traditional SOA” – EDA: Table of Characteristics Draft Summary Comparison – Under Construction Asynchronous Event Triggers Synchronous Service Invocation Application Interaction Flow Control Faster Sense/Respond Service Component Reuse Primary Technical Goal Reduced Costs and Increased Visibility Reduce Costs and Time-to-Market Primary Business Goal One-to-One, One-to-Many, Many-to-Many One-to-One Process Communication Models Publish/Subscribe Orchestration Process Management Producer Consumer Process Trigger No Scheduler Scheduler Required Process Coordination Strongly Decoupled Loosely Coupled Application Interaction EDA SOA Architectural Characteristic
  • FYI: Event Processing (EP) and CEP Visualized P 1 P 8 RE P 4 P 5 P 2 P 7 P 62 P 3 P 61 Events Events Events Events Events Events Events Events Many-to-One Asynchronous Events Processing with Rules Engine (RE)
  • SOA + EDA + CEP Visualized (with Rules Engine) Orchestrated Process 1 in an SOA Orchestrated Process 2 in an SOA Processing Not Orchestrated in an EDA Note: Request/Reply Implied in “ Orchestrated Processes” Event 10 Event 9 Synchronous & Asynchronous, Loosely Coupled & Decoupled, Managed, Orchestrated, Not Orchestrated, Consumer-Driven, Producer-Driven P 11 P 33 P 12 P 31 P 32 P 13 P 34 RE P 14 P 22 Event 1 Event 5 Event 2 Event 6 Event 3 Event 4 Event 8 Event 7
  • Key SOA + EDA Takeways
    • SOA, “Advanced SOA,” EDA and CEP are all Event Driven Architectures
    • Gartner’s Principal Analysts Now Define SOA as follows:
      • “Advanced SOA” = distributed computing services (based on interfaces)
      • In this “new definition” SOA and EDA are combined (EDA is a part of SOA).
    • SOA / EDA Overview
      • EDA was generally message interaction between strongly decoupled or very loosely coupled applications.
      • SOA was “traditionally” managed or “orchestrated” interaction between loosely coupled applications, modeled as services
      • “Advanced SOA” is simply, “distributed computing services with interface definitions”
    • Both SOA and EDA are Required for Business Optimization
    • SOA and EDA are Complimentary Architectures
  • Our Agenda
    • Introduction
    • Event Driven Architecture
      • Overview of SOA + EDA + CEP = “SOA on Steroids!”
      • High Level Overview of Decision Making and BusinessEvents™
      • Event-Decision Reference Architecture
    • Event Driven CEP Scenarios, Use Cases and Application
  • Overview of IT and Decision Making What is a High Level View of How Businesses Make Decisions?
    • Facts
    • Rules
    • Procedures
    • Historical Data/
    • Historical Events
    • Real-Time Data
    • Real-Time Events
    • Statistical
    • Financial
    • Optimization
    • Simulation
    Document- Driven
    • Unstructured Docs
    • Distributed Computing
    • Publish-Subscribe
    • Collaboration
    Knowledge- Driven Decision Making Communications- Driven Model- Driven Data- Driven
  • TIBCO BusinessEvents™ Solutions Overview TIBCO BusinessEvents™ Solutions Space Data: Events & Databases -Real-Time & Historical Data Models: Statistical Financial Optimization Comms: Pub/Sub Messaging Queues Topics UIs Knowledge: Facts & Rules
  • A Business Optimization Perspective What Classes of Rule-Based Problems Do Businesses Need to Solve? Rule-Based
    • Pattern Recognition
    • Anomaly Detection
    • Track and Trace
    • Monitoring (BAM)
    • Dynamic Resource Allocation
    • Adaptive Resource Allocation
    • Constraint Satisfaction (CSP)
    • Dynamic CSP
    • Adaptive Marketing
    • Dynamic CRM
    • Fault Management
    • Impact Assessment
    Detection Prediction Scheduling
    • Fraud Detection
    • Intrusion Detection
    • Fault Detection
    • Rule-Based Access Control
    • Exception Management
    • Compliance Work Flow
    • Risk Management
    • Fault Analysis
    • Impact Assessment
    Solving a Broad Class of Complex Problems
  • Event-Decision Hierarchy 22 Impact Assessment Situational Assessment Relationship of Events Identify Events Location, Times and Rates of Events of Interest Existence of Possible Event of Interest Data/Event Cloud Analysis of Situation & Plans Contextual and Causal Analysis Causal Analysis, Bayesian Belief Networks, NNs, Correlation, State Estimation, Classification Use of Distributed Sensors for Estimations Raw Sensor Data (Passive and Active) Adapted from: Waltz, E. & Llinas, J., Multisensor Data Fusion, 1990 HIGH LOW MED
  • Event-Decision High Level Architecture 22 EVENT CLOUD (DISTRIBUTED DATA SET) KS KS KS KS KS KS KS KS KS KS KS KS KS KS Adapted from: Engelmore, R. S., Morgan, A.J., & and Nii, H. P., Blackboard Systems, 1988 & Luckham, D., The Power of Events, 2002
    • Sensors
      • Systems that provide data and events to the inference models and humans
    • Actuators
      • Systems that take action based on inference models and human interactions
    • Knowledge Processors
      • Systems that take in data and events, process the data and events, and output refined, correlated, or inferred data or events
    HLA - Knowledge Sources KS KS KS
  • Event-Decision Reference Architecture SOA + EDA + CEP + BPM + The User Experience 24 Adapted from JDL: Steinberg, A., & Bowman, C., Handbook of Multisensor Data Fusion, CRC Press, 2001 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction Event-Decision Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
  • Structured Processing for Event-Decision
    • Multi-level inference in a distributed event-decision architectures
      • Level 5 – User Interface
        • Human visualization, interaction and situation management
      • Level 4 – Process Refinement
        • Decide on control feedback, for example resource allocation, sensor and state management, parametric and algorithm adjustment
      • Level 3 – Impact Assessment
        • Impact threat assessment, i.e. assess intent on the basis of situation development, recognition and prediction
      • Level 2 – Situation Refinement
        • Identify situations based on sets of complex events, state estimation, etc.
      • Level 1 – Event Refinement
        • Identify events & make initial decisions based on association and correlation
      • Level 0 – Event Preprocessing
        • Cleansing of event-stream to produce semantically understandable data
    Level of Inference Low Med High
  • EP Level 0 – Event Preprocessing
    • Cleanse/Refine/Normalize Data for Upstream Processing
    • Calibrate Raw Event Cloud:
      • Web Server Farm Event Stream Example -
        • Group HTTP REQUESTS and RESPONSES
        • Reduce and Extract Required Data from Transaction
        • Format into Event for Upstream Processing
      • Intelligent Agent Fraud Detection Event Steam Example -
        • Receive Event Stream from Purpose-Built FD Application
        • Reduce and Extract Required Event from Event Stream
        • Format for Upstream Processing
    • Reduces System Load by Preprocessing Events
    • Enables Upstream to Concentrate on Most Relevant Events
    • Focuses on Objects/Events
  • EP Level 1 – Event Refinement
    • Problem: Which Events in the Event Stream Are “Interesting”?
    • Event Refinement Example (Association & Classification):
      • Hypothesis Generation (HG)
        • Processing incoming events, data and reports
        • Hypothesis: This Group of Events May Represent Fraud
        • Output: Fraud Detection Scorecard or Matrix
      • Hypothesis Evaluation (HE)
        • Evaluates Scorecard/Matrix for likelihood comparison
        • Rank Evaluation: These Events have a Higher Likelihood of Fraud
        • Output: Fills Scorecard/Matrix with relative likelihood estimation
      • Hypothesis Selection (HS)
        • Evaluates Scorecard/Matrix for best fit into “badges of fraud”
        • Evaluation: Provide an Estimate (Name) of the Fraudulent Activity
        • Output: Assignment of fraudulent activity estimate to event
  • EP Level 2 – Situation Refinement
    • What is the Context of the Identified Events?
    • Focuses on Relationships and States Among Events
    • Situation Refinement
      • Event-Event Relationship Networks
      • Temporal and State Relationships
      • Geographic or Topological Proximity
      • Environmental Context
        • Example: Brand currently used by phishing site in Internet increasing probability of fraud and identity theft
    • Event / Activity Correlation – Relational Networks
    • Pattern, Profile and Signature Recognition Processing
    • Question: Do “Complex Events” == “Situations”?
  • EP Level 3 – Impact Assessment
    • Predict Intention of Subject (Fraudster example)
      • Make changes to account identity information?
      • Transfer funds out of account?
      • Test for access and return at later time?
    • Estimate Capabilities of Fraudster
      • Organized Gang or Individual Fraudster?
      • Expert or Novice?
    • Estimate Potential Losses if Successful
    • Identify Other Threat Opportunities
  • EP Level 4 – Process Refinement
    • Evaluate Process Performance and Effectiveness
      • Exception Detection, Response Efficiency and Mitigation
      • Knowledge Development
    • Identify Changes to System Parameters
      • Adjust Event Stream Processing Variables
      • Fine Tune Filters, Algorithms and Correlators
    • Determine If Other Source Specific Resources are Required
    • Recommend Allocation and Direction of Resources
  • EP - Database Management Examples
    • Reference Database
      • User Profiles
      • Activity and Event Signatures and Profiles
      • Environmental Profiles
    • Inference Database
      • Subject Identification
      • Situation and Threat Assessment
      • Knowledge Mining
    • Referential Mapping Database Examples
      • Mapping Between IP Address and Domain
      • Mapping Between Known Anonymous Proxies
  • EP Level 5 – User Interface / Interaction
    • Operational Visualization at all “Levels”
      • Dynamic Graphical Representations of Situations
      • Supports the Decision Making Process of Analytics Personnel
    • Process and Resource Control
      • Supports Resource Allocation and Process Refinement
    • Display Control & Personalization
      • Different Operator Views Based on Job Function and Situation
  • TIBCO’S Event-Decision Reference Architecture Combining SOA + EDA + CEP + BPM + BAM ++ Flexible SOA and Event-Driven Architecture
  • Our Agenda
    • Introduction
    • Event Driven Architecture
      • Overview of SOA + EDA + CEP = “SOA on Steroids!”
      • High Level Overview of Decision Making and BusinessEvents™
      • Event-Decision Reference Architecture
    • Event Driven CEP Scenarios, Use Cases and Application
  • Event Processing / CEP Application Scenarios A Few Examples of Detection-Prediction Scenarios We Solve for Customers Predictive Consumer Information Management Financial Services Adaptive ESB (Declarative v Procedural) Financial Services Intrusion and Fraud Detection Financial Services Supply Chain Monitoring Logistics (including RFID) Supply Chain Monitoring Manufacturing Anti Money Laundering and More. Financial Services and Government Power Grid Monitoring Energy Track & Trace / Scheduling Transportation Service Monitoring Telecommunications Track & Trace Supply Chain - Logistics Network & Applications Management Telecommunications CEP Application Scenarios Example Industry Area
  • Use Case One: Identity Theft Detection / Phishing Example Fraud Detection Scenario Uses Proxy Alert Service Account Lockout Profile Mismatch Brand Phishing Alert Security Alert Customer Known Fraud IP Identity Theft Login Success Phishing Alert Brand Misuse Source: Bass, T., TIBCO Software Inc., January 2006
  • Use Case Two: Fusion-Based IDS High Level Event-Driven Architecture (EDA) – Early Phase JAVA MESSAGING SERVICE (JMS) DISTRIBUTED QUEUES (TIBCO EMS) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE ) SENSOR NETWORK RULES NETWORK NIDS BW JMS LOGFILE JMS BW LOGFILE JMS BW LOGFILE JMS BW IDS JMS BW HIDS JMS BW SQL DB BW JMS ADB SQL DB BW JMS ADB MESSAGING NETWORK TIBCO PRODUCTS SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM
  • Overview of a Solutions Architecture for Enterprise IDS
    • Fusion of IDS information across Customer’s Enterprise, including:
      • Log files
      • Existing Customer’s IDS (host and network based) devices
      • Network traffic monitors (as required)
      • Host statistics (as required)
    • Secure, standards-based JAVA Messaging Service (JMS) for messaging:
      • Events parsed into JMS Application Properties (and Extended JMS Headers)
      • SSL transport for JMS messages
    • TIBCO technology for next-generation detection, prediction, rule-based intrusion response, and adaptive control
      • TIBCO Business Works™ as required, to transform, map or cleanse data
      • TIBCO BusinessEvents™ for rule-based IDS analytics
      • TIBCO Active Database Adapter as required
      • TIBCO Hawk™ for distributed application (and network) monitoring
  • Potential Extensions to Solutions Architecture
    • Extension of IDS to rules-based access control
      • Integration of IDS with access control
      • TIBCO BusinessEvents™ for rule-based access control
    • Extension of IDS and access control to incident response
      • Event-triggered work flow
      • TIBCO iProcess™ BPM for incident response
      • TIBCO iProcess™ BPM security entitlement work flow
      • TIBCO BusinessEvents™ for rule-based access control
    • Extensions for other risk and compliance requirements
      • Basel II, SOX, and JSOX - for example
      • Other possibilities to be discussed later
    • Extensions for IT management requirements
      • TIBCO Hawk™ for distributed application (and network) command and control (C2)
  • Event-Driven Operational Risk Management An Enterprise View of Risk and Asset Management with Events Control evaluation (SOX) Operational Risk (Basel II) Security Outsourcing Privacy Business Continuity Planning Event-Driven Operational Risk Assessment & Management
  • “Enterprise Architecture View” SOA + EDA + CEP + BPM + BAM (User Interaction) EVENTS EVENTS
  • Key Takeaways
    • “ Advanced SOA” is Evolving Toward SOA + EDA + CEP ++
    • “ Advanced SOA” Requires a Number of Technologies:
      • Distributed Computing, Publish/Subscribe and SOA
      • Event Driven Architecture for Complex Situations and Event Processing
      • Hierarchical, Cooperative Inference and Rules Processing
      • High Speed, Real Time Processing with State Management
    • Event Processing Communities are Working On a Common Vocabulary and Functional Reference Architecture based on Mature, Industry-Standard Event Processing / Inference Models
    • TIBCO is one of the Leading EDA / SOA Focused Commercial Software Companies
  • Thank You! Tim Bass, CISSP Principal Global Architect, Director [email_address] Complex Event Processing at TIBCO