Combating Fraud and Intrusion Threats with Event Processing

Combating Fraud and Intrusion Threats with Event Processing Tim Bass, CISSP Principal Global Architect, Director Emerging Technologies Group TIBCO Software Inc.

TUCON Session Information

Fortunately, one of the most promising applications of complex event processing (CEP) is in the area of security event management (SEM), an evolving science that can provide businesses with advanced threat detection and warning capabilities.

Learn more about how to apply advanced SEM concepts to the security of your business.

Threats to your online business are everywhere!

Our Agenda

Trends in Cyber attacks, Threats & Vulnerabilities

Security Event Management (SEM) Overview

How Complex Event Processing (CEP) Helps

TIBCO BusinessEvents™ and CEP

Question & Answers

Threats Are Everywhere! Source: www.cert.org Intruders High Low 1980 1985 1990 1995 2000+ Intruder Knowledge Attack Sophistication cross site scripting password guessing self-replicating code password cracking exploiting known vulnerabilities disabling audits back doors hijacking sessions sweepers sniffers packet spoofing GUI automated probes/scans denial of service www attacks Tools “ stealth” / advanced scanning techniques burglaries network mgmt. diagnostics distributed attack tools staged auto coordinated / bots

Malicious Code Trends

Malicious Code – The Numbers

IE Critical Vulnerabilities

FireFox Critical Vulnerabilities

Global Distribution of On-Line Banking

Global Distribution of Phishers

Vulnerabilities Exponentially Increasing?

Our Agenda

Question & Answers

SEM Functionality

Log collection from heterogeneous devices - the capability to read, parse, normalize, and gather security events from a variety of heterogeneous event sources

Situation detection - the capacity to detect and refine threat-related situations automatically and priorities based on an automatic impact assessment, optimizing staff performance to focus on preventing the most important threats

Threat prevention and remediation - generate alerts and automated responses based upon high probability threat scenarios and manage the life cycle of the threat

Report generation – automate reports that support post-threat investigation, regulatory compliance and update visualizations and dashboards

Scalable, distributed architecture – the architecture must manage millions of logs per day, distribute the processing load, and with service-oriented services for transformation, event tracking, correlation, updates, remediation and visualizations

Overview of IDS & FDS Systems Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Security Event “Stovepipes” Centralized Distributed Fraud and Intrusion Detection Systems, Logs Agent Based

No Shortage of “Event Aggregators” !

What is Missing from this SEM Architecture?

SEM Illustrated

SEM: Key Take-Aways

NO ESB – there is no secure, standards-based communications infrastructure for distributed event management in current SEM solutions

WEAK or NO ANALYTICS - there is limited capability to detect and refine threat-related situations with high probability using state-of-the-art analytics

WEAK or NO EDA - not standard generated alerts and automated responses to kick off workflow, compliance and other remediation activities

WEAK REPORTING – dashboards and reports tend to be" event aggregators” that do not filter out the “noise”

UNSCALEABLE, CENTERALIZED ARCHITECTURES – current SEM architectures cannot manage millions events in a heterogeneous, distributed architecture

Our Agenda

Question & Answers

How Does CEP Helps with SEM?

What is an Event?

An Event is a significant change in state.

State 1 State 2 Your on-line banking application is normal A threat to your on-line system was detected Event

What is an Event Driven Architecture?

EDA is an architectural style that manages and executes rules of the form:

WHEN reality deviates from expectations

THEN update expectations and initiate response.

EDA Characteristics Aggregate events across multiple sources; compare reality with expectations Analyze Detect events across extended environment in real-time Sense Update expectations; Invoke distributed services in real-time Respond

Detecting Situations from Events

Anticipated event (pattern matching/detection)

Specify pattern of the anticipated event and the appropriate response

Unanticipated event (anomaly detection)

Specify patterns of normality; event is deviation from pattern

when reality doesn’t fit “normality” then alert business user.

Event Processing Characteristics

Asynchronous Timing : The timing of events are not controlled by the enterprise.

Noise: External event data is noisy.

Complex Event Processing: The significant state-change for the enterprise is detected by fusing data from multiple sources.

Managing Uncertainty

Asynchronous Timing :

Integrate request-response SOA with asynchronous EDA

Noise:

Manage uncertainty about errors; both false positives and false negatives.

Multisensor Event Fusion:

Extreme decoupling .

Houston Denver Edmonton London Sydney NY, NY Trader Dashboards Risk Manager Houston Corporate VP, Risk Risk Management Dashboards Scheduler Dashboards

Key Take-Aways on Events

Event Processing Characteristics:

Sense and Respond : Respond quickly when reality deviates from expectation or plans.

Asynchrony : The timing of events are not controlled by the enterprise.

Global situational awareness: Can be achieved only by correlating multiple sources of data from outside the enterprise with enterprise data.

Errors : Events are “noisy.”

CEP Illustrated Detecting Threats with Complex Event Processing

Complex Event Processing " Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 " --- Gartner July 2003 Situation Detection

Event Processing Reference Architecture 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction Event-Processing Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM

Situational Awareness via Event Processing

Multi-level inference in a distributed event-decision architectures

User Interface (Dashboards, BAM, Visualization, Portals)

Human visualization, monitoring, interaction and situation management

Level 4 – Process Refinement (Adaptive BPM)

Decide on control feedback, for example resource allocation, sensor and state management, parametric and algorithm adjustment

Level 3 – Impact Assessment (Predictive Analytics)

Impact assessment, i.e. assess intent on the basis of situation development, recognition and prediction

Level 2 – Situation Refinement (Situational Detection)

Identify situations based on sets of complex events, state estimation, etc.

Level 1 – Event Refinement (Event Tracking)

Identify events & make initial decisions based on association and correlation

Level 0 – Event Preprocessing

Cleansing of event-stream to produce semantically understandable data

Level of Inference Low Med High

Event Processing Characteristics

Event Processing Agents, Sensors and ESB

“System” can “learn” expectations from positive and negative examples

Users can specify expectations using:

SQL-like queries

Fuzzy matches

Statistical operators

Regular expressions and rules

CEP

Our Agenda

Question & Answers

TIBCO BusinessEvents™ Solutions Overview BusinessEvents™ Solutions Space Data: Events & Databases -Real-Time & Historical Data Models: Statistical Financial Optimization Comms: Pub/Sub Messaging Queues Topics UIs Knowledge: Facts & Rules

TIBCO BusinessEvents™ Overview

High performance, low latency business rules engine.

Top down business process modeling.

Real-time event processing.

Cross-application and cross-process integration.

Analytical and predictive models .

Modeling Tools, Statefulness, Business Rules and Process Integration UML Conceptual UML State Business Rules Business Users Event Analyzer

TIBCO BusinessEvents™ Overview Collection, Normalization Metric of Managed Objects, Normalized Non-Contextual Events Metadata Repository Semantic Model Events Rules Design Environment State Model Event Management, Correlation, Aggregation, Inference and Analysis Correlated, Analyzed, Contextual Dialogue Events Rules, Knowledge, Patterns, Models Visualization, Reporting, Alert Management Application Interface Feeds Visualization: Detection Metrics Agents Synthetic Warehouse Visualization: Process View Dialogue Manager Inference Engine FDS/IDS Logfiles Edge Devs Sensors

BusinessEvents™ Components Enterprise Metadata (Concepts, Properties, State Models, XML Schemas, Business Rules) BusinessEvents Workbench (Designtime) BusinessEvents Engine (Runtime) Business User Interface Business User Language Decision Tables Runtime Viewer Management Server

Runtime – BusinessEvents™ Engine Engine Inference Engine

forward chaining – optimized Rete based rule inferencing

history of objects – calculation of real-time time-series

persistence – virtual memory, 100% failsafe

performance – 10s of 1000s of rules per second

Models

ontology - objects, events, inheritance, relationships, properties

state model – objects life cycle, event patterns, time, alerts, reports

kpi model - real-time calculation, thresholds / alerts

Monitor and Management Channels Embedded DB

On-Line Fraud Detection Use Case Approx. 12,000 Hits Per Second During Peak Period Across the Three Sites – One Instance Of TIBCO BusinessEvents™ Capable of Handling Maximum Hits Overall 100 Million Hits Handled Between 3PM – 4 PM Peak Approx. 250 Million Hits Per Day Across the Three Sites TIBCO EMS™ TIBCO Business Events™ Session Info Three Server Farms ~600-700 Application Servers

Wrap Up: TIBCO’s CEP-Based SEM

ESB – a secure, standards-based communications infrastructure for distributed event management for SEM

STRONG ANALYTICS - extensible event-driven rules-engine detect and refine threat-related situations with high probability using state-of-the-art analytics

EDA - standards compliant messages, ESB, and alerts and automated responses to kick off workflow, compliance and other remediation activities with BPM suite