Next Generation Security Event Management (SEM) with Complex Event Processing (CEP)

Next Generation Security Event Management (SEM) with Complex Event Processing (CEP) Tim Bass, CISSP www.thecepblog.com [email_address] +66 (0) 832975101

Our Agenda

Trends in Cyber attacks, Threats & Vulnerabilities

Security Event Management (SEM) Overview

Complex Event Processing (CEP) for Next Generation SEM

Note: Due to the 30 minute time constraint we will not cover all these slides today. However, all slides are included in the CDIC 2007 conference binder.

Threats are More Complex High Low 1980 1985 1990 1995 2000 Intruder Knowledge Attack Sophistication Cross site scripting password guessing self-replicating code password cracking exploiting known vulnerabilities disabling audits back doors hijacking sessions sweepers sniffers packet spoofing GUI automated probes/scans denial of service www attacks Tools “ stealth” / advanced scanning techniques burglaries network mgmt. diagnostics distributed attack tools Staged Auto Coordinated Intruders

Malicious Code Trends

Malicious Code Zoo – The Numbers

IE Critical Vulnerabilities

FireFox Critical Vulnerabilities

Vulnerabilities Are Increasing

Our Agenda

Complex Event Processing (CEP) and SEM

Wrap Up

SEM Functionality / Requirements

Log collection from heterogeneous devices - the capability to read, parse, normalize, and gather security events from a variety of heterogeneous event sources

Situation detection - the capacity to detect and refine threat-related situations automatically and priorities based on an automatic impact assessment, optimizing staff performance to focus on preventing the most important threats

Threat prevention and remediation - generate alerts and automated responses based upon high probability threat scenarios and manage the life cycle of the threat

Report generation – automate reports that support post-threat investigation, regulatory compliance and update visualizations and dashboards

Scalable, distributed architecture – the architecture must manage millions of logs per day, distribute the processing load, and with service-oriented services for transformation, event tracking, correlation, updates, remediation and visualizations

Overview of Cyber Defense Systems Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Security Event “Stovepipes” Centralized Distributed Cyber Defense Systems, Logs Agent Based

There is No Shortage of “Simple Event Aggregators”

What is Missing from this SEM Architecture?

SEM Today: Key Take-Aways

NO ESB – there is no secure, standards-based communications infrastructure for distributed event management in current SEM solutions

WEAK or NO ANALYTICS - there is limited capability to detect and refine threat-related situations with high probability using state-of-the-art analytics

WEAK or NO EDA - not standard generated alerts and automated responses to kick off workflow, compliance and other remediation activities

WEAK REPORTING – dashboards and reports tend to be" event aggregators” that do not filter out the “noise”

UNSCALEABLE, CENTERALIZED ARCHITECTURES – current SEM architectures cannot manage millions events in a heterogeneous, distributed architecture

Our Agenda

Complex Event Processing (CEP) and SEM

Wrap Up

Key EDA and CEP Concepts Aggregate events across multiple sources; correlate with historical data, refine Detect events across the enterprise in real-time. Normalize and contextualize. Manage resources, processes; Invoke actions in real-time Analyze & Visualize Sense Respond ©2007 , Tim Bass

Complex Event Processing " Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 " --- Gartner July 2003 Situation Detection

CEP Reference Architecture 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Functional Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM Adapted by Tim Bass from the JDL: Steinberg, A., & Bowman, C., Handbook of Multisensor Data Fusion, CRC Press, 2001

Cyberspace Situational Awareness

Multi-level inference in a distributed event-decision architectures

User Interface (Dashboards, BAM, Visualization, Portals)

Human visualization, monitoring, interaction and situation management

Level 4 – Process Refinement (Adaptive BPM)

Decide on control feedback, for example resource allocation, sensor and state management, parametric and algorithm adjustment

Level 3 – Impact Assessment (Predictive Analytics)

Impact assessment, i.e. assess intent on the basis of situation development, recognition and prediction

Level 2 – Situation Refinement (Situational Detection)

Identify situations based on sets of complex events, state estimation, etc.

Level 1 – Event Refinement (Event Tracking)

Identify events & make initial decisions based on association and correlation

Level 0 – Event Preprocessing

Cleansing of event-stream to produce semantically understandable data

Level of Inference Low Med High

Event Processing Characteristics

Event Processing Agents, Sensors and ESB

“System” can “learn” expectations from positive and negative examples

Users can specify expectations using:

SQL-like queries

Fuzzy matches

Statistical operators

Regular expressions and rules

CEP

SEM/CEP Solutions Overview SEM/CEP Solutions Space Data: Events & Databases -Real-Time & Historical Data Models: Statistical Financial Optimization Comms: Pub/Sub Messaging Queues Topics UIs Knowledge: Facts & Rules

Real Time Fuzzing Detection CEP Bayesian Classifier with Rules Engine Edge SSL Decryption, Extraction and Preprocessing Fuzzing for Web Application Security Vulnerabilities ESB CEP Engine Fuzzer Run Time Session Info Web Server Farms Application Servers CEP Engine

Wrap Up: CEP-Based SEM

ESB – a secure, standards-based communications infrastructure for distributed event management for SEM

STRONG ANALYTICS - extensible event-driven CEP engine detect to detect and refine threat-related situations with high probability using flexible, easy to implement state-of-the-art analytics

EDA - standards compliant messages, ESB, and alerts and automated responses to kick off workflow, compliance and other remediation activities with BPM workflow and compliance platforms

CUSTOM REPORTING – dashboards and reports easily customized with Rich Internet Application (RIA) visualization tools

SCALEABLE, DISTRIBUTED ARCHITECTURE – event-driven, cooperative agents to manage millions events in a heterogeneous, distributed architecture

Wrap Up: Key Takeaways

One of the most promising applications of complex event processing (CEP) is in the area of security event management (SEM), an evolving science that can provide businesses with advanced threat detection and warning capabilities.