We developed Sybase BAM many years ago that is very similar to the current streaming middleware.

1. Sybase BAM
Overview
Xu, Jiang (BAM/Rules Team)
March 20, 2007
Sybase Confidential Proprietary.

2. Sybase Confidential 2
Agenda
• Technology Background
• Analytic Model
• Architecture
• Main Features
• Demo

3. Sybase Confidential 3
Background - Overview
• Business Activity Monitoring (BAM)
• Complex Event Processing / Event Stream Processing
• Two approach of CEP/ESP
• Real Time Business Intelligence
• Two approach of RTBI

4. Sybase Confidential 4
Background - BAM
"Business activity monitoring" (BAM) is Gartner's
term defining how we can provide real-time access
to critical business performance indicators to
improve the speed and effectiveness of business
operations.
Unlike traditional real-time monitoring, BAM draws
its information from multiple application systems
and other internal and external sources, enabling a
broader and richer view of business activities.

5. Sybase Confidential 5
Background – ESP/CEP
“Event Stream Processing” (ESP) is software technology
that allows applications to monitor streams of event data,
analyze those events, and act upon opportunities and
threats in real time. ESP systems often utilize, or include,
event databases and event visualization tools, event-driven
middleware, and event processing languages
“Complex Event Processing” (CEP) is a key element of
ESP that provides language elements that allows
applications to express the complex patterns among events
it's looking for. CEP provides constructs that include event
correlation, event abstraction, event hierarchies, and the
ability to express relationships between events such as
causality, membership, and timing.

6. Sybase Confidential 6
Two approach of CEP/ESP – SQL
Based Approach I
Some people coming from RDBMS development
have extended SQL to provide CEP/ESP.
• The SQL processing in traditional RDBMS is
“data is static and query is dynamic”.
• The SQL processing in CEP/ESP is “data is
dynamic and query is static”.
• Because the event data may be overflow, it is
necessary to introduce “time window” to SQL

7. Sybase Confidential 7
Two approach of CEP/ESP – SQL
Based Approach II
SELECT I1.SourceIP As SourceIP, I1.AttackKind As AttackKind, V1.Virus As Virus
FROM InIDSAlerts As I1 KEEP 30 SECONDS, InVirusAlerts As V1 KEEP 30
SECONDS
WHERE I1.SourceIP=V1.SourceIP
Join
Projection
.
.
.
.
.
.
Scan Scan
.
.
.
I1 KEEP 30 SEC I2 KEEP 30 SEC
I1 I2

8. Sybase Confidential 8
Two approach of CEP/ESP – Rule
Based Approach I
Some people come from integration development
have extend rule engine to provide CEP/ESP.
Sybase BAM chooses this approach.
The key of this approach is to add complex state
management and corresponding operator to the
traditional rule engine that can support complex
event pattern and event correlation.

9. Sybase Confidential 9
Two approach of CEP/ESP – Rule
Based Approach II
Rules
States
Event
Actions

10. Sybase Confidential 10
Background – RT BI
“Real time business intelligence” (RT BI) is the
process of delivering information about business
operations without any latency.
While traditional business intelligence presents
historical information to users for analysis, real
time business intelligence compares current
business events with historical patterns to detect
problems or opportunities automatically.

11. Sybase Confidential 11
Two approach of RT BI
• Event driven, Real time Business Intelligence
Real time Business Intelligence systems are event driven,
and use ESP/CEP techniques to enable events to be
analyzed without being first transformed and stored in a date
warehouse.
This approach is better for BAM.
• Real time Data warehouse
An alternative approach to event driven architectures is to
increase the refresh cycle of an existing data warehouse to
update the data more frequently. These real time data
warehouse systems can achieve near real time update of
data, where the data latency typically is in the rage from
minutes to hours out of date.
This approach is better for ETL.

12. Sybase Confidential 12
Analytic Model - Overview
Fields: Abstract states definition.
Key, Unbound, Bound, Aggregation
Rules: Intelligence
If condition Then action
Actions: Behavior
Update, Aggregation, Alert, Timer, SQL, Java
Script, Purge
Timers: Scheduler
If timer arrive Then action
Binder: Concrete states storage
BAMDB, UserDB, RefAM

13. Sybase Confidential 13
Analytic Model – Processing
Fields
Key
Bound
Bound
Bound
Aggregate
Unbound
Rules Actions
• Update
• Aggregate
• SQL
• Alert
• Java Script
• Timer control
• Purge
if…
if…
1. Keys, (some) other field passed into Analytic Model
2. Historical values found based on keys
3. Rules applied to data
4. Actions performed, update data
5. Repeat 3, 4 as needed
6. New values stored

14. Sybase Confidential 14
Analytic Model – in SOA
Input Fields
-----
-----
-----
-----
Output Fields
-----
-----
Monitor Service
Fields
• Key
• Unbound
• Bound
• Aggregate
Rules / Actions
• Update
• Aggregate
• Send Alert
• SQL
• Java Script
• Timer Control
• Purge
Analytic Models / Analytic Objects
Fields Rules / Actions
Fields Rules / Actions

15. Sybase Confidential 15
Analytic Model - Functionality
• Monitor services interact with multiple Analytic Models,
setting key fields to define specific object instance.
• Within Analytic Object, multiple rule calls trigger actions
that further update object and perform other activities.
• Any field set in one Analytic Object is then available to
subsequent objects, as determined by the Monitor Service.
• If there is implicate or explicate key fields setting between
different Analytic Objects, record the cross correlation of
Analytic Objects.
• Service output fields may be return result of any field from
any Analytic Object.

16. Sybase Confidential 16
Architecture - Overview
Monitor
Service Editor Monitor Service
WSDL
Monitor
Command and
Control
Monitor
Analytic
Model Editor
Dashboard
Business
Process
External
Client
SOAP,
JMS, etc
Monitor
Service
BAM-Defined
Database
Binding
User Defined
Database
Binding
SCS Container
Analytic
Object
Access
Library
Rules
Timed Event
Daemon

17. Sybase Confidential 17
Architecture - Components
• BAM Engine
§ Analytic Object Access Library
§ BAM Rule Engine
§ Timed Event Daemon
§ Monitor Service WSIF Provider
• BAM Tooling
§ Analytic Model Editor
§ Monitoring Service Editor
• BAM Web GUI
§ Monitoring Console
§ Dashboard

18. Sybase Confidential 18
Runtime Processing of BAM
Queu
e
SCS
JMS
WSHF
Provider
CSB
Monitor
Service
WSIF
Provider
Optimus
Analytic
Object
Access
Library
DB
Timed Event
Daemon

19. Sybase Confidential 19
Main Features - Overview
• Complex Event Processing Support
• Real Time Business Intelligence Support
• Comprehensive Alert Capability
• Intuitive Visualization for Monitoring and Analysis
• Metadata-Driven Design Tooling
• Service Oriented Architecture Support
• High Volume

20. Sybase Confidential 20
Main Features - Complex Event
Processing Support I
• Event-Condition-Action (ECA) model
§ Event Triggering, Rule Evaluation, Execute Action
• Event Transport/Triggering
§ JMS, HTTP, Email, File, Timer
• Event Parsing/Transformation
§ XML, CWF, SOAP
• Event Routing
§ Body-based, Header-based, Endpoint-based

21. Sybase Confidential 21
Main Features - Complex Event
Processing Support II
• Event States
§ Stateless, Stateful, Historical
• Event Correlation
§ Correlate low-level events to high-level event
§ Key correlation, Cross correlation, History correlation
• Event Reprocess
§ Take corrective action for closed loop integration
• Complex Event Pattern Support
§ Based on ECA model + Event States + Event Correlation.

22. Sybase Confidential 22
Main Features - Real Time Business
Intelligence Support I
• Rule-based intelligence
§ Light-weight BAM Rule Engine (BRE)
§ Patent-pending Boolean Network Rule Engine (BNRE)
• Analyzing real-time data in the context of historic
information
§ Reference contextual data from ASE, IQ, EII

23. Sybase Confidential 23
Main Features - Real Time Business
Intelligence Support II
• Time windowed aggregation / computation
§ User-defined computation expression
§ Extensible Aggregator: Average, Rate, Standard Deviation
§ Sliding Time Window / Fixed Time Window
• Multi-dimensional analysis support
§ Based on Event Correlation + Aggregation + Computation

24. Sybase Confidential 24
Main Features – Comprehensive Alert
Capability
• Publish-subscribe model
§ XML Messages Publish via JMS
§ Customized Subscription
• Multiple Delivery Target
§ JMS, JMX, Email
• Alert escalation
§ Timer, On-demand
• Alert lifecycle
§ Active, Canceled, Completed, Escalated, Suppressed

25. Sybase Confidential 25
Main Features - Intuitive Visualization
for Monitoring and Analysis
• Dashboard
§ Visual objects of Key Performance Indicator (KPI) is changed
dynamically as events occur in real time
• Monitoring
§ Real time event is displayed in tabular forms
§ Drill-down from high-level event to low-level events
• Alerting
§ View and resolve alerts

26. Sybase Confidential 26
Main Features - Metadata-Driven
Design Tooling
• Based on Eclipse and EMF (Eclipse Modeling
Framework)
• Fully integrated and conformed to Sybase
WorkSpace

27. Sybase Confidential 27
Main Features – SOA Support
• BAM is exposed as “Monitoring Service” in
Sybase Service Container

28. Sybase Confidential 28
Main Features - High Volume
• High Performance
§ BAM engine can process about 2000 messages/sec on a 2
CPU machine
• Linear Scalability
§ BAM engine is linear scalability
§ Single BAM DB is linear scalability with CPU number
§ Multiple BAM DB are linear scalability with machine number

29. Sybase Confidential 29
Reference
Business Activity Monitoring
http://en.wikipedia.org/wiki/Business_activity_monitoring
Complex Event Processing
http://en.wikipedia.org/wiki/Complex_event_processing
Event Stream Processing
http://en.wikipedia.org/wiki/Event_Stream_Processing
Real-time Business Intelligence
http://en.wikipedia.org/wiki/Real_time_business_intelligence
BI 2.0: The Next Generation
http://www.dmreview.com/article_sub.cfm?articleId=1066763
BAM: Event-Driven Business Intelligence for the Real-Time Enterprise
http://www.dmreview.com/article_sub.cfm?articleId=8177
Data Integration—the Foundation of a Robust Enterprise Architecture
http://www.informatica.com/company/featured_articles/data_integration_foundation_082004.htm