Driving Operational Responsiveness with Complex Event Processing

Driving Operational Responsiveness with Complex Event Processing
Dr Giles Nelson
Senior Director of Strategyand Co-inventor, Apama
5 June 2009

Events are everywhere
Achieving Operational Responsiveness requires:

Recognition of problems and opportunities among these large numbers of events
Responding to the problems and opportunities in time to be effective

Trade settlement
5 days
2 hrs.
Business Velocity Is Key to Efficiency
mail express fax e-mail Document transfer
Business
Requirement
Algorithmic trading
100 ms
20 ms
Reduce
processing time
Airline operations
20 min
30 sec
Design Strategy
Call center inquiries
8 hr
10 sec
Track financial position
5 min
1 day
STP, zero-latency enterprise
15 min
1 day
Supply chain updates
Phone activation
3 days
1 min
Refresh data warehouse
1 week
0.5 hour
Build-to-order PC
4 weeks
1 day
Typical Business SLAs106 105 104 103 100 10 1 0Seconds

We’re told that information is not timely enough
► 83% believe IT has a major influence on the ability of businesses to adapt and change
► 77% believe availability and timeliness of information is inadequate to support this change
► The velocity of information is not keeping up with the speed of business
Survey of 500 European Business and IT Leaders
Source: Vanson Bourne Research, 2008

Conventional Ways of Examining Business Information
Always looking in therear view mirror
How can I examine what is happening and react now?

Complex Event Processing
What has happened?
Traditional Processing - Acts on data from the past“What were the average call exceptionrates from yesterday?”
What is happening now (CEP)?
Event Streams
time
1
2
3
4
5
6
7
8
9
Complex Event Processing – acts on data in motion “When 2 or more call exceptions occur during a 1 minute period, send an alert.”

Principles of CEP-based Systems
Treat any business update as an “event”
Enable users to rapidly define event-based rules to identify patterns indicating opportunities and threats to the business
Rules are loaded into a real-time engine that offers analysis and response with low latency
Engine is permanently connected to multiple event sourcesanddestinations
Events can be captured and preserved in time-order for historical pattern analysis and root-cause analysis

CEP Market Size
Now: $141M globally
2012: $803M
Compound Annual Growth of 55%

Progress Customer Examples

Apama Today
Leader in CEPFrom IDC report: “Complex Event Processing Opportunity Analysis and Assessment of Key Products”, February 2009
CEP is well established in financial services
Expanding into Telco, Energy, Transport & Logistics, Manufacturing
120+ customer deployments

Customer Case: Logistics
The Challenge
Royal Dirkzwager
Maritime Information and Service Provider
Massive domain
More than 90,000 ship movements a year with lots of cargo coming in to Rotterdam
Massive data volumes
Land- and sea-based AIS and LRIT systems
How to optimize ship movement and berth space for a lean and green approach?

Customer Case:Royal Dirkzwager
The world hasbecome smaller
Space-basedAIS and LRIT makeit possible to havecomplete control
The Challenge

GISService
MessageService
NotificationService
Customer Case:Royal Dirkzwager
The Result: Complex Event Processing
Business Rules
Dashboards
Adapter
AIS Events
LRIT Events

Top ten global bank, EMEA headquartered
Current Situation:
Business Processes are not simple...
Problems happen, and can have a big negative impact on the business
Problems are usually detected too late
The consequences of these delays usually means lost of revenue, non-fulfillment of period, non-fulfillment of agreements, un-satisfied customers, …

Top ten global bank, EMEA headquartered
Critical Aspects
No end-to-end visibility of the process
Incidents discovered too late
Difficulty to quantify the operational impact of a problem
Root-cause: Difficult to find
Every day “fighting fires”
Complexity
Big number of systems and applications
Processes and workflows complex
Multiple functions and responsibilities
Volume is increasing
Response time: Not enough
Clients are adding pressure

Retail Bank Process Monitoring
Architecture
Event Engine
Architect
Management
Process monitoringscenarios
Dashboards
Process specific information sources
Existing BPM system

Apama

A real-time event correlation engine with management and high-availability
Integrated, real-time dashboards
Eclipse-based Development Environment: Apama Studio
Research Studio

What is Apama CEP?
A complete platform for building real-time, event-driven applications. It consists of:

How does Apama Work?
MANAGE/MONITOR
It starts with events
Adapter
Events
Events
Events
Events

Integration
PointSystems
DBMS
Middleware
Other…
Building and Deploying an Apama Application
Adapters connect the event processing engine to the IT environment. Pre-packaged adapters can be dynamically deployed, configured, and modified. New adapters can be built quickly

Smart BlockAnalytics
Integration
PointSystems
DBMS
Middleware
Other…
CEP Smart Blocks encapsulate common analytics and domain specific functionality (for example, libraries of statistical analytics)

Scenariomodelling
Integration
PointSystems
DBMS
Middleware
Other…
Event based scenarios can be modelled using a GUI tool.
Business analysts can develop, configure and deploy scenarios themselves

Customers &partners
End-User Dashboards
Scenariomodelling
Integration
PointSystems
Database
Middleware
Other…

Adopting CEP: Progress’ approach
A proof of concept
With real business use case
Consultancy and Education
Success!

Summary