Presentation from IBM Impact 2012 with Nastel Technologies and Verdande illustrating two-tier analytics with CEP and CBR for optimal root cause analysis for application performance
Grateful 7 speech thanking everyone that has helped.pdf
The Case for Reasoning: Monitoring Bank Infrastructure with AutoPilot and Verdande
1. The Case for Reasoning
A Demo: Nastel AutoPilot and Verdande
Nastel Technologies Confidential
2. The Story: A Bank at risk
2
ACME Financial
Services The Risk to NASD
Reporting Deadlines
Monitoring must be monitored
ACME Bond
Trading
3. The Story: A Bank at risk
3
Monitoring Bond Trades
FIXML trade representation
ACME Financial
– mandatory fields
Services
Message flow monitored
Network events
Monitoring
Market Volume - spikes
Monitoring IT Infrastructure
Economic events from
Bloomberg
Time
ACME Bond
ACT Monitoring
Trading
4. The Story: A Bank at risk
4
ACME Financial
Services A firm may be fined by the
NASD when these bond
Monitoring
trades requirements are
violated
All price, volume and
transactions must be
reported to NASD within 15
ACME Bond minutes
Trading
5. AutoPilot Monitors Bank’s Infrastructure
5
Bank Infrastructure Servers
TIBCO WMQ
J2EE/.NET DB
Cloud
DataPower
Application
Solace System Z
Servers
Other tools
forwarding events
to AutoPilot Bloomberg
CEP
6. AutoPilot Monitors Bank’s Infrastructure
6
Infrastructure Events
Alerts sent when
situation is outside
business-normal
Events from other Tools
Cloud
Bloomberg Feed
Composite Events/
Situational Analysis
from AutoPilot CEP
CEP
7. AutoPilot with CEP - Pattern Analysis
7
Low-latency pattern analysis with good “signal-to-noise ratio”
Rapidly find the patterns..
Scales to handle millions of events/second…
Noise removed, event
data is actionable…
Situations
Turns Big Data
Cloud
into Small
Big Data Small Data
Meaningful Data
CEP
Lots of noise in this data…
8. The Two-step Analysis
8
1. AutoPilot with CEP analyzes and predicts IT situations
2. Filtered data sent to Verdande to analyze business impact
IT Problems Analysis Business Impact Analysis
Situations
Cloud Small Data Verdande
CEP
9. Scenario: ACME Financial Services has a Service Alert
9
CEP
Service Alert
FIXML
ACT Status = Online ACT Monitor
Report Time = 12:34:21 Report Time
Trade ID = 1238624 Trade ID
Volume = 85,000 Volume
Value=19 Avg.=189 Max=236 Min=24 Last-Update=4/10/2012 Queue Status
10. Q with problem impacts the Bond Trading App
10
CEP
Transaction Detail
Transaction Service Start Server Application Duration Status
Q status Bonds 502 Prod BondTrade 5.23s Missed SLA
11. Spent too much time in the Q for Bond Trading App
11
CEP
CICS
Bond-Queues
(40% of total
time)
WebSphere Message WMQ
Broker
Traders AS
Bond Trading
Application WebLogic
12. MGET Operation took too long
12
CEP
MGET Bond Trading and TradeVerification Applications
2012-04-29 14:06:08 Elapsed Time of 40000 Message
Age 2540030
Support WMQ Missed SLA
Automatic deep-dive
Analysis
13. Determines progress through the business process
13
CEP
Verification step in business process has been impacted
14. AutoPilot’s Municipal Bond - IT Ops Analysis passes to Verdande
14
CEP
ACT Status = Online
Report Time = 12:34:21
Trade ID = 1238624
Volume = 85,000
Value=19 Avg.=189 Max=236 Min=24 Last-Update=4/10/2012
15. Input Data Sources Unifed Data Structure Data Analysis Agents Situational Description
Municipal Bond-IT Ops
Analysis from AutoPilot Time-Index Data
15
Current
Case
Static Data
Visualization Case Search
Current
Stored Case
Case
16. Input Data Sources Unifed Data Structure Data Analysis Agents Situational Description
Municipal Bond-IT Ops
16
Analysis from AutoPilot Current Case
Time-Index Data
Current
Feature Value Case
Trade Type Data
Static
Municiple Bond
Trade Detail
Trade ID 1238624
Visualization Case Search
asd
ACT Status Online
Queue Status
Current
Stored Case
Case
Report Time 12:34:21
Service Alert
Volume 85,000
Economic Activity
17. Input Data Sources Unifed Data Structure Data Analysis Agents Situational Description
Municipal Bond-IT Ops
Analysis from AutoPilot Time-Index Data
17
Current
Case
Static Data
Visualization Case Search
Current
Stored Case
Case
18. Retrieved Case Current Case
Feature Value Feature Value
Trade Type Corporate Bond 78% Trade Type Municple Bond
Trade Detail 74% Trade Detail
Trade ID 3762347 n/a Trade ID 1238624
ACT Status asdPending 33% ACT Status asdOnline
Queue Status Queue Status
75%
Report Time 08:34:21 69% Report Time 10:44:21
Service Alert 70% Service Alert
Volume 1,000 60% Volume 9,560
Economic Activity 100% Economic Activity
18
19. Input Data Sources Unifed Data Structure Data Analysis Agents Situational Description
Municipal Bond-IT Ops
Analysis from AutoPilot Time-Index Data
19
Current
Case
Static Data
Visualization Case Search
Current
Stored Case
Case
20. The result of the case
search is displayed as a dot
20 on the case radar in the
center of the screen.
The current case had a
number of close similarities
with the retrieved case that
resulted in a weighted
match of 69%.
Each dot represents one
case. The cases that are
more similar that some
threshold is put on this
radar in such a way that
they are closer to the center
of the radar the higher the
similarity between that case
and the current situation.