Market Surveillance using statistical techniques and graph analytics

1. Market Surveillance – Going Beyond
Rules
Taking Surveillance to Next Level
Raja Das, Chief Architect, Financial Crimes Solutions
Sept 2017

2. Solution Value Proposition

3. 3
Surveillance Insight for Financial Services
Fuse insights to detect
suspicious activities
View linkages with instant
drill-down & playbacks
Build employee profiles
with personality &
behavior traits
See alerts & investigations
from continuously updated
risk models
Order
Trade
Execution
Reference
News
Social
Email
Chat
Voice
Reduced cost of employee
non-compliance & misconduct
Faster detection of
sophisticated scenarios
Risk based prioritization of
alerts and reduced false
positives
Transactions
Communications
External
Surveillance

4. 4
In addition to extending the ‘base-level’ controls, clients are also seeking to improve the accuracy
and effectiveness of their surveillance controls through a holistic and cognitive approach

5. 5
Surveillance Solution - Value proposition summary
Reasoning
Engine which goes
beyond rules based
alert system
Fusion of information
from multiple data
sources (ex: Trade,
Electronic
Communication, Voice,
Activity etc.)
Learning models which
are continuously
updated
Providing effective and
easy to use visualization
tools Identify new predictive patterns
by linking various information
with Trader profiles "i.e. know
your trader”
Improve accuracy of alerts -
Reduce false positives and
negatives
Improve efficiency of
investigations
IBM’s Advanced Analytics
(NLP, Sentiment analysis, Semantic Analysis, Behavior analysis, Emotion Analysis etc.)
Real-Time & Scalable Analysis Capabilities
Real-time supervision and
surveillance

6. 6
Solution High level view
6
Data Ingestion Pre-built
Analytics
Insight
Delivery
• Market data as CSV
in FIX format
• REST based
adaptor to accept
Incoming message
payload
• Pump/Dump detection
engine
• Spoofing Engine
• Bayesian Inference
Engine for reasoning
• UIMA based analytic
pipeline
• Dashboards
• REST based
Alert Service
IBM Industry Analytics Solutions
WHAT WE ARE DELIVERING
End-to-End Pre-built Capabilities
Business
Users
Trading
Data
Trade
Quote
Order
Execution
Market /
Customer data
Communication
data
Chat
Voice
Email

7. Solution Architecture

8. 8
Order
Position
Market
Social
Reference
Trade
ChatEmail
Bayesian Network, Markov
Processes
Reasoning Engine
People Data
(Brokers, Dealers, Traders,
Customers)
Data Store
• Metadata
• Results
• Raw DataVoice
Unstructured Data Ingestion
Semantic Analysis
Behavior Analysis
Social Analysis
Emotion Analysis
Compliance
Workbench
Raw Data
Anomaly Info.
Anomaly
Scores
Graph
Database
Real-time Analysis Platform
Normalize Analyze
Alert
Generation
In-memory Database
Integrated Trade Surveillance System – Logical View of the Foundation
Salient Features
- Single solution handling
Trade, eComms, &
Voice
- Real-Time, Near Real-
Time & Batch Analysis
- Cognitive Analysis and
Reasoning Engine
- Natural Language
Processing
- Advanced visualization

9. 9
Order
Positio
n
Market
Social
Referen
ce
Trade
ChatEmail
Bayesian Network,
Markov Processes
Reasoning
Engine
People Data
(Brokers, Dealers,
Traders,
Customers)
Data Store
• Metadata
• Results
• Raw DataVoice
Unstructured Data Ingestion
Semantic
Analysis
Behavior Analysis
Social Analysis
Emotion Analysis
Compliance
Workbench
Raw Data
Anomaly Info.
Anomaly
Scores
Graph
Databas
e
Real-time Analysis
Platform
Normalize Analyze
Alert
Generation
In-memory
Database
9
Integrated Trade Surveillance System - System Capability Mapping
Real-time Analysis Platform
• Normalize structured trade
data
• Analyze trade data to detect
anomalous signals
• Load data into data storage
• Speech to Text Technology
• Low-Level:
Codecs/Channel/Handset
Compensation; SNR
Estimation
• Mid-Level: Emotion and
Speech Recognition,
Speaker Diariaziation,
Age/Gender Estimation
• High_level: Sentiment
Analysis, Topic Detection,
Speaker Profile
Data Storage
• Stores both raw data, meta data, temporary
results/signals and the Case Alerts
• SQL Interface
Reasoning Engine
• Based on Bayesian Network
• Models Learn over time
Unstructured Data (eComms) Load
• Load email, chat, twitter, etc. data
• Load Message Board data
• Load other activity data
Compliance Workbench
• Integrated UI for both structured and unstructured data
• Executive Dashboard
• Details of individual Alerts
• Explore and analyze individual indicators/signal
Analysis Pipeline
Social Analysis
• Social communication graph
analysis
Semantic Analysis
Behavior Analysis
• Early Fusion of email
characteristics
• Feature based outlier detection
Emotion Analysis
• Risk Emotion Detection – analyze
user written text
APIs
• Integrate data channels (e-Comms)
• Integrate with Case Management
• Integrate with external Visualization
Tools
1
1
2
2
3
3
4
4
5 5
6
7
77
6

10. 10
Trade Surveillance Tool and Data Flow

11. Using Graph Analytics To Detect Market
Abuse

12. 12
Graph Database: storing the network structure in a graph database allows
for deep quantitative analytics
Native Graph DB stores nodes and
relationships directly, It makes
retrieval efficient.
Retrieving multi-step
relationships is a
'graph traversal' problem
• Native Graph Store for High Performance Traversal, Analytics & Visualization
• Generic Graph Query Language for Flexibility & Extensibility
In Relational DB, relationships are
distributed and stored as tables

13. Insider Trading Graph Analysis Example: page 1 of 5
Analyzing complex layered activities distributed across multiple exchanges and asset classes can be extremely
convoluted. Using a native graph structure can dramatically increase the efficiently of mining comparable activities.
Designing workflows to automatically mine similarities within historical broker activity can synthesize scenarios for
business agent review.
FFIV [ is a ]
US Equity [ in ]
NASDAQ [ with ]
Derivative Contracts [ in ]
AMEX, BATS, BOX, C2, CBOE, PHLX, ISE
Graph Legend:
Equity OrderDerivativeBroker
BUY SELL
Single Native Daily Market Graph (≈ 40B Nodes)
Relationship of Orders
This example is meant to demonstrate how
fundamental market activity can be stored in an
innate graph structure that intuitively replicates how
the market activity occurs.
Here we find equity FFIV which is listed on the
NASDAQ but has derivatives contracts sold
through multiple exchanges. We can structure
the market activity as a vast network of
orders. Each encoding information on edges as
well as order properties improves efficiency.
The relationship between securities, orders, and brokers is paramount. Entities are
modeled:
Equity Order BrokerDaily:
≈ 10,000 ≈ 40
Billion
≈ 650,000
Additional degrees of separate such as derivatives/securities can be modeled.
Equity Order BrokerDerivativeDerivative
Directionally, enormous amounts of information can be encoded into the graph.
FFIV Order
Broker
A123
FFIV Order
Broker
B234
BUY
SELL
exchange: BATS
counterparty linkage
exchange: BATS
volume: 10,000
volume: 10,000

14. Insider Trading Graph Analysis Example: page 2 of 5
By modeling the market as a network of connected trades previous analytic computations can be replaced with queries.
Now attributes of the network are considered degrees of connectedness whereas before they would have been stored in
separate tables and looked up / aggregated independently.
Call Order
FFIV
FFIV – A123 Connection: April 2, 2016
Here we can examine the degree of separation and (source / sink
paths) to learn features from the network.
Strike Price
Property
Expiration Property
Broker
A123
Put
$105
Order
Strike Price
Property
Jun 15 Expiration Property
Order
volume: 10,000
volume: 5,000
volume: 1,000
exchange:
NASDAQ
property value
Order Summary:
 1) Enter large position in stock
 2) Enter large position in Call
 3) Enter large position in Call (second order)
 4) Wrote large position of Put
Example Queries:
 Parking: Do round-trip paths exist between two brokers and a single security?
 Front-Running: Do paths exist with better price execution on house accounts
rather than client accounts?
Apr 15
$103
Order
time
BUY
SELL
1
2
3
4

15. Insider Trading Graph Analysis Example: page 3 of 5
Major Public Announcement: April 3, 2016
Broker A: Annotation
Max Gain: $5,000,000
Position Value: $20,000
Volume: 500
PutFFIV PutFFIV PutFFIV
Broker B: Annotation
Max Gain: $10,000,000
Position Value: $40,000
Size: 1000
Broker C: Annotation
Max Gain: $5,000,000
Position Value: $20,000
Size: 500
5M
Kick-off Historical Graph Query Batch Process
Candidate List: A123, B324, C543
Given the traversal object g of a graph that reflects a specific date, and the symbol of the
targeted instrument 'FFIV', return a list of all broker names who have positions with Max
Profit property of greater than $500,000
g.V().has('symbol','FFIV').repeat(out()).times(2).where(values('max_profit').max().is(gt(500,0
00))).out().values('name')
=> A123, B324, C543
Query 1
Broker
A123
Broker
B324
Broker
C543
Max Profit Property*
Order
10M
Order
5M
Order
* Max Profit is a static property given at trade execution. It is the maximum profit from Put trade. Other properties or analytics are eligible in this scenario as well.
Max Profit Property Max Profit Property

16. Insider Trading Graph Analysis Example: page 4 of 5
Using the candidate list from before a graph traversal across historical snapshots can be done to find similar instances.
Put Order
Broke
r
A123
Historic Snapshot: January 1, 2016
Put
Broke
r B324
Traversal Result: IBM Put Position
IBM
Start
Match
Given the name of a broker 'A123' in the result returned for
query 1, and given the traversal object g of a graph that for a
specific date, return a list of broker names who have positions
for the same instrument as the targeted broker, and the Max
Profit values for the orders of these brokers and the targeted
broker on the instrument are all over $500,000:
g.V().match(
__.as('broker1').has('name','ABC123’),
__.as(‘broker1’).in().as('order1’),
__.as('order1').values('max_profit').max().is(gt(500,000)),
__.as(‘order1').repeat(in()).times(2).as('instrument'),
__.as('instrument').repeat(out()).times(2).as(‘order2'),
__.as('order2').values('max_profit').max().is(gt(500,000)),
__.as(‘order2’).out().as(‘broker2’)).
select('broker2').dedup().by('name')
=> B324
Iteratively, queries such as this can return matches for a
compliance officer to review:
Result:
Query 2
1
2
3
4
5
1
4
4
Equity
Exchang
e
Expiratio
n
Strike Broker
Order
Size
Max Profit
IBM BOX 15-Jan $150 A123 300 $4.5M
IBM BATS 15-Jan $155 B324 400 $6.2M
5 75
Strike: $150
Strike: $155
4.5M ✓
✓
2
6
3
Order
6.2M
Put Order
Broke
r
A123
Historic Snapshot: February 14, 2016
Put
Broke
r B324
Traversal Result: FB Put Position
FB
Start
Match
1
4
4
5 75
Strike: $80
Strike: $70
1.6M ✓
✓
2
6
3
Order
2.1M
6
7

17. Insider Trading Graph Analysis Example: page 5 of 5
Looking at the results it appears Broker B324 is the common link across all matches.
Broker
A123
Broker
B324
Broker
C543
Sub-match Investigation:
Results of investigation discovered that Broker B324
was the source of non-public information in the ring
and received information from a colleague who covers
Technology for a major Investment Bank in New York
City.
Initiate
Investigation
Date Equity XCh Exp Ins Strike Broker Size
Max
Gain
A123, B324, C543
3-Apr-16 FFIV BOX 15-Apr P $100 A123 500 $5M
3-Apr-16 FFIV C2 6-May P $100 B324 1000 $10M
3-Apr-16 FFIV CBOE 6-May P $100 C543 500 $5M
A123, B324
1-Jan-16 IBM BOX 15-Jan P $150 A123 300 $4.5M
1-Jan-16 IBM BATS 15-Jan P $155 B324 400 $6.2M
14-Feb-15 FB BOX 15-Mar P $80 A123 200 $1.6M
14-Feb-15 FB BATS 7-Apr P $70 B324 300 $2.1M
B324, C543
5-Dec-15 GRPN BOX 30-Dec P $4 B324 1600 $640K
5-Dec-15 GRPN BATS 30-Dec P $5 C543 2000 $1M
14-Oct-15 TWTR ISE 15-Oct P $25 B324 400 $1M
14-Oct-15 TWTR AMEX 15-Oct P $30 C543 500 $1.5M
Sample Report
Call Order
Broker
B324
Historic Snapshot: December 5, 2015
Call
Broker
C543
Traversal Result: GRPN Put Position
GRPN
Start
Match
1
4
4
5 75
Strike: $4
Strike: $5
640K ✓
✓
2
6
3
Order
1M
Put Order
Broker
B324
Historic Snapshot: October 14, 2015
Put
Broker
C543
Traversal Result: TWTR Put Position
TWTR
Start
Match
1
4
4
5 75
Strike: $25
Strike: $30
1M ✓
✓
2
6
3
Order
1.5M

18. Questions ?