1. By Alan L. Paris, Director Financial Crime and
Compliance Analytics (FCC)
CRISIL LLC, Global Research & Analytics
CRISIL LLC is an S&P Global Company
Compliance Analytics
A New Era…Insight, Efficiency, and Control:
Applying Quantitative Analytic Solutions to Regulatory Compliance problems

2. 2
Agenda
• Industry Challenges
• Solution Framework
• Examples of the Analytical Approach
• Risk & Regulatory Analytics: An Overview
• AML Model Validation: A Qualitative and Quantitative Approach
• Case Studies
− AML Transaction Monitoring
− CRR
− Fraud Detection

3. 3
Compliance Industry Challenges
Many banks have received
regulatory actions relating to
compliance, fraud,
sanctions, & money
laundering
Matters Requiring Board Attention (“MRBA”)
Cease & Desist Orders (“C&D”)
Examination findings & recommendations
Deferred Prosecution Agreements (“DPA”)
Typical shortfalls include
• Lack of proper customer identification and due diligence on high risk and other entities
• Inability to monitor relationships on a bank-wide basis
• Poorly administered risk classification methodology
• Inadequate periodic client entity review processes and high error rates
• Lack of data control and accuracy
• Reliance on other systems
• Lack of independent review and oversight
Written Agreement

4. 4
Compliance Solution Framework
What have our clients
done to address
these issues?
Control Metrics and Reporting
Analytics beyond “Tune Up/Tune Down”
Enhanced Data Management & Visualization
Learning from Risk Management Disciplines
Connecting The Dots…
• Taking a Data Driven and Analytical Approach
• AML Model Validation (Qualitative & Quantitative Approach)
• Transaction Monitoring – Alert Management, Metadata, Customer segmentation
• CDD & COB – Customer Risk Rating methodology
• Data Visualization & Reporting – Moving beyond Descriptive & Prescriptive to Predictive
• Fraud detection and prevention strategies
• Optimizing False Positive Ratios
Attribution Analysis

5. 5
Applying Analytics to KYC/AML
Threshold Tuning
• Setting the initial thresholds, reviewing and fine-
tuning the thresholds using statistical analyses
• Reducing false positives or irrelevant scenarios to
improve the system’s robustness
• Above/below the line testing
• Reviewing and comparing alerts with the red flags
and Suspicious Activity Reports (SAR)
Segmentation
• Data Management – identifying relevant data,
extracting, cleansing, grouping, etc.
• Identifying proper attribute variables through
attribute profiling analysis and business logic
• Creating new segments based on select variables
• Clustering segments using various techniques such
as K-means algorithm
Analytics in AML/KYC
Validation
• Key Validation Exercises:
– Thresholds on L1, L2, L3 alert-based rules
– Scenario summary tables
– Alerts generated by scenarios
– Transactions of each alerts
• Generating reports for MIS and audit
requirements
Scenario Development
• Developing new scenarios, fine-tuning existing
scenarios based on newly-derived or existing
thresholds
• Replicating the scenarios from other systems
(Norkom, Actimize, Mantas)
• Developing codes on the behavior detection
techniques based on the minimum transaction
amounts and initial threshold values set for
different parameters

6. 6
• Setting initial or reviewing existing thresholds
• Tuning using combination of rules for rules and
alert score threshold tuning to lower false
positives
• Management of scoring scale and algorithm to
manage alert volumes
• Dashboards for model and rules efficiency
• monitoring
• Validating model per OCC 2011-12 guidelines
• Extensive experience in validating –
• Thresholds on L1, L2, L3 alert-based rules
• Scenario summary tables
• Generating reports for MIS and audit
requirements
• Account policy rules management on Policy
Manager
• Development of specific user defined policies
• Usage of Risk Case Manager to file FIU reports
• Form creation and report generation
• Modification of core model parameter for
behavior/peer group/anticipatory etc.
• Suspicious behavior models
• AML-EFT, AML-CLF, AML-EMF, etc.
• Unusual behavior models
• AML-ULN, AML-UCR, etc.
• User defined - scoring factors, scales, algorithm,
etc.
Applying Analytics to Suspicious Activity Monitoring
Core Detection
Models
Scoring & Alert
Consolidation
System
ValidationPolicy & Risk
Case
Manager
Analytics in
Suspicious Activity Monitoring
Scoring & Alert Consolidation
System ValidationPolicy & Risk Case Manager
Core Detection Models

7. 7
Data Visualization
Tying together multiple sources of disparate data

8. 8
Risk & Regulatory Analytics – Overview
Stress Test Modeling
• Develop models for various PPNR
components
• Develop loss forecasting models for
various loan accrual portfolios
• Model macro-to-micro relationships
between economic variables and risk
parameters
• Forecast economic variables using
statistical modeling methods
Model Development
• Market Risk: VaR, Specific VaR, Stressed
VaR, RNIV
• Credit Risk: PD/LGD/EAD estimation, IFRS
9 modelling
• Operational Risk: Advanced
Measurement Approach (AMA)
• Pillar II Risk Modeling:
 Business risk modeling
 Economic Risk Capital
Model Validation and Documentation
• Validate regulatory capital models,
economic capital models, stress test
models and valuation models
• Data quality assessment and model
testing
• Challenge model assumptions and
highlight limitations
• Perform back-testing and benchmarking
• Adherence to FRB’s SR 11-7 guidelines
Change Management
• Regulatory Change Management: FRTB,
BCBS 239, UMR, IFRS 9 and MIFID II
• Business analysis and testing
• Gap & impact analysis of existing
processes and new processes
• Planning, implementation and manage
large-scale change programs
• Risk data aggregation
Compliance Analytics
• Financial crime analytics across
financial services business lines and
products
• Transaction monitoring
• Name screening & KYC analytics
RISK &
REGULATORY
ANALYTICS
Activities
• AML Model Development and
Validation
• Fraud Detection and Modeling
• Transaction Monitoring and Analytics
• Name Screening & KYC Analytics
• Compliance Analytics and Reporting
• Customer Risk Rating Calculation
• Data Visualization
• Building Scenario Models, Analytical
Libraries and Tools
• Regulatory Change Management

9. 9
Conceptual Soundness Review
Conceptual robustness of model is
validated in light of Model Risk
Management policy and SR 11-7
guidelines :
• Regulatory alignment
 Ability to capture Risk factors
and score appropriately for
every risk tier
 Monitor deviation in
transactions’ expected value and
volume
 Ability to detect suspicious
transactions
• Appropriateness of
 User Acceptance Testing results
and approvals by stakeholders
 Model development and Model
performance Monitoring and
Maintenance plan
 Reasoning behind performance
tuning conducted
 Methodology assumptions
Data Validation
Accuracy and integrity of the input data is
verified by validating the following controls:
• Appropriate mapping of data fields from the
internal data to vendor data structure
• Data mapped in timely manner
• Sufficiency of controls around data
• Ability of a model to
 Rate a customer with incomplete
information
 Detect duplicate accounts
Process Validation
Validate the existence of the following:
• Internal written policies and
procedures to comply with BSA
• Ongoing employee training
program
• Roles and Responsibilities for
model implementation, use, and
validation
• Model performance Monitoring
and Maintenance plan with
appropriate metrics
• Well defined quality tuning process
• Exception requests
• Change logs tracking the change in
the parameters and settings during
tuning
• Management approval to the
adjustments, overlays, overrides
made to the model’s settings,
parameters and outputs
• Tracking and reporting the trend in
percentage of overrides made in
different risk tiers
System Validation
Validating System Integration Testing (SIT)
results to ensure that Input files from across
various systems received on a timely manner
Validate the existence of the following:
• SIT summary interpretation
• SIT execution outcome (pass/fail) and parties
executing the test cases
• Approvals by appropriate stakeholders
Model Risk Focus: AML Model Validation – Qualitative
Solutions
AML Model Qualitative Validation Activities

10. 10
Model Risk Focus: AML Model Validation – Quantitative
Solutions
AML Model Quantitative Validation Activities
Alerts
SAR
Overrides
Risk-factors
Alerts analysis
The bank might be using several AML models ( eg. for CDD, Suspicious
activity transactions monitoring, watch list screening etc.) Analysis of
the alerts generated from different system can be performed.
• Alert mismatch: Comparison of alerts generated by different
models/ tools
• Time series analysis of the alerts generated
• Any significant change in the trend of alerts generated and SAR filed
would call for model investigation.
Alert optimization and Parameter tuning
analysis
• The quality and quantity of alerts can be
optimized by modifying the model
parameter scores / thresholds.
• A comparison of alerts and SAR filed before
and after the tuning of the AML model
parameters/ thresholds gives insights about
the effectiveness of model tuning procedure
Risk factor analysis
Each risk factor contributing to the AML model can be studied
independently.
• Risk factor stability : Analysis of distribution of alerts among the different
risk levels of a particular risk factor
• Any significant change in alerts distribution indicates model deterioration
Sensitivity Analysis of Risk Factors
• The sensitivity analysis of risk factors
can be performed by shocking the
scores of each risk factors and
observing its impact on the number of
alerts generated
SAR analysis
SAR (Suspicious Activity Reports) are generated from the
output of screening of model alerts. SAR can be analyzed
as follows:
• Percentage of SAR filed in each risk tier. This is expected
to have monotonous increasing trend as the risk tier
increases.
Benchmarking analysis
• The percentage of SAR filed using the current model can
be compared with some previous tool/ model used for
AML monitoring
• Benchmarking analysis can give insights about the
advantages and disadvantages of using the current
model.
• Percentage of overrides can be tracked on monthly basis.
• Model parameters / thresholds need to be modified based on the overrides analysis.

11. 11
Case Studies – Financial Crime Analytics
• Tools
• Case Study 1: AML Transaction Monitoring
• Case Study 2: Customer Risk Rating (CRR) Calculation
• Case Study 3: Advanced Matching for Identifying Active and Dormant Frauds

12. 12
AML Tools
The Usual
Suspects
A Different
Way of Seeing
Mantas/Oracle AML
A Host of AML tools providing unique features are available in the market
Actimize SAS AML
Teradata/SQLTIBCO Spotfire LexiNexis
FortentFICO Falcon Norkom
Machine LearningAI
RPAQlikSense
CrowdSourcing
DataViz

13. 13
Case Study 1 – AML Transaction Monitoring (1/2)
Description
• Develop and validate the current standardized customer segmentation
model for a large bank holding company headquartered in the UK
• Develop a methodology to determine thresholds for each segment
• Develop and execute scenarios to generate alerts and transactions, by
tuning the parameters for thresholds
Execution Highlights
Segmentation Validation:
• Generalized Validation Framework
– Developed a suite for validating the segmentation model for
standardized segments and risks for different portfolios
– Gained an in-depth understanding of various input parameters for
different portfolios to derive common ‘plug-in’ solution for validation
tasks
• Automated Reports
– Produced automated validation reports for different portfolios
– Automated deep dives for root-cause analysis for segmentation
validation failures
– Provided scalable reporting aspects to monitor/include additional KPIs
Threshold Review & Tuning
• Determine relevant basis population for each scenario based on
– Transactional level scenarios
– Non-transactional level scenarios
• Derive and validate thresholds based on tunable parameters to ensure
alert rules for scenarios are met
• Have a QC process to verify the threshold bands selected and to verify
events and alerts generation
Portfolio II
Portfolio X Portfolio IV
Portfolio V
Portfolio iII Portfolio I
Automation
Parameterized
Standardization
CRISIL
Validation
Framework
Validation
Reports
Root Cause
Analysis

14. 14
Case Study 1 – AML Transaction Monitoring (2/2)
Threshold Tuning Approach
Scenario Tuning Approach
Scenario Testing & Tuning
• Scenario development/modifications based on the following
parameters (not extensive)
– Customer segment
– Geographies, channels and risk levels
– Line of banking business
– Amendable transaction patterns
• Scenario Validation to confirm
– Business requirements, specifications and transaction/alert counts
are met within the select scenario
– Coverage and redundancy aspects between scenarios
• Develop a threshold table for each scenario to analyze or modify
parameter threshold based on feedback
Client Impact
• The bank was able to meet the timelines of this validation
• The suite of tools developed could be reused (with little or no
modifications) for future validation exercises
• Risks associated with failed segments were quickly mitigated
• Coverage redundancy reduced due to improved segmentation and
scenario tuning
• False positives reduced by 5% due to threshold tuning
Thresholds
• Variation in parameters’
thresholds
Scenarios
• Tunable parameters
changed
Alerts
• Genuine
alerts
• False
positives
Region Channel
Risk
Business
Activity
Customer
Data
• Script development
• Script validation
• Default and below threshold analysis
Scenario
Development
• Accuracy
• False Positives
• Coverage Redundancy
QC & Tuning

15. 15
Case Study 2 - Customer Risk Rating (CRR) (1/5)
• Building of in-house name screening algorithms
• Tolerance setting and validation (feedback)
• Extension to residential and document level matches
• Development of automated compliance reports
CRISIL GR&A
also supports
CRR Calculation Process
• Identification of data sources and data extraction
• Reviewing the data quality
Data Collection
• Computation of individual factor scores
• Aggregation of account-level data to provide a holistic view of the customersCustomer
Aggregation
• Calculating the final CRR score as a weighted sum of individual dimension scores
• Defaulting the MSB, PEP and SAR customers as identified by the client to a high risk scoreCustomer
Analysis
• Data standardization to remove inconsistencies such as white spaces, dashes, variable date formats
• Normalization of key fields for scoring (e.g., Citizenship: Yes – US, No – Non-US)Data Standardi-
zation
• Calculation of the CRR factor scores at account level
Account Analysis

16. 16
Case Study 2 – Customer Risk Rating (CRR) Calculation
(2/5)
Approach & Methodology
• Customers should be categorized into risk-assessed groups titled
risk classes. Risk classes may be simple such as low, medium,
medium-high, high, and very high.
• Once a customer has been assigned a risk class, a specific
tolerance limit is applied to the expected incoming and outgoing
volumes for profile monitoring purposes.
• The tolerance limit is inversely related to the risk class; hence,
the higher the risk, the lower the tolerance.
– Low Risk - (LR) – 80% tolerance
– Medium Risk - (MR) – 50% tolerance
– High Risk - (HR) – 20% tolerance
Activities Performed in Risk Assessment Process
• Data loading and cleansing
• Standardization (including the application of fuzzy search)
• Computation of factor weights at the account level
• Roll-up the account level information to customer level
• Calculation of dimension weights and CRR weight at the
customer level
Background
• Determining the risk rating for each customer is a key
component of a financial institution’s Bank Secrecy
Act/Anti-Money Laundering (BSA/AML) program.
• Regulators expect financial institutions to have a holistic
understanding of each customer so that they can
appropriately monitor suspicious activities and report
them.
• By knowing each customer’s risk rating, banks can
decide on the appropriate level and frequency of due
diligence needed to manage that risk.
Business Objective
• Build a comprehensive risk assessment process using
data analytics techniques to:
– Carry out detailed risk assessment of business,
focusing on customer behavior, delivery channels,
etc.
– Design and put in place controls and systems to
manage and reduce the impact of risks

17. 17
CRR Calculation Process – Overview
Case Study 2 – Customer Risk Rating (CRR) Calculation
(3/5)
CRR Calculation Process
1. Data Collection
• The customer data was primarily provided by the client
• Onboarding/transaction data was collected from various source
systems
2. Data Standardization
• The data files contained missing values and multiple data types.
The following elements were standardized:
• Identification of data sources and data extraction
• Reviewing the data quality
Data Collection
• Computation of individual factor scores
• Aggregation of account-level data to provide a
holistic view of the customersCustomer
Aggregation
• Calculating the final CRR score as a weighted sum
of individual dimension scores
• Defaulting the MSB, PEP and SAR customers as
identified by the client to a high risk scoreCustomer
Analysis
• Data standardization to remove inconsistencies
such as white spaces, dashes, variable date
formats
• Normalization of key fields for scoring (e.g.,
Citizenship: Yes – US, No – Non-US)
Data Standardi-
zation
• Calculation of the CRR factor scores at account
level
Account Analysis
Citizenship
• Where citizenship was missing, primary country was checked for US, if
primary state was one of the 50 states and territories (DC, PR, VI),
citizenship was set to Yes.
Data consistency
• Product descriptions matched to the Client Enterprise Risk Assessment
(ERA) Product Risk Rating
• Country codes matched to ERA High Risk Country Codes
• Zip codes matched to ERA HIDTA/HIFCA ZIP Codes
• TIN/SSN, account numbers, client, and account names matched to PEP
and SAR lists
Search for keywords pertaining to
• High-risk nature of business
• Legal formation
• Product descriptions

18. 18
Case Study 2 – Customer Risk Rating (CRR) Calculation
(4/5)
3. Account Analysis
• CRR factor scores are calculated at account level
4. Customer Aggregation and Factor Scoring
• CRISIL identified key attributes for both business and
individual customers (“Factors”)
• Each factor is mapped to standardized set values.
– Accounts with US Citizenship - “Yes”
– Accounts with Non-US Citizenship - “No”
• The standardized set values are assigned a risk score
– Accounts with a Citizenship value “Yes” (US Citizens)
are given a score of 1 (low)
– Accounts with a Citizenship value “No” (Non-US
Citizens) are given a score of 5 (high)
• Factor scores are computed for all the factors for all the
accounts and are then rolled up to customer level
– When rolling up, the highest score for every factor is
selected as the customer’s factor score.
5. CRR Scoring and Customer Analysis
• A “Dimension” made up of multiple factors was given a certain
aggregation criteria
– Geography dimension for individuals is made up of the
maximum score for “Country of Primary Address”, “Country
of Mailing Address”, and “Country of Statement Address”
– Client dimension for businesses calculated using 60% of the
“Legal Form/Ownership” score and 40% of the “Nature of
Business” score
– “Money Service Businesses” (MSB), “Politically Exposed
Persons” (PEP) and customers identified in “Suspicious
Activity Reporting” (SAR) were defaulted to the highest risk
score (“5”)
• Every dimension was given a weight and the final CRR score was
calculated by combining the dimension weight with dimension
scores.
• The highest risk dimension scores are selected for a customer’s
underlying accounts
• PEP, SAR and MSB customers identified by CRISIL are defaulted
to a high risk score
Account A
Channel Score: 5
Account B
Channel Score: 1
Customer X
Channel Score: 5

19. 19
• Each factor has a set of pre-defined values which are assigned a risk score
– E.g., The possible values for Citizenship are YES (score of 1), NO (score of 5), and BLANK (score of 1)
• Each dimension is made up of single or multiple factor(s) grouped together
– E.g., Geography dimension is composed of
 Country of Primary Address
 Mailing Address
 The maximum score of those 2 factors will determine the Geography score
• CRR score is calculated as the weighted sum of various dimension scores
Sample CRR
Calculation
Case Study 2 – Customer Risk Rating (CRR) Calculation
(5/5)
Client Type Dimension
Dimension
Weight
Factor
Factor
Weight
Standardized Value
Factor
Score
Dimension
Score
CRR
Score
Individual
Client 25% Citizenship 100% NO 5 5
4.2
Geography 30%
Country of Primary Address
100%
HR 5
5 (Max of
Country
Scores)
Country of Mailing Address MR 3
Country of Statement Mailing MR 3
Product 25% Product 100% HR 5 5
Delivery Channel 10% Delivery Channel 100% FACE-TO-FACE 1 1
Length of Relationship 10% Length of Relationship 100% >5 YEARS 1 1
(Client Score of 5*25%) + (Geography Score of 5*30%) + (Product Score of 5*25%) +
(Delivery Channel Score of 1*10%) + (Length of Relationship Score of 1*10%)
generates a final score of 4.2``

20. 20
Case Study 3 – Advanced Matching for Identifying Active
and Dormant Frauds (1/2)
Approach & Methodology
• Extraction of confirmed fraud data for 24 months
• Chronological assessment of relationship between frauds
• Include SME, corporate and commercial accounts
• Understand key relationship parameters for successive fraudsBasis Creation
• Monitor the rules and provide feedback on capture rates
• Monitor false negatives to improve capture rates
Monitoring
• Adjust matching tolerance to control the number of alerts
generated
• Analyze false positive cases to build exclusion rules
• Update known fraud data for future runsFeedback
• Identify the key matching parameters between a pair of
customers
– Transaction volume, value and velocity
– Contact details such as address, phone, email (fuzzy matching)
– KYC related information (deterministic matching)
– For commercial account, additional information of ownership
Key Variables
• Determine suspects by using the above logic on latest fraud data
• High risk
– Satisfies all matching criterion
• Medium/Low Risk
– Satisfies one or two matching criterionRanking Alerts
Background
• A large BHC in the EU faced problems in dealing with
sleeper and dormant frauds
• Existing processes and controls were unable to detect
subtle changes in personal information
• Fraudsters were misrepresenting or giving misleading
information that appeared genuine
Business Objective
• Build a scalable framework using data analytics
techniques to
 Identify new and existing suspects based on their
KYC, transactional and personal/contact data
 Rank probability of suspects committing lending
fraud or AML-related offence
 Add new sources of known fraud/AML list
• Develop a mechanism to monitor performance and
recommend changes, if required

21. 21
Key Insights
Case Study 3 – Advanced Matching for Identifying Active
and Dormant Frauds (2/2)
Implementation Approach
• The higher the number of mappings, the higher is the priority
• Process run fortnightly to generate alerts
• False positive cases are added to exclusion
• Operational activity is client confidential
Client Impact
• Adjusted transactional matching process led to the closure of
~70% dormant/sleeper accounts
• Advanced matching algorithms helped in achieving a fraud
capture rate of ~55% for demographic and static matching
• Low volume and high quality leads reduced operational costs
Transactional Mapping
• Volume/value/velocity of transaction volatility
• Low transaction volume but high credit/debit
volume
• High probability of fraud; Cases few in number
Contact Mapping
• Fuzzy matching of contact details especially
address
• Commercial accounts checked for ownership
• Large number of cases are risk ranked
KYC Mapping
• Advanced matching of names, DoB, KYC etc
• Identifies shared/fabricated documents
• Risk ranking used as # of cases are large

22. 22
Key Takeaways
• Although Compliance tends to be Policy, Process and Procedure driven, getting
good results is “all about Data”.
• The Convergence of Risk and Compliance disciplines yields new analytical,
visual, and automated solutions to vexing problems.
• Applying a disciplined approach to Data Management and Analytics will get
you the most value for your compliance dollar