2. AGENDA
Transactions landscape
Transactions data
Problems to tackle
Transactions analytics
Types of techniques
Performance measurement
Target definition
Fraud
Credit risk
Attrition
Deploying the solution
Using the scores in production
Monitoring the production system
2
4. TRANSACTIONS DATA
Credit/Debit cards
Authorisations
Payments
Statements
Non-monetary data
Bureau data
Demographic data
Campaign data
Clickstream data
Wire transfers
Financial transactions
4
5. PROBLEMS TO TACKLE
First party fraud
Second party fraud
Third party fraud
Credit risk, bankruptcy
Product offers, pricing
Money laundering
Financial trading violations
Bio-terrorism
Intrusion detection
5
8. FIRST PARTY FRAUD
Committed on own account
Victimless fraud
Examples
Fictitious identities
Check kiting
Bust out fraud
Tax under-filing
8
9. SECOND PARTY FRAUD
Committed by someone known to or close to
genuine account holder
Examples
Employee abuse of corporate cards
Relatives abusing cards/data of children, siblings,
parents
Caregivers abusing cards/data of senior citizens
9
10. THIRD PARTY FRAUD
Committed on some one else’s account
Impersonation of genuine identity
Examples
Identity theft
Lost/stolen cards/accounts
Stolen data/account information
Online fraud with infected PCs
10
13. US CARD FRAUD LOSSES
13
Source: Kansas City Federal Reserve
14. CARD FRAUD LOSSES FOR SELECT COUNTRIES
14
Source: Kansas City Federal Reserve
15. CREDIT RISK
Existing accounts
Serious delinquency
Bankruptcy
Charge-off
New accounts
Delinquency in first six months
Bankruptcy in first six months
Charge-off in first six months
15
24. TYPICAL RULE BASED SYSTEM
Pros
Easy to understand
Can be a batch or automated system
Effective in catching the obvious cases
Cons
Too many false-positives
Likely to miss many risky cases
Does not provide priority for investigation
Difficult to maintain
24
25. RULES FOR MEASURING SUCCESS
All
”goods”
and
“bads”
unknown
All
”goods”
and
“bads”
known
25
26. PERFORMANCE MEASURES
How good is the score at separating the two classes of goods and
bads?
Information value
Kolmogorov–Smirnov statistic
Lift curve
ROC curve
Gini coefficient
Somer’s D-concordance statistic
How good is the score as a probability forecast?
Binomial and Normal tests
Hosmer-Lemeshow test
How good is the score and cut-offs in business decisions?
Error rates
Swap set analysis
26
30. LIFT CURVE
Plots percentage bads rejected versus percentage rejected
Ideal score given by ABC where B represents population
bad rate
Random score represented by AC
Accuracy ratio AR=2(Area of curve above diagonal)/Area of
ABC
30
31. ROC CURVE
ABC represents ideal score
Diagonal represents random score
Gini coefficient (GC) measures twice
the ratio of area between curve and
diagonal to area ABC
GC=1 corresponds to perfect score
GC=0 represents random score
Somer’s D-concordance (SD)
If “good” and “bad” chosen at
random, good will have lower
score/probability of being bad than
bad
AUROC is area under ROC curve
GC=2AUROC-1=SD
31
32. BINOMIAL TEST
Checks if predicted bad rate in a given bin i is
correct versus underestimated
Let there be bads in the observations of
bin i and the probability of a borrower in that
band being good
The predicted bad rate in bin i is correct if it the
number of bads k in bin i is less than or equal
to
32
33. NORMAL TEST
Approximation of Binomial
The predicted bad rate in bin i is correct if it the
number of bads k in bin i is less than
Where is the inverse of the cumulative
normal distribution
33
34. HOSMER-LEMESHOW TEST
Assess whether observed bad rates match
expected bad rates in each of ten bins
A chi-square test statistic
Let
34
37. ERROR RATES
Account False Positive Ratio (AFPR): The ratio of good to bad
accounts for a given cut-off score
A ratio of 10:1 would indicate that out of 11 accounts, 1 is bad, 10 are
good
Account Detection Rate (ADR): The ratio of bad accounts to the total
number of bad accounts for the period at a given cut-off score.
If there are 100 bad accounts in the time period and 30 of them are at
or above the cut-off score at some time during the period, the ADR is
30%
Value Detection Rate (VDR): Percentage of dollars saved on detected
bad accounts for a given cut-off score
Assuming that the total losses on all accounts are $1,000,000 and that
$600,000 of these are saved by the system, the VDR would,
consequently, be equal to 60%
37
38. SWAP SET ANALYSIS
Used to compare two competing scores
Choose top x% accounts using score1 and
score2
Eliminate the common bads and goods
Compare the two data sets to identify bads
caught by score1 but not score 2 and vice
versa
Score1 is better than score2 if it has a higher
bad rate in the swap set
38
39. TARGET DEFINITION: CARD FRAUD
39
Pre-fraud Fraud window Post Block
Date/time of
first fraudulent
transactions
Block date/time
All transactions are
declined / blocked
Fraud activities has not been detected or confirmed yet. The approved fraudulent
transactions during this window are the fraud losses. Legitimate transactions could
exist in this period. (For the fraud case with no loss, there is no fraud window.)
All transactions are
legitimate
40. TARGET DEFINITION: CREDIT RISK
Bad: Account becomes at any time during the outcome
window
3+ cycles delinquent
Bankrupt
Charged-off
Indeterminate accounts
Maximum of 2 cycles delinquent in the outcome window
Fraud or Transfer status in the outcome window
Inactive accounts
Indeterminate accounts will be excluded from off-sample
validation and off-time validation
Other accounts are Good
40
41. 41
TARGET DEFINITION: ATTRITION RISK
Account closure
Many banks/vendors use this to define “Bad” accounts
Silent attrition
Many banks/vendors use this to define “Bad” accounts
Silent attrition defined as a sharp and lasting drop in economic value
(balance and activity) of accounts that were valuable in prior periods
Many banks/vendors refine this definition to exclude accounts that have
other reasons for change in economic value of account
Many banks/vendors use both to define “Bad” accounts
All other non-fraudulent active current accounts are classified
as “Good” accounts
42. Using the scores in production
Monitoring the production system
42
DEPLOYING THE SOLUTION
43. 43
SCORE USES
Typical use of scores is in strategies to manage decisions
concerning:
Whether to approve/decline authorizations
Whether to approve/decline over-limit requests
Actions to make delinquent accounts current
Increase/decrease credit limits
Whether to reissue credit cards
Collections related actions
Credit risk score is the most frequently used score for above
strategies. Some banks also use attrition, revenue and profit
scores
Scores also used in other strategies such as retention, balance
transfer, balance build, convenience checks, and cross-sell/up-
sell optimization
Fraud scores are used for approve/decline/refer decisions
45. WHY DO BOTH RULES AND SCORING?
Rules allow the input of client specific intellectual property and operation
constraints
Rules allow tracking and adjustments for new or short term risk patterns
Models pick up non-obvious risk patterns and behaviors
Output from advanced models easy to translate into probability & log odds
scores
Scores can be used very easily to rank order entities
The combination of rules and scores provides better detection rate and
better quality referral
Business implication - with the same amount of resource,
Catching more risk activity
Catching them earlier
Faster way to get a good ROI
45
49. CREDIT LIMIT STRATEGY
Risk Score Low Medium High
Credit Limit Utilization Low High Low High
Delinquency Status Clean Dirty Clean Dirty Clean Dirty Clean Dirty
Credit Line Inc. 0 500 0 1000 500 1500 1000 5000 2500
Implemented in the form of decision trees/strategies
Champion/Challenger framework for improving strategies over time
Randomly assign accounts to champion or challenger strategy
Measure performance over time
Takes a six to twelve months to evaluate each challenger strategy
A very small number of potential champion strategies can be tested at a
given time
Difficult to analyze why a particular challenger strategy worked
49
50. EXPANDING BEYOND THE “COMFORT ZONE”
Risk Score Low Medium High
Credit Limit Utilization Low High Low High
Delinquency Status Clean Dirty Clean Dirty Clean Dirty Clean Dirty
Champion Credit Line Inc. 0 500 0 1000 500 1500 1000 5000 2500
Test Group 1 0 0 0 500 0 500 0 2500 1000
Test Group 2 0 0 0 500 0 1500 0 3000 1500
Test Group 3 0 0 0 1500 0 2000 1500 4000 2000
Test Group 4 500 1000 500 2500 1000 3000 2000 7000 3000
Test Group 5 500 1500 1000 3000 1500 4000 2500 8000 4000
Test Group 6 500 2000 1500 4000 2500 5000 3000 9000 5000
50
51. NON-LINEAR PROGRAMMING EXAMPLE (A)
Credit limit increases are a
continuous variable
Randomly choose a small
number of accounts for
optimization
Use Lagrangian relaxation
techniques
Adding more constraints can
make solution more difficult
Map optimal solution to a
decision tree to score all
accounts
Deploying decision tree in
lieu of solution can result in
significant loss in benefit of
the whole effort
51
52. LINEAR PROGRAMMING EXAMPLE I (B)
Only discrete credit limit
increases allowed
Subset of LP problem has
integer solutions most of the
time
Account level optimization
possible
Solve relaxed LP problem
and check feasibility for
remaining constraints
No need to map optimal
solution to a score
52
53. MONITORING THE SYSTEM
Monitoring the scoring system
Stability index of score
Stability index of input fields
Remedies for score deterioration
Monitoring the portfolio
Population stability report
Characteristic analysis report
Final score report
Delinquency distribution list
Roll rates
Vintage analysis
Reports by portfolio segments, risky segments
53
55. REMEDIES FOR SCORE DETERIORATION
Score shelf life depends upon the problem
Fraud scores have lower shelf life because fraudsters constantly change
techniques
Credit scores have longer shelf life because causes do not change much
over time
Remedies
Recalibrate the score
Least expensive, easiest to implement
A table mapping the old score to a new score
Retrain the model
More expensive, straightforward to implement
Keep same variables, simply change the weights/coefficients
Rebuild the model
Most expensive, needs the full implementation cycle
New models with new variables and new weights/coefficients
55
56. QUARTERLY REPORTS
Population stability report
Measures change in score distribution over time
Characteristic analysis report
Measures changes in individual input fields over time
Final score report
Measures how closely the score is used in production
E.g., show number of accepts and rejects by application score
band
Delinquency distribution report
Measures the portfolio quality by different score ranges
56
