This document discusses different types of models used to predict customer behavior and make business decisions. Generic models use broad customer data to predict outcomes, while custom models are tailored to a specific company's portfolio. The document compares the two approaches and provides an example of validating a custom model's ability to accurately predict which accounts will become delinquent. It also demonstrates how a model can be used to maximize profits by adjusting credit limits or requiring deposits based on risk scores.

1. 1 © Experian 19/09/2019
Sip and Solve
Choosing the right model:
custom or generic
Featuring:
Marsha Silverman
Lead Analytic Consultant
Experian Commercial Decision Sciences

2. 2 © Experian Sip and Solve: Choosing the right model
What is a model?
An engine or mathematical formula that distinguishes the relationships between “random
information” and an event.
• Used to guide a business area to make decisions at key points in the customer life cycle.
• Helps give the likelihood of particular ‘outcome’ for individual accounts or transaction.
• Based on the historic behavior of similar types of accounts.
• Examples of use cases:
• Will an account go delinquent within the next 12 months?
• What is the likelihood a customer will respond to a marketing offer?
• Who is more likely to pay its delinquent debt?

3. 3 © Experian
Modeling terminology
• Target variable: the event or behavior we try to predict
– Also referred to as the objective function, dependent variable, or outcome variable, event, Y
– Example: 90+ days past due in the next 12 months, response to offer or not
• Score: the estimation of the target variable as produced by the model
– Also referred to as Prediction, P hat, Output, Probability, Estimation, Y hat
– Example: Probability of 90+ days past due in the next 12 months, Probability of Response
• Predictors: variables that are expected to be somehow mathematically linked with the target variable
– Also referred to as Independent Variables, X’s, Inputs, Factors
– Examples:
▪ BizAggs
▪ SBCS Aggs
▪ SBFE Aggs
▪ Premiere Attributes
Sip and Solve: Choosing the right model

4. 4 © Experian
Analytic Spaces
Account
Acquisition
Account
Management
Account
Loss
• Front-End Risk
• Pre-Screen Risk
• Fraud
• Bankruptcy
• 1st Payment
Default
• Behavioral
• Line Management
• Roll Rate
• Delinquency
•Transactional
Fraud
• Early Stage
Collections
• Late Stage
Collections
• Recovery
• Response
• Activation
• Conversion
• Purchase
Propensity
• Profitability
• Usage
• Activation /
Reactivation
• Balance Transfer
• Cross-Sell
• Attrition / Churn
• Pre-Payment
• Retention
Risk
Management
Marketing
Efforts
Experian data can be used as predictive tools across the lifecycle of accounts for both Risk & Marketing
Sip and Solve: Choosing the right model

5. 5 © Experian
Custom or generic model?
• Generic models are more general, developed on a broader population than any
one specific portfolio
• Generic models may or may not be predictive of a specific portfolio
• A score validation can determine how well a generic score predicts behavior of a
specific portfolio
One size fits all Custom fitvs.
Sip and Solve: Choosing the right model

6. 6 © Experian
Model Validation Example
Starting population – Accounts opened in Q1 2018
Jan – Mar 2018
Observation period
Jan – Mar 2019
12-month rolling performance window from opening
Performance definitions
Bad
Businesses with C/O or
91+ delinquency
Good
Businesses without Bad
statuses
End of performance period
Accounts would be retro-scored as of the observation point to determine how well the score
predicts the actual behavior experienced during the performance window.
Sip and Solve: Choosing the right model

7. 7 © Experian
How do we know if a model “works”?
K-S or Kolmogorov-Smirnov measures performance of classification models.
• It is a measure of the degree of separation between the positive and negative distributions.
• In most classification models, the K-S will fall between 0 and 100, where the higher the value, the
better the model is at separating the positive from negative cases.
Cumulative%bads
Cumulative % goods
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Sip and Solve: Choosing the right model

8. 8 © Experian
How do we know if a score “works”? - Odds Chart
The FSR score performs well at accurately scoring future Bads in the worst score ranges.
Nearly 40% of all future bads are captured in the bottom 20%
KS = 24
FSR
score
Total Bads Goods
Bad rate
int
decum %
pop
Bad cap
decum
Good cap
decum
KS
97 - 100 1291 39 1252 3.0% 100.0% 100.0% 100.0% 0.00
90 - 96 1499 48 1451 3.2% 91.4% 96.8% 90.9% 0.06
79 - 89 1690 64 1626 3.8% 81.4% 92.9% 80.4% 0.13
69 - 78 1423 70 1353 4.9% 70.1% 87.6% 68.5% 0.19
42 - 68 1526 115 1411 7.5% 60.6% 81.9% 58.7% 0.23
26 - 41 1501 119 1382 7.9% 50.4% 72.5% 48.4% 0.24
15 - 25 1466 143 1323 9.8% 40.4% 62.8% 38.4% 0.24
9 - 14 1421 144 1277 10.1% 30.6% 51.1% 28.8% 0.22
7 - 8 1162 152 1010 13.1% 21.1% 39.3% 19.5% 0.20
1 - 6 1999 328 1671 16.4% 13.3% 26.8% 12.1% 0.15
14978 1222 13756 8.2%
Sip and Solve: Choosing the right model

9. 9 © Experian
Financial Stability Risk validation
maximizing profit
Maximum net profit of $2,988,655 is at FSR of 9+
This is $390k over the total population ($2,988,655 - $2,598,563)
For one year of applicants, the net profit gain is $1,170,000
[$390k * 3]
Profit can be further increased by assessing deposits for those
applicants scoring below 9
FSR
score
Total Bads Goods
Bad rate
int
decum %
pop
97 - 100 1291 39 1252 3.0% 100.0%
90 - 96 1499 48 1451 3.2% 91.4%
79 - 89 1690 64 1626 3.8% 81.4%
69 - 78 1423 70 1353 4.9% 70.1%
42 - 68 1526 115 1411 7.5% 60.6%
26 - 41 1501 119 1382 7.9% 50.4%
15 - 25 1466 143 1323 9.8% 40.4%
9 - 14 1421 144 1277 10.1% 30.6%
7 - 8 1162 152 1010 13.1% 21.1%
1 - 6 1999 328 1671 16.4% 13.3%
14978 1222 13756 8.2%
profit margin on goods 15%
co rate 20%
Total good
balance
Total bad
balance
CO balance
Annual
profit est
Est loss
Net profit
cum.
3,174,489$ 148,235$ 10,025$ 493,962$ 39,672$ 454,290$
3,892,563$ 237,564$ 38,541$ 612,392$ 86,054$ 980,628$
4,005,358$ 241,308$ 40,530$ 629,761$ 88,792$ 1,521,597$
4,812,652$ 342,173$ 48,532$ 762,959$ 116,967$ 2,167,589$
3,491,868$ 1,018,134$ 142,930$ 645,956$ 346,557$ 2,466,988$
3,402,776$ 1,267,648$ 212,735$ 662,534$ 466,265$ 2,663,258$
3,307,563$ 1,110,626$ 178,008$ 629,410$ 400,133$ 2,892,534$
2,885,293$ 1,526,637$ 214,542$ 615,990$ 519,869$ 2,988,655$
2,322,419$ 1,790,791$ 233,032$ 563,258$ 591,190$ 2,960,723$
4,171,432$ 4,451,173$ 631,781$ 1,159,856$ 1,522,016$ 2,598,563$
35,466,413$ 12,134,289$ 1,750,656$ 6,776,077$
Sip and Solve: Choosing the right model

10. 10 © Experian 19/09/2019
Thank you!
Have questions or want to learn more about choosing the right model?
• Chat with us: bit.ly/sip-solve-chat
Upcoming Sip and Solves -
• Join the list to stay in the loop: bit.ly/sip-and-solve