This document summarizes a presentation about assessing CECL models. It discusses that CECL requires estimating lifetime expected credit losses using multiple components and economic scenarios, which makes the models complex and outcomes difficult to assess. It emphasizes that back-testing, sensitivity analysis, and scenario analysis are important to evaluate whether the models perform reasonably under different economic conditions and assess the impact of assumptions. It also stresses the need to review outcomes and test assumptions at granular levels by risk drivers and perform qualitative adjustments through model risk management.

1. Current Expected Credit Loss (CECL)
Performance Assessment
October 24th, 2018
Xiaoling (Sean) Yu
SVP, Director of Model Validation
KeyBank
3rd Edition CECL 2018 Congress

2. 2
Disclaimer
The views expressed in this presentation are those of the
speaker and do not necessarily reflect the views of
KeyCorp or its subsidiaries in all respects.
3rd Edition CECL 2018 Congress

3. 3
A Complex Process
CECL is a complex process consisting of multiple components. And the principles-based
instead of prescriptive rule means many options to choose from and many decisions to
make.
3rd Edition CECL 2018 Congress
Output Reversion
PD (PIT)
LGD (PIT)
EAD (PIT)
Prepayment (PIT)
Economic
Scenario(s) (PIT)
R&S Period Reversion Historical Average
Reversion
Approach
PD (TTC)
LGD (TTC)
EAD (TTC)
Prepayment (TTC)
PD (PIT)
LGD (PIT)
EAD (PIT)
Prepayment (PIT)
Reversion
Approach
Economic
Scenario(s) (PIT)
Economic
Scenario (TTC)
R&S Period Reversion Historical Average
Input Reversion
Or

4. 4
Hard to Assess by Design
3rd Edition CECL 2018 Congress
PIT TTC
EL
UL
Econ.
Path
R&S
Y1 Y2 Y3 Y4 Y5 Y6 Y7
Reversion Historical Avg.
CECL estimates lifetime expected loss that reflects a hybrid view on cycle sensitivity, i.e.
point-in-time (PIT) for R&S period but through-the-cycle (TTC) for period beyond R&S, which
makes it hard to assess the appropriateness of the final outcomes.

5. 5
Components: Back-testing
• While focusing on ensuring proper conservativeness during stress periods for CCAR
models, back-testing for CECL models focuses on whether outcomes are reasonable
and unbiased through the cycle.
• Within R&S period, back-testing of credit loss models and prepayment assumptions are
conducted to understand the potential ranges of errors or any delays in capturing the
trends.
• If leveraging the input reversion approach, back-testing is conducted to assess whether
feeding TTC economic scenario into the PIT credit loss models and prepayment
assumptions will give reasonable outcomes (i.e. the non-linearity concern).
• When the input reversion approach is leveraged, should the PIT credit loss models and
prepayment assumptions be tested over the lifetime of the loans?
3rd Edition CECL 2018 Congress

6. 6
Components: Back-testing (cont’d)
• The Federal Open Market Committee
(FOMC)’s Summary of Economic
Projections (SEP) reports the root mean
squared errors (RMSEs) of real-time
forecasts over the past 20 years made by a
group of leading private and public sector
forecasters.
• Shaded bands show median SEP forecasts
(as of September 2016) ± average
historical RMSE at the appropriate forecast
horizon, which cover approximately 70%
future outcomes assuming future
prediction errors are stable, unbiased and
normally distributed.
Source: Reifschneider and Tulip (2017)
3rd Edition CECL 2018 Congress
• The accuracy of economic forecasting also need to be assess to understand the potential
range of errors, especially the ability to forecast the turning points.

7. 7
Components: Sensitivity and Scenario Analyses
• While focusing on assessing proper separation between different scenarios for CCAR
models, scenario analysis for CECL models should focus on whether the models will
make reasonable projections under different economic cycles.
• A key limitation of CECL (and CCAR) models is that most banks have just one or two
cycles of historical data. Good back-testing results can only ensure proper model
performance if the future will resemble the past.
3rd Edition CECL 2018 Congress
• However, every cycle is a bit unique.
Sensitivity and scenario analyses are
critical to assess the robustness of the
models and help establish the boundaries
of model performance, which provide the
foundation for proper ongoing monitoring
and model overlay/adjustment processes.
Statistics of US Business Cycles since 1945
Contraction Expansion
Peak to Trough
Previous trough
to
this peak
Trough from
Previous
Trough
Peak from
Pervious
Peak
Mean 11 60 71 71
STD 4 34 33 35
Min 6 12 28 18
Max 18 120 128 128
Data Source: https://www.nber.org/cycles.html
Cycle

8. 8
Putting Everything Together
• “Back-testing”: Estimating lifetime loss under actual economic scenarios (at different
points of the cycle) for R&S period, and reverting to historical average as designed
• Scenario Analyses: Estimating lifetime loss under hypothetic economic scenarios (at
different points of the cycle) for R&S period, and reverting to historical average as
designed
• Sensitivity Analyses of qualitative options/decisions:
o Impacts of different R&S Periods, Reversion Periods, and Reversion approaches
o Impacts of economic forecast errors
o Comparison between single or multiple economic forecast scenarios
3rd Edition CECL 2018 Congress

9. 9
Granular Level Assessments
• CECL outcomes need to be assessed at granular level:
o To support the disclosure requirements about credit risk by credit quality indicators
and years of the asset’s origination (i.e. vintage)
o To support attribution analyses of quarter over quarter (QoQ) variations by key risk
drivers (e.g. FICO, LTV, internal risk rating, industry, etc.)
• The aforementioned back-testing, sensitivity and scenario analyses need to be reviewed
at granular level by key risk drivers for any counterintuitive outcomes.
• Many modeling assumptions against data limitations (e.g. applying models developed on
legacy data to newly acquired portfolios) need to be assessed in a similar fashion.
3rd Edition CECL 2018 Congress

10. 10
Granularity vs. Stability
• In many cases, the demand on proper model outcomes at granular level and the need of
stable/robust models have to be balanced.
• For instance, geographic effects can be captured by modeling regional economic
variables directly or handled by introducing regional dummy variables.
3rd Edition CECL 2018 Congress
o Modeling regional economic variables
could lead to more QoQ variations, as
forecasts at regional level could be more
volatile than forecasts at national level.
o Leveraging dummy variables is based on
a strong assumption that the relationships
between regional and national economic
variables are relatively stable.

11. 11
Qualitative Adjustments
• Most of the current qualitative adjustment factors (e.g. changes in lending practices,
economic conditions, portfolio credit quality, etc.) are covered by the CECL models.
• Although not directly covered by the CECL models, the remaining factors (e.g. changes
in the ability of lending management, the quality of loan review system, and regulatory
requirements) all indirectly affect the fit-for-use of the models, and therefore should be
closely monitored from model risk management perspective.
• How to effectively integrate model risk management and ALLL for overseeing qualitative
adjustments, from governance and procedure perspectives, needs to be thought
through.
• The aforementioned performance assessments can help evaluate the needs and
prioritize the efforts.
3rd Edition CECL 2018 Congress

12. 12
THANK YOU
3rd Edition CECL 2018 Congress