2. The Need for Model Management and Governance
Federal guidance, spurred by the great recession, enhanced IT capabilities, the need for greater
security, an increased acceptance of financial theory and the overall pursuit of best practice have all
elevated the level of detail and precision which financial modeling in the business world has pursued.
Even in corporate environments outside of financial services and federal regulation, companies often
desire the proper tools and knowledge to confirm their decision-making models are accurate and
appropriate. This presentation provides a pathway towards those needs and introduces two model
management objectives:
1. An understanding of modern model governance standards and management,
2. Knowledge of proper validation and monitoring techniques.
Rob Trippe, MBA
Corporate Financial Model Management
3. Definition of a Financial Model
Rob Trippe, MBA
Federal Reserve issued guidance to banks (SR 11-7 among other documents) defined a model for the purpose of
model risk management after the wake of the great recession in 2008. It provides a framework for best in class
financial model development, validation and monitoring.
SR 11-7 outlines six criteria a financial tool must possess to be a model.
1. The analysis must employ academic theory
2. Three components: inputs-calculation processes-outputs
3. The analysis must transform data into useful business information
4. Must be used repetitively
5. Output must be quantitative in nature
6. Subject matter expertise is required
7. Possible qualitative model inputs or output
Data Calculation Process Output
4. Definition of a Financial Model
Rob Trippe, MBA
Models are not limited to one programming language environment or application. The term “model” should be
viewed “wing to wing” from initial data inputs to final model output. Numerous programming environments may
be involved as with numerous data sources. “Downstream” models should be identified.
5. What Are Corporate Finance Models Used For?
Rob Trippe, MBA
Financial models inform decision makers and also assist in complying with regulatory requirement.
The three primary corporate finance activities are:
1. Investment Decisions - estimating and comparing cash flows, risks and returns to requirements and
expectations.
2. Financing Decisions – determining capital structure; what type of instruments will we use to finance the
assets, what are the terms of this financing (e.g. length, terms) and to what degree can each asset be
levered?
3. Dividend Policy - should we invest for growth and replacement or should we dividend cash flows to
investors and what are investor’s expectations?
6. Influencing Model Factors
Rob Trippe, MBA
Models improve in integrity and effectiveness as the result of several guiding influences.
Proper Theory and Statement Analysis Model Governance and SR 11-7
Industry Rules such as FAST for Excel
7. Types of Models
Rob Trippe, MBA
Deterministic models use assumptions (independent variables) which are believed to be “known”, that is they
are static. Run a deterministic model ten times and you will get the same result.
%
Inflation Assumption Through Time
Stochastic models use assumptions (independent variables) which are only believed to be “known”, within a set
of parameters or “boundaries”. Run a stochastic model ten times and you will get ten different results.
Dependent variable output therefore has a random element to it. Models are often then run using Monte Carlo
simulation for numerous results and an average of these results is used as the final model output.
%
Inflation Assumption Through Time
8. Other Model Classifications
Rob Trippe, MBA
Extrapolative Modeling utilizes prior period results in time series fashion, year over year growth, for example.
Index Modeling utilizes outside data to estimate future internal results.
Disaggregated Modeling uses disparate data (line items on financial information, for example) to
estimate future outcomes.
Extrapolative Indexed Disaggregated
Deterministic
Stochastic
9. The Model Lifecycle
Rob Trippe, MBA
Models follow a what is called the “model lifecycle”. The model lifecycle has four components. In order, they are:
Development Validation Implementation
Ongoing Monitoring
10. Policy and procedure formalizes risk management activities for implementation. SR
11-7 recommends an emphasis be placed on testing and analysis with a key goal of
promoting accuracy. These roles and responsibilities can be divided among
ownership, control and compliance.
Strong governance coordinates processes and model output across functional areas
and validates final model output. It provides a venue for sharing ideas across areas
and business units.
Introduction to Model Governance
Rob Trippe, MBA
11. Model Governance
Rob Trippe, MBA
Model Governance concerns itself with:
A. Identifying models
B. Developing internal policy and procedure
C. Developing and enforcing model standards
D. Interpreting and abiding by regulatory guidance and rule
E. Researching and identifying best practice
F. Ensuring proper documentation
G. Assisting in tracking models and ensuring proper remediation when required
H. Overseeing model testing
I. Risk management
J. Change management
13. Model Acceptance
Rob Trippe, MBA
Models are hard work. But, in the end, are they accepted both in build and results?
Here are some key thoughts to remember:
• Does your model comply to accepted model development or are they simply proprietary? (FAST standards)
• Was the model developed with “management buy-in?”
• Do key model users buy into the theory behind the model? If not, should the model attempt to better
support the theory driving model calculations?
• Does model development theory still hold?
• Does model output “marry” to downstream use? If not, how can this be improved?
• What approval processes are in place for the ongoing use and approval of the model?
• Does the data input and model final output remain relevant?
14. Model Definitions
Rob Trippe, MBA
1 Back-test Use of historic data as a test to model output validity.
2 Benchmark The comparison of model output to the output of an outside and
independent source.
3 Emerging Risk Unforeseeable risk arising further in time and model execution.
4 FAST Set of rules for financial model design. Flexible, Appropriate,
Structured and Transparent.
5 Impact Analysis Assessment of cost, timing, scope and quality of a model-
consequence.
6 In-Sample Historical data used in model development.
7 Model Quantitative method, system, or approach that applies statistical,
economic, financial, or mathematical theories, techniques, and
assumptions to process input data into quantitative estimates.
8 Out of Sample Historical data not used in model development.
9 Outcomes Analysis The comparison of model output to actual outcomes. Back-testing is
one example.
10 Parameter Numerical characteristic of a set or population of numbers.
Here are some great definitions to know:
15. Model Definitions
Rob Trippe, MBA
11 Calibration Adjustment of data and assumptions.
12 Residual Risk Remaining risk after a risk mitigation action has been performed.
13 Risk Appetite Largest tolerable degree of uncertainty acceptable.
14 Scenario Multiple changes to inputs to reflect a given set of circumstances.
15 Secondary Risk Risk arising from a risk response.
16 Sensitivity Impact of a change to an input relative to the change in output.
17 Stress Test Assessment of model stability by employing hypothetical data inputs
or drivers.
18 Threshold Measure of uncertainty or impact worthy of attention.
19 Tolerance Degree of deviation within which a model still functions properly.
20 Validation A set of processes and activities intended to verify that a model
performs as intended and as expected.
16. Model Standards
Rob Trippe, MBA
Here is one set of standard topics for non-regulated corporate finance models. These ensure ownership,
improves accuracy and reduces key person dependence. Regulatory requirement may come into play as
well for certain financial institutions which ups the ante.
1. Executive Summary 2. Model Development 3. Operating Procedure 4. Ongoing Monitoring and Governance
Model Use Theory and Approach Model Operation Environment Change Management
Model Theory and Calculation Data and Assumptions Model Owners and Users Risk Management Oversight - 2nd Line of
Defense
Model Resources Language Application Code and Calculations Model Recalibration Audit – 3rd Line of Defense
Model Inputs Model Output User Process - Inputs and Assumptions
Model Output Risk and Control Points Risks and Controls – 1st Line of Defense
Impact to Downstream Models and Analysis Model Weaknesses and Limitations Model Output
Model Limitations Technical Specifications
Testing and Validation
17. Model Participants
Rob Trippe, MBA
There are numerous players and business units involved through the model lifecycle. They include:
Model Developers
Model Owners
Model Users
Model Testers
Model Validators
Those who develop the
model using systems,
theories, formulas and data.
Those responsible for model
use in “real time”.
Those who implement the
model and its output.
Usually IT, who test model
development results and
manage model change.
Those who confirm a model
is working as intended,
often using mathematical
approaches.
Business Units
Model Governance
Risk Management
Compliance
Audit
Business Units
implementing the model
and utilizing the model
output.
Those who see that models
are tracked, implementing
monitored, inventoried and
comply to standards.
Those who monitor and
manage the risk element of
a business’s activities.
Those who confirm
activities meet regulatory
requirements.
A last internal line of
defense.
18. Model Risk Management
Rob Trippe, MBA
Virtually each of the activities mentioned in this presentation relate to development and subsequent
model risk management.
Model risk refers to the chance of unintended consequences resulting from model development, inputs or
outputs.
Risk Activities
• Development
• Implementation
• Use
• Change
• Retirement
• Validation
• Review
Key Risk Definitions
• Appetite
• Thresholds
• Tolerance
• Secondary Risk
• Operational Risk
• Risk and Control Points
• Emergent Risk
• Ongoing Monitoring
19. Good documentation is a key component to model risk management and should be viewed as such and not
simply a compliance and requirement exercise.
Advisory bulletin 2013-07 supports this assertion and adds that documents should be the responsibility of the
model developer (development) and owner (operating procedures, ongoing monitoring), thereby providing a first
line of defense and control between model developers, users and model owners. Proper documentation is
cumbersome and takes time, but it is critical to a model’s overall success by reducing risks and costs. Benefits
include:
Reduced key person dependence
Mitigated transitional challenges as models pass from user to user and output need to output need as
conditions and requirements change
A more streamlined validation and audit process
Reduction in potential user error
Avoidance of potential misuse
Effective communication of challenging theoretical concepts – “comfort”
Model Documentation
Rob Trippe, MBA
20. Model Documentation
Rob Trippe, MBA
The documentation of financial models is a critical element of model risk management.
It behooves developers, users and owners to spend the time and effort to properly document a model through
its life cycle. Documentation, though time consuming and cumbersome, aids in validation, monitoring and
audit and provides a platform for model developers to effectively communicate their model and its conceptual
soundness to all areas of an institution, from data downloaders to executive management. Federal agencies
will look favorably upon models which have comprehensive and well-conceived documents to support their
models.
21. Model Flow Charting
Rob Trippe, MBA
Flow charts are powerful tools. Flow charts can come in many forms and there is no one exact manner to flow a
model. From my experience, two flow charts stand out as invaluable;
1. System flow chart “wing to wing” (input to output)
2. Development flow chart
A system flow chart will show, left to right, a model’s data inputs and IT/business unit environments, calculation
processes, and model output and IT/business unit environments.
A development flow chart will visualize the exact mathematical methodologies and techniques employed to
transform input data into useful business information and will generally focus on only one business environment.
22. Model Flow Charting
Rob Trippe, MBA
Flow charting financial statement builds provide the user a quick gauge on what data is at hand versus what
is needed to complete a model. A flow chart will help identify the required builds and environments from
which data will be acquired. Flow charting will also help visualize core components within and across models
and are an ideal medium for identifying and communicating risk and control points. Flow charts will
dramatically improve model buy-in and provide a path for solid structure. Dead ends, duplicity and
unmanageable audit trails now become visual.
23. Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Numerous organizations have developed modeling rules for the use of Excel. These rules provide
structure and commonality among spreadsheet design and use. Many rules and guidelines can apply to
other languages as well.
Excel is an application and not a language. And it is only one of many possible languages and applications
which may be incorporated into a model. Due to its wide-spread use and applicability to recent college
graduates, this section focuses on best practice for Excel.
Excel Standards Organizations
• FAST (Flexible, Appropriate, Structured, Transparent)
• Smart Financial Modeling
• BPM (Best Practice Modeling)
Common Rule Categories
• Workbook Design
• Worksheet Design
• Line Item Design
• Cell Design and Use
• Assumption Design and Use
• Use of Features
• Formatting Rules
24. Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Saving and Printing
Use naming protocol
Use saving protocol
Print format for all pages
Print then proof, never off the screen only
Color
Use color protocol
Use color sparingly
Inputs are blue
Calcs are black
External links are green
Trouble spots are red
The purpose of these rules is to provide clarity, accuracy, safety and a clear audit trail. Only create work
which you would gladly show on a blackboard in a court of law.
25. Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Brevity
Keep formulas short
Assumptions shown on same page as calc
Keep audit trails short
Avoid daisy chains
Never replicate math, except as proofs
Avoid external links, use exported tabs instead
Highlight external links when required to use
Avoid circular references when possible
When creating a model, you are telling a
story. As the FAST standards say, you are
really writing sentences, paragraphs and
chapters. That approach will help get your
model “read” and more easily accepted.
Models are not political tools.
26. Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Rows, Columns and Tabs
One column, one purpose
One row, one purpose
No line or column breaks between calculations
Avoid merging cells
Forecast periods move left to right
Statements move top to bottom
No breaks between rolling forecast periods
Don’t hide rows or columns
Tabs flow left to right
Use summary sheets, clean of build ups
Formulas, Calculations and Functions
Use statement accounting sign convention as you would read
from a 10K
Subtotals presented below
Use balance and sanity checks
Show all math
Anchor repetitive calcs left to right and top to bottom
Macros should be used sparingly
Avoid named ranges
Create dedicated calculation areas
Anchor “drivers”
27. Thought to form and structure is critical for all models. From experience, we know that such thought
is simply not always the case. All financial models follow the same logical order. They:
1. Compile data
2. Adjust and conform data as required
3. Transform data through calculation
4. Present final model output such as a value, a table or a forecast
Use this commonality to develop discipline in how models are built and structured among and
across users and functional areas. Create an intuitive and easy to follow work flow, such as using
tabs left-to-right in Excel. Models in MatLab code can leverage replicable building blocks.
Model Form and Structure
Rob Trippe, MBA
29. Model Validation Techniques
Rob Trippe, MBA
Validation of model results can be performed several ways:
Sensitivity Analysis – Vary inputs and assumptions to compare against other input and assumption
changes to see if a proper correlation exists in model output.
Back-testing – Use of historic data as a test to model output validity. This is also a great way to develop a
model. Simply build your model using historic data and solve for a known answer.
Benchmarking – As previously discussed, this verifies your model’s output by comparing it to another source,
either internal or external.
Scenario Analysis – Run your model using various inputs and assumptions to test for model
integrity and reasonableness.
Use Test – Has the model stood the test of time?
30. Back-Testing
Rob Trippe, MBA
Assuming we use a three year average to forecast accounts receivable for the year 2018. Is
our result consistent with out of sample actuals at a tolerable level? Here, it appears so.
Out of Sample In Sample Forecast
2010 2011 2012 2013 2014 2015 2016 2017 2018
Revenue $1,000 $1,050 $1,100 $1,075 $1,094 $1,200 $1,250 $1,275 $1,350
Accounts Receivable $75 $68 $89 $78 $78 $90 $91 $101 $102
A/R % Revenue 7.5% 6.5% 8.1% 7.3% 7.1% 7.5% 7.3% 7.9% 7.6% (3 Year Average)
Trailing 3 Year Average 7.4% 7.3%
Variance to Actual 0.1% 0.1%
% Variance 1.4% 2.0%
31. Validation Benchmarks
Rob Trippe, MBA
• Prior performance
• Expectation
• Competitor results
• Industry standards
• Analyst forecasts
• Internal consensus
Model output can be compared to numerous benchmarks. They include:
32. Sensitivity Analysis
Rob Trippe, MBA
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10
Line Chart
Rise over Run = 1
$-
$10.00
$20.00
$30.00
$40.00
$50.00
$60.00
$70.00
$80.00
$90.00
1 2 3 4 5 6 7 8 9 10
NPV - Parabolic
Discount Rtae = 20%
For deterministic models, simple
line charts flush out model output
inconsistencies. There is no
“scatter” around a fitted line,
since deterministic charts have no
random variable.
33. Common Validation Ratios
Rob Trippe, MBA
Profitability Ratios
EBIT Margin EBIT / Revenue
Net Margin Net Income / Revenue
NCF % Revenue Net Cash Flow / Revenue
NCF % of Net Income Net Cash Flow / Net Income
Return on Assets Net Income / [[Total Assets Beg. Balance + Total Assets Ending Bal.]/2]
Return on Equity Net Income / [[Equity Beg. Balance + Equity Ending Bal.]/2]
Growth Ratios
Revenue Growth [[Revenue2 / Revenue1] -1]
Earnings Growth (EBIT) [[EBIT2 / EBIT1] -1]
Earnings Growth (Net Income) [[Net Income2 / Net Income1] -1]
These ratios provide sanity checks for validation. Is model output reasonable and does it follow accepted
patterns? Ratios not only reflect the financial position of a firm but may also reflect a model’s calculation
abilities and errors.
34. Common Validation Ratios
Rob Trippe, MBA
Expense Ratios
Operating Expense % of Revenue Operating Expenses / Revenue
Turnover Ratios
Receivables Turnover Revenue / [[A/R Beg. Balance + A/R Ending Bal.]/2]
Fixed Asset Turnover Revenue /[ [Fixed Asset Beg. Balance + Fixed Asset Ending Bal.]/2]
Total Asset Turnover Revenue / [[Total Assets Beg. Balance + Total Assets Ending Bal.]/2]
Risk / Leverage Ratios
Current Ratio Current Assets / Current Liabilities
Debt / Equity Total Interest Bearing Debt / Total Equity
Interest Coverage EBIT / Interest Expense
These metrics may also help verify inputs and assumptions and also validate a model’s results:
35. Validation Trends
Rob Trippe, MBA
Through a company’s life cycle, there are several common types of model pattern output depending on
assumptions and their effect on earnings and/or cash flow growth. Look for irregularities (sudden drops or
steep inclines) and see if your model follows traditional forecast paths.
g
t
g
t
g
t
No Growth Three Phase Three Phase - Declining
36. Ongoing Monitoring
Rob Trippe, MBA
Development Validation Implementation
Ongoing Monitoring
Ongoing monitoring plays a vital role in model integrity. Are models bask-tested annually? What business conditions
and/or data sources have changed? What external factors may have changed affecting the model’s relevancy? Is the
regulatory environment adopting new rules and standards?
37. Helpful Resources
Rob Trippe, MBA
Here are links to model documentation and other guidance:
SR 11-7
https://www.federalreserve.gov/bankinforeg/srletters/sr1107.htm
FHFA AB 2013-07
https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/AB-2013-07-Model-Risk-Management Guidance.aspx
FHFA AB 2009-03
https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/Rescinded-Advisory-Bulletins.aspx
OCC 2011-2012
https://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12.html
FAST modeling standards:
http://www.fast-standard.org/
The Global Association of Risk Professional has numerous articles on model risk and validation:
https://www.garp.org