SlideShare a Scribd company logo
1 of 38
Introduction to Model Management
for
Traditional FP&A
Rob Trippe, MBA
The Need for Model Management and Governance
Federal guidance, spurred by the great recession, enhanced IT capabilities, the need for greater
security, an increased acceptance of financial theory and the overall pursuit of best practice have all
elevated the level of detail and precision which financial modeling in the business world has pursued.
Even in corporate environments outside of financial services and federal regulation, companies often
desire the proper tools and knowledge to confirm their decision-making models are accurate and
appropriate. This presentation provides a pathway towards those needs and introduces two model
management objectives:
1. An understanding of modern model governance standards and management,
2. Knowledge of proper validation and monitoring techniques.
Rob Trippe, MBA
Corporate Financial Model Management
Definition of a Financial Model
Rob Trippe, MBA
Federal Reserve issued guidance to banks (SR 11-7 among other documents) defined a model for the purpose of
model risk management after the wake of the great recession in 2008. It provides a framework for best in class
financial model development, validation and monitoring.
SR 11-7 outlines six criteria a financial tool must possess to be a model.
1. The analysis must employ academic theory
2. Three components: inputs-calculation processes-outputs
3. The analysis must transform data into useful business information
4. Must be used repetitively
5. Output must be quantitative in nature
6. Subject matter expertise is required
7. Possible qualitative model inputs or output
Data Calculation Process Output
Definition of a Financial Model
Rob Trippe, MBA
Models are not limited to one programming language environment or application. The term “model” should be
viewed “wing to wing” from initial data inputs to final model output. Numerous programming environments may
be involved as with numerous data sources. “Downstream” models should be identified.
What Are Corporate Finance Models Used For?
Rob Trippe, MBA
Financial models inform decision makers and also assist in complying with regulatory requirement.
The three primary corporate finance activities are:
1. Investment Decisions - estimating and comparing cash flows, risks and returns to requirements and
expectations.
2. Financing Decisions – determining capital structure; what type of instruments will we use to finance the
assets, what are the terms of this financing (e.g. length, terms) and to what degree can each asset be
levered?
3. Dividend Policy - should we invest for growth and replacement or should we dividend cash flows to
investors and what are investor’s expectations?
Influencing Model Factors
Rob Trippe, MBA
Models improve in integrity and effectiveness as the result of several guiding influences.
Proper Theory and Statement Analysis Model Governance and SR 11-7
Industry Rules such as FAST for Excel
Types of Models
Rob Trippe, MBA
Deterministic models use assumptions (independent variables) which are believed to be “known”, that is they
are static. Run a deterministic model ten times and you will get the same result.
%
Inflation Assumption Through Time
Stochastic models use assumptions (independent variables) which are only believed to be “known”, within a set
of parameters or “boundaries”. Run a stochastic model ten times and you will get ten different results.
Dependent variable output therefore has a random element to it. Models are often then run using Monte Carlo
simulation for numerous results and an average of these results is used as the final model output.
%
Inflation Assumption Through Time
Other Model Classifications
Rob Trippe, MBA
Extrapolative Modeling utilizes prior period results in time series fashion, year over year growth, for example.
Index Modeling utilizes outside data to estimate future internal results.
Disaggregated Modeling uses disparate data (line items on financial information, for example) to
estimate future outcomes.
Extrapolative Indexed Disaggregated
Deterministic 
Stochastic
The Model Lifecycle
Rob Trippe, MBA
Models follow a what is called the “model lifecycle”. The model lifecycle has four components. In order, they are:
Development Validation Implementation
Ongoing Monitoring
Policy and procedure formalizes risk management activities for implementation. SR
11-7 recommends an emphasis be placed on testing and analysis with a key goal of
promoting accuracy. These roles and responsibilities can be divided among
ownership, control and compliance.
Strong governance coordinates processes and model output across functional areas
and validates final model output. It provides a venue for sharing ideas across areas
and business units.
Introduction to Model Governance
Rob Trippe, MBA
Model Governance
Rob Trippe, MBA
Model Governance concerns itself with:
A. Identifying models
B. Developing internal policy and procedure
C. Developing and enforcing model standards
D. Interpreting and abiding by regulatory guidance and rule
E. Researching and identifying best practice
F. Ensuring proper documentation
G. Assisting in tracking models and ensuring proper remediation when required
H. Overseeing model testing
I. Risk management
J. Change management
Confidential Draft - Not for Distribution - © Rob Trippe - 2018
All Rights Reserved.
 Challenge to model development and verification of validation testing.
 Developing and disseminating model standards which identify the critical elements of model development,
implementation, use and monitoring and provide a road map of expectations for success.
 Naming and ensuring secured models.
 Model inventorying which should be taken on a periodic basis noting both a model’s rank of importance and
where each model is in the development-use-monitoring lifecycle.
 Implementation and use governance (use of links, data dumps, input identification).
 Shared control tools and techniques for users; watch windows, balancing checks, backups, etc.
 Flow chart guidance which will visually point out risk and control points which may not be obvious to a user in a
live environment.
 Qualitative risk analysis is a good starting point in identifying risk points and controls. Consider brainstorming,
Nominal Group or Delphi Technique which includes both upstream and downstream model owners and users.
Model Governance Activities
Rob Trippe, MBA
Model Acceptance
Rob Trippe, MBA
Models are hard work. But, in the end, are they accepted both in build and results?
Here are some key thoughts to remember:
• Does your model comply to accepted model development or are they simply proprietary? (FAST standards)
• Was the model developed with “management buy-in?”
• Do key model users buy into the theory behind the model? If not, should the model attempt to better
support the theory driving model calculations?
• Does model development theory still hold?
• Does model output “marry” to downstream use? If not, how can this be improved?
• What approval processes are in place for the ongoing use and approval of the model?
• Does the data input and model final output remain relevant?
Model Definitions
Rob Trippe, MBA
1 Back-test Use of historic data as a test to model output validity.
2 Benchmark The comparison of model output to the output of an outside and
independent source.
3 Emerging Risk Unforeseeable risk arising further in time and model execution.
4 FAST Set of rules for financial model design. Flexible, Appropriate,
Structured and Transparent.
5 Impact Analysis Assessment of cost, timing, scope and quality of a model-
consequence.
6 In-Sample Historical data used in model development.
7 Model Quantitative method, system, or approach that applies statistical,
economic, financial, or mathematical theories, techniques, and
assumptions to process input data into quantitative estimates.
8 Out of Sample Historical data not used in model development.
9 Outcomes Analysis The comparison of model output to actual outcomes. Back-testing is
one example.
10 Parameter Numerical characteristic of a set or population of numbers.
Here are some great definitions to know:
Model Definitions
Rob Trippe, MBA
11 Calibration Adjustment of data and assumptions.
12 Residual Risk Remaining risk after a risk mitigation action has been performed.
13 Risk Appetite Largest tolerable degree of uncertainty acceptable.
14 Scenario Multiple changes to inputs to reflect a given set of circumstances.
15 Secondary Risk Risk arising from a risk response.
16 Sensitivity Impact of a change to an input relative to the change in output.
17 Stress Test Assessment of model stability by employing hypothetical data inputs
or drivers.
18 Threshold Measure of uncertainty or impact worthy of attention.
19 Tolerance Degree of deviation within which a model still functions properly.
20 Validation A set of processes and activities intended to verify that a model
performs as intended and as expected.
Model Standards
Rob Trippe, MBA
Here is one set of standard topics for non-regulated corporate finance models. These ensure ownership,
improves accuracy and reduces key person dependence. Regulatory requirement may come into play as
well for certain financial institutions which ups the ante.
1. Executive Summary 2. Model Development 3. Operating Procedure 4. Ongoing Monitoring and Governance
Model Use Theory and Approach Model Operation Environment Change Management
Model Theory and Calculation Data and Assumptions Model Owners and Users Risk Management Oversight - 2nd Line of
Defense
Model Resources Language Application Code and Calculations Model Recalibration Audit – 3rd Line of Defense
Model Inputs Model Output User Process - Inputs and Assumptions
Model Output Risk and Control Points Risks and Controls – 1st Line of Defense
Impact to Downstream Models and Analysis Model Weaknesses and Limitations Model Output
Model Limitations Technical Specifications
Testing and Validation
Model Participants
Rob Trippe, MBA
There are numerous players and business units involved through the model lifecycle. They include:
Model Developers
Model Owners
Model Users
Model Testers
Model Validators
Those who develop the
model using systems,
theories, formulas and data.
Those responsible for model
use in “real time”.
Those who implement the
model and its output.
Usually IT, who test model
development results and
manage model change.
Those who confirm a model
is working as intended,
often using mathematical
approaches.
Business Units
Model Governance
Risk Management
Compliance
Audit
Business Units
implementing the model
and utilizing the model
output.
Those who see that models
are tracked, implementing
monitored, inventoried and
comply to standards.
Those who monitor and
manage the risk element of
a business’s activities.
Those who confirm
activities meet regulatory
requirements.
A last internal line of
defense.
Model Risk Management
Rob Trippe, MBA
Virtually each of the activities mentioned in this presentation relate to development and subsequent
model risk management.
Model risk refers to the chance of unintended consequences resulting from model development, inputs or
outputs.
Risk Activities
• Development
• Implementation
• Use
• Change
• Retirement
• Validation
• Review
Key Risk Definitions
• Appetite
• Thresholds
• Tolerance
• Secondary Risk
• Operational Risk
• Risk and Control Points
• Emergent Risk
• Ongoing Monitoring
Good documentation is a key component to model risk management and should be viewed as such and not
simply a compliance and requirement exercise.
Advisory bulletin 2013-07 supports this assertion and adds that documents should be the responsibility of the
model developer (development) and owner (operating procedures, ongoing monitoring), thereby providing a first
line of defense and control between model developers, users and model owners. Proper documentation is
cumbersome and takes time, but it is critical to a model’s overall success by reducing risks and costs. Benefits
include:
 Reduced key person dependence
 Mitigated transitional challenges as models pass from user to user and output need to output need as
conditions and requirements change
 A more streamlined validation and audit process
 Reduction in potential user error
 Avoidance of potential misuse
 Effective communication of challenging theoretical concepts – “comfort”
Model Documentation
Rob Trippe, MBA
Model Documentation
Rob Trippe, MBA
The documentation of financial models is a critical element of model risk management.
It behooves developers, users and owners to spend the time and effort to properly document a model through
its life cycle. Documentation, though time consuming and cumbersome, aids in validation, monitoring and
audit and provides a platform for model developers to effectively communicate their model and its conceptual
soundness to all areas of an institution, from data downloaders to executive management. Federal agencies
will look favorably upon models which have comprehensive and well-conceived documents to support their
models.
Model Flow Charting
Rob Trippe, MBA
Flow charts are powerful tools. Flow charts can come in many forms and there is no one exact manner to flow a
model. From my experience, two flow charts stand out as invaluable;
1. System flow chart “wing to wing” (input to output)
2. Development flow chart
A system flow chart will show, left to right, a model’s data inputs and IT/business unit environments, calculation
processes, and model output and IT/business unit environments.
A development flow chart will visualize the exact mathematical methodologies and techniques employed to
transform input data into useful business information and will generally focus on only one business environment.
Model Flow Charting
Rob Trippe, MBA
Flow charting financial statement builds provide the user a quick gauge on what data is at hand versus what
is needed to complete a model. A flow chart will help identify the required builds and environments from
which data will be acquired. Flow charting will also help visualize core components within and across models
and are an ideal medium for identifying and communicating risk and control points. Flow charts will
dramatically improve model buy-in and provide a path for solid structure. Dead ends, duplicity and
unmanageable audit trails now become visual.
Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Numerous organizations have developed modeling rules for the use of Excel. These rules provide
structure and commonality among spreadsheet design and use. Many rules and guidelines can apply to
other languages as well.
Excel is an application and not a language. And it is only one of many possible languages and applications
which may be incorporated into a model. Due to its wide-spread use and applicability to recent college
graduates, this section focuses on best practice for Excel.
Excel Standards Organizations
• FAST (Flexible, Appropriate, Structured, Transparent)
• Smart Financial Modeling
• BPM (Best Practice Modeling)
Common Rule Categories
• Workbook Design
• Worksheet Design
• Line Item Design
• Cell Design and Use
• Assumption Design and Use
• Use of Features
• Formatting Rules
Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Saving and Printing
Use naming protocol
Use saving protocol
Print format for all pages
Print then proof, never off the screen only
Color
Use color protocol
Use color sparingly
Inputs are blue
Calcs are black
External links are green
Trouble spots are red
The purpose of these rules is to provide clarity, accuracy, safety and a clear audit trail. Only create work
which you would gladly show on a blackboard in a court of law.
Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Brevity
Keep formulas short
Assumptions shown on same page as calc
Keep audit trails short
Avoid daisy chains
Never replicate math, except as proofs
Avoid external links, use exported tabs instead
Highlight external links when required to use
Avoid circular references when possible
When creating a model, you are telling a
story. As the FAST standards say, you are
really writing sentences, paragraphs and
chapters. That approach will help get your
model “read” and more easily accepted.
Models are not political tools.
Best Practice Modeling Rules for Excel
Rob Trippe, MBA
Rows, Columns and Tabs
One column, one purpose
One row, one purpose
No line or column breaks between calculations
Avoid merging cells
Forecast periods move left to right
Statements move top to bottom
No breaks between rolling forecast periods
Don’t hide rows or columns
Tabs flow left to right
Use summary sheets, clean of build ups
Formulas, Calculations and Functions
Use statement accounting sign convention as you would read
from a 10K
Subtotals presented below
Use balance and sanity checks
Show all math
Anchor repetitive calcs left to right and top to bottom
Macros should be used sparingly
Avoid named ranges
Create dedicated calculation areas
Anchor “drivers”
Thought to form and structure is critical for all models. From experience, we know that such thought
is simply not always the case. All financial models follow the same logical order. They:
1. Compile data
2. Adjust and conform data as required
3. Transform data through calculation
4. Present final model output such as a value, a table or a forecast
Use this commonality to develop discipline in how models are built and structured among and
across users and functional areas. Create an intuitive and easy to follow work flow, such as using
tabs left-to-right in Excel. Models in MatLab code can leverage replicable building blocks.
Model Form and Structure
Rob Trippe, MBA
Confidential Draft - Not for Distribution - © Rob Trippe - 2018
All Rights Reserved.
Collaboration
In financial modeling, no one functional area reigns supreme. Example: a financial model solves for
a balance sheet answer. That means keeping accountants involved in the model process, wing-to-
wing. Statistical methodologies and techniques may be employed in data transformation, but stats
are just part of a broader model framework.
At every possible step speak in terms of the language used in a model’s final output environment
(finance, accounting, etc.). Functional areas will often use the same terminology to mean different
things. Be aware of this and agree on definitions first. When a term is used for the first time, stop
and define it for all involved. One company I worked with made investments in equity securities
using a portion of equity. Imagine the potential for confusion as model output is passed from entity
to entity to entity! “Corporate”, owners and business units do not always coordinate and come to
terms on basic terminologies and builds and often work under differing constraints.
Rob Trippe, MBA
Model Validation Techniques
Rob Trippe, MBA
Validation of model results can be performed several ways:
Sensitivity Analysis – Vary inputs and assumptions to compare against other input and assumption
changes to see if a proper correlation exists in model output.
Back-testing – Use of historic data as a test to model output validity. This is also a great way to develop a
model. Simply build your model using historic data and solve for a known answer.
Benchmarking – As previously discussed, this verifies your model’s output by comparing it to another source,
either internal or external.
Scenario Analysis – Run your model using various inputs and assumptions to test for model
integrity and reasonableness.
Use Test – Has the model stood the test of time?
Back-Testing
Rob Trippe, MBA
Assuming we use a three year average to forecast accounts receivable for the year 2018. Is
our result consistent with out of sample actuals at a tolerable level? Here, it appears so.
Out of Sample In Sample Forecast
2010 2011 2012 2013 2014 2015 2016 2017 2018
Revenue $1,000 $1,050 $1,100 $1,075 $1,094 $1,200 $1,250 $1,275 $1,350
Accounts Receivable $75 $68 $89 $78 $78 $90 $91 $101 $102
A/R % Revenue 7.5% 6.5% 8.1% 7.3% 7.1% 7.5% 7.3% 7.9% 7.6% (3 Year Average)
Trailing 3 Year Average 7.4% 7.3%
Variance to Actual 0.1% 0.1%
% Variance 1.4% 2.0%
Validation Benchmarks
Rob Trippe, MBA
• Prior performance
• Expectation
• Competitor results
• Industry standards
• Analyst forecasts
• Internal consensus
Model output can be compared to numerous benchmarks. They include:
Sensitivity Analysis
Rob Trippe, MBA
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10
Line Chart
Rise over Run = 1
$-
$10.00
$20.00
$30.00
$40.00
$50.00
$60.00
$70.00
$80.00
$90.00
1 2 3 4 5 6 7 8 9 10
NPV - Parabolic
Discount Rtae = 20%
For deterministic models, simple
line charts flush out model output
inconsistencies. There is no
“scatter” around a fitted line,
since deterministic charts have no
random variable.
Common Validation Ratios
Rob Trippe, MBA
Profitability Ratios
EBIT Margin EBIT / Revenue
Net Margin Net Income / Revenue
NCF % Revenue Net Cash Flow / Revenue
NCF % of Net Income Net Cash Flow / Net Income
Return on Assets Net Income / [[Total Assets Beg. Balance + Total Assets Ending Bal.]/2]
Return on Equity Net Income / [[Equity Beg. Balance + Equity Ending Bal.]/2]
Growth Ratios
Revenue Growth [[Revenue2 / Revenue1] -1]
Earnings Growth (EBIT) [[EBIT2 / EBIT1] -1]
Earnings Growth (Net Income) [[Net Income2 / Net Income1] -1]
These ratios provide sanity checks for validation. Is model output reasonable and does it follow accepted
patterns? Ratios not only reflect the financial position of a firm but may also reflect a model’s calculation
abilities and errors.
Common Validation Ratios
Rob Trippe, MBA
Expense Ratios
Operating Expense % of Revenue Operating Expenses / Revenue
Turnover Ratios
Receivables Turnover Revenue / [[A/R Beg. Balance + A/R Ending Bal.]/2]
Fixed Asset Turnover Revenue /[ [Fixed Asset Beg. Balance + Fixed Asset Ending Bal.]/2]
Total Asset Turnover Revenue / [[Total Assets Beg. Balance + Total Assets Ending Bal.]/2]
Risk / Leverage Ratios
Current Ratio Current Assets / Current Liabilities
Debt / Equity Total Interest Bearing Debt / Total Equity
Interest Coverage EBIT / Interest Expense
These metrics may also help verify inputs and assumptions and also validate a model’s results:
Validation Trends
Rob Trippe, MBA
Through a company’s life cycle, there are several common types of model pattern output depending on
assumptions and their effect on earnings and/or cash flow growth. Look for irregularities (sudden drops or
steep inclines) and see if your model follows traditional forecast paths.
g
t
g
t
g
t
No Growth Three Phase Three Phase - Declining
Ongoing Monitoring
Rob Trippe, MBA
Development Validation Implementation
Ongoing Monitoring
Ongoing monitoring plays a vital role in model integrity. Are models bask-tested annually? What business conditions
and/or data sources have changed? What external factors may have changed affecting the model’s relevancy? Is the
regulatory environment adopting new rules and standards?
Helpful Resources
Rob Trippe, MBA
Here are links to model documentation and other guidance:
SR 11-7
https://www.federalreserve.gov/bankinforeg/srletters/sr1107.htm
FHFA AB 2013-07
https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/AB-2013-07-Model-Risk-Management Guidance.aspx
FHFA AB 2009-03
https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/Rescinded-Advisory-Bulletins.aspx
OCC 2011-2012
https://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12.html
FAST modeling standards:
http://www.fast-standard.org/
The Global Association of Risk Professional has numerous articles on model risk and validation:
https://www.garp.org
The Author
Rob Trippe is a financial modeling veteran. With over fifteen years’ experience, Rob has developed corporate finance models
for valuation, M&A, forecasting and performance monitoring. He is widely respected for his deep understanding of corporate
finance theory, lectures at university and has worked with some of the world’s largest and most respected firms including the
investment bank Houlihan Lokey, Hertz and GE Capital. He also was the managing director and an owner of a coveted minor
league professional sports franchise at a young age. Rob was accredited in valuation in 2008 and holds an MBA, Finance from
Boston College.
Robert M. Trippe MBA, AVA Page 1
Corporate Finance Author
Currently developing a commissioned case study for a prominent University of Michigan affiliated academic publishing firm. Case study
scope relates to M&A and valuation of a limited partnership with diversified international holdings and interests. 2018
FP&A-Trends Author Model Governance Applied Corporate Financial Modeling 2018
FP&A-Trends Author Corporate Finance Budgeting - Bridging Past and Future 2017
FP&A-Trends Author Model Risk 20 Financial Modeling Definitions to Memorize 2017
FP&A-Trends Author Model Governance SR 11-7 and Corporate Finance 2017
Florida State University Author Multinational Finance Authored Exam 2017
Wall Street Journal Contributing Author Valuation NFL Franchise Valuations 1993
USA Today Contributing Author Valuation NFL Franchise Valuations 1993
Corporate Finance Visiting Lecturer
Recurring visiting lecturer on corporate finance modeling, pro forma analysis and cost of capital at Florida State University. Topics have
included forecasting integrated statements, internal and sustainable growth, discount rates, CAPM, weighted cost of capital and F/X
effects in capital planning.
Florida State University Visiting Lecturer Corporate Finance CAPM, WACC 2015
Florida State University Visiting Lecturer Corporate Finance Financial Modeling and Pro Forma Analysis 2014
Babson College Visiting Speaker Corporate Finance Fundamentals of Valuation 1994
Confidential Draft - Not for Distribution - © Rob Trippe - 2018
All Rights Reserved.
38
Confidential Draft - Not for Distribution - © Rob Trippe - 2018
All Rights Reserved. Rob Trippe, MBA

More Related Content

Similar to Model Management for FP&A

Model Risk Management: Using an infinitely scalable stress testing platform f...
Model Risk Management: Using an infinitely scalable stress testing platform f...Model Risk Management: Using an infinitely scalable stress testing platform f...
Model Risk Management: Using an infinitely scalable stress testing platform f...QuantUniversity
 
Independent models validation and automation
Independent models validation and automationIndependent models validation and automation
Independent models validation and automationSohail_farooq
 
Chappuis Halder - Model validation review
Chappuis Halder - Model validation reviewChappuis Halder - Model validation review
Chappuis Halder - Model validation reviewAugustin Beyot
 
Validating Qualitative Models
Validating Qualitative ModelsValidating Qualitative Models
Validating Qualitative ModelsJacob Kosoff
 
IFRS 9 Model Risk Management - Given the Short Shift ?
IFRS 9 Model Risk Management - Given the Short Shift ?IFRS 9 Model Risk Management - Given the Short Shift ?
IFRS 9 Model Risk Management - Given the Short Shift ?Sandip Mukherjee CFA, FRM
 
Be aers-fara-modellinginsolvency-nov2010
Be aers-fara-modellinginsolvency-nov2010Be aers-fara-modellinginsolvency-nov2010
Be aers-fara-modellinginsolvency-nov2010Dodi Mulyadi
 
Model Risk Management in U.S. Regional Banks
Model Risk Management in U.S. Regional BanksModel Risk Management in U.S. Regional Banks
Model Risk Management in U.S. Regional BanksMwestergaard
 
Enterprise 360 degree risk management
Enterprise 360 degree risk managementEnterprise 360 degree risk management
Enterprise 360 degree risk managementInfosys
 
FitchLearning QuantUniversity Model Risk Presentation
FitchLearning QuantUniversity Model Risk PresentationFitchLearning QuantUniversity Model Risk Presentation
FitchLearning QuantUniversity Model Risk PresentationQuantUniversity
 
Zou_Resume_2015_Dec_Q
Zou_Resume_2015_Dec_QZou_Resume_2015_Dec_Q
Zou_Resume_2015_Dec_QXiaorong Zou
 
CHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docxCHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docxtiffanyd4
 
CHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docxCHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docxmccormicknadine86
 
Stochastic Modeling - Model Risk - Sampling Error - Scenario Reduction
Stochastic Modeling - Model Risk - Sampling Error - Scenario ReductionStochastic Modeling - Model Risk - Sampling Error - Scenario Reduction
Stochastic Modeling - Model Risk - Sampling Error - Scenario ReductionRon Harasym
 
White paper model risk sept 2011
White paper model risk sept 2011White paper model risk sept 2011
White paper model risk sept 2011Bank Risk Advisors
 
DEFINITION.docx
DEFINITION.docxDEFINITION.docx
DEFINITION.docxAbdetaImi
 

Similar to Model Management for FP&A (20)

Model Risk Management: Using an infinitely scalable stress testing platform f...
Model Risk Management: Using an infinitely scalable stress testing platform f...Model Risk Management: Using an infinitely scalable stress testing platform f...
Model Risk Management: Using an infinitely scalable stress testing platform f...
 
Independent models validation and automation
Independent models validation and automationIndependent models validation and automation
Independent models validation and automation
 
Chappuis Halder - Model validation review
Chappuis Halder - Model validation reviewChappuis Halder - Model validation review
Chappuis Halder - Model validation review
 
ROI-Institute-Brochure1
ROI-Institute-Brochure1ROI-Institute-Brochure1
ROI-Institute-Brochure1
 
Validating Qualitative Models
Validating Qualitative ModelsValidating Qualitative Models
Validating Qualitative Models
 
IFRS 9 Model Risk Management - Given the Short Shift ?
IFRS 9 Model Risk Management - Given the Short Shift ?IFRS 9 Model Risk Management - Given the Short Shift ?
IFRS 9 Model Risk Management - Given the Short Shift ?
 
Be aers-fara-modellinginsolvency-nov2010
Be aers-fara-modellinginsolvency-nov2010Be aers-fara-modellinginsolvency-nov2010
Be aers-fara-modellinginsolvency-nov2010
 
Model Risk Management in U.S. Regional Banks
Model Risk Management in U.S. Regional BanksModel Risk Management in U.S. Regional Banks
Model Risk Management in U.S. Regional Banks
 
Enterprise 360 degree risk management
Enterprise 360 degree risk managementEnterprise 360 degree risk management
Enterprise 360 degree risk management
 
FitchLearning QuantUniversity Model Risk Presentation
FitchLearning QuantUniversity Model Risk PresentationFitchLearning QuantUniversity Model Risk Presentation
FitchLearning QuantUniversity Model Risk Presentation
 
Zou_Resume_2015_Dec_Q
Zou_Resume_2015_Dec_QZou_Resume_2015_Dec_Q
Zou_Resume_2015_Dec_Q
 
1120 track1 grossman
1120 track1 grossman1120 track1 grossman
1120 track1 grossman
 
CHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docxCHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docx
 
CHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docxCHAPTER Modeling and Analysis Heuristic Search Methods .docx
CHAPTER Modeling and Analysis Heuristic Search Methods .docx
 
Levelling Up - Five Real World Examples of Incremental Maturity
Levelling Up - Five Real World Examples of Incremental MaturityLevelling Up - Five Real World Examples of Incremental Maturity
Levelling Up - Five Real World Examples of Incremental Maturity
 
Roi Certification Kraków May 2017
Roi Certification Kraków May 2017Roi Certification Kraków May 2017
Roi Certification Kraków May 2017
 
Project selection
Project selection Project selection
Project selection
 
Stochastic Modeling - Model Risk - Sampling Error - Scenario Reduction
Stochastic Modeling - Model Risk - Sampling Error - Scenario ReductionStochastic Modeling - Model Risk - Sampling Error - Scenario Reduction
Stochastic Modeling - Model Risk - Sampling Error - Scenario Reduction
 
White paper model risk sept 2011
White paper model risk sept 2011White paper model risk sept 2011
White paper model risk sept 2011
 
DEFINITION.docx
DEFINITION.docxDEFINITION.docx
DEFINITION.docx
 

Recently uploaded

Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...
Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...
Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...First NO1 World Amil baba in Faisalabad
 
BPPG response - Options for Defined Benefit schemes - 19Apr24.pdf
BPPG response - Options for Defined Benefit schemes - 19Apr24.pdfBPPG response - Options for Defined Benefit schemes - 19Apr24.pdf
BPPG response - Options for Defined Benefit schemes - 19Apr24.pdfHenry Tapper
 
Call Girls In Yusuf Sarai Women Seeking Men 9654467111
Call Girls In Yusuf Sarai Women Seeking Men 9654467111Call Girls In Yusuf Sarai Women Seeking Men 9654467111
Call Girls In Yusuf Sarai Women Seeking Men 9654467111Sapana Sha
 
call girls in Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️
call girls in  Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️call girls in  Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️
call girls in Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
Chapter 2.ppt of macroeconomics by mankiw 9th edition
Chapter 2.ppt of macroeconomics by mankiw 9th editionChapter 2.ppt of macroeconomics by mankiw 9th edition
Chapter 2.ppt of macroeconomics by mankiw 9th editionMuhammadHusnain82237
 
VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130
VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130
VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130Suhani Kapoor
 
AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...
AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...
AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...yordanosyohannes2
 
The Triple Threat | Article on Global Resession | Harsh Kumar
The Triple Threat | Article on Global Resession | Harsh KumarThe Triple Threat | Article on Global Resession | Harsh Kumar
The Triple Threat | Article on Global Resession | Harsh KumarHarsh Kumar
 
government_intervention_in_business_ownership[1].pdf
government_intervention_in_business_ownership[1].pdfgovernment_intervention_in_business_ownership[1].pdf
government_intervention_in_business_ownership[1].pdfshaunmashale756
 
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
Monthly Market Risk Update: April 2024 [SlideShare]
Monthly Market Risk Update: April 2024 [SlideShare]Monthly Market Risk Update: April 2024 [SlideShare]
Monthly Market Risk Update: April 2024 [SlideShare]Commonwealth
 
Malad Call Girl in Services 9892124323 | ₹,4500 With Room Free Delivery
Malad Call Girl in Services  9892124323 | ₹,4500 With Room Free DeliveryMalad Call Girl in Services  9892124323 | ₹,4500 With Room Free Delivery
Malad Call Girl in Services 9892124323 | ₹,4500 With Room Free DeliveryPooja Nehwal
 
Quantitative Analysis of Retail Sector Companies
Quantitative Analysis of Retail Sector CompaniesQuantitative Analysis of Retail Sector Companies
Quantitative Analysis of Retail Sector Companiesprashantbhati354
 
House of Commons ; CDC schemes overview document
House of Commons ; CDC schemes overview documentHouse of Commons ; CDC schemes overview document
House of Commons ; CDC schemes overview documentHenry Tapper
 
High Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur Escorts
High Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur EscortsHigh Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur Escorts
High Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur Escortsranjana rawat
 
How Automation is Driving Efficiency Through the Last Mile of Reporting
How Automation is Driving Efficiency Through the Last Mile of ReportingHow Automation is Driving Efficiency Through the Last Mile of Reporting
How Automation is Driving Efficiency Through the Last Mile of ReportingAggregage
 
Q3 2024 Earnings Conference Call and Webcast Slides
Q3 2024 Earnings Conference Call and Webcast SlidesQ3 2024 Earnings Conference Call and Webcast Slides
Q3 2024 Earnings Conference Call and Webcast SlidesMarketing847413
 
(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办
(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办
(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办fqiuho152
 
Interimreport1 January–31 March2024 Elo Mutual Pension Insurance Company
Interimreport1 January–31 March2024 Elo Mutual Pension Insurance CompanyInterimreport1 January–31 March2024 Elo Mutual Pension Insurance Company
Interimreport1 January–31 March2024 Elo Mutual Pension Insurance CompanyTyöeläkeyhtiö Elo
 
fca-bsps-decision-letter-redacted (1).pdf
fca-bsps-decision-letter-redacted (1).pdffca-bsps-decision-letter-redacted (1).pdf
fca-bsps-decision-letter-redacted (1).pdfHenry Tapper
 

Recently uploaded (20)

Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...
Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...
Authentic No 1 Amil Baba In Pakistan Authentic No 1 Amil Baba In Karachi No 1...
 
BPPG response - Options for Defined Benefit schemes - 19Apr24.pdf
BPPG response - Options for Defined Benefit schemes - 19Apr24.pdfBPPG response - Options for Defined Benefit schemes - 19Apr24.pdf
BPPG response - Options for Defined Benefit schemes - 19Apr24.pdf
 
Call Girls In Yusuf Sarai Women Seeking Men 9654467111
Call Girls In Yusuf Sarai Women Seeking Men 9654467111Call Girls In Yusuf Sarai Women Seeking Men 9654467111
Call Girls In Yusuf Sarai Women Seeking Men 9654467111
 
call girls in Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️
call girls in  Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️call girls in  Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️
call girls in Nand Nagri (DELHI) 🔝 >༒9953330565🔝 genuine Escort Service 🔝✔️✔️
 
Chapter 2.ppt of macroeconomics by mankiw 9th edition
Chapter 2.ppt of macroeconomics by mankiw 9th editionChapter 2.ppt of macroeconomics by mankiw 9th edition
Chapter 2.ppt of macroeconomics by mankiw 9th edition
 
VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130
VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130
VIP Call Girls Service Begumpet Hyderabad Call +91-8250192130
 
AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...
AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...
AfRESFullPaper22018EmpiricalPerformanceofRealEstateInvestmentTrustsandShareho...
 
The Triple Threat | Article on Global Resession | Harsh Kumar
The Triple Threat | Article on Global Resession | Harsh KumarThe Triple Threat | Article on Global Resession | Harsh Kumar
The Triple Threat | Article on Global Resession | Harsh Kumar
 
government_intervention_in_business_ownership[1].pdf
government_intervention_in_business_ownership[1].pdfgovernment_intervention_in_business_ownership[1].pdf
government_intervention_in_business_ownership[1].pdf
 
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(DIYA) Bhumkar Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Monthly Market Risk Update: April 2024 [SlideShare]
Monthly Market Risk Update: April 2024 [SlideShare]Monthly Market Risk Update: April 2024 [SlideShare]
Monthly Market Risk Update: April 2024 [SlideShare]
 
Malad Call Girl in Services 9892124323 | ₹,4500 With Room Free Delivery
Malad Call Girl in Services  9892124323 | ₹,4500 With Room Free DeliveryMalad Call Girl in Services  9892124323 | ₹,4500 With Room Free Delivery
Malad Call Girl in Services 9892124323 | ₹,4500 With Room Free Delivery
 
Quantitative Analysis of Retail Sector Companies
Quantitative Analysis of Retail Sector CompaniesQuantitative Analysis of Retail Sector Companies
Quantitative Analysis of Retail Sector Companies
 
House of Commons ; CDC schemes overview document
House of Commons ; CDC schemes overview documentHouse of Commons ; CDC schemes overview document
House of Commons ; CDC schemes overview document
 
High Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur Escorts
High Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur EscortsHigh Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur Escorts
High Class Call Girls Nagpur Grishma Call 7001035870 Meet With Nagpur Escorts
 
How Automation is Driving Efficiency Through the Last Mile of Reporting
How Automation is Driving Efficiency Through the Last Mile of ReportingHow Automation is Driving Efficiency Through the Last Mile of Reporting
How Automation is Driving Efficiency Through the Last Mile of Reporting
 
Q3 2024 Earnings Conference Call and Webcast Slides
Q3 2024 Earnings Conference Call and Webcast SlidesQ3 2024 Earnings Conference Call and Webcast Slides
Q3 2024 Earnings Conference Call and Webcast Slides
 
(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办
(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办
(办理原版一样)QUT毕业证昆士兰科技大学毕业证学位证留信学历认证成绩单补办
 
Interimreport1 January–31 March2024 Elo Mutual Pension Insurance Company
Interimreport1 January–31 March2024 Elo Mutual Pension Insurance CompanyInterimreport1 January–31 March2024 Elo Mutual Pension Insurance Company
Interimreport1 January–31 March2024 Elo Mutual Pension Insurance Company
 
fca-bsps-decision-letter-redacted (1).pdf
fca-bsps-decision-letter-redacted (1).pdffca-bsps-decision-letter-redacted (1).pdf
fca-bsps-decision-letter-redacted (1).pdf
 

Model Management for FP&A

  • 1. Introduction to Model Management for Traditional FP&A Rob Trippe, MBA
  • 2. The Need for Model Management and Governance Federal guidance, spurred by the great recession, enhanced IT capabilities, the need for greater security, an increased acceptance of financial theory and the overall pursuit of best practice have all elevated the level of detail and precision which financial modeling in the business world has pursued. Even in corporate environments outside of financial services and federal regulation, companies often desire the proper tools and knowledge to confirm their decision-making models are accurate and appropriate. This presentation provides a pathway towards those needs and introduces two model management objectives: 1. An understanding of modern model governance standards and management, 2. Knowledge of proper validation and monitoring techniques. Rob Trippe, MBA Corporate Financial Model Management
  • 3. Definition of a Financial Model Rob Trippe, MBA Federal Reserve issued guidance to banks (SR 11-7 among other documents) defined a model for the purpose of model risk management after the wake of the great recession in 2008. It provides a framework for best in class financial model development, validation and monitoring. SR 11-7 outlines six criteria a financial tool must possess to be a model. 1. The analysis must employ academic theory 2. Three components: inputs-calculation processes-outputs 3. The analysis must transform data into useful business information 4. Must be used repetitively 5. Output must be quantitative in nature 6. Subject matter expertise is required 7. Possible qualitative model inputs or output Data Calculation Process Output
  • 4. Definition of a Financial Model Rob Trippe, MBA Models are not limited to one programming language environment or application. The term “model” should be viewed “wing to wing” from initial data inputs to final model output. Numerous programming environments may be involved as with numerous data sources. “Downstream” models should be identified.
  • 5. What Are Corporate Finance Models Used For? Rob Trippe, MBA Financial models inform decision makers and also assist in complying with regulatory requirement. The three primary corporate finance activities are: 1. Investment Decisions - estimating and comparing cash flows, risks and returns to requirements and expectations. 2. Financing Decisions – determining capital structure; what type of instruments will we use to finance the assets, what are the terms of this financing (e.g. length, terms) and to what degree can each asset be levered? 3. Dividend Policy - should we invest for growth and replacement or should we dividend cash flows to investors and what are investor’s expectations?
  • 6. Influencing Model Factors Rob Trippe, MBA Models improve in integrity and effectiveness as the result of several guiding influences. Proper Theory and Statement Analysis Model Governance and SR 11-7 Industry Rules such as FAST for Excel
  • 7. Types of Models Rob Trippe, MBA Deterministic models use assumptions (independent variables) which are believed to be “known”, that is they are static. Run a deterministic model ten times and you will get the same result. % Inflation Assumption Through Time Stochastic models use assumptions (independent variables) which are only believed to be “known”, within a set of parameters or “boundaries”. Run a stochastic model ten times and you will get ten different results. Dependent variable output therefore has a random element to it. Models are often then run using Monte Carlo simulation for numerous results and an average of these results is used as the final model output. % Inflation Assumption Through Time
  • 8. Other Model Classifications Rob Trippe, MBA Extrapolative Modeling utilizes prior period results in time series fashion, year over year growth, for example. Index Modeling utilizes outside data to estimate future internal results. Disaggregated Modeling uses disparate data (line items on financial information, for example) to estimate future outcomes. Extrapolative Indexed Disaggregated Deterministic  Stochastic
  • 9. The Model Lifecycle Rob Trippe, MBA Models follow a what is called the “model lifecycle”. The model lifecycle has four components. In order, they are: Development Validation Implementation Ongoing Monitoring
  • 10. Policy and procedure formalizes risk management activities for implementation. SR 11-7 recommends an emphasis be placed on testing and analysis with a key goal of promoting accuracy. These roles and responsibilities can be divided among ownership, control and compliance. Strong governance coordinates processes and model output across functional areas and validates final model output. It provides a venue for sharing ideas across areas and business units. Introduction to Model Governance Rob Trippe, MBA
  • 11. Model Governance Rob Trippe, MBA Model Governance concerns itself with: A. Identifying models B. Developing internal policy and procedure C. Developing and enforcing model standards D. Interpreting and abiding by regulatory guidance and rule E. Researching and identifying best practice F. Ensuring proper documentation G. Assisting in tracking models and ensuring proper remediation when required H. Overseeing model testing I. Risk management J. Change management
  • 12. Confidential Draft - Not for Distribution - © Rob Trippe - 2018 All Rights Reserved.  Challenge to model development and verification of validation testing.  Developing and disseminating model standards which identify the critical elements of model development, implementation, use and monitoring and provide a road map of expectations for success.  Naming and ensuring secured models.  Model inventorying which should be taken on a periodic basis noting both a model’s rank of importance and where each model is in the development-use-monitoring lifecycle.  Implementation and use governance (use of links, data dumps, input identification).  Shared control tools and techniques for users; watch windows, balancing checks, backups, etc.  Flow chart guidance which will visually point out risk and control points which may not be obvious to a user in a live environment.  Qualitative risk analysis is a good starting point in identifying risk points and controls. Consider brainstorming, Nominal Group or Delphi Technique which includes both upstream and downstream model owners and users. Model Governance Activities Rob Trippe, MBA
  • 13. Model Acceptance Rob Trippe, MBA Models are hard work. But, in the end, are they accepted both in build and results? Here are some key thoughts to remember: • Does your model comply to accepted model development or are they simply proprietary? (FAST standards) • Was the model developed with “management buy-in?” • Do key model users buy into the theory behind the model? If not, should the model attempt to better support the theory driving model calculations? • Does model development theory still hold? • Does model output “marry” to downstream use? If not, how can this be improved? • What approval processes are in place for the ongoing use and approval of the model? • Does the data input and model final output remain relevant?
  • 14. Model Definitions Rob Trippe, MBA 1 Back-test Use of historic data as a test to model output validity. 2 Benchmark The comparison of model output to the output of an outside and independent source. 3 Emerging Risk Unforeseeable risk arising further in time and model execution. 4 FAST Set of rules for financial model design. Flexible, Appropriate, Structured and Transparent. 5 Impact Analysis Assessment of cost, timing, scope and quality of a model- consequence. 6 In-Sample Historical data used in model development. 7 Model Quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates. 8 Out of Sample Historical data not used in model development. 9 Outcomes Analysis The comparison of model output to actual outcomes. Back-testing is one example. 10 Parameter Numerical characteristic of a set or population of numbers. Here are some great definitions to know:
  • 15. Model Definitions Rob Trippe, MBA 11 Calibration Adjustment of data and assumptions. 12 Residual Risk Remaining risk after a risk mitigation action has been performed. 13 Risk Appetite Largest tolerable degree of uncertainty acceptable. 14 Scenario Multiple changes to inputs to reflect a given set of circumstances. 15 Secondary Risk Risk arising from a risk response. 16 Sensitivity Impact of a change to an input relative to the change in output. 17 Stress Test Assessment of model stability by employing hypothetical data inputs or drivers. 18 Threshold Measure of uncertainty or impact worthy of attention. 19 Tolerance Degree of deviation within which a model still functions properly. 20 Validation A set of processes and activities intended to verify that a model performs as intended and as expected.
  • 16. Model Standards Rob Trippe, MBA Here is one set of standard topics for non-regulated corporate finance models. These ensure ownership, improves accuracy and reduces key person dependence. Regulatory requirement may come into play as well for certain financial institutions which ups the ante. 1. Executive Summary 2. Model Development 3. Operating Procedure 4. Ongoing Monitoring and Governance Model Use Theory and Approach Model Operation Environment Change Management Model Theory and Calculation Data and Assumptions Model Owners and Users Risk Management Oversight - 2nd Line of Defense Model Resources Language Application Code and Calculations Model Recalibration Audit – 3rd Line of Defense Model Inputs Model Output User Process - Inputs and Assumptions Model Output Risk and Control Points Risks and Controls – 1st Line of Defense Impact to Downstream Models and Analysis Model Weaknesses and Limitations Model Output Model Limitations Technical Specifications Testing and Validation
  • 17. Model Participants Rob Trippe, MBA There are numerous players and business units involved through the model lifecycle. They include: Model Developers Model Owners Model Users Model Testers Model Validators Those who develop the model using systems, theories, formulas and data. Those responsible for model use in “real time”. Those who implement the model and its output. Usually IT, who test model development results and manage model change. Those who confirm a model is working as intended, often using mathematical approaches. Business Units Model Governance Risk Management Compliance Audit Business Units implementing the model and utilizing the model output. Those who see that models are tracked, implementing monitored, inventoried and comply to standards. Those who monitor and manage the risk element of a business’s activities. Those who confirm activities meet regulatory requirements. A last internal line of defense.
  • 18. Model Risk Management Rob Trippe, MBA Virtually each of the activities mentioned in this presentation relate to development and subsequent model risk management. Model risk refers to the chance of unintended consequences resulting from model development, inputs or outputs. Risk Activities • Development • Implementation • Use • Change • Retirement • Validation • Review Key Risk Definitions • Appetite • Thresholds • Tolerance • Secondary Risk • Operational Risk • Risk and Control Points • Emergent Risk • Ongoing Monitoring
  • 19. Good documentation is a key component to model risk management and should be viewed as such and not simply a compliance and requirement exercise. Advisory bulletin 2013-07 supports this assertion and adds that documents should be the responsibility of the model developer (development) and owner (operating procedures, ongoing monitoring), thereby providing a first line of defense and control between model developers, users and model owners. Proper documentation is cumbersome and takes time, but it is critical to a model’s overall success by reducing risks and costs. Benefits include:  Reduced key person dependence  Mitigated transitional challenges as models pass from user to user and output need to output need as conditions and requirements change  A more streamlined validation and audit process  Reduction in potential user error  Avoidance of potential misuse  Effective communication of challenging theoretical concepts – “comfort” Model Documentation Rob Trippe, MBA
  • 20. Model Documentation Rob Trippe, MBA The documentation of financial models is a critical element of model risk management. It behooves developers, users and owners to spend the time and effort to properly document a model through its life cycle. Documentation, though time consuming and cumbersome, aids in validation, monitoring and audit and provides a platform for model developers to effectively communicate their model and its conceptual soundness to all areas of an institution, from data downloaders to executive management. Federal agencies will look favorably upon models which have comprehensive and well-conceived documents to support their models.
  • 21. Model Flow Charting Rob Trippe, MBA Flow charts are powerful tools. Flow charts can come in many forms and there is no one exact manner to flow a model. From my experience, two flow charts stand out as invaluable; 1. System flow chart “wing to wing” (input to output) 2. Development flow chart A system flow chart will show, left to right, a model’s data inputs and IT/business unit environments, calculation processes, and model output and IT/business unit environments. A development flow chart will visualize the exact mathematical methodologies and techniques employed to transform input data into useful business information and will generally focus on only one business environment.
  • 22. Model Flow Charting Rob Trippe, MBA Flow charting financial statement builds provide the user a quick gauge on what data is at hand versus what is needed to complete a model. A flow chart will help identify the required builds and environments from which data will be acquired. Flow charting will also help visualize core components within and across models and are an ideal medium for identifying and communicating risk and control points. Flow charts will dramatically improve model buy-in and provide a path for solid structure. Dead ends, duplicity and unmanageable audit trails now become visual.
  • 23. Best Practice Modeling Rules for Excel Rob Trippe, MBA Numerous organizations have developed modeling rules for the use of Excel. These rules provide structure and commonality among spreadsheet design and use. Many rules and guidelines can apply to other languages as well. Excel is an application and not a language. And it is only one of many possible languages and applications which may be incorporated into a model. Due to its wide-spread use and applicability to recent college graduates, this section focuses on best practice for Excel. Excel Standards Organizations • FAST (Flexible, Appropriate, Structured, Transparent) • Smart Financial Modeling • BPM (Best Practice Modeling) Common Rule Categories • Workbook Design • Worksheet Design • Line Item Design • Cell Design and Use • Assumption Design and Use • Use of Features • Formatting Rules
  • 24. Best Practice Modeling Rules for Excel Rob Trippe, MBA Saving and Printing Use naming protocol Use saving protocol Print format for all pages Print then proof, never off the screen only Color Use color protocol Use color sparingly Inputs are blue Calcs are black External links are green Trouble spots are red The purpose of these rules is to provide clarity, accuracy, safety and a clear audit trail. Only create work which you would gladly show on a blackboard in a court of law.
  • 25. Best Practice Modeling Rules for Excel Rob Trippe, MBA Brevity Keep formulas short Assumptions shown on same page as calc Keep audit trails short Avoid daisy chains Never replicate math, except as proofs Avoid external links, use exported tabs instead Highlight external links when required to use Avoid circular references when possible When creating a model, you are telling a story. As the FAST standards say, you are really writing sentences, paragraphs and chapters. That approach will help get your model “read” and more easily accepted. Models are not political tools.
  • 26. Best Practice Modeling Rules for Excel Rob Trippe, MBA Rows, Columns and Tabs One column, one purpose One row, one purpose No line or column breaks between calculations Avoid merging cells Forecast periods move left to right Statements move top to bottom No breaks between rolling forecast periods Don’t hide rows or columns Tabs flow left to right Use summary sheets, clean of build ups Formulas, Calculations and Functions Use statement accounting sign convention as you would read from a 10K Subtotals presented below Use balance and sanity checks Show all math Anchor repetitive calcs left to right and top to bottom Macros should be used sparingly Avoid named ranges Create dedicated calculation areas Anchor “drivers”
  • 27. Thought to form and structure is critical for all models. From experience, we know that such thought is simply not always the case. All financial models follow the same logical order. They: 1. Compile data 2. Adjust and conform data as required 3. Transform data through calculation 4. Present final model output such as a value, a table or a forecast Use this commonality to develop discipline in how models are built and structured among and across users and functional areas. Create an intuitive and easy to follow work flow, such as using tabs left-to-right in Excel. Models in MatLab code can leverage replicable building blocks. Model Form and Structure Rob Trippe, MBA
  • 28. Confidential Draft - Not for Distribution - © Rob Trippe - 2018 All Rights Reserved. Collaboration In financial modeling, no one functional area reigns supreme. Example: a financial model solves for a balance sheet answer. That means keeping accountants involved in the model process, wing-to- wing. Statistical methodologies and techniques may be employed in data transformation, but stats are just part of a broader model framework. At every possible step speak in terms of the language used in a model’s final output environment (finance, accounting, etc.). Functional areas will often use the same terminology to mean different things. Be aware of this and agree on definitions first. When a term is used for the first time, stop and define it for all involved. One company I worked with made investments in equity securities using a portion of equity. Imagine the potential for confusion as model output is passed from entity to entity to entity! “Corporate”, owners and business units do not always coordinate and come to terms on basic terminologies and builds and often work under differing constraints. Rob Trippe, MBA
  • 29. Model Validation Techniques Rob Trippe, MBA Validation of model results can be performed several ways: Sensitivity Analysis – Vary inputs and assumptions to compare against other input and assumption changes to see if a proper correlation exists in model output. Back-testing – Use of historic data as a test to model output validity. This is also a great way to develop a model. Simply build your model using historic data and solve for a known answer. Benchmarking – As previously discussed, this verifies your model’s output by comparing it to another source, either internal or external. Scenario Analysis – Run your model using various inputs and assumptions to test for model integrity and reasonableness. Use Test – Has the model stood the test of time?
  • 30. Back-Testing Rob Trippe, MBA Assuming we use a three year average to forecast accounts receivable for the year 2018. Is our result consistent with out of sample actuals at a tolerable level? Here, it appears so. Out of Sample In Sample Forecast 2010 2011 2012 2013 2014 2015 2016 2017 2018 Revenue $1,000 $1,050 $1,100 $1,075 $1,094 $1,200 $1,250 $1,275 $1,350 Accounts Receivable $75 $68 $89 $78 $78 $90 $91 $101 $102 A/R % Revenue 7.5% 6.5% 8.1% 7.3% 7.1% 7.5% 7.3% 7.9% 7.6% (3 Year Average) Trailing 3 Year Average 7.4% 7.3% Variance to Actual 0.1% 0.1% % Variance 1.4% 2.0%
  • 31. Validation Benchmarks Rob Trippe, MBA • Prior performance • Expectation • Competitor results • Industry standards • Analyst forecasts • Internal consensus Model output can be compared to numerous benchmarks. They include:
  • 32. Sensitivity Analysis Rob Trippe, MBA 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Line Chart Rise over Run = 1 $- $10.00 $20.00 $30.00 $40.00 $50.00 $60.00 $70.00 $80.00 $90.00 1 2 3 4 5 6 7 8 9 10 NPV - Parabolic Discount Rtae = 20% For deterministic models, simple line charts flush out model output inconsistencies. There is no “scatter” around a fitted line, since deterministic charts have no random variable.
  • 33. Common Validation Ratios Rob Trippe, MBA Profitability Ratios EBIT Margin EBIT / Revenue Net Margin Net Income / Revenue NCF % Revenue Net Cash Flow / Revenue NCF % of Net Income Net Cash Flow / Net Income Return on Assets Net Income / [[Total Assets Beg. Balance + Total Assets Ending Bal.]/2] Return on Equity Net Income / [[Equity Beg. Balance + Equity Ending Bal.]/2] Growth Ratios Revenue Growth [[Revenue2 / Revenue1] -1] Earnings Growth (EBIT) [[EBIT2 / EBIT1] -1] Earnings Growth (Net Income) [[Net Income2 / Net Income1] -1] These ratios provide sanity checks for validation. Is model output reasonable and does it follow accepted patterns? Ratios not only reflect the financial position of a firm but may also reflect a model’s calculation abilities and errors.
  • 34. Common Validation Ratios Rob Trippe, MBA Expense Ratios Operating Expense % of Revenue Operating Expenses / Revenue Turnover Ratios Receivables Turnover Revenue / [[A/R Beg. Balance + A/R Ending Bal.]/2] Fixed Asset Turnover Revenue /[ [Fixed Asset Beg. Balance + Fixed Asset Ending Bal.]/2] Total Asset Turnover Revenue / [[Total Assets Beg. Balance + Total Assets Ending Bal.]/2] Risk / Leverage Ratios Current Ratio Current Assets / Current Liabilities Debt / Equity Total Interest Bearing Debt / Total Equity Interest Coverage EBIT / Interest Expense These metrics may also help verify inputs and assumptions and also validate a model’s results:
  • 35. Validation Trends Rob Trippe, MBA Through a company’s life cycle, there are several common types of model pattern output depending on assumptions and their effect on earnings and/or cash flow growth. Look for irregularities (sudden drops or steep inclines) and see if your model follows traditional forecast paths. g t g t g t No Growth Three Phase Three Phase - Declining
  • 36. Ongoing Monitoring Rob Trippe, MBA Development Validation Implementation Ongoing Monitoring Ongoing monitoring plays a vital role in model integrity. Are models bask-tested annually? What business conditions and/or data sources have changed? What external factors may have changed affecting the model’s relevancy? Is the regulatory environment adopting new rules and standards?
  • 37. Helpful Resources Rob Trippe, MBA Here are links to model documentation and other guidance: SR 11-7 https://www.federalreserve.gov/bankinforeg/srletters/sr1107.htm FHFA AB 2013-07 https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/AB-2013-07-Model-Risk-Management Guidance.aspx FHFA AB 2009-03 https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/Rescinded-Advisory-Bulletins.aspx OCC 2011-2012 https://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12.html FAST modeling standards: http://www.fast-standard.org/ The Global Association of Risk Professional has numerous articles on model risk and validation: https://www.garp.org
  • 38. The Author Rob Trippe is a financial modeling veteran. With over fifteen years’ experience, Rob has developed corporate finance models for valuation, M&A, forecasting and performance monitoring. He is widely respected for his deep understanding of corporate finance theory, lectures at university and has worked with some of the world’s largest and most respected firms including the investment bank Houlihan Lokey, Hertz and GE Capital. He also was the managing director and an owner of a coveted minor league professional sports franchise at a young age. Rob was accredited in valuation in 2008 and holds an MBA, Finance from Boston College. Robert M. Trippe MBA, AVA Page 1 Corporate Finance Author Currently developing a commissioned case study for a prominent University of Michigan affiliated academic publishing firm. Case study scope relates to M&A and valuation of a limited partnership with diversified international holdings and interests. 2018 FP&A-Trends Author Model Governance Applied Corporate Financial Modeling 2018 FP&A-Trends Author Corporate Finance Budgeting - Bridging Past and Future 2017 FP&A-Trends Author Model Risk 20 Financial Modeling Definitions to Memorize 2017 FP&A-Trends Author Model Governance SR 11-7 and Corporate Finance 2017 Florida State University Author Multinational Finance Authored Exam 2017 Wall Street Journal Contributing Author Valuation NFL Franchise Valuations 1993 USA Today Contributing Author Valuation NFL Franchise Valuations 1993 Corporate Finance Visiting Lecturer Recurring visiting lecturer on corporate finance modeling, pro forma analysis and cost of capital at Florida State University. Topics have included forecasting integrated statements, internal and sustainable growth, discount rates, CAPM, weighted cost of capital and F/X effects in capital planning. Florida State University Visiting Lecturer Corporate Finance CAPM, WACC 2015 Florida State University Visiting Lecturer Corporate Finance Financial Modeling and Pro Forma Analysis 2014 Babson College Visiting Speaker Corporate Finance Fundamentals of Valuation 1994 Confidential Draft - Not for Distribution - © Rob Trippe - 2018 All Rights Reserved. 38 Confidential Draft - Not for Distribution - © Rob Trippe - 2018 All Rights Reserved. Rob Trippe, MBA