Model Audit

Model Audit – do you know what risk you are taking on? Andy Hucknall – Navigant Consulting 18 May 2009 If you have been involved with a project / infrastructure finance or PPP/PFI deal that has a financial model (i.e. all of them!), you may have heard that the model needs auditing, but why is it needed, what is a model audit and how is it carried out? It may not sound like the most exciting part of transaction, but that does not make it any less important. The model audit market in the UK has been dominated by three players; but we at Navigant Consulting are the new entrants, led by a senior team with more than 15 years of combined experience. We have a different approach, focussing on risk rather price, and the feedback that we are getting from both sponsors, and particularly funders, is that this is long overdue. Model auditors for the past few years have been primarily selected on the basis of the cheapest quote, leading to a “de-scoping” of the model audit. Model auditors have been doing the minimum amount of work possible in order to keep the costs down. Typically, it is the detailed ‘bottom up’ review of the financial model’s formulae that is avoided, with reliance placed upon a ‘shadow model’ that seeks to re-perform the calculations. Given the current riskier environment, and competitive dialogue procurement requiring project sponsors to commit to their numbers at an earlier stage, we believe that “de-scoping” of the model audit is a risky move. We have always advocated the approach of combining the bottom-up approach of reviewing every unique formula with high level analytical review, re-performance of key calculations, sensitivity analysis and documentation review to ensure that the model is consistent with the project documentation. Our team considers this to be less risky than using the shadow modelling approach. Why do you need a model audit? Banks require a model audit as a condition precedent to lending, but it is the project sponsors who pay for the model audit. It has been in the project sponsors’ interest to keep the cost down, so they have pushed to de-scope the model audit as much as possible, relying upon the model auditor’s professional indemnity (PI) insurance if something goes wrong. We are increasingly seeing the banks demanding a full scope model audit as they see the model audit as a key part of the risk management process. There are numerous documented cases of undetected critical errors in spreadsheets leading to ill-informed decisions that can result in mis-pricing, lower returns on investment than expected and misinformation being provided to stakeholders and regulatory bodies. All may result in financial loss or reputational damage. Do you really want to take on a significant financial risk based upon a potentially erroneous financial model? Assuming that the answer is no, de-scoping the model audit is a false economy.

What is a model audit? A model audit is an independent review of whether a financial model meets an agreed set of objectives. The model auditor, thus, reports whether those objectives have been met, and any exceptions. A typical set of objectives may be that: • the calculations in the model are arithmetically correct • the results are materially reliable, accurate, complete and consistent with the assumptions contained in the model • certain ratios are calculated correctly and in line with the definitions from the financing agreement • the accounting treatments and assumptions applied within the Model are consistent with current local GAAP and with key provisions of the Project’s financing documentation • the tax assumptions applied within the Model are consistent with current local tax legislation and with key provisions of the Project’s financing documentation as provided • any unexplained trends or variations in key financial and banking indicators in Model outputs are identified through analytical review • any unexplained inconsistent or unintuitive cash-flow trends (including revenues, costs, taxes, depreciation) or variations in key financial indicators based on the inputs and the Project’s commercial structure are identified through analytical review of Model outputs • the results produced from changes to underlying assumptions accurately and completely reflect the potential impact of those changes If a model auditor is de-scoping, which of the above objectives can be removed? It is doubtful whether any of the above should be removed, thus other techniques are used to limit the scope (and the hence the cost) such as limiting the number of iterations of the model and documentation that are included in the fee (with excessive cost for additional iterations), charging for “extras” that should be included, such as third party duty of care letters, or providing less affirmative final reports (audit opinion). The last point is particularly important. A clear, affirmative statement in the final report that the objectives have been met provides comfort to those relying on the report. If the model audit has been de-scoped, the final report wording may simply state the agreed procedures were performed, thus potentially absolving the model auditor of liability for omissions. How should a model audit be undertaken? A model audit should, first and foremost, be tailored to achieve the agreed objectives, and thus, each model audit should be planned individually. It should combine a detailed ‘bottom-up’ review of each unique formula with ‘top-down’ analysis, such as: • the reperformance of key calculations based upon the Project’s documentation • analytical review of trends in key outputs • sensitivity analysis • commercial sense checking This combination of detailed ‘bottom up’ review with ‘top down’ analysis provides the best possible chance of identifying material errors. Experience shows that the detailed ‘bottom up’ review and ‘top down’ analysis identify different errors that using only way technique may not identify, thus de-scoping the model

audit by ignoring the detailed ‘bottom up’ review significantly increases the risk of material errors not being identified. Academic research by the expert in this field, Prof. Ray Panko (University of Hawai’i), in his paper “Spreadsheet Errors: What We Know, What We Think We Can Do” to the European Spreadsheet Risk Interest Group (EuSpRIG) Symposium 2000 demonstrated that the independent cell by cell inspection is almost the only way of systematically identifying errors. We believe that the model audit should be assessed and planned by senior, experienced staff, and that the senior staff should undertake the parts of the model audit identified as higher risk during the planning stage. Our senior team are all qualified accountants with a statutory audit background, and bring this risk management approach to model auditing, whilst our team are all experienced professionals with a corporate finance background. We just do not believe that model audit is something that can be undertaken by graduates in their first job without wider finance experience. In many cases, the model auditor will simply limit costs by both avoiding work such as the detailed review of formulae, and by using more junior, inexperienced staff, relying upon their PI cover. Shadow modelling – the risk Some model auditors dismiss the detailed review of formulae as a “tick and bash” exercise, even calling it ‘traditional’ model audit as if imply to imply it is out of date. The alternative proposed by a few model auditors is ‘shadow modelling’ whereby project inputs are put into their own model, and results compared, i.e. re-performance. That is not to say that re-performance is not useful, rather it should be one of a range of techniques that is used, rather than the only one. Shadow modelling, thus, is limited to the “top down” elements of a full scope review, saving cost, but increasing risk. Your choice? Given that shadow modelling is being used by several firms, always ask what approach your model auditor will be using! If you do choose to de-scope the model audit to save money, be aware of the extra risk that you are taking on, and ask is it worth it? The model audit is a key financial risk management tool, not a box ticking exercise. As such, it needs to include a proper scope of services to mitigate the financial risk that is being committed to. Given that financial commitment is occurring earlier in the process under competitive dialogue, we are increasingly seeing some independent review taking place earlier in the process, often known as a ‘pre-preferred bidder’. This is common practice in the rest of Europe, but we hope and expect that project sponsors will require some independent review of their financial models before preferred bidder. Above all, you need to know what risk you are taking on - particularly if you are prepared to risk a de-scoped model audit using shadow modelling. Andy Hucknall is an Associate Director at Navigant Consulting.

Model Audit

by Andy Hucknall on May 18, 2009

Accessibility

Upload Details

Usage Rights

2 Embeds 41

Statistics

Model Audit Document Transcript

http://www.lmodules.com	38
http://www.linkedin.com	3