Model Audit - is a good model more expensive to
audit?
Andy Hucknall – Navigant Consulting
08/07/2009


There have been di...
One of the consequences of the FAST methodology is, thus, an increased number of
unique formulae. This debate is, therefor...
The assertion is that if a client's, or their advisor's, quality assurance procedures in
building a financial model are be...
As with the planning of a statutory audit, the amount of work undertaken is
dependent on the assessment of the risk of a m...
Upcoming SlideShare
Loading in …5
×

Model Audit - is a good model more expensive to audit?

568 views
498 views

Published on

Financial Models built to the FAST standard have more unique formulae than other financial models, but are better built. In this article, I discuss why a good model should not be more expensive to model audit.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
568
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Model Audit - is a good model more expensive to audit?

  1. 1. Model Audit - is a good model more expensive to audit? Andy Hucknall – Navigant Consulting 08/07/2009 There have been discussions recently on the client forum of Financial Mechanics as to whether building a financial model to best practice leads to an increase in the cost of the model audit. This may sound counter-intuitive at first, but let me explain why this discussion has arisen.</P< span> Financial Mechanics, a financial modelling specialist, is a member of the FAST Modelling Alliance, with 'FAST', being: • FLEXIBLE - model design and modelling techniques must allow models to be adapted easily and quickly when new information becomes available • ACCURATE - models must reflect key business assumptions directly and faithfully without being over-built or cluttered with unnecessary detail • STRUCTURED - rigorous consistency in layout and organisation is essential to retain a model's logical integrity over time, particularly as a model's author may change • TRANSPARENT - modelling approach is founded on simple, clear calculations that can be understood by other modellers and non-modellers alike Financial models built to best practice have formulae that are consistent across a row of the model, and often across a whole block. The formula on the left of the row, or the top left of the block, is said to be the unique formula. A model auditor is, thus, able to review the unique formula rather than every formula in a row or block, considerably reducing the workload and cost. What the FAST methodology advocates is simplicity in the formulae. Where a non- FAST model may have formulae incorporating multiple logic tests, FAST advocates short, simple, direct and readily understood formulae. The result is that complicated, multiple logic formulae are written as separate, linked formulae with a single logic test in each formula.
  2. 2. One of the consequences of the FAST methodology is, thus, an increased number of unique formulae. This debate is, therefore, confined to FAST and non-FAST financial models that are built with 'left-right' and 'top-down' consistency, and not between FAST methodology and badly structured models. The question on the Fi-Mech forum was "Does the FAST Method drive up model audit costs?" on the premise that the number of unique formulae is greater. On the assumption that the number of unique formulae is greater, yet the risk of material errors should be lower, is would seem perverse that the model audit cost should go up. There are two things that the model auditor can take into account to offset the increased number of formulae: • complexity factors • capability risk Complexity Factors The complexity factor is a measure of how complicated a formula is, and from that, an assessment can be made as to how long a formula takes to review - the greater the complexity, the greater the time. Many model audit software packages provide an assessment of the model in terms of the number of unique formulae, and the complexity of each formula. A model built using the FAST method is likely have lower complexity factor formulae, although more of them, than other models. Whether these two elements offset will depend on individual circumstances. Capability Risk The terminology "Capability Risk" came from Gavin Townshend, head of financial modelling at John Laing plc. It is a good way of expressing how a model auditor will assess a financial model beyond the statistics of unique formulae and complexity factors. Using Gavin's words, is "the more robust a client's own internal review processes are (e.g. running sensitivity tests and analysing the results, commercial sense checking, etc.) and the more capable the client's own people are at performing such tasks, the lower the risk of significant model errors", and therefore, the lower the Capability Risk
  3. 3. The assertion is that if a client's, or their advisor's, quality assurance procedures in building a financial model are better, there is a lower risk of material errors in the financial model. As the model auditor uses a risk-based approach to planning and undertaking the model audit, if the Capability Risk is lower, then the amount of work needed to be undertaken by a model auditor to manage the risk should be lower, and hence, less costly. Combination The building of a financial model to FAST modelling standards in itself will, other things being equal, lead a model auditor to assess the capability risk to be lower. Even if the combination of more unique formulae, but at a lower complexity factor, leads to an increased total length of time to undertake the detailed 'bottom up' review of each unique formulae, the reduced capability risk will mean less time is needed to undertake the 'top down' analysis. It should be noted that the detailed 'bottom up' review of unique formulae is less risky than the 'top down' analysis, and hence can be undertaken by less senior, and hence, lower cost, staff. So even if the time required for 'bottom up' review is marginally longer for FAST method built financial models, the extra cost is more than offset by the time saving, at higher hourly rates, in 'top down' analysis. To reiterate from my earlier article "Model audit - do you know what risk you are taking on?" (IJ Online, 18 May 2009), the 'top down' analysis will include: • the re-performance of key calculations based upon the project's documentation • analytical review of trends in key outputs • sensitivity analysis • commercial sense checking The amount of time budgeted for 'top down' analysis is a judgemental assessment by the model auditor based upon experience, of which Capability Risk is a major influence. Proper planning (and risk assessment) prevents poor performance (and mitigates the risk of missing material errors)
  4. 4. As with the planning of a statutory audit, the amount of work undertaken is dependent on the assessment of the risk of a material error. The same applies to model auditing, and thus, the lower the capability risk, the lower the amount of work that needs to be undertaken to mitigate the risk of material errors. This highlights the need for proper risk assessment in planning the model audit, and advantages of the planning being undertaken by experienced model auditors, ideally a qualified accountant with statutory audit experience. Summary So to answer the question "Does the FAST Method drive up model audit costs?" My answer would be that it shouldn't do. More than that, reducing the Capability Risk by building a financial model to best practice should reduce the cost and financial risk to the client themselves as the model will be easier to update and optimise in the run up to financial close with less likelihood of material error. Author: Andy Hucknall is an associate director at Navigant Consulting. To contact him, click here...

×