Model Audit - Shadow Modelling

Model Audit – there’s even more to it! Jerome Brice, July 2009 A version of this article appeared on IJ online Key terms: model audit, project finance, financial modelling, PFI, PPP http://www.mazars.co.uk/projectfinance jerome.brice@mazars.co.uk I read with great interest Andy Hucknall’s recent article ‘Model Audit – do you know what risks you are taking on’ which appeared on these pages. I very much welcome a debate on how to best obtain risk assurance from your model auditor and share his concerns that the standard of model audit service can be at best variable. However, while I agree with his observations on the symptoms of poor model audit practice and indeed to the spirit of his suggested solutions – I do take issue with three key elements of his diagnosis. Firstly, I do not believe that the ‘inspection’ methodology he advocates is superior to ‘shadow modelling’ ; instead I believe that shadow modelling as we at Mazars perform it is a more robust process that provides the end-to-end test of model logic that cell inspection cannot. Secondly, Andy advocates a ‘risk rather than price’ based approach and believes a ‘full audit’ should always be undertaken. I believe this is too simplistic; as every project financier knows risk and price go together. What is needed at the outset of a model audit is an informed discussion between auditor and client on the scope of work and a shared understanding of the assurance this will provide. Finally, Andy states that a model audit should be ‘assessed’ by ‘senior, experienced staff’ and mentions that his team includes ‘qualified accountants’ with a ‘corporate finance’ background. I would go further, model audits should be ‘undertaken’ by experienced staff with a ‘project finance’ background, the work should be ‘assessed’ at Partner level. The team should also have full access to technical teams who can give informed opinions on the latest developments in tax legislation and accounting standards. ‘Shadow modelling’ versus ‘inspection’ The first misconception I would like to correct is that shadow modelling is limited to the ‘top down’ elements of a full scope review. This, at least as undertaken at Mazars, is simply not the case. When we create a shadow model of the model under test the task is to fully reconcile all of the model logic – therefore this technique is just as granular or ‘cell-by-cell’ as the alternative ‘inspection’ approach.

The key advantage of shadow modelling is that if we use the same inputs as the model under test we expect to get the same answer. If the answer in any cell which impacts upon model outputs is different, we keep asking the question ‘Why?’ until either any error in the model under test is amended or we have a complete explanation as to why differences arise. Contrast this with the inspection approach where the auditor is comparing the model under test with a model in his or her head – my experience is that the model in my head is easily influenced if I am looking at the logic that someone else has created! Another key difference is the audit trail. At the end of the shadow modelling process we have a shadow model which fully reconciles any differences in the model under test – this forms a full proof of model logic from inputs through to outputs. Contrast this with the inspection approach – at the end of the process all that you have is a printed list of formula with some ticks! There is a further major disadvantage of the inspection approach - it is far too boring! The human brain is not designed to cope with repetitive checking – and errors will inevitably be missed. The shadow modelling process in contrast is an intellectual challenge, and even if the shadow modeller should become bored – the process doesn’t allow short cuts, the lack of proof of reconciliation is there for all to see. This observation is borne out by the research; Prof Ray Panko (the same expert Andy refers to) in his paper “What We Know About Spreadsheet Errors” updated in May 2008 refers to survey evidence which suggests individuals undertaking ‘code inspection’ can be expected to capture around 60% of spreadsheet errors. This would surely leave you with sleepless nights considering the 40% of errors that may be missed? While I do not claim that the shadow modelling is a panacea for all spreadsheet errors I do believe it is by far the best solution to the audit of project finance models. ‘Risk rather than Price’ or ‘Risk and Price’ In his article, Andy outlines a standard set of objectives for a model audit and suggests that this should always be followed – he views proposed changes to this set of objectives as an auditor ruse to ‘de-scope’ and save cost. These objectives are indeed familiar as the scope of work often set out by our clients and is one that we are happy to undertake. However, this does not mean that this scope of work cannot be discussed. Our main clients are leading banks and equity investors who are undoubtedly ‘informed’; bringing as they do experience of multiple deals backed by a team of lawyers specialising in ensuring that risks are properly allocated. As such, they are not naïve and cannot be hoodwinked into accepting a de-scoped service they don’t want. However, what they do welcome is a commercial discussion on the appropriate scope of work for the assurance they require. For example, they may wish to omit the tax requirement if they have retained separate tax advisors or alternatively, they may wish to reinforce areas where they have project specific concerns. In such cases we would of course have an informed discussion on the balance of risk versus price – this is a sensible commercial approach from both client and auditor perspective - one size does not fit all.

The ability to offer informed opinions One of the areas where I do agree with Andy is in the importance of the involvement of senior staff in the model audit process. There is a wider point here and while I have dwelt on the importance of a robust process to model logic, my experience is that this isn’t the major source of material error in models. The main sources or error are inconsistency of the model with contract documentation and non-compliance of the model with accounting standards and tax legislation. My main recommendations for avoiding errors related to contract documentation is to ensure that document review is undertaken by experienced staff who combine an understanding of commercial imperatives at financial close with the knowledge of the financial realities that the project company will face when operational. The second is to ensure that a clear process is agreed to ensure that a full suite of document is made available to the auditor prior to financial close. As clients of Mazars will recognise we facilitate this process by ensuring that a Partner or Director is fully involved and visible at all stages of the audit including guaranteeing our attendance at financial close. The other types of error which we see all too often in the operational environment relate to accounting and tax. We frequently see models which have had to be restated as the statutory auditor has not agreed the accounting treatment employed at financial close and we have also been contacted by project companies in difficulties because the tax assumptions have not held because they were either too simplistic or plain wrong. On the basis of this experience, I do not believe it will continue to be tenable for model auditors to sign off the ‘accounting’ and ‘tax’ if they do not have access to expert teams who fully understand the latest developments in international accounting standards and tax legislation. It is simply not enough to merely have ‘qualified accountants’ in the team or ‘access’ to a tax expert. For example, the impact of the implementation of International Financial Reporting Standards has far reaching consequences for infrastructure companies. Such developments can only be properly assessed by technical teams who liaise directly with the standard setters and who regularly work on international GAAP on conversion projects. Similarly, as HM Revenue and Customs becomes increasingly vigilant it is important concession companies have the most up-to-date advice on both corporation and indirect taxes. This sort of information cannot be the responsibility of one ‘go-to’ team member, it must reside within fully resourced expert technical teams. In concluding, I am aware that I have been robust in my responses to some of the assertions that Andy makes – while I feel this is justified, I am happy to accept that there may be different approaches that achieve the same objective of concrete model assurance. In addition, there are certainly two important views that I am sure Andy and I both share. The first is that model audits are very necessary; material mistakes in models are common and whichever model auditor is employed they are likely to add considerable comfort as to the deliverability of a transaction. The second is the observation that the choice of model auditor is all too often made solely on price. As the discussion above has demonstrated there

are key differences to the way that the leading model audit firms approach an audit and the value they add – to quote Andy once more, in selecting you model auditor ‘above all, you need to know what risk you are taking on’.

Model Audit - Shadow Modelling

by Jerome Brice

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 7

Statistics

Model Audit - Shadow Modelling Document Transcript

http://www.lmodules.com	5
http://www.linkedin.com	2