How To Forecast Accurately In Uncertain Times

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    6 Favorites

    How To Forecast Accurately In Uncertain Times - Presentation Transcript

    1. How to Forecast Accurately in Uncertain Times A guide to the key issues that reduce the accuracy of forecasts and how Finance functions can address them Date – December 2008 Rational Strategies Scientifically better decisionsTM
    2. MANAGEMENT SUMMARY In today’s uncertain economic conditions the ability to forecast accurately is vitally important • A recent survey by CFO magazine revealed that the ability to produce accurate forecasts was the second greatest internally focussed concern of CFOs: – CFO magazine’s findings can be found at http://www.cfo.com/article.cfm/12492577?f=related • While the uncertain conditions do make it more difficult to forecast accurately, it is nevertheless possible to improve on their accuracy: – And the key to doing so is to understand the key issues that reduce the accuracy of forecasts, the majority of which are actually present regardless of whether conditions are steady, or uncertain. • To help finance functions do just that, this document outlines the key issues that impact on forecast accuracies and suggests how to address them: – It’s not intended to be a “how to” guide to forecasting, as that depends critically on the particular type of forecast a finance function is trying to produce, the approaches they’re currently using, the availability of data etc. Rather this document provides a starting point for finance functions, suggesting areas to investigate. • This document is structured into the following sections: – An overview of the key issues reducing forecast accuracies; – Actions that Finance functions can take to improve the accuracy of existing forecasting approaches; – An overview of a quantitative forecasting technique suited to dealing with uncertainty. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 2
    3. So, why is it so difficult to forecast accurately? - The key factors that reduce forecast accuracies
    4. THE KEY FACTORS REDUCING FORECAST ACCURACY The accuracy of many forecasts are reduced by issues in the following three key areas... The Generation • Forecast models, whether financial or economic, are built upon input data and the assumptions of Forecast made about that data. • Uncertainties in input data, assumptions that aren’t sufficiently rigorously challenged and human Inputs nature failings in the ways that we perceive information, all reduce forecast accuracy. • Many Finance departments rely on spreadsheets to produce financial forecasts and other outputs - unfortunately, spreadsheet based models (of any type) are inherently error-prone. Forecasting • Additionally most finance professionals haven’t received appropriate spreadsheet training, and Model Issues spreadsheet design guidelines are rarely used and applied. • Many spreadsheet based models are overly-complex, which can reduce forecast accuracy, particularly when there’s uncertainties in the input data. The Way • When forecast models produce unpalatable results an occasional reaction is to “massage” the results, or worse, to request that analysts rework their models to produce “acceptable” results. Forecast • If the forecast models are valid then either their forecasts should be regarded as valid, or the Outputs Are model abandoned completely, in favour of qualitative forecasting approaches. Used • Massaging the message or reworking models simply reduces forecasting accuracy. We’ll briefly examine each of these areas over the next few pages, to gain an insight into the actions Finance functions can take to improve the accuracy of their forecasts. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 4
    5. THE KEY FACTORS REDUCING FORECAST ACCURACY – THE GENERATION OF FORECAST INPUTS Starting with forecast inputs, there are three major issue areas that can significantly reduce forecast accuracies The inputs to forecasts are frequently undermined by …and depending on the situation, these can the following issues… significantly reduce forecast accuracies. Uncertainties, • Inevitably there will be shortcomings in input business data , particularly data gaps and uncertainties: Ambiguities and – For example, in today’s volatile business environment there will Gaps in Input be significant uncertainties around any data based on projections of historic performance (eg. sales, order volumes). Data – There will also be outright gaps in the input data, for example when entering new markets, or launching new products. – As outlined later in this paper, rigorous handling of these data shortcomings will help ensure high forecast accuracy. • While much effort is expended obtaining input data, the The Generation Insufficiently same attention is rarely paid to key input assumptions: of Forecast Rigorous – All forecasts are built on key assumptions and the validity and Inputs Assumptions accuracy of any forecast can only be assessed when presented alongside those assumptions. For example, a forecast based on growing product sales by 10% would have a very different validity than another assuming growth of 2%. – Identifying and challenging key assumptions therefore needs to form a core part of the forecasting process. • Behavioural economics reveals that human beings are Human Nature rather less rational than we might think: Failings – Particularly with respect to how we perceive data and information in the light of what we’d like to believe. – As explained on the following page these human failings have important implications for any attempts at forecasting. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 5
    6. THE KEY FACTORS REDUCING FORECAST ACCURACY – THE GENERATION OF FORECAST INPUTS Similarly to the three wise monkeys, human beings don’t see, seek, or use information in the way that we think we do Behavioural economics research has recently revealed shortcomings in the way that human beings interact with information, in particular: 1: • That we fail to “see” key information, particularly when we’re busy: – For example, Merck disclosed to regulators in 2000 major problems with the drug Vioxx. In spite of this, an additional 1 million prescriptions were written by doctors between then and when the drug was withdrawn from the market in 2004, contributing to a n estimated 25,000 heart attacks and strokes. – Doctors, like corporate decision makers, are busy, imperfect decision makers and overlooked these critical findings. • That we fail to “seek” information that contradicts our viewpoints: – Frequently decision makers favour a particular outcome and so fail to seek out contradictory information. – The most devastating recent example of this led to the Spaceship Challenger’s destruction. The night before the launch NASA executives argued about whether that night’s low temperatures would affect the O-rings but concluded that it wouldn’t. Subsequent investigations into the disaster revealed that had they checked the O-ring data on the prior 24 launches they would have discovered that the probability of a disaster exceeded 99%. • That we fail to “use” information: – For example, information about competitors. Research at Carnegie Mellon University demonstrated that decision makers focus on how well they can perform a task but tend to ignore how well the competition can do the same task. As a result decision makers have a tendency to enter product domains offering easy access, and to enter more difficult product domains too infrequently. – And as all the competitors tend to behave in the same way then competition is increased in the “easy markets” and decreased in the more difficult markets. The upshot of which is that even when we think we’re rationally building the most accurate forecast model we can, we may be overlooking key data items, or moulding the data to fit our viewpoints, or simply not making appropriate use of the information we’ve got. 1. Source: Adapted from the Harvard Business Review, Decision Making, January 2006. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 6
    7. THE KEY FACTORS REDUCING FORECAST ACCURACY – FORECASTING MODEL ISSUES Having considered input related issues, we now need to consider issues relating to forecasting models themselves Forecasting model issues fall into two major categories, those relating to the spreadsheet technologies underpinning most forecasts and those relating to the complexities of the models used: • Spreadsheets are an inherently error-prone technology: – Data can be accidentally over-written or misreferenced, cell formulas can be incorrectly written or copied and can be hard to read and audit; – Hard coded values such as tax rates can be buried deep within sheets and missed when updates are needed. • They’re poorly suited to dealing with uncertainties and ambiguity: Spreadsheet – Spreadsheets force the entry of single values into cells, making it complicated to allow for uncertainties and ambiguities in data; the generation Related Issues of multiple scenarios to allow for these ambiguities can be time consuming . • These issues are compounded by the fact that most financial practitioners lack appropriate spreadsheet training: Forecasting – Frequently resulting in poorly structured, hard to audit models, where the logic, assumptions and calculation approaches are hard to follow and difficult Model Issues to subsequently change without introducing errors. • And further compounded by the typical lack of in-house spreadsheet design guidelines and standards: Overly Complex – With individual financial practitioners developing their own personal Models approaches to developing spreadsheet models, increasing the risk of errors and reducing their auditability. • The upshot of which is that there are pervasive levels of errors in spreadsheets: – For example, separate surveys carried out by KPMG and Coopers and Lybrand in the late 90’s found that 91% of corporate finance spreadsheets contained significant errors. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 7
    8. THE KEY FACTORS REDUCING FORECAST ACCURACY – FORECASTING MODEL ISSUES Having considered input related issues, we now need to consider issues relating to forecasting models themselves (continued) Forecasting model issues fall into two major categories, those relating to the spreadsheet technologies underpinning most forecasts and those relating to the complexities of the models used: • In most cases, increasing the complexity of a forecast model will actually lead to a decrease in the accuracy of its forecasts. The reasons for this include: • The uncertainties in the input data: – As noted earlier, there are always uncertainties in input data. As the number of variables in a forecast model rises, these uncertainties will combine in Spreadsheet unpredictable ways, potentially significantly degrading forecast accuracies. Related Issues • Overlooked correlations between input data items: – The correlations between input data items needs to be factored into forecast models and as the number of data items rises, correlations are likely to overlooked. Forecasting – For example if a sales forecast model includes items relating to individual Model Issues product sales and to product commissions paid to sales people, then the correlations between those two items needs to be factored in, as changes to commission structures will clearly have a correlated impact on product sales. Overly Complex • The reduced clarity as to what the major drivers are underlying the Models forecasts: – As model complexities increase, it becomes increasingly difficult to determine which elements of the model are driving it’s key forecast characteristics, therefore also making it more difficult to refine the model. – In addition, when it’s not possible to determine the key drivers of forecast outputs, then decision makers are deprived of key levers that they can use to drive the business’ performance. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 8
    9. THE KEY FACTORS REDUCING FORECAST ACCURACY – THE WAY FORECAST OUTPUTS ARE USED Finally, the value and accuracy of forecasts can be significantly reduced by the way in which they’re used This is a complex issue and it’s sufficient to simply note that: • Overt or subtle pressure placed on analysts can reduce the accuracy of forecast models: – Overt pressure simply taking the form of senior personnel or colleagues not believing the outputs of a model and telling analysts to rework them, to deliver more “acceptable” results. – More subtle pressures include peer expectations. For example during the housing bubble in the US and the UK between 2005 and 2007, few housing analysts were comfortable forecasting house price falls, in spite of the fundamental indicators pointing in that direction (credit over-stretch, surplus housing stocks etc). • Forecast models have, by their nature, to deal with probable outcomes, as the future is never 100% predictable: – If this aspect of a forecast model is not understood by the output users, then erroneous decisions may be made. – For example if a model forecasts that sales will be, say, £10m in a given region for Q1 2010 and a firm only has capacity to deal with £7.5m of sales, then it may seem reasonable to invest £500k in sales capacity to realise that extra £2.5m in sales. Reasonable that is, unless the forecast is only made with a 60% certainty level, in which case significantly different management decisions might be taken. So, having identified the major issues that reduce forecast accuracies, let’s now consider the actions we can take to improve the accuracy of our forecasting approaches. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 9
    10. What can we do to improve the accuracy of our existing forecasting approaches?
    11. IMPROVING THE ACCURACY OF EXISTING FORECASTING APPROACHES Working through each identified issue in turn, there are actions we can take to improve the accuracy of existing approaches Issues Reducing Actions Finance Can Take To Additional Comments Forecast Accuracy Address These • Spreadsheet models are ill-suited to • Use expert input and /or benchmark dealing with uncertainties, so either move Inputs – data data to close gaps and remove to qualitative forecast approaches, or if 1 uncertainties, ambiguities and gaps ambiguities, logging these as assumptions but importantly investigating quantitative outputs are needed then consider using Monte Carlo techniques their impacts on the forecasts. to deal with the uncertainty (see page 14). • Identify key assumptions, review and • These issues are best tackled together, challenge them in light of current and by altering existing forecasting future circumstances. processes, critically incorporating key Inputs – Insufficiently 2 rigorous assumptions • Investigate their impacts on forecasts. challenge stages as the inputs and • Provide key assumptions alongside assumptions are being reviewed and forecasts, so that forecast validity can be using rigorous group review processes to assessed. try and avoid group think and human failings. •Remedying this issue is exceedingly • Edward De Bono’s works, particularly difficult as it applies both to individuals his “Thinking Course” provide tools and and groups (group think issues). 3 Inputs – Human failings • Rigorous challenge and review techniques that can be used to ensure rigour in the inputs. processes are the most pragmatic http://www.amazon.com/Bonos-Thinking- approach to trying to address these. Course-Revised/dp/0816031789 Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 11
    12. IMPROVING THE ACCURACY OF EXISTING FORECASTING APPROACHES Working through each identified issue in turn, there are actions we can take to improve the accuracy of existing approaches (continued) Issues Reducing Actions Finance Can Take To Additional Comments Forecast Accuracy Address These •Introduce in-house spreadsheet design • Best practice spreadsheet design guidelines and provide training in guidance can be found throughout the Forecasting Model spreadsheet modelling and design. web. A useful starting point is the 4 Issues – Spreadsheet related issues • Have IT convert key forecasting spreadsheets into VBA based Institute of Chartered Accountants guide to spreadsheet best practice, which can spreadsheets – they’ll run faster and with be found at: fewer errors. http://www.eusprig.org/smbp.pdf • Rebuild simpler forecast models from the ground up, focussing on business • Partial model rebuilds and refocusing Forecasting Model drivers. are tricky and there’s no guarantee that 5 Issues – Overly complex models • Where time or resources prevent this, focus on identifying key business drivers the resulting models will be more accurate than the original models. in existing models and making those model elements as robust as possible. • Again this is a difficult issue to address as it’s impacted on by the corporate decision making culture. The Way Forecast 6 Outputs are Used • Presenting forecasts along with their assumptions and their likely probabilities are pragmatic approaches to dealing with this problem. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 12
    13. And if I need to produce the most accurate possible quantitative forecasts in today’s uncertain environment? - Which forecasting techniques are the most accurate in these conditions?
    14. ACCURATE QUANTITATIVE FORECASTING TECHNIQUES FOR VOLATILE CONDITIONS From a Finance perspective, the single most difficult forecasting issue to deal with is that of uncertainty in inputs Conventional forecasting and budgeting ...unfortunately in volatile conditions, many approaches require certainty in inputs... inputs are likely to be uncertain • For example inventory volumes will depend on sales volumes and these are likely to lie in an estimated range: – For example, perhaps lying between £10m and £15m, with a best guess most likely out turn of £12m. – Conventional forecast approaches like the one represented on the left require a single best guess estimate to be entered for this input, say £12m, which if sales come in at the bottom of the range, will have overstated sales by £2m, or if sales come in at the top of the range, will have underestimated sales by £3m, both being poor outcomes. • While uncertainties in a single input can be dealt with by generating multiple forecasts, this is impractical for higher numbers of inputs: – Forecasters sometimes allow for the uncertainties in a range of inputs by creating multiple forecasts (eg. high, medium, low) that try to encompass all the possible outcomes of all the uncertainties. – Given the complexities in the way that uncertainties (or probabilities) combine and aggregate up, then the likelihood that such aggregate forecasts will effectively capture the total range of outcomes, and most likely outcomes, from such combined uncertainties, is almost vanishingly small. – However, generating sufficient numbers of forecasts to allow for the range of possible outcomes in each of the input items is impractical when dealing with more than 4 or 5 uncertain inputs (as, for example, generating 3 forecasts for each of 3 uncertain inputs would require the generation of 27 separate scenarios to capture all the possible outcomes). Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 14
    15. ACCURATE QUANTITATIVE FORECASTING TECHNIQUES FOR VOLATILE CONDITIONS The most accurate way of dealing with these uncertainties is to use “Monte Carlo” modelling techniques • Quantitative modelling techniques that can be incorporated into spreadsheet models: What They Are – Importantly allowing input information to be entered as ranges (probabilities) and using these to determine the range of likely outcomes and the probabilities with which they’ll occur, for example: Monte Carlo Where and • They’re well established, widely used techniques: Modelling How They’re – Being used in such widely disparate fields as finance, project management, energy, manufacturing, engineering, research and Techniques Used development, insurance, oil and gas, transportation, and the environment. • There are many benefits to using such approaches when Benefits Of developing forecasts: Using Them – The ability to effectively (in time and resources) identify the most probable outcomes, along with the full range of outcomes and their probabilities of occurring. – The ability to determine the key factors that drive the forecast outcomes and hence provide decision makers with the levers with which to influence their business outcomes. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 15
    16. ACCURATE QUANTITATIVE FORECASTING TECHNIQUES FOR VOLATILE CONDITIONS While commercial packages are available that simplify Monte Carlo modelling, care is still needed! • Two major vendors of spreadsheet (Excel) compatible Monte Carlo modelling packages are Palisade Corporation and Oracle: – Palisade Corporation sells the DecisionSuite stable of products, one of which, @Risk, is a Monte Carlo simulation package. Information on Palisade’s products can be found at http://www.palisade.com/ – Oracle sells a package of similar capability, Crystal Ball, details of which can be found at http://www.oracle.com/crystalball/index.html • While these and similar packages can help produce more accurate forecasts, they do introduce another issue that needs to be approached with care – that of model risk: – Quite simply these packages allow relatively complex forecasting and financial models to be built with relative ease and in inexperienced hands, seemingly realistic forecasts can be produced that are in fact wildly inaccurate; – For example, the uncertainties in the input data need to be specified with care and the pre-defined distributions supplied with such packages make it easy for users to select inappropriate and unrealistic representations of those uncertainties; – Similarly, correlations between input parameters need to be handled with care if they’re not to undermine the model’s forecast accuracy. And that brings our overview of forecasting in uncertain times to an end. If you’d like further information on any of the topics covered in this paper then our contact details are noted on the following pages. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 16
    17. About Rational Strategies
    18. ABOUT RATIONAL STRATEGIES Rational Strategies helps Institutions use modelling approaches to make better business decisions • Rational Strategies brings academic rigour combined with For further information on real world pragmatism: any of the topics covered in – Dr Giulio Galassi, Rational Strategies’ founder, is a leading expert in the use of this paper, contact: modelling and simulation techniques to help institutions make better business decisions. Dr Giulio Galassi – Having completed a doctorate in Theoretical Physics, where he gained world Rational Strategies class experience in the use of modelling and simulation techniques, Dr Galassi entered the Banking Industry, initially putting his modelling and simulation Email: expertise to work on the Royal Bank of Scotland’s dealing floor. ggalassi@RationalStrategies.co.uk – Dr Galassi’s career subsequently progressed through increasingly senior roles in major Blue Chip companies and leading consultancies, including stints leading Abbey’s internal consulting team and heading up CSC’s Financial Services consulting group. – In late 2007 Dr Galassi set up Rational Strategies to focus on helping clients make better business decisions using the modelling and simulation approaches in which he’s a leading expert. – With over 15 years experience in industry, Dr Galassi has an excellent understanding both of the challenges facing senior industry executives and the difficulties facing them in making major decisions, when information is lacking, incomplete, there’s a high degree of complexity or changes in market conditions throw up uncertainties. – In spite of his deep academic background Dr Galassi is a strong believer in using pragmatic approaches to help clients make their decisions and resolve their issues. Having been an academic in the past he’s able to avoid the trap of “analysis paralysis” that hits so many institutions, making sure that his clients get pragmatic, actionable insight from his work with them, rather than analyses that are overly academic or simply not actionable. Rational Strategies • Proprietary and Confidential How To Forecast Accurately in Uncertain Times | 18
    SlideShare Zeitgeist 2009

    + Giulio GalassiGiulio Galassi Nominate

    custom

    1168 views, 6 favs, 2 embeds more stats

    In today's uncertain economic conditions, Finance f more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1168
      • 1164 on SlideShare
      • 4 from embeds
    • Comments 0
    • Favorites 6
    • Downloads 80
    Most viewed embeds
    • 2 views on http://www.visualcv.com
    • 2 views on http://www.brijj.com

    more

    All embeds
    • 2 views on http://www.visualcv.com
    • 2 views on http://www.brijj.com

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories