The Amazing Science and Delicate Art of Forecasting

The Amazing Science and
Delicate Art of Forecasting
Cyrille Betant, March 2015

Summary
1. The notion of forecasting
2. The mechanical tools for forecasting
3. Scaling your forecast
4. Looking beyond the numbers
5. Communicating the forecast
6. Rolling forecast
7. The value of crowd forecasting
8. Important factors to remember

1. The Notion of
Forecasting
What is a forecast?

Forecasting, an Overview and Brief Definition
If the plan is your destination,…
…the forecast is the road leading to it.
- Creating a forecast is literally engineering the road that is going
to lead you to your target goal, your destination.
- That road, however, will have to take into account the landscape it goes through and
may very well change as your project unfolds.
- To know what the road looks like, you need to know where you are and where you
are going.
- Hence, the forecast is the financial expression of what you anticipate will happen in
your operations on top of what you know already happened.
- This is a representation of what your financials are going to look like at the end
of your forecasting horizon (whatever your budget is), where the road leads.

In Short
From the dawn of humanity, a disease, with one of the
highest rate of mortality ever, has been plaguing every human
life and every human endeavor:
Wishful Thinking
Whether the cave man thinking that they can outrun
that nasty saber-tooth tiger…
…or the financial trader thinking that the
market will continue to go up forever,
all are suffering from the same devastating disease that
only a good forecast can cure.

The 5 Key Questions
In essence, to create a forecast, you need to answer these
five key questions:
1) Who
2) What
3) When
4) Where
e.g. through its agreement with pharmacy distributors
e.g. the consumer product division
e.g. will introduce a new toothpaste
e.g. next quarter
e.g. in all major North American markets
5) How

Rules of the Road
Many of your forecasters will not have a finance education or background and
might be unfamiliar with what the forecast exercise is really about. It is always
good to remind them these basic tenets:
A Forecast IS A Forecast IS NOT
- A candid view of your
operational results
- An opportunity to re-negotiate
your budget
- A way to hold on to previously
planned spending
- A prediction of what your
financial results are likely to be
- An opportunity to assess
actions needed to achieve your
budget goals
- A change in your budget
goals
- A mechanism to voice as
early as possible operational
risks in your plans
- A tool to massage your
numbers for messaging
purpose

2. The Mechanical Tools for
Forecasting
How do you create a forecast?

Extrapolation, Projection and Forecast
Different operational or tactical needs call for different tools and
processes to project future results with the accurate granularity.
An extrapolation is the prediction of a future state where every element
known today to be relevant to that prediction will remain unchanged.
A projection is the prediction of a future state where elements known
today to be relevant to that prediction will change.
A forecast is the prediction of a future state where elements known today
to be relevant to that prediction, as well as new elements will change.
This is if nothing at all changed, usually not a likely scenario
This is a useful scenario to reflect known future changes (e.g.
the rent will go up)
This is the most complex scenario, it integrate changes in unknown
future factors (e.g. a competitor will open shop next door)

The Notion of Forces
Let’s represent the elements that affect our predictions as “forces”.
These forces act on our results in a neutral, positive or negative way. In
addition, there is an intensity to these competing forces that will influence
the results (e.g. a mild negative force vs. an enormous positive force).
1 – A force can be:
2 – In addition, a
force can be:
Positive (e.g.
increase in
customers)
Negative (e.g.
increase in
costs)
Or Neutral (e.g.
change of
landlord)
Small (e.g. a
0.3% increase
in rent
Medium (e.g. a
3% change in
web traffic
Or Big (e.g. a
10% decrease
in revenue
3 – Lastly, a force
can be:
… ..| … … … …
Limited in time Permanent Cyclical

Aggregating the Data
Once you have defined the right tools/formulas and quantified the force factors
influencing your forecast, you need to aggregate your data to see the “big picture”
Current
Market
Future
Market
InflationInflation
Distributor
agreements
Distributor
agreements
Distributor
agreements
Distributor
agreements

In Summary
When building your forecast, regardless of whether you represent it
graphically or numerically, it is important to understand the influence of the
following criteria:
1 – Determine all the existing forces that will influence your results and that
you know about (e.g. changes in relationship, pricing, etc.).
3 – Determine how these forces will influence your results, when they start
having an influence and how much they will influence your operations.
4 – The better you are at defining the big picture in your current and future
environment, the easier it is going to be to translate it into sensible
numbers.
As a consequence, you should always ask yourself, not whether your
numbers are right, but whether your assumptions are.
2 – Determine all the new forces that you anticipate will emerge or existing
ones that will stop being relevant (e.g. introduction of a new product by a
competitor, obsolescence of your equipment or technology, etc.).

3. Scaling Your Forecast
What order of magnitude is
relevant?

Time Sensitivity
When is the right time to do a forecast and what is the right time
horizon to consider?
Here are 4 criteria to keep in mind:
1 – Determine what you want to measure (e.g. revenue, attendance,
inventories, employee attrition, customer turn-over, etc.)
3 –Determine the volatility of your operations, in other words, how often your
measured data is likely to change, to determine how often you should forecast.
4 –Determine how far in advance you need to predict your operations (the end
of the quarter, the end of the year, 12 months ahead, etc.) to determine the
time-horizon of your forecast.
2 – Determine when the previously available data becomes obsolete to define
when you should start your forecast (e.g. when your budget becomes less of a
realistic plan, is it on month 1 or after the 1st quarter?).

Appropriate Level of Detail
When building your forecast, one of the early
questions you have to ask yourself is: What is the
right granularity, should I predict my results to the million or to the cent?
1 – Determine the level of details at which the numbers will be looked at (e.g.
is it a summary income statement rounded at the million level, or a detailed
group P&L rounded at the ten-thousand dollars level?
3 – Determine who in your constituency should submit their forecast (e.g. all
units below the division, all entities that have a project manager, only the main
units?).
4 – Always remember that the answer might be different for each month,
unit and line item (e.g. unit A should always report, but unit B only if there
is a change of more than 10%, new trips don’t need to be reported but new
CAPEX purchase always do, etc.).
2 – Determine the level at which your constituents can make their own
unplanned transactions without impacting your results (e.g. any travel below
$1,000, any purchase approved in your operations below $10,000?).

4. Looking Beyond the
Numbers
How do I ensure that my forecast
is reasonable?

Keeping Everybody Honest
Now you have produced a forecast, but is it accurate?
Here are 5 factors that will help you ensure that your numbers make sense:
1) Proof read the numbers: Your forecast arithmetically works, but needs to be
validated to ensure reasonableness. It can be as simple as checking against a run-
rate, or as complex as validating against a predictive algorithm, but there needs to be
a “mechanical” boundary against which benchmarking the forecast.
2) Check assumptions compatibility: Check that assumptions are not mutually
exclusive (e.g. incremental marketing investment is going to grow my market, and I
will save everywhere to fund the extra manufacturing capacity I need).
3) Validate forecast feasibility: Use common sense to validate your forecast as a
whole (e.g. your market may very well double, but if you don’t have the capacity to
address it, it is not going to move your numbers).
4) Probe operating plan: Probe your forecasters for the operational details of how
they are going to deliver their forecast (e.g. what is the game-changer to triple sales?)
5) It is OK to be the villain: Everybody likes to tell a good story, and people want to
be heroes, not villains. To counter that pressure and a natural tendency to optimism,
give your forecaster a safe environment where they can be candid about their
forecast and comfortable telling you the bad news. Cyrille Betant, March 2015

Probability and Reasonableness
After you have established the forces at play in your forecast, determined the
right time horizon, and set the right granularity, you have produced a forecast.
But is it really what is going to happen, does it make sense?
No matter how complex your modeling is, no matter how scientifically proven your
method is, you should always look at your forecast with skeptical eyes (e.g. are
costs really doubling, sales tripling, people spend 50% more at my shop, etc.).
Consider narrowing/eliminating ranges (e.g. coming to an average reasonable
answer instead of a wide range, 1000 visitors/day, not between 512 and 1,634).
It is very important to define values for your assumptions instead of ranges, to do
so you can:
1 - Use the average/mean of the top and bottom value in your range,
2 - Use the most likely value in your range,
3 - Use the historical pattern in your range (seasonality, growth rate, etc.),
4 - Use your best judgment to adjust values that seem extreme), etc.

5. Communicating your
Forecast
What to say about your forecast?

Coming to An Answer
You have formulated your forecast, but what to say about it and how to say it? These
seem like trivial questions, but they are often the difference between a fruitful and a
fruitless exercise. Here are important factors to keep in mind when deciding how to
relay your findings:
1 - Build the forecast for your audience. Both content, granularity, and time
horizon are important to adjust for your audience (e.g. summarized P&L for the year
rounded to the million, or monthly detail by line-item rounded to the thousand?).
2 - Always give a number (however rounded), not a range. It is impossible to
measure accuracy and drive accountability if you forecast a range. A range is an
incentive for the measured units to make it as broad as possible in order for the
forecast to be accurate, thus defeating the original purpose of the forecast.
3 - The aim of the forecast is to measure how far your are from your goal. It is
imperative for the forecast to be presented candidly, even if (especially) it diverges
from the plan. The forecast ensures that appropriate and timely actions are taken in
order to either close the gap with the plan, or adjust the expectations set in the plan.

Coming to the Wrong Answer
Many a time, your forecast will give you an answer that is not what you wanted to
see. It is of course important to be able to go back to your sources and validate all the
assumptions made as well as the calculations. However, it is crucial to remember that
the forecast is a reflection of where you are, not where we want to be. It is very
important to resist the temptation to adjust the numbers to a scenario closer to our
wishes, as doing so could be extremely damaging to your organization.
If your forecast (the road you have built) doesn’t lead you to your plan (the
destination you have set), don’t try to re-build your model, look at what the gap is and
use your forecast as an opportunity to take the actions necessary to close it.

6. Rolling Forecast
How to keep your forecast fresh?

Doing It All Over Again
After your forecast has been done and communicated, the next phase starts
immediately: doing it again!
For a forecast to be effective, it needs to be:
1- Current (with the most recent assumptions)
To deliver this, you must constantly refresh your forecast. That way, as time
passes, you know that you always have a current forecast to guide you.
It is important to do it on a recurring basis, as one of the roles of the forecast is to
be an early indicator of variances compared to your plan. The more often you
refresh your forecast, the more chances you have to catch a changing trend that is
going to affect your results.
2- Sensible (with all assumptions validated by those close to your operations)
4- Answering the questions you asked in the first place (if you are worried about
delivering your annual revenue commitment, there is no point centering your
forecast around your payroll benefits)
3- Realistic (no matter what your forecast tells you, it needs to make sense)

Principle of a Rolling Forecast
One appealing option is to forecast to a constant time window (usually 12 to 18
months) so that, no matter where you are in your fiscal year, you always have the
same horizon in front of you. A rolling forecast gives you the additional work of only
planning for one more month and gives you a constant “time-buffer” in your planning.
It also gives you a solid basis for your budget or long-term plan when the time comes.
A rolling forecast also gives you more comparison points for the same period in time.
Here are the basic mechanics of a rolling forecast:
1 – Every month, change the current month financials from forecast to actuals data.
2 – Refresh your previous forecast for all the other periods.
3 – Add one month of forecast at the end of the previously forecasted period.
Mth 1 Mth 2 Mth 3 … Mth 12
Forecast Period 1 Jul Act Aug Fcst Sep Fcst … Jun Fcst
Forecast Period 2 Aug Act Sep Fcst Oct Fcst … Jul Fcst
Forecast Period 3 Sep Act Oct Fcst Nov Fcst … Aug Fcst
…

7. The Value of Crowd
Forecasting
IARPA case study

IARPA Case
Part of the Office of the Director of National Intelligence (ODNI) is the Intelligence
Advanced Research Projects Activity (IARPA). Part of IARPA is the Office of
Anticipating Surprise (OAS).
IARPA’s OAS wanted to know how to improve the predictive accuracy, precision, and
timeliness of their forecast.
IARPA funded the Good Judgment Project as part of their ACE (Aggregative
Contingent Estimation) project.
The project, co-led by the University of Pennsylvania and UC Berkeley, let academics
and industry groups form teams of forecasters to predict the outcome of hundreds of
international political, military, and economic scenarios with the objective of beating
the accuracy of a control group over a 4-year period.
Thousands of volunteers joined into this project.

IARPA Results
After only 2 years of the 4-year project, the overwhelming evidence showed that:
Teams of average citizens consistently beat the forecast accuracy of the best
analysts, intelligence officers, experts, and pundits.
The average predictions are more accurate even when the experts have access to
classified information not available to the average team.
The forecasters have to be generally well informed, but don’t have to be subject
matter experts.
If the group is large enough, the average prediction will beat the expert’s one by a
significant margin.

8. Important Factors to
Remember
How to make the best of your
forecast?

Important Factors to Remember
As a conclusion, here are important factors to keep in mind when engaging in a
forecasting exercise:
1) Forecasting is an art more than a science: You need to have your math right
and a lot of sophisticated algorithms can help you tremendously, but in the end, it is
your knowledge of the subject, the relationships with your operation managers and
analysts, and your judgment that will make the difference.
2) Segregate the duties: Separate the tasks of input, modeling, analysis, and review
to maintain as impartial a look as possible on everybody's contribution to your
forecast. Do not let people review their own work as it is psychologically too difficult to
probe or contradict your own assumptions.
3) Collaborate across disciplines: Use inputs and analysis from as diverse a pool
as possible. Nobody has all the answers all the time, so your only insurance against
wishful thinking is to make sure that as many current of thoughts are represented in
your analysis before you decide what the forecast should be.

The Amazing Science and Delicate Art of Forecasting

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (9)

Similar to The Amazing Science and Delicate Art of Forecasting

Similar to The Amazing Science and Delicate Art of Forecasting (20)

The Amazing Science and Delicate Art of Forecasting