Ideas for Artificial Intelligence in Demand Planning and forecasting

1. Forecast Performance KPIs and Metrics
for Demand Planning and Forecasting
A SET OF INDICATORS, MEASURES AND METRICS TO ENSURE A SUCCESSFUL AND
SUSTAINABLE DEMAND PLANNING PROCESS.
IDEAS FOR ARTIFICIAL INTELLIGENCE AUTOMATION AND INSIGHT TRACKING THAT
CAN ENHANCE THE FORECAST QUALITY AT SIGNIFICANTLY HIGHER EFFICIENCY
LEVELS

2. Demand
Planning
Process
Compliance
Forecast
Completeness
Indicators
Forecast
Performance
KPIs
Trended
Forecast Error
24-36 Month
Rolling Horizon
% DFUs with
Automated
Statistical Baselines
Assumptions
Documentation for
manual
adjustments
All Active Products
(incl. New Products)
Action Agreed
Record
& Follow-up
Regular Rhythm
Wheel attended by
the quorum
Trended
Forecast Bias
All Demand Sources
Demand
Planning
Analytics &
Metrics
Demand Patterns
Is there seasonality,
lumpiness or
intermittency?
Forecast Stability
Is there too much
noise?
Forecast Evolution
How does the forecast
change over time?
Predictability
What is the Statistical
Fit?

3. Forecast
Performance
KPIs
Trended
Forecast Bias
Forecast Bias % is a measure of whether the forecast is consistently too
high or too low and is in need of correction, either downwards or
upwards, to remove the bias going forward. The Forecast is captured 3
months lag before the Actual month, in line with the execution horizon,
before calculation. Multiple Lags are recommended e.g. M02, M01, etc.
𝐴𝐺𝐺 𝑆𝐾𝑈 𝐴𝑐𝑡𝑢𝑎𝑙 𝐷𝑒𝑚𝑎𝑛𝑑 3 𝑃𝑒𝑟. 𝑀𝑜𝑣. 𝐴𝑣𝑔.
𝐴𝐺𝐺 𝑆𝐾𝑈 𝐹𝑜𝑟𝑒𝑐𝑎𝑠𝑡 𝑀03 3 𝑃𝑒𝑟. 𝑀𝑜𝑣. 𝐴𝑣𝑔.
1
-19.1%
-1.3%
-14.3%
-7.6%
-9.8%
-8.5%
-11.4% -10.6%
-23.6%
-12.8%
-17.9%
-14.6%
-19.2%
-15.0%
-30.0%
-25.0%
-20.0%
-15.0%
-10.0%
-5.0%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
08.2013 09.2013 10.2013 11.2013 12.2013 01.2014 02.2014 03.2014 04.2014 05.2014 06.2014 07.2014 08.2014 09.2014
Forecast Bias % 3 per. Mov. Avg. (Forecast Bias %)
…expressed as a %

4. Forecast
Performance
KPIs
Trended
Forecast Error
Forecast Error % is a measure of the absolute differences between the
actual demand and the forecast over the actual demand. These results can
be aggregated to the any level of the product hierarchy and then
calculated. As an prime indicator, this helps to show how much variation
and phasing issues there are at the lowest level of the SKU hierarchy.
𝐴𝐺𝐺 𝐴𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝐷𝑒𝑚𝑎𝑛𝑑 − 𝐹𝑜𝑟𝑒𝑐𝑎𝑠𝑡 3 𝑃𝑒𝑟. 𝑀𝑜𝑣. 𝐴𝑣𝑔.
𝐴𝐺𝐺 𝐴𝑐𝑡𝑢𝑎𝑙 𝐷𝑒𝑚𝑎𝑛𝑑 3 𝑃𝑒𝑟. 𝑀𝑜𝑣. 𝐴𝑣𝑔.
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
08.2013 09.2013 10.2013 11.2013 12.2013 01.2014 02.2014 03.2014 04.2014 05.2014 06.2014 07.2014 08.2014 09.2014
Forecast Error % 3 per. Mov. Avg. (Forecast Error %)
…expressed as a %

5. Forecast
Completeness
Indicators
24-36 Month
Rolling Horizon
All Active Products
(incl. New Products)
All Demand Sources

6. Forecast
Completeness
Indicators
24-36 Month
Rolling Horizon
All Active Products
(incl. New Products)
All Demand Sources
-
10,000
20,000
30,000
40,000
50,000
60,000
70,000
08.2013 10.2013 12.2013 02.2014 04.2014 06.2014 08.2014 10.2014 12.2014 02.2015 04.2015 06.2015 08.2015 10.2015 12.2015 02.2016 04.2016 06.2016 08.2016
Actual Demand Forecast M03 Forecast Projection Linear (Actual Demand)
The Forecast must be completely extended to the entire planning
horizon (this can vary from 2-5years) every period. This ensures that
longer lead-time capabilities are recognised and projects started in time
for increasing Demand to be satisfied. Or that with a declining trend,
cost can be removed and profit maintained through the end-of life phase
of a product. As you can see below this forecast is both biased and
incomplete.
?

7. Demand
Planning
Process
Compliance
% DFUs with
Automated
Statistical Baselines
Assumptions
Documentation for
manual
adjustments
Action Agreed
Record
& Follow-up
Regular Rhythm
Wheel attended by
the quorum

8. Demand
Planning
Process
Compliance
% DFUs with
Automated
Statistical Baselines
Distribution Centre
Outlet
Daily
Weekly
Monthly
SKU
Family
Category
Time Axis
Product Axis
Network Axis
The Forecast Cube is a visualization of how
aggregation & disaggregation works in demand
planning. Historical data comes into the system
at the lowest level that suits the business.
Complete Real-time PoS data being Nirvana. The
forecasting system can aggregate to any level
that has been defined in the axis hierarchies.
Possibly a 4th Axis could be
Customer/Geography/Sales Office/Region etc…
DFU

9. Demand
Planning
Process
Compliance
% DFUs with
Automated
Statistical Baselines
The definition of a DFU or ‘Demand Forecasting Unit’ is different to that of an SKU which is a
‘Stock Keeping Unit’ or Product at a Location within the entire Supply Chain Network. The usual
methodology in forecasting is to aggregate demand from the SKUs to a level in the Forecast Cube
that will return a significant statistical fit and projection. This may incur transformations with
UOMs or Units of Measure to help align to true consumption.
Once this exercise is completed and the DFUs identified, Statistical models can then be tested on
this resulting aggregated and transformed historical data set. The best fit models are then
selected to project and disaggregate via proportional factors to the contributing SKUs. This
disaggregated forecast is then used to drive the replenishment process. The disaggregated
forecast is also used to measure Forecast Bias and Error %.
Automating the process of Statistical modelling is the key to reducing the workload on demand
planners but this has to be carefully tracked and reported.
The percentage of DFUs that are automated vs the total number of DFUs can give you an
indication of the ‘Forecastability’ or predictability of the product portfolio. Obviously the higher
the number the better. Or whether you have the correct level in the product hierarchy for your
DFUs. Typically, as you travel up the product hierarchy, the ‘Law of Big Numbers’ kicks in and
Forecastability improves. This would bring better statistical fits at a higher level but more
attention to detail at the SKU level would be needed to track any pattern differences e.g.
switching or step-changes.
For example, aggregating all locations may be false, as there maybe a particular demand pattern
at each location. In this case, you would aggregate SKUs up the product axis across similar
products or families to find a DFU that would be more accurate than at SKU level. This is a
continuous task of demand planners to find the right DFUs that will be best.
I sense that this task could be completed by Intelligent Algorithms in the future thus removing a
difficult and laborious job for Demand Planners. Food for thought…

10. Demand
Planning
Process
Compliance
Assumptions
Documentation for
manual
adjustments
Action Agreed
Record
& Follow-up
Regular Rhythm
Wheel attended by
the quorum
Every manual adjustment that are overlaid onto the baseline must be documented
with an underlying assumption supporting that adjustment. There can be many
assumptions about the future demand of a product from a promotion, an external
factor such as the economic outlook or competitor activity. A ‘Book of
Assumptions’ is kept and referred back to when diagnosing significant forecast
errors.
The largest risk is the compounding effect when multiple adjustments are made on
the same product at the same time. This must be discussed & moderated by the
quorum of the Demand Planning Team. In many cases, the key assumptions that
are left out are the ‘post promotional effects’, a dip in sales as the promoted stock
depletes in the down stream nodes of the supply chain and based on underlying
product consumption. E.g. if you sell a jar of mayonnaise with an extra jar free,
consumption is unlikely to rise much but the shopper will not buy another jar for a
longer time.
All actions that have been agreed to amend the forecast must be logged and
tracked for completeness. An action must have an owner and a completion date and
the Demand Planner is responsible to following-up on the due tasks.
The Demand Planning cycle must be agreed and diarized for the next 12 months
rolling. All member of the Quorum (core members or their substitutes) must attend
these meeting reliably. An attendance record should be kept to institutionalize the
process within the community. Their focus is the constant update of the latest
demand (warts & all) and to take corrective action to change the forecast for the
better. One of my favourite mindset quote is…

11. Demand
Planning
Analytics &
Metrics
Demand Patterns
Is there seasonality,
lumpiness or
intermittency?
Forecast Stability
Is there too much
noise?
Forecast Evolution
How does the forecast
change over time?
Predictability
What is the Statistical
Fit?

12. Once a Statistical Model has been selected to become the baseline for the
forecast projection, there are statistical measures that describe the ‘Goodness-
of-Fit’ of model to the data sample.
A Regression model fit statistic is called the 𝑟2
and is represented by a %. 100%
being that every data point fits the regression. In Linear regressions, level &
trend only, tracking this measure is an excellent way of keeping a eye on your
baselines without having to look at them whenever you get a new data point.
You only look at 𝑟2
values that are decreasing significantly. A good 𝑟2
value is
50% or greater for a linear regression. Good for regular non-seasonal driven
consumption
A Non-linear regression model also has an 𝑟2
stat but it is also necessary to
combine that with the F-Test of Statistical Significance. You could have a high fit
of 90% but with low or no significance so another type of model would be
needed. Non-Linear regressions are good for regular seasonally driven
consumption patterns. e.g. Ice Cream, Suntan Lotion, Beer, the list is endless...
Demand
Planning
Analytics &
Metrics
Demand Patterns
Is there seasonality,
lumpiness or
intermittency?
Forecast Stability
Is there too much
noise?
Forecast Evolution
How does the forecast
change over time?
Predictability
What is the Statistical
Fit?

13. The most difficult demand to forecast are Lumpy or Intermittent patterns
characterized by spikes of irregular quantities and timings, together with gaps of
zero demand. See example below…
One approach here is to aggregate up along the Period Axis to a rolling 12
month or more. This approach only provides one number and as a forecast
driving supply in the usual way is extremely inaccurate.
There are multiple ways to manage these products. From holding Finished Goods
stock with a reorder point methodology to offering Make-to-Order only with
long lead-times.
Testing for intermittency can be performed by the system or, if the capability
doesn’t exist within the system, is to use a Hilbert-Huang Transform (HHT)
algorithmic analysis.
A clever program could then test for intermittency and set the appropriate
Replenishment methodology without any human interaction. This is nice because
a lot of tail SKUs usually have this peculiar demand pattern. *excited with stars
in eyes emoji*
Demand
Planning
Analytics &
Metrics
Demand Patterns
Is there seasonality,
lumpiness or
intermittency?
Forecast Stability
Is there too much
noise?
Forecast Evolution
How does the forecast
change over time?
Predictability
What is the Statistical
Fit?
-
10
20
30
40
50
60
08.201310.201312.201302.201404.201406.201408.201410.201412.201402.201504.201506.201508.201510.201512.201502.201604.201606.201608.2016
Actual Demand Forecast M03 Forecast Projection Linear (Actual Demand)

14. Demand
Planning
Analytics &
Metrics
Demand Patterns
Is there seasonality,
lumpiness or
intermittency?
Forecast Stability
Is there too much
noise?
Forecast Evolution
How does the forecast
change over time?
Predictability
What is the Statistical
Fit?
Noise is characterized by recognizing an increase to the forecast then a decrease
to the forecast and then an increase yet again. As you can probably imagine this
is not healthy for the Supply Planning process as orders are expedited then DE-
expedited every time the forecast is amended. This is particularly damaging in
the short-term execution horizon, so it is best to measure the noise within the
cumulative lead-times. Noisy products can then be highlight to the Demand
Planning Team to take care of.
As the Demand Planning Team and the Statistical models for DFUs change the
forecast regularly, we must measure whether the changes are NOT causing
problems to the up stream Supply Chain. When increases or decreases to the
forecast are consistent, then we must also track this Evolution and highlight to
the DP Team. The size of these changes are also significant and must be
highlighted immediately to the Supply Planning process. E.g. doubling the
forecast may remove all your projected stock within days, or halving you stock
may cause excessive inventory and/or wastage.
Differentiating the difference between REAL Evolution and Noise could be
completed by Intelligent Algorithms (AI) given the correct data and criteria.
Keeping a track on every SKU and how the forecast changes over time is an
impossible task for a human mind. When AI support works here, it would make a
giant leap for the Demand Planning Team in keep forecasts under control. *huge
grinning smiley emoji*
"Statistics are like bikinis. What they show is interesting. What they hide is vital."
- Aaron Levenstein