Forecast technique and case study

1. Demand Planning &
Forecasting

2. 4-Dimensions Demand
Forecasting Framework

3. 4-Dimensions Demand Forecasting
Framework
Granularity Temporality
Process Metrics
How to set up a demand forecasting
process
We forecast to take action

4. Granularity and Temporality
Material. Per product, segment,
brand, weight, money?
Geographical. Per country,
warehouse, market, region, sales
channel, store, client?
Time Buckets. Which temporal
aggregation: day, week, month?
Horizon. Weeks, months, quarters,
years ahead?
Products
Brands
All
Stores
Regions
All
Days
Months
Year

5. Complexity
Granularity and Temporality
Complexity
How do we choose?
We forecast to take action
Accuracy
Granularity
Horizon
Horizon and granularity
cost accuracy
Accuracy

6. Granularity, Temporality
and Decisions
Short-term (Operations)
• How much to ship?
• How much to produce?
• How many extra hours do we need?
Mid-term (Tactical)
• How much to produce (seasonal
products)?
• How many interims should we hire?
Long-term (Strategy)
• Should we launch this new product?
• Should we open a new plant?
• How many people should we hire?
Logistic
Manager
I need to deploy
products to shops
Plant
Manager
I need to plan
production
Marketing
Strategist
I plan new
products’
introduction
Horizon
Granularity
SKU x Stores x Weeks
SKU x Global x Months
Brand x Global x Quarter

7. Metrics

8. Accuracy and Bias
Bias. Tendency to over- or
under forecast.
Accuracy. Amplitude of
the forecast error.
We need to align the
forecasting metrics with
supply chain objectives.
Biased
Unbiased
Accurate Not Accurate

9. Accuracy and Bias
• We can have inaccurate,
unbiased forecasts (orange).
• As well as accurate, biased
forecasts (grey).
• Choosing the best
forecasting model is often
more relying on choice
theory than on
straightforward selection.
Bias: -1%
Accuracy: 69%
Bias: -12%
Accuracy: 84%
Volume
Time
Forecast #1 Forecast #2 Demand

10. Metrics Based on Interpretable
Sensitivity to
bigger errors
Scaled Not Biased
Sensitivity to
outliers
MAE |e| ✓    ✓
MAPE |e| ✓  ✓  ✓
RMSE e²  ✓  ✓ 
MASE |e|   ✓  ✓
RMSSE e²  ✓ ✓ ✓ 
Accuracy Metrics
We need to align
the forecasting
metrics with supply
chain objectives
Bad
MAPE
Good
MAE & Bias
Better
WMAE & Bias
Even better?
WMAE, WMASE, WRMSE, WMRSSE & Bias
Scaled Metrics
Each item is scaled
based on its
forecastability
Weighted Metrics
Each item is weighted
based on its
importance

11. Process

12. S&OP Process
Forecast
Baseline
Demand
Planners
Sales
Team
Consensus
Meeting
✓ Update thousands of
SKUs
✓ Increase total volumes
✓ Increase total volumes
again
We need to take care
of new products. We
received info from
our main clients.
We want to
secure stock
We want to
match yearly
budget
✓ Slightly decrease total
volumes
Problem 1. You do not want
any team member to spend
too much time editing the
forecast.
Problem 2. You do not want
any team member to spend
time editing the forecast
while not improving it.

13. Forecast Value Added
Process Step
Person-
hour
RMSE FVA MAE FVA Bias FVA
Benchmark 88% 52% -1%
Baseline 72% +16 45% +7 -3% -2
Planners 72 68% +4 43% +2 1% +2
Sales team 20 68% +0 45% -2 5% -4
Consensus 8 69% -1 45% +0 4% +1
Gilliland, M. (2010). The Business Forecasting Deal: Exposing Myths, Eliminating
Bad Practices, Providing Practical Solutions. John Wiley & Sons, Hoboken, N.J.
Solution. Track the added (or destroyed)
value of each step in the forecasting process
(compared to the previous one). Compare
the added value with the time spent on the
forecast.

14. Weighted Metrics & Forecast Value
Added
Solution. Looking at a portfolio of
products/SKU requires smart weighted
KPIs.
• The errors should be weighted based
on costs, profits, importance, raw
material utilization.
• Combining FVA with weighted metrics
will allow you to focus on the most
critical SKU while being efficient.
Product Forecast Demand Error |Error|
Hammer 150 100 50 50
Nail 1.000 1.500 - 500 500
Total 1.150 1.600 - 450 550
-28% 34%
Product Forecast Demand
Profits
(per piece)
Weighted
Error
Weighted
|Error|
Hammer 150 100 5,00 250 250
Nail 1.000 1.500 0,01 - 5 5
Total 1.150 1.600 245 255
48% 50%
Vandeput, N. (2021). Data Science for Supply Chain
Forecasting. Berlin, Boston: De Gruyter.
Problem 3. You want your team to pay
more attention to a few critical products,
not the trivial many.

15. Short walk through of
M5 challenge

16. M5 Challenge
10 stores in 3 US states
3049 SKUs per store
12 hierarchy levels
42840 time series
Goal: Forecast next 28 days

17. Level Aggregation Level Number of series
1 Unit sales of all products, aggregated for all stores/states 1
2 Unit sales of all products, aggregated for each State 3
3 Unit sales of all products, aggregated for each store 10
4 Unit sales of all products, aggregated for each category 3
5 Unit sales of all products, aggregated for each department 7
6 Unit sales of all products, aggregated for each State and category 9
7 Unit sales of all products, aggregated for each State and department 21
8 Unit sales of all products, aggregated for each store and category 30
9 Unit sales of all products, aggregated for each store and department 70
10 Unit sales of product x, aggregated for all stores/states 3,049
11 Unit sales of product x, aggregated for each State 9,147
12 Unit sales of product x, aggregated for each store 30,490
Source:
https://mofc.unic.ac.cy/wp-content/uploads/2020/03/M5-Competitors-Guide-Final-10-March-2020.docx

18. Solution
Our solution was
• 20.3% better than best statistical benchmark
• 13.5% better than Google AutoML solution. (w.r.t.
competition metric)
Sources:
• https://ai.googleblog.com/2020/12/using-automl-for-time-series-forecasting.html?m=1
• “The M5 Accuracy competition: Results, findings and conclusion”, Makridakis et. al.
• https://www.kaggle.com/c/m5-forecasting-accuracy/leaderboard

19. Intermittent demand Continuous demand
vs.

20. 12 levels to forecast… and reconcile
Global
N-Beats ensemble
Global
lgb ensemble per store
Supply chain
C-level

21. Forecast Reconciliation
• Reconciled forecast is a must-have
• Equal error contribution across all twelve levels 
should not over optimize one specific level

22. N-Beats ensemble
Level1 forecast
lgb ensemble forecasts
aggregated to Level1
Selection based on lowest
deviation in mean error and RMSE
Different model sets shifted by
asymmetric loss

23. • Do we have (mostly) intermittent or continuous time-series?
• What time frequency are we interested in ?
• Do we need reconciled/consistent forecasts ?

24. Usual Engagement mode
PoC to structure project and limit risk
Project step
Person
days
Kickoff workshop to align all stakeholders and find common
understanding of goals and deliverables (including preparation)
4
Review of data and consistency (checks with feedback loop with
client).
Benchmark setting.
4
Data inspection and selection of most promising models.
Training and test of models (2-4 days per model approach).
15-25
Report on model results and most promising forecasting and supply
chain improvement levers.
4
Usual PoC range
€ 30.000 -
€ 55.000
Pricing differs based on data
quality, project scope, travel
etc.
Obviously our goal is to find
the best possible solution for
your needs and not to offer
the lowest price

25. Discussion
nicolas.vandeput@supchains.com
matthiasanderer@googlemail.com