Ideas on how to improve time series models Human beings have shown fascination for what the future would hold since the beginning of time. In data science and statistics, the process of forecasting is based on time series data, which can be defined as a sequence of observations taken sequentially in time. Statistical forecasting tries to predict what future observations will be, based on past ones. A clear example of how this is beneficial in practice is sales forecasting. Companies make use of sales forecasting to try and understand their customer demand, in order to improve inventory management and cut down production costs. (...)

1. SFSCon 2020
Statistical forecasting
with exogenous factors
Ideas on how to improve time series models
Geri Skenderi
Prof. Marco Cristani

2. Outline
Introduction to Forecasting
Data and Statistical analysis
Method and Experiments
Conclusion

3. Outline
Introduction to Forecasting
Data and Statistical analysis
Method and Experiments
Conclusion

4. Introduction
Ancient forecasting

5. Introduction
Modern forecasting
George E.P. Box
1919 - 2013
y’t = c + 𝜙1y’t-1.... 𝜙py’t-p+ 𝜃1 𝜀t-1.... 𝜃q 𝜀t-q + 𝜀t
ARIMA forecast equation for time point t
Rob J. Hyndman
1967

6. Introduction
Forecasting methods

7. Outline
Introduction to Forecasting
Data and Statistical analysis
Method and Experiments
Conclusion

8. Data
Time series
• Forecasts are performed on time series data, which can be defined as a sequence of
observations taken over regular time intervals.
• This data can be univariate or multivariate.
• In this presentation, we will see how to harness the power of multivariate data, in an
era where data is everywhere.

9. Data
The dataset
• Raw sales data provided from a big Italian
fast-fashion company.
• Multiple features to pose a forecasting
problem.
• We will focus on stores located in three
major cities: Milan, Turin and Rome.

10. Data
The dataset
Data from two shops, both located inside malls.

11. Data
The dataset
Data from five shops, four in a mall and one on
the street.

12. Data
The dataset
Data from five shops, one in a mall and four on
the street.

13. Where do we go from here?
Use exogenous data to pose a multivariate
forecasting problem

14. Statistical Analysis
Milan Data
Sales and seasons
Check if certain periods of the year lead to
more/less sales.
Sales and the weather
Check if the weather leads to more/less
sales.
Sales and the day of the week
Check if certain days of the week lead to
more/less sales.

15. Statistical Analysis
Sales and day of the week

16. Statistical Analysis
Sales and seasons

17. ● Intuition: The weather affects our mood and habits, so it could have an impact on sales
and shopping behavior.
● More data pre-processing involved. The external weather signals contain
measurements, but also weather phenomena as categorical data:
1. Integer encode weather phenomena.
2. Create a binary weather signal: "rainy": -1 or "not rainy": 1.
3. Original sales signal must be supplementally cleaned and modified.
Statistical Analysis
Sales and weather

18. ● For extensive homogeneity, high sales seasons are removed. Then the signal is taken
by year, divided into ordinary seasons and further into work and off days.
● This structure gives rise to the following set of variables, for each shop:
○ 4 months(March-June, September-December).
○ Approximately 4 weeks per month.
○ 4 days for the working days (Mon-Thu), 3 days for the weekend (Fri-Sun)
● Undersampling is used as a final step to remove statistical inconsistencies.
Statistical Analysis
Sales and weather

19. Statistical Analysis
Sales and weather
# Samples Mean sold t-test p-
value
Rainy 22 104.50
Not-Rainy 22 60.95
0.002
# Samples Mean sold t-test p-
value
Rainy 16 171.56
Not-Rainy 16 122.25
0.02
# Samples Mean sold t-test p-
value
Rainy 26 140.42
Not-Rainy 22 100.58
0.01
# Samples Mean sold t-test p-
value
Rainy 18 266.00
Not-Rainy 18 185.28
0.001
Milan spring-summer 2017 workdays Milan spring-summer 2017 off-days
Milan spring-summer 2019 workdays Milan spring-summer 2019 off-days
Independent two sample statistical t-test used to determine if there is a significant difference.

20. Outline
Introduction to Forecasting
Data and Statistical analysis
Method and Experiments
Conclusion

21. Method
Long Short-Term Memory

22. Method
EXO-LSTM
An LSTM network which makes use of exogenous temporal and weather data and
uses it for forecasting.
E
N
C
O
D
E
Temporal data
Weather data

23. ● The data is transformed into input - output sequences.
● Standardization is applied: Z = (x - 𝜇)/σ
● By the end of the whole procedure, the data is normalized, divided into sequences and
possesses the original signal, along with the exogenous temporal and weather signals.
t-7 t-6 t-5 t-4 t-3 t-2 t-1 t
Method
Feeding data to the LSTM

24. ● Uses weekly data, with the goal of having an efficient way of checking possible trends
during the year.
● SARIMA and SARIMAX used as classical methods while the LSTM is used as a Deep
Learning representative
● Four LSTM architectures were built:
○ MONO-LSTM: Trained on the univariate total sales signal.
○ EXO1-LSTM: Trained on the total sales signal and the exogenous temporal signal.
○ EXO2-LSTM: Trained on the total sales signal and the exogenous weather signal.
○ EXO3-LSTM: Trained on all the possible features, so sales, weather and temporal signals.
Experiments
Long term forecasting

25. Dataset SARIMA SARIMAX MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM
Milan 299.299334 320.983730 269.000266 259.300154 294.289433 276.278388
Rome 107.928035 103.974567 149.296986 95.339082 127.586807 117.149285
Turin 137.123418 142.099703 146.110333 137.757753 174.867309 163.300861
Dataset SARIMA SARIMAX MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM
Milan 6.099036 8.041129 7.412993 7.515069 8.366195 8.250909
Rome 12.846811 12.232047 16.026479 10.029953 12.068225 12.251298
Turin 11.014910 12.382435 13.085100 12.106654 13.937898 13.659325
Measure SARIMA SARIMAX MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM
RMSE 181.450262 189.019333 188.135862 164.132330 198.914516 185.576178
SMAPE 9.986919 10.885204 12.174857 9.883892 11.457439 11.387177
RMSE of weekly forecasting based on location
SMAPE of weekly forecasting based on location
RMSE and SMAPE of weekly forecasting based on all the data

26. The effects of the weather are not as crucial and understandable, while temporal
information like the period of the year is key.

27. ● Uses daily data, with the goal of knowing what is going to happen daily and fill the shelves
with products.
● We use only the four LSTM architectures here, as we have already seen they surpass the
baselines.
● This forecasting strategy along with the different EXO-LSTM models can also aid us in
understanding customer behaviour.
Experiments
Short term forecasting

28. Dataset MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM
Milan 92.681814 97.419602 92.756764 87.843425
Rome 103.320297 101.16842 107.342307 99.044623
Turin 40.815775 34.641631 35.821523 36.963587
Dataset MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM
Milan 17.082440 19.892359 20.611555 16.801701
Rome 23.468583 21.877664 22.764811 22.043665
Turin 20.751887 18.803312 18.672907 17.939686
Measure MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM
RMSE 78.939295 77.743218 78.640198 74.617212
SMAPE 20.434304 20.191111 20.683091 18.928351
RMSE of daily forecasting based on location
SMAPE of daily forecasting based on location
RMSE and SMAPE of daily forecasting based on all the data

29. The effects of the weather and time together become clear and are important for
improving forecasts and understanding customer behaviour.
Ground Truth MONO-LSTM EXO1-LSTM EXO2-LSTM EXO3-LSTM

30. Here we can see how knowing that it is going to rain helps the EXO-LSTM forecast more sales
than when it does not possess this information (MONO).

31. Outline
Introduction to Forecasting
Data and Statistical analysis
Method and Experiments
Conclusion

32. Conclusion
Key takeaways: How to improve time series models?
1. Make sure you are making the best use out of your data and other available data.
2. Analyze your data to see if you can extract “hidden” information.
3. Use neural models when dealing with multivariate problems.
4. Don’t be afraid of trying new things.

33. Conclusion
Future work you could focus on
Various Neural
Architectures
Ensemble
Methods
Feature
Selection

34. Conclusion
Code and experiment details
Available for free in my GitHub:
https://github.com/geriskenderi/exo-lstm

35. Get in Touch
We would love to hear your thoughts
LINKEDIN
@geriskenderi
TWITTER
@geriskenderi
EMAIL
geri.skenderi@univr.it
TWITTER
@MarcoCristani
EMAIL
marco.cristani@univr.it

36. THANK YOU