In the 46 years since the seminal work of Bates and Granger on the combination of forecasts, research has been rather consistent in concluding both empirically and theoretically, that the combination of forecasts improves accuracy over selecting a single ‘best’ forecast. However, nearly all of research has focused on the combination of forecasts ignoring the obvious question which arises; what about combining the forecast models themselves. In this paper we try to answer this question: Are there benefits to be gained from the combination of individual forecast models rather than the forecasts themselves. By exploring the combination of forecast models we focus on the combination of model elements, such as parameters and components, resulting in a single forecast model as compared to multiple forecast models whose forecasts are then combined. This approach promises savings in terms of reduced computational time in producing forecasts, and simplicity and interpretability from the use of a single (combined) forecast model, while mitigating modelling uncertainty. Using the family of exponential smoothing algorithms and the ARIMA methodology and the Bagging approach based on bootstrapping, we generated multiple forecast models (and forecasts). We assess the impact of Bagging in terms of the diversity of forecast model structures generated and investigate the use of the mean and median combination methods in combining the resulting forecasts and forecast models using real and simulated time series.

1. D R . D E V O N B A R R O W
D R . N I K O L A O S K O U R E N T Z E S
3 5 T H I N T E R N A T I O N A L S Y M P O S I U M O N F O R E C A S T I N G
M A R R I O T T R I V E R S I D E C O N V E N T I O N C E N T R E
2 2 – 2 4 J U N E 2 0 1 5
To combine forecasts or
forecast models (Parameters)

2. Outline
22/07/2015To combine forecasts or forecast models
2
 Forecast combination and model uncertainty
 Research questions
 Experimental Design
 Results
 Conclusion
“Everything should be made as simple as possible, but no simpler.” Albert
Einstein (Supposedly)

3. Model uncertainty
22/07/2015To combine forecasts or forecast models
3
 Model building process
 Model formulation (or model specification)
 Model fitting ( or model estimation)
 … (Chatfield, 1995)
 Sources of uncertainty
 Model structure
 Model parameter estimates
 Unexplained random variations (Draper et al., 1987; Hodges, 1987)

4. Forecast combinations
22/07/2015To combine forecasts or forecast models
4
 Given multiple models
 Select a single (forecast) model
 Create multiple forecasts
 Take a simple average of all forecasts
 Does combination work?
 Generally leads to improved accuracy (Stock and Watson, 2004 ;Fildes, Nikolopoulos et al.
2008) etc…
 More robust and accurate than individual forecast (Newbold and Granger,74; Palm and
Zellner,92) etc...
 M3-Competition (Makridakis et al. 2000)  simple average (Comb S-H-D)
outperforms others
 Nearly 50 years of forecast combination research focused on
 Combining forecasts
 Combining model parameters almost neglected

5. Combinations and model uncertainty
22/07/2015To combine forecasts or forecast models
5
 Why combinations work?
 Usually the model form is unknown to the forecaster i.e. ‘true
model’
 Data generating process may not be of a simple functional
form
 Things change with time e.g. seasonality and/or trend may
disappear, structural breaks
 Outliers and anomalies may distort structure
 A finite number of observations, often small samples
 Large effects are easier to identify than smaller effects

6. Outline
22/07/2015To combine forecasts or forecast models
6
 Forecast combination and model uncertainty
 Research questions
 Experimental Design
 Results
 Conclusion

7. Research questions
22/07/2015To combine forecasts or forecast models
7
 Bagged exponential smoothing (Christoph, Hyndman & Benitez, 2014)
 Perform STL decomposition
 Bootstrapped residuals of the decomposition
 Recombine to obtain new series
 Estimate a model for each bootstrapped series
 Shown to work well – especially for monthly data
 Combine forecasts from a family of closely related methods
 Consequently the model estimated to be best can
vary from data set to data set
 To combine forecasts or forecast models
(parameters)?

8. Research questions
22/07/2015To combine forecasts or forecast models
8
 Nearly 50 years of research focused on
 Combining forecasts
 Combining model parameters – almost neglected
 Bagged exponential smoothing (Christoph, Hyndman & Benitez, 2014)
 Perform STL decomposition
 Bootstrapped residuals of the decomposition
 Recombine to obtain new series
 Estimate a model for each bootstrapped series
 Shown to work well – especially for monthly data
 Combine forecasts from a family of closely related methods
 Consequently the model estimated to be best can vary
from data set to data set
 To combine forecasts or forecast models (parameters)?

9. Research questions
22/07/2015To combine forecasts or forecast models
9
 [image of bootstrap series]
 [Image of changing models]

10. Research questions
22/07/2015To combine forecasts or forecast models
10
 RQ1: Are there benefits to be achieved from
combining forecast model parameters rather than
model forecasts themselves?
 RQ 1.1: Given the same forecast model structure?
 RQ 1.2: When the forecast model structure is allowed to vary?
 RQ2: How can differences in performance be
explained?

11. Outline
22/07/2015To combine forecasts or forecast models
11
 Forecast combination and model uncertainty
 Research question
 Experimental Design
 Results
 Conclusion

12. Experimental design - setup
22/07/2015To combine forecasts or forecast models
12
 Combinations
 Parameter combination
 Forecast combination
 Combination methods
 Mean, Median, Mode
 Forecast model generation
 Bootstrap
 Simulation
 Conditions
 Model structure fixed
 Model structure varies

13. Experimental design - setup
22/07/2015To combine forecasts or forecast models
13
 Family of exponential smoothing methods
 Model uncertainty
 A general class of models where the true model is a special case
 Models of different structures
Source: Hyndman et al. 2002 based on (Pegels ,1969 ; Gardner 1985)

14. Experimental design - benchmarks
22/07/2015To combine forecasts or forecast models
14
 Automatic model selection based on AIC (Hyndman et al. 2002)
 Bagged ETS (Christoph, Hyndman & Benitez, 2014)
 Perform STL decomposition
 Bootstrapped residuals of the decomposition
 Recombine to obtain new series
 Estimate a model for each bootstrapped series
 Points of comparison:
 Model selection versus combination
 Model structure varies across bootstrapped series
 Forecasts and not parameters are combined
Source: Hyndman et al. 2002 based on (Pegels ,1969 ; Gardner 1985)

15. Experimental design - evaluation
22/07/2015To combine forecasts or forecast models
15
 Research Question 1:
 Empirical evaluation
 M3 Competition data
 Forecast accuracy using SMAPE and GMRAE
 Research Question 2:
 Bias-variance decomposition

16. Outline
22/07/2015To combine forecasts or forecast models
16
 Forecast combination and model uncertainty
 Research question
 Experimental Design
 Results
 Conclusion

17. Results
22/07/2015To combine forecasts or forecast models
17

18. Outline
22/07/2015To combine forecasts or forecast models
18
 Forecast combination and model uncertainty
 Research question
 Experimental Design
 Results
 Conclusion

19. Conclusion
22/07/2015To combine forecasts or forecast models
19