The document provides an overview of Box-Jenkins (ARIMA) methodology for time series modeling and forecasting. It discusses autoregressive (AR), moving average (MA), and autoregressive moving average (ARMA) models. It also covers model identification using autocorrelation (ACF) and partial autocorrelation (PACF) functions, as well as model estimation, checking, selection and forecasting. Examples are provided to illustrate the methodology.

1. Chapter 9: Box-Jenkins (ARIMA)
Methodology

2. ARIMA Models
• The Box-Jenkins methodology refers to a set of
procedures for identifying, fitting and checking
ARIMA models with time series.
•
• The AR in ARIMA refers to Autoregressive models
• The MA in ARIMA refers to Moving Average
models
• The I in ARIMA refers to the number of lags used
in differencing the data

3. Autoregressive Models
Yt = 0 + 1Yt-1 + 2Yt-2 … + pYt-p + et,
• where t = coefficients to be estimated and
p = number of lags
• The number of lags (p) used in the model is a
parameter and its value must be determined
by the user. An autoregressive model with a
lag of two will be denoted as AR(2).
•

4. Moving Average Models
• Yt =  + et - w1et-1 - w2et-2 - … wqet-q,
• where wt = coefficients to be estimated, and et
are the error terms
• The number of error terms used in the model, q,
is a parameter and its value must be determined
by the user. A moving average model with two
error terms will be denoted as MA(2).
•

5. ARMA models
• Combining AR and MA models, an ARMA(p,q)
model is as follows:
• Yt = 0 + 1Yt-1 + 2Yt-2 … + pYt-p + et, - w1et-1
- w2et-2 - … wqet-q,

6. Differences
• Differences of the time series may be used if it is not
stationary. In some cases, a difference of the
differences may be necessary before a stationary data
is obtained. We use the notation “d” to indicate the
number of times the time series is differenced to
obtain a stationary series.
•
• ARIMA Notation: ARIMA(p,d,q) = An ARIMA model
with the time series differenced d times as the
response variable with a p-order autoregressive model
mixed with q-order moving average model.
•

7. Model Identification
• We use ACF (Autocorrelation function) and
PACF (Partial Autocorrelation function) . ACF
measures the correlation between a time
series and its past values at different time
lags. (i.e.corr(yt,yt−k),k=1,2,...)
• PACF measures the autocorrelation between
Yt and Yt-k, when the effects of other time lags,
1, 2, .., k-1, are removed.

8. AR(1):Yt=0+ 1Yt-1+t
18.05.2023
8
18.05.2023
-1
1
0 k
-1
1
0 k



    
-1
1
0 k
-1
1
0 k










ACF PACF

9. AR(2):Yt=0+ 1Yt-1+ 2Yt-2 +t
18.05.2023
9
18.05.2023
-1
1
0 k
-1
1
0 k



    
-1
1
0 k
-1
1
0 k










ACF PACF



10. MA(1):Yt=+t- 1t-1
18.05.2023
10
18.05.2023
-1
1
0 k
-1
1
0 k

-1
1
0 k
-1
1
0 k










ACF PACF

11. MA(2):Yt=+t- 1t-1 - 2t-2
18.05.2023
11
18.05.2023
-1
1
0 k
-1
1
0 k
-1
1
0 k
-1
1
0 k

ACF PACF














12. ARMA(1,1):Yt= 0+ 1Yt-1 +t- 1t-1
18.05.2023
12
-1
1
0 k
-1
1
0 k
Auto Correlation Partial Auto Correlation










13. ARMA(1,1):Yt= 0+ 1Yt-1 +t- 1t-1
18.05.2023
13
18.05.2023
-1
1
0 k
-1
1
0 k
Auto Correlation Partial Auto Correlation

14. AR, MA or ARMA?
Autocorrelations Partial
Autocorrelations
MA(q) Cut off after the
order of q of the
process
Die out
AR(p) Die out Cut off after the
order of p of the
process
ARMA(p,q) Die out Die out

15. Model Building Strategy
• Step 1: Model identification
Plot the time series/ACF and examine whether it is
stationary. If not, try some transformation and or
differencing, until the data seems stationary. Compare ACF
and PACF of the time series data and identify the ARIMA
model to be used. To judge the significance of
autocorrelation and partial autocorrelation, the
corresponding sample values may be compared with ±2/ .
Use the principle of parsimony.
• Step 2: Model estimation
Use SPSS or other package to estimate the model
parameters. t-test may be used to judge whether a
parameter may be dropped from the model.
n
n

16. Model Building Strategy
• Step 3: Model checking
• The model will be considered adequate if the residuals
are random. The following three procedures may be used.
• Residual plots as in regression may be used,
• rk(e) must be within ±2/ of zero, and
• L-Q test may be used to test whether a group
autocorrelation of lags 1, 2,.. m, is significant.
•
• Step 4: Model forecasting
• SPSS generates forecasts for a given number of future
periods.
n

17. Model Selection Criteria
• Akaike Information Criterion (AIC) selects the best
model from a group of candidate models as the
one that minimizes
• Bayesian Information Criterion (BIC) selects the
best model e that minimizes
where σ2 residual variance
2 2
AIC=ln r
n
 
2 lnn
BIC=ln r
n
 

18. Example 1
-1
-0.5
0
0.5
1
0 2 4 6 8 10 12 14 16
lag
ACF for Y
+- 1.96/T^0.5
-1
-0.5
0
0.5
1
0 2 4 6 8 10 12 14 16
lag
PACF for Y
+- 1.96/T^0.5

19. Example 2
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0 5 10 15 20
lag
ACF for C1
+- 1.96/T^0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0 5 10 15 20
lag
PACF for C1
+- 1.96/T^0.5

20. Example 3
18.05.2023
20
18.05.2023
-0.23 -0.2 -1.93 -0.97 0.1
0.63 -0.21 1.87 0.83 -0.62
0.48 0.91 -0.97 -0.33 2.27
-0.83 -0.36 0.46 0.91 -0.62
-0.03 0.48 2.12 -1.13 0.74
1.31 0.61 -2.11 2.22 -0.16
0.86 -1.38 0.7 0.8 1.34
-1.28 -0.04 0.69 -1.95 -1.83
0 0.9 -0.24 2.61 0.31
-0.63 1.79 0.34 0.59 1.13
0.08 -0.37 0.6 0.71 -0.87
-1.3 0.4 0.15 -0.84 1.45
1.48 -1.19 -0.02 -0.11 -1.95
-0.28 0.98 0.46 1.27 -0.51
-0.79 -1.51 -0.54 -0.8 -0.41
1.86 0.9 0.89 -0.76 0.49
0.07 -1.56 1.07 1.58 1.54
0.09 2.18 0.2 -0.38 -0.96

21. Example 3
18.05.2023
21
18.05.2023
Let’s look at ACF and PACF

22. Example 3
18.05.2023
22
18.05.2023
-3
-2
-1
0
1
2
3
1 9 17 25 33 41 49 57 65 73 81 89

23. Example 3
18.05.2023
23
18.05.2023
-1
1
0 k
-1
1
0 k

Auto Correlation Partial Auto Correlation

24. ATR şirketi üretim hedefleri Öngörüsü
18.05.2023
24
18.05.2023
MA(1):Yt=+t+ 1t-1
1
5875
.
0
1513
.
0
ˆ


 t
Y 

25. ATR şirketi üretim hedefleri Öngörüsü
18.05.2023
25
18.05.2023
Relative change in each estimate less than 0.0010
Final Estimates of Parameters
Type Coef SE Coef T P
MA 1 0.5875 0.0864 6.80 0.000
Constant 0.15129 0.04022 3.76
0.000
Mean 0.15129 0.04022
Number of observations: 90
Residuals: SS = 74.4933 (backforecasts excluded)
MS = 0.8465 DF = 88
Modified Box-Pierce (Ljung-Box) Chi-Square statistic
Lag 12 24 36 48
Chi-Square 9.1 10.8 17.3 31.5
DF 10 22 34 46
P-Value 0.524 0.977 0.992 0.950
Forecasts from period 90
95% Limits
Period Forecast Lower Upper Actual
91 0.43350 -1.37018 2.23719
92 0.15129 -1.94064 2.24322

26. Example 3
18.05.2023
26
18.05.2023

27. Example 3
18.05.2023
27
18.05.2023
Obs. # Actual Forecast Error
1 -0.23 0.101969 -0.331969
2 0.63 0.346323 0.283677
3 0.48 -0.015371 0.495371
4 -0.83 -0.139741 -0.690259
5 -0.03 0.556819 -0.586819
.
.
.
.
.
.
.
.
.
.
.
.
86 -0.51 1.06829 -1.57829
87 -0.41 1.07854 -1.48854
88 0.49 1.02581 -0.53581
89 1.54 0.46608 1.07392
90 -0.96 -0.47964 -0.48036
91forecast
92Forecast
90
91 5875
.
0
1513
.
0
ˆ 


Y )
4804
.
0
(
5875
.
0
1513
.
0 

 4335
.
0

0.4335
0.1513
91
92 5875
.
0
1513
.
0
ˆ 


Y )
000
.
0
(
5875
.
0
1513
.
0 
 1513
.
0

0.4335 0.0000

28. Example
• The analyst for the ISC Corporation was asked to
develop forecasts for the closing prices of ISC stock.
• The stock has been languishing for some time with
little growth, and senior management wanted some
projections to discuss with the board of directors.
• The ISC stock prices are plotted in the following
slide.
Dr. Mohammed Alahmed
28

29. Example 4:Stock prices of ISC
18.05.2023
29
18.05.2023
235 200 250 270 275
320 290 225 240 205
115 220 125 275 265
355 400 295 225 245
190 275 250 285 170
320 185 355 250 175
275 370 280 310 270
205 255 370 220 225
295 285 250 320 340
240 250 290 215 190
355 300 225 260 250
175 225 270 190 300
285 285 180 295 195

30. Example 4: ISC
Index
ISC
60
54
48
42
36
30
24
18
12
6
1
400
350
300
250
200
150
100
Time Series Plot of ISC corporation Stock
Dr. Mohammed Alahmed
30

31. Example 4
• The plot of the stock prices suggests the series is
stationary.
• The stock prices vary about a fixed level of
approximately 250.
• Is the Box-Jenkins methodology appropriate for this
data series?
• The ACF and PACF for the stock price series are
reported in the following two slides.
Dr. Mohammed Alahmed
31

32. Example 4
Lag
Autocorrelation
20
18
16
14
12
10
8
6
4
2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
Autocorrelation Function for ISC
(with 5% significance limits for the autocorrelations)
Dr. Mohammed Alahmed
32
Lag
Partial
Autocorrelation
20
18
16
14
12
10
8
6
4
2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
Partial Autocorrelation Function for ISC
(with 5% significance limits for the partial autocorrelations)

33. Example 4
• The sample ACF alternate in sign and decline to zero
after lag 2.
• The sample PACF are similar are close to zero after
time lag 2.
• These are consistent with an AR(2) or ARIMA(2,0,0)
model
• AR(2) model is fit to the data.
• WE include a constant term to allow for a nonzero
level.
Dr. Mohammed Alahmed
33

34. Example 4
18.05.2023
34
18.05.2023 Pazarlıoğlu-Güneş
Looks like AR (2)
-1
1
0 k
-1
1
0 k












35. Example 4
• The estimated coefficient 2 is not significant (t=1.75) at
5% level but is significant at the 10 % level.
• The residual ACF and PACF are given in the following
two slides.
• The ACF and PACF are well within their two standard
error limits.
Dr. Mohammed Alahmed
35
Final Estimates of Parameters
Type Coef SE T P
AR 1 -0.3243 0.1246 -2.60 0.012
AR 2 0.2192 0.1251 1.75 0.085
Constant 284.903 6.573 43.34 0.000

36. Example 4
36
AR(2):Yt=0+ 1Yt-1+ 2Yt-2 +t
2
t
1
t
t Y
219
.
0
Y
324
.
0
9
.
284
Ŷ 
 



37. Example 4
Lag
Autocorrelation
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
ACF of Residuals for ISC
(with 5% significance limits for the autocorrelations)
Dr. Mohammed Alahmed
37
Lag
Partial
Autocorrelation
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
PACF of Residuals for ISC
(with 5% significance limits for the partial autocorrelations)

38. Forecast for Example 4
38
Obs. No Actual Forecast Error
1 235 248.416 -13.416
2 320 269.842 50.158
3 115 232.664 -117.664
4 355 317.771 37.229
5 190 195.006 -5.006
.
.
.
.
.
.
.
.
.
.
.
.
61 340 271.141 68.8586
62 190 223.987 -33.9866
63 250 297.837 -47.8371
64 300 245.496 54.5041
65 195 242.438 -47.4377
66Forecast
67Forecast
64
65
66 Y
219
.
0
Y
324
.
0
9
.
284
Ŷ 


287.4
234.5
287.445 -0.0450
)
300
(
219
.
0
)
195
(
324
.
0
9
.
284 

 4
.
287

65
66
67 Y
219
.
0
Y
324
.
0
9
.
284
Ŷ 

 )
195
(
219
.
0
)
4
.
287
(
324
.
0
9
.
284 

 5
.
234


39. Example 5
Minutes
Number
of
Users
100
90
80
70
60
50
40
30
20
10
1
240
220
200
180
160
140
120
100
80
Time Series Plot of Number of Users
Dr. Mohammed Alahmed
39

40. Example 5
Lag
Autocorrelation
20
18
16
14
12
10
8
6
4
2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
Autocorrelation Function for Number of Users
(with 5% significance limits for the autocorrelations)
Dr. Mohammed Alahmed
40
Lag
Partial
Autocorrelation
20
18
16
14
12
10
8
6
4
2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
Partial Autocorrelation Function for Number of Users
(with 5% significance limits for the partial autocorrelations)

41. Example 5
• The gradual decline of ACF values indicates non-
stationary series.
• The first partial autocorrelation is very dominant and
close to 1, indicating non-stationarity.
• The time series plot clearly indicates non-stationarity.
• We take the first differences of the data and
reanalyze.
Dr. Mohammed Alahmed
41

42. Example 5
Minutes
first
difference
100
90
80
70
60
50
40
30
20
10
1
15
10
5
0
-5
-10
-15
Time Series Plot of first difference
Dr. Mohammed Alahmed
42

43. Example 5
Lag
Autocorrelation
20
18
16
14
12
10
8
6
4
2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
Autocorrelation Function for first difference
(with 5% significance limits for the autocorrelations)
Dr. Mohammed Alahmed
43

44. Model identification
Lag
Partial
Autocorrelation
20
18
16
14
12
10
8
6
4
2
1.0
0.8
0.6
0.4
0.2
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
Partial Autocorrelation Function for first difference
(with 5% significance limits for the partial autocorrelations)
Dr. Mohammed Alahmed
44
• ACF shows a mixture of exponential decay and sine-wave pattern
• PACF shows three significant PACF values.
• This suggests an AR(3) model.
• This identifies an ARIMA(3,1,0).

45. Seasonality and ARIMA models
• The ARIMA models can be extended to handle seasonal
components of a data series.
• The general shorthand notation is
ARIMA (p, d, q)(P, D, Q)s
• Where s is the number of periods per season.
•
Dr. Mohammed Alahmed
45

46. Seasonality and ARIMA models
• The seasonal lags of the ACF and PACF plots show the
seasonal parts of an AR or MA model.
• Examples:
1. Seasonal MA model:
• ARIMA(0,0,0)(0,0,1)12
– will show a spike at lag 12 in the ACF but no other significant
spikes.
– The PACF will show exponential decay in the seasonal lags
i.e. at lags 12, 24, 36,…
2. Seasonal AR model:
• ARIMA(0,0,0)(1,0,0)12
– will show exponential decay in seasonal lags of the ACF.
– Single significant spike at lag 12 in the PACF.
Dr. Mohammed Alahmed
46

47. 3000
2500
2000
1500
1000
500
0
1 12 23 34 45 56 67 78 89 100 111
Example 6:Sales
47

48. Auto correlation dies out after first lag.
But it is different than 0 at 12,24,36 lag
Series may not be stationary. Let’s take seasonal difference
12.12.2018
Dr
.M.HanifiVAN 48

49. 12.12.2018
Dr
.M.HanifiVAN 49
12Yt Yt Yt12
Seasonal Difference

50. 500
400
300
200
100
0
-100
-200
-300
1 12 23 34 45 56 67 78 89 100 111
12.12.2018
Dr
.M.HanifiVAN 50
Seasonal Difference
After seasonal difference, series looks like
stationary.

51. Coeffient at lag 12 is significantly different
than 0.
12.12.2018
Dr
.M.HanifiVAN 51

52. 12.12.2018
41
Coeffients at lag 12 and 24 are significantly
different than 0.
Model look like seasonal MA(1)

53. -1
1
0 k
-1
1
0 k
Auto Correlation Partial Auto
Correlation
12.12.2018
53

54. • Model for ARIMA(p,d,q)(P,D,Q)12 will be
• ARIMA(0,0,0)(0,1,1)12 .
• We have D=1 as we take seasonal
difference
•Yt Yt12 t 1t12
• Yt Yt12 85.457t 0.8180t12

55. Forecast
12.12.2018
Dr
.M.HanifiVAN 55
Yt Yt12 85.457t 0.818t12
Ŷ116  Y104  85.457  0.818104
Ŷ116  227585.457  0.818(72.418)  2419.7
Ŷ117  Y105  85.457  0.818105
Ŷ116  2581.885.457 0.818(199.214) 2504.3