Econometrics is the application of statistical methods to economic data to empirically test economic theories. The main goals of econometrics are to find estimators that have desirable statistical properties like unbiasedness, efficiency, and consistency. These estimators are used to assess economic theories, forecast macroeconomic indexes, predict revenue, and estimate the impact of economic changes. Common econometric tools include linear regression models, which make assumptions about the properties of the error term or residuals. Autoregressive models forecast variables using their own lagged values.

1. Time Series
Vitalii Radchenko

2. What is Econometrics?

3. What is Econometrics?
• is the application of statistical methods to economic data
and is described as the branch of economics that aims to
give empirical content to economic relations

4. What is Econometrics?
• is the application of statistical methods to economic data
and is described as the branch of economics that aims to
give empirical content to economic relations
• Basic tools:

5. What is Econometrics?
• is the application of statistical methods to economic data
and is described as the branch of economics that aims to
give empirical content to economic relations
• Basic tools:
• linear regression models

6. What is Econometrics?
• is the application of statistical methods to economic data
and is described as the branch of economics that aims to
give empirical content to economic relations
• Basic tools:
• linear regression models
• statistical theory

7. The main goals

8. The main goals
• To find estimators that have desirable statistical properties:

9. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness

10. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency

11. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency

12. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for

13. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for
• assessing economic theories

14. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for
• assessing economic theories
• forecasting macroeconomic indexes

15. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for
• assessing economic theories
• forecasting macroeconomic indexes
• predicting revenue

16. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for
• assessing economic theories
• forecasting macroeconomic indexes
• predicting revenue
• estimating the impact of something

17. The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for
• assessing economic theories
• forecasting macroeconomic indexes
• predicting revenue
• estimating the impact of something
Interpretability and Statistical robustness

18. Government
The main goals
• To find estimators that have desirable statistical properties:
• unbiasedness
• efficiency
• consistency
• Applied for
• assessing economic theories
• forecasting macroeconomic indexes
• predicting revenue
• estimating the impact of something
Interpretability and Statistical robustness

19. Basic approach

20. Basic approach
Linear regression

21. Basic approach
Linear regression
• Residuals assumptions:

22. Basic approach
Linear regression
• Residuals assumptions:
1. Mean equals 0

23. Basic approach
Linear regression
• Residuals assumptions:
1. Mean equals 0
2. Deviation is constant (homoscedasticity)

24. Basic approach
Linear regression
• Residuals assumptions:
1. Mean equals 0
2. Deviation is constant (homoscedasticity)
3. Independent residuals (covariation equals 0)

25. Basic approach
Linear regression
• Residuals assumptions:
1. Mean equals 0
2. Deviation is constant (homoscedasticity)
3. Independent residuals (covariation equals 0)
4. Independence of residuals and regressors

26. Basic approach
Linear regression
• Residuals assumptions:
1. Mean equals 0
2. Deviation is constant (homoscedasticity)
3. Independent residuals (covariation equals 0)
4. Independence of residuals and regressors
5. Residuals should be normally distributed

27. Gauss-Markov theorem
If all five assumptions are satisfied for a simple linear
regression, then the variance of the OLS estimates will be
the smallest among all unbiased estimates

28. Hypothesis estimation
• If the assumption of the residuals normality is satisfied,
then we test the hypotheses by comparing with the
values of the Fisher distribution
• If not satisfied – Chi-square

29. Hypothesis

30. Hypothesis
• The adequacy of the model (H0: R^2 = 0)

31. Hypothesis
• The adequacy of the model (H0: R^2 = 0)
• The significance of the correlation between the variables
(H0: rxy = 0)

32. Hypothesis
• The adequacy of the model (H0: R^2 = 0)
• The significance of the correlation between the variables
(H0: rxy = 0)
• The significance of the regression coefficients (H0: B = 0)

33. Hypothesis
• The adequacy of the model (H0: R^2 = 0)
• The significance of the correlation between the variables
(H0: rxy = 0)
• The significance of the regression coefficients (H0: B = 0)
• Multicollinearity (VIF, Farr-Glauber criterion)

34. Hypothesis
• The adequacy of the model (H0: R^2 = 0)
• The significance of the correlation between the variables
(H0: rxy = 0)
• The significance of the regression coefficients (H0: B = 0)
• Multicollinearity (VIF, Farr-Glauber criterion)
• Check of the functional form (criterion RESET)

35. Heteroskedasticity

36. Heteroskedasticity
• Identification: Golffred-Quondt criterion, White criterion,
Broyush-Pagan criterion, Glaser criterion

37. Heteroskedasticity
• Identification: Golffred-Quondt criterion, White criterion,
Broyush-Pagan criterion, Glaser criterion
• Second condition is not satisfied for the variance equality

38. Heteroskedasticity
• Identification: Golffred-Quondt criterion, White criterion,
Broyush-Pagan criterion, Glaser criterion
• Second condition is not satisfied for the variance equality
• Weighted least squares

39. Heteroskedasticity
• Identification: Golffred-Quondt criterion, White criterion,
Broyush-Pagan criterion, Glaser criterion
• Second condition is not satisfied for the variance equality
• Weighted least squares

40. Autocorrelation

41. Autocorrelation
• Correlated residuals

42. Autocorrelation
• Correlated residuals

43. Autocorrelation
• Correlated residuals
• Identification: the Durbin-Watson criterion, Broych-Godfrey

44. Autocorrelation
• Correlated residuals
• Identification: the Durbin-Watson criterion, Broych-Godfrey
• The generalized least-squares method

45. Autocorrelation
• Correlated residuals
• Identification: the Durbin-Watson criterion, Broych-Godfrey
• The generalized least-squares method
• Select correlation coefficient:

46. Autocorrelation
• Correlated residuals
• Identification: the Durbin-Watson criterion, Broych-Godfrey
• The generalized least-squares method
• Select correlation coefficient:
• Durbin-Watson estimate

47. Autocorrelation
• Correlated residuals
• Identification: the Durbin-Watson criterion, Broych-Godfrey
• The generalized least-squares method
• Select correlation coefficient:
• Durbin-Watson estimate
• Cochrane-Orcatte method

48. Autocorrelation
• Correlated residuals
• Identification: the Durbin-Watson criterion, Broych-Godfrey
• The generalized least-squares method
• Select correlation coefficient:
• Durbin-Watson estimate
• Cochrane-Orcatte method
• Hildretha-Lou method

49. Time Series Patterns

50. Time Series Patterns
Trend

51. Time Series Patterns
Trend
• A trend exists when there is a long-term increase or decrease in the data. It does
not have to be linear. Sometimes we will refer to a trend “changing direction”
when it might go from an increasing trend to a decreasing trend.

52. Time Series Patterns
Trend
• A trend exists when there is a long-term increase or decrease in the data. It does
not have to be linear. Sometimes we will refer to a trend “changing direction”
when it might go from an increasing trend to a decreasing trend.
Seasonal

53. Time Series Patterns
Trend
• A trend exists when there is a long-term increase or decrease in the data. It does
not have to be linear. Sometimes we will refer to a trend “changing direction”
when it might go from an increasing trend to a decreasing trend.
Seasonal
• A seasonal pattern exists when a series is influenced by seasonal factors (e.g.,
the quarter of the year, the month, or day of the week). Seasonality is always of a
fixed and known period.

54. Time Series Patterns
Trend
• A trend exists when there is a long-term increase or decrease in the data. It does
not have to be linear. Sometimes we will refer to a trend “changing direction”
when it might go from an increasing trend to a decreasing trend.
Seasonal
• A seasonal pattern exists when a series is influenced by seasonal factors (e.g.,
the quarter of the year, the month, or day of the week). Seasonality is always of a
fixed and known period.
Cyclic

55. Time Series Patterns
Trend
• A trend exists when there is a long-term increase or decrease in the data. It does
not have to be linear. Sometimes we will refer to a trend “changing direction”
when it might go from an increasing trend to a decreasing trend.
Seasonal
• A seasonal pattern exists when a series is influenced by seasonal factors (e.g.,
the quarter of the year, the month, or day of the week). Seasonality is always of a
fixed and known period.
Cyclic
• A cyclic pattern exists when data exhibit rises and falls that are not of fixed
period. The duration of these fluctuations is usually of at least 2 years.

56. Time series decomposition

57. Time series decomposition
• additive model
yt = St +Tt + Et
St - seasonal component, Tt – trend-cycle component, Et – reminder

58. Time series decomposition
• additive model
• multiplicative model
yt = St +Tt + Et
yt = St iTt i Et
St - seasonal component, Tt – trend-cycle component, Et – reminder

59. Time series decomposition
• additive model
• multiplicative model
yt = St +Tt + Et
yt = St iTt i Et
St - seasonal component, Tt – trend-cycle component, Et – reminder

60. Time Series Decomposition
• What is the problem here?

61. Time Series Decomposition
• What is the problem here? Seasonal component is not meaningful

62. Stationarity and differencing

63. Stationarity and differencing
• Stationary data:

64. Stationarity and differencing
• Stationary data:
• mean is a constant

65. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant

66. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time

67. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time

68. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:

69. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)

70. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)

71. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)

72. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)
Problem is that the absence of a unit root is not a proof of stationarity

73. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)
• Make data stationary:
Problem is that the absence of a unit root is not a proof of stationarity

74. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)
• Make data stationary:
• log-transformation
Problem is that the absence of a unit root is not a proof of stationarity

75. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)
• Make data stationary:
• log-transformation
• differencing
Problem is that the absence of a unit root is not a proof of stationarity

76. Stationarity and differencing
• Stationary data:
• mean is a constant
• variance is a constant
• covariance is not a function of time
• Tests:
• Augmented Dickey-Fuller (a unit root is present)
• KPSS (trend stationary)
• Make data stationary:
• log-transformation
• differencing
• log-transformation and differencing
Problem is that the absence of a unit root is not a proof of stationarity

77. Autoregressive models

78. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable

79. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable
yt = c +φ1yt−1 +φ2yt−2 +…+φpyt−p + et

80. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable
• For an AR(1) model:
yt = c +φ1yt−1 +φ2yt−2 +…+φpyt−p + et

81. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable
• For an AR(1) model:
• When ϕ1=0, yt is equivalent to white noise
yt = c +φ1yt−1 +φ2yt−2 +…+φpyt−p + et

82. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable
• For an AR(1) model:
• When ϕ1=0, yt is equivalent to white noise
• When ϕ1=1 and c=0, yt is equivalent to a random walk
yt = c +φ1yt−1 +φ2yt−2 +…+φpyt−p + et

83. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable
• For an AR(1) model:
• When ϕ1=0, yt is equivalent to white noise
• When ϕ1=1 and c=0, yt is equivalent to a random walk
• When ϕ1=1 and c≠0, yt is equivalent to a random walk with drift
yt = c +φ1yt−1 +φ2yt−2 +…+φpyt−p + et

84. Autoregressive models
• forecast the variable of interest using a linear combination of past
values of the variable
• For an AR(1) model:
• When ϕ1=0, yt is equivalent to white noise
• When ϕ1=1 and c=0, yt is equivalent to a random walk
• When ϕ1=1 and c≠0, yt is equivalent to a random walk with drift
• When ϕ1<0, yt tends to oscillate between positive and negative
values.
yt = c +φ1yt−1 +φ2yt−2 +…+φpyt−p + et

85. Moving average models
• Rather than use past values of the forecast variable in a
regression, a moving average model uses past forecast
errors in a regression-like model
yt = c + et +θ1et−1 +θ2et−2 +…+θpet−p

86. ARIMA

87. ARIMA
• combine differencing with autoregression and a moving average model, we obtain
a non-seasonal ARIMA model
yt
'
= c +φ1yt−1
'
+…+φpyt−p
'
+θ1et−1 +…+θqet−q + et

88. ARIMA
• combine differencing with autoregression and a moving average model, we obtain
a non-seasonal ARIMA model
• Information Criteria
yt
'
= c +φ1yt−1
'
+…+φpyt−p
'
+θ1et−1 +…+θqet−q + et

89. ARIMA
• combine differencing with autoregression and a moving average model, we obtain
a non-seasonal ARIMA model
• Information Criteria
• Akaike’s Information Criterion (AIC)
yt
'
= c +φ1yt−1
'
+…+φpyt−p
'
+θ1et−1 +…+θqet−q + et
AIC = −2log(L)+ 2(p + q + k +1)

90. ARIMA
• combine differencing with autoregression and a moving average model, we obtain
a non-seasonal ARIMA model
• Information Criteria
• Akaike’s Information Criterion (AIC)
• Bayesian Information Criterion (BIC)
yt
'
= c +φ1yt−1
'
+…+φpyt−p
'
+θ1et−1 +…+θqet−q + et
AIC = −2log(L)+ 2(p + q + k +1)
BIC = AIC + (log(L)− 2)(p + q + k +1)

91. ARIMA
• combine differencing with autoregression and a moving average model, we obtain
a non-seasonal ARIMA model
• Information Criteria
• Akaike’s Information Criterion (AIC)
• Bayesian Information Criterion (BIC)
• corrected Akaike’s Information Criterion (AICc)
yt
'
= c +φ1yt−1
'
+…+φpyt−p
'
+θ1et−1 +…+θqet−q + et
AIC = −2log(L)+ 2(p + q + k +1)
BIC = AIC + (log(L)− 2)(p + q + k +1)
AICc = AIC +
2(p + q + k +1)(p + q + k + 2)
T − p − q − k − 2

92. Other econometrics models

93. Other econometrics models
• SARIMA – ARIMA with seasonal component

94. Other econometrics models
• SARIMA – ARIMA with seasonal component
• ARFIMA – ARIMA allowing non-integer values in differencing
parameter

95. Other econometrics models
• SARIMA – ARIMA with seasonal component
• ARFIMA – ARIMA allowing non-integer values in differencing
parameter
• VAR – vector autoregression (allows to predict multiple target
variables and to learn dynamic relationships between them)

96. Other econometrics models
• SARIMA – ARIMA with seasonal component
• ARFIMA – ARIMA allowing non-integer values in differencing
parameter
• VAR – vector autoregression (allows to predict multiple target
variables and to learn dynamic relationships between them)
• ARCH (GARCH) – (assume that we have heteroskedastisity,
predicting mean and deviation separately)

97. Other econometrics models
• SARIMA – ARIMA with seasonal component
• ARFIMA – ARIMA allowing non-integer values in differencing
parameter
• VAR – vector autoregression (allows to predict multiple target
variables and to learn dynamic relationships between them)
• ARCH (GARCH) – (assume that we have heteroskedastisity,
predicting mean and deviation separately)
• Hierarchical time series (predict high and low level)

98. Other econometrics models
• SARIMA – ARIMA with seasonal component
• ARFIMA – ARIMA allowing non-integer values in differencing
parameter
• VAR – vector autoregression (allows to predict multiple target
variables and to learn dynamic relationships between them)
• ARCH (GARCH) – (assume that we have heteroskedastisity,
predicting mean and deviation separately)
• Hierarchical time series (predict high and low level)
Forecasting: principles and practice (Rob J Hyndman)

99. Why econometrics models
are good?

100. Why econometrics models
are good?
Simple

101. Why econometrics models
are good?
Simple
Statistically robust

102. Why econometrics models
are good?
Simple
Statistically robust
Interpretable

103. Why econometrics models
are bad?

104. Why econometrics models
are bad?
Simple

105. Why econometrics models
are bad?
• mostly linear and can catch non-linear dependencies
Simple

106. Why econometrics models
are bad?
• mostly linear and can catch non-linear dependencies
• takes long time to optimize and train models
Simple

107. Why econometrics models
are bad?
• mostly linear and can catch non-linear dependencies
• takes long time to optimize and train models
• accuracy is not very good
Simple

108. Why econometrics models
are bad?
• mostly linear and can catch non-linear dependencies
• takes long time to optimize and train models
• accuracy is not very good
• the same features could be generated manually and used
with more complex models
Simple

109. Neural Networks

110. Neural Networks
• RNN is a hype theme

111. Neural Networks
• RNN is a hype theme
• Sequence-to-Sequence modeling
.. 3 2 1
LSTM
1 2 3 ..

112. Neural Networks
• RNN is a hype theme
• Sequence-to-Sequence modeling
• Experiment with number of lag features and adding “external” data
.. 3 2 1
LSTM
1 2 3 ..
.. 3 2 1
LSTM
1 2 3 ..
Dense
External features

113. Neural Networks
• RNN is a hype theme
• Sequence-to-Sequence modeling
• Experiment with number of lag features and adding “external” data
• Works with multiple time series with a long history
.. 3 2 1
LSTM
1 2 3 ..
.. 3 2 1
LSTM
1 2 3 ..
Dense
External features

114. Neural Networks
• RNN is a hype theme
• Sequence-to-Sequence modeling
• Experiment with number of lag features and adding “external” data
• Works with multiple time series with a long history
• Usually works worse than linear or boosting models
.. 3 2 1
LSTM
1 2 3 ..
.. 3 2 1
LSTM
1 2 3 ..
Dense
External features

115. Stacking
Validation
Linear Regressions
Boosting

116. Stacking
Train Test
Train Test
Train Test
Train Test
Fold 1
Fold 2
Fold 3
Fold 4

117. Stacking
Train Test
Train Test
Train Test
Train Test
Fold 1
Fold 2
Fold 3
Fold 4Step 1
• train linear regression and optimize parameters on Fold 1

118. Stacking
Train Test
Train Test
Train Test
Train Test
Fold 1
Fold 2
Fold 3
Fold 4Step 1
• train linear regression and optimize parameters on Fold 1
Step 2
• predict train and test on Fold 2. Use predictions as a new feature and apply boosting

119. Stacking
Train Test
Train Test
Train Test
Train Test
Fold 1
Fold 2
Fold 3
Fold 4Step 1
• train linear regression and optimize parameters on Fold 1
Step 2
• predict train and test on Fold 2. Use predictions as a new feature and apply boosting
Step 3
• train linear regression and optimize parameters on Fold 2

120. Stacking
Train Test
Train Test
Train Test
Train Test
Fold 1
Fold 2
Fold 3
Fold 4Step 1
• train linear regression and optimize parameters on Fold 1
Step 2
• predict train and test on Fold 2. Use predictions as a new feature and apply boosting
Step 3
• train linear regression and optimize parameters on Fold 2
Step 4
• predict train and test on Fold 2. Use predictions as a new feature and apply boosting
…

121. Stacking
Train Test
Train Test
Train Test
Train Test
Fold 1
Fold 2
Fold 3
Fold 4Step 1
• train linear regression and optimize parameters on Fold 1
Step 2
• predict train and test on Fold 2. Use predictions as a new feature and apply boosting
Step 3
• train linear regression and optimize parameters on Fold 2
Step 4
• predict train and test on Fold 2. Use predictions as a new feature and apply boosting
…
Last step
• validate final results on Fold 2, Fold 3 and Fold 4

122. Interpretation

123. Interpretation
• Linear regression - the easiest way to interpret features

124. Interpretation
• Linear regression - the easiest way to interpret features
• MARS (Earth) – a flexible regression method that automatically searches for interactions and non-linear
relationships

125. Interpretation
• Linear regression - the easiest way to interpret features
• MARS (Earth) – a flexible regression method that automatically searches for interactions and non-linear
relationships
• parameters: number of interactions, regularization, smoothing etc

126. Interpretation
• Linear regression - the easiest way to interpret features
• MARS (Earth) – a flexible regression method that automatically searches for interactions and non-linear
relationships
• parameters: number of interactions, regularization, smoothing etc
• ELI5 – support Lightgbm and Xgboost sklearn api

127. Interpretation
• Linear regression - the easiest way to interpret features
• MARS (Earth) – a flexible regression method that automatically searches for interactions and non-linear
relationships
• parameters: number of interactions, regularization, smoothing etc
• ELI5 – support Lightgbm and Xgboost sklearn api
• show weights and explain predictions

128. Interpretation
• Linear regression - the easiest way to interpret features
• MARS (Earth) – a flexible regression method that automatically searches for interactions and non-linear
relationships
• parameters: number of interactions, regularization, smoothing etc
• ELI5 – support Lightgbm and Xgboost sklearn api
• show weights and explain predictions
• as it is linear approximation, <BIAS> is usually big (misleading)

129. Summary

130. Summary
• Choose metric and validation approach based on business task

131. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation

132. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation
• Create a simple baseline

133. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation
• Create a simple baseline
• Generate default features

134. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation
• Create a simple baseline
• Generate default features
• Try linear and boosting models (and RNN)

135. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation
• Create a simple baseline
• Generate default features
• Try linear and boosting models (and RNN)
• Add more features

136. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation
• Create a simple baseline
• Generate default features
• Try linear and boosting models (and RNN)
• Add more features
• Don’t forget about ensembles and stacking :)

137. Summary
• Choose metric and validation approach based on business task
• Do basic EDA and start with log-transformation
• Create a simple baseline
• Generate default features
• Try linear and boosting models (and RNN)
• Add more features
• Don’t forget about ensembles and stacking :)
• Check feature weights and prediction explanation

138. Contacts
• ODS-slack: @vradchenko
• Email: radchenko.vitaliy.o@gmail.com
• Facebook: https://www.facebook.com/
vitaliyradchenko127

139. Thank you