This document discusses the key assumptions of linear regression models: (1) linearity between dependent and independent variables, (2) independent and normally distributed errors, (3) homoscedasticity or constant variance of errors. It describes how to check these assumptions by examining residual plots and statistical tests, and provides examples of what abnormal patterns may indicate violations of assumptions. Remedies discussed include transforming variables to achieve linearity and normality or working with smaller intervals of data to address heteroscedasticity.

1. SLR Model Assumptions
SPSS-Model Check
Introduced by
Dr. Nermin Osman
Assistant Lecturer of Medical Statistics,
MRI, Alexandria University
United Nations System Staff College
Intern-UNITAR

3. Skewness vs. Kurtosis

4. http://www.leanmath.com/bl
og-entry/coefficient-variation

5. Z score

6. Four principal assumptions which justify the
use of linear regression models for purposes
of prediction
(1) linearity of the relationship between dependent and
independent variables
(2) Independence of the errors (no serial correlation)
(3) homoscedasticity (constant variance) of the errors
(a) versus time
(b) versus the predictions (or versus any independent variable)
(4) normality of the error distribution.

7. Other Potential assumption violations
Multicollinearity: X variables that are nearly linear
combinations of other X variables in the equation
Outliers/ Influentials: apparent non-normality by a few data
points

8. Analysis of Residuals
RESIDUAL
)ˆ( yyei 

9. N.B. Errors and residuals
The error of an observed value is the deviation of
the observed value from the true value (for
example, a population mean)
The residual of an observed value is the difference
between the observed value and
the estimated value of the quantity of interest (for
example, a sample mean).

10. Residuals
Residuals are deviations of observed values from model fitted values.
10
Residual

11. Violations of linearity: How to detect?
Nonlinearity is usually most evident in a plot of:
◦ Residuals versus predicted values (Ŷ) (The points should be
symmetrically distributed around a horizontal line )

17. Abnormal Patterns in Residual Plots
Figures a). and b). suggest
non-linear relationship
between X and Y.
Fig. c). Suggests
autocorrelation.
Fig. d). Suggests variance is
not the same since the
spread of Y values is far
greater for larger values of
X.

18. Analysis of Residuals
Assumption2: Independent
residuals

19. Analysis of Residuals
Assumption2: Independent residuals
Residuals
0
1
-1
2
-2
3
-3
(a) Independent Residuals
Residuals
0
1
-1
2
-2
3
-3
(b) Residuals Not Independent

20. Independent residuals :How is it
Detected?
Residual Plots (Z Residual vs. Predicted value)
Durbin-Watson Test
20

26. Assumption 3: Violation of normality
26

27. -Normally distributed
residuals
Note
It’s the residuals that have to be normally distributed
Not
the predictors

28. Mean and Standard Deviation
of Normal Density
7: NORMAL PROBABILITY
DISTRIBUTIONS
28
μ
σ

29. 68-95-99.7 Rule
68% of
the data
95% of the data
99.7% of the data

31. Violation of normality:
How to fix?
violations of normality often arise either because :
◦ (a) the distributions of the dependent and/or independent variables are
themselves significantly non-normal,
◦ (b) the linearity assumption is violated.
In both cases, a nonlinear transformation of variables might cure both
problems.

32. Normality Assumption:How is it
Detected?
Histogram
P-P plot
K-S test for residuals
32

43. What to do about Non-Normality
Transform data
◦ positive skew/ high kurotosis  log transform
◦ negative skew/ low kurotosis - - power X square
43

44. Assumption 4:
Homoscedististy: Constant (equal) variance of
the residuals
44

45. Analysis of Residuals
Homoscedististy
Residuals
0
1
-1
2
-2
3
-3
(c) Constant Variance
x1

46. HeteroscedasticityResiduals
0
1
-1
2
-2
3
-3
x1
(a) Variance Decreases as x Increases

47. Heteroscedasticity
This assumption is a about heteroscedasticity of
the residuals
◦ Hetero=different
◦ Scedastic = scattered
◦ we want our data to be homoscedastic
◦ (Funnelling in / Funnelling Out)
47

48. Effects of Violations of homoscedasticity
make it difficult to depict the true standard deviation of the
forecast errors, usually resulting in confidence intervals that are
too wide or too narrow.
E.g. if the variance of the errors is increasing over time,
confidence intervals for out-of-sample predictions will tend to be
unrealistically narrow.
Heteroscedasticity may also have the effect of giving too much
weight to small subset of the data (namely the subset where the
error variance was largest) when estimating coefficients.

49. Why is heteroscedasticity a problem?
Heteroscedasticity does not give us biased estimates of
the coefficients--however, it does make the standard
errors of the estimates unreliable.
Due to the aforementioned problem, t-tests cannot be
trusted. We run the risk of rejecting a null hypothesis that
should not be rejected.

50. Heteroscedasticity:
How to detect?
look at plots of residuals versus time and residuals versus predicted
value,
You might also want to plot residuals versus some of the independent
variables.)

56. Heteroscedasticity:
How to fix?
a simple fix would be to work with shorter intervals of data
in which variance is more nearly constant.
Heteroscedasticity can also be a byproduct of a significant
violation of the linearity and/or independence assumptions,
in which case it may also be fixed as a byproduct of fixing
those problems.

57. What to do about Non-Normality
Transform data
◦ positive skew –  log transform
◦ negative skew - - power X square
57

58. Thank You