Injustice - Developers Among Us (SciFiDevCon 2024)
Multiple regression (1)
1. Multiple Regression
(1)
Slide 1
Shakeel Nouman
M.Phil Statistics
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
2. 11
Slide 2
Multiple Regression (1)
• Using Statistics
• The k-Variable Multiple Regression Model
• The F Test of a Multiple Regression Model
• How Good is the Regression
• Tests of the Significance of Individual
•
•
Regression Parameters
Testing the Validity of the Regression
Model
Using the Multiple Regression Model for
Prediction
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
3. 11
Slide 3
Multiple Regression (2)
• Qualitative Independent Variables
• Polynomial Regression
• Nonlinear Models and Transformations
• Multicollinearity
• Residual Autocorrelation and the DurbinWatson Test
• Partial F Tests and Variable Selection
Methods
• The Matrix Approach to Multiple
Regression Analysis
•Multiple Regression (1) By Shakeel Nouman M.Phil Statisticsof Terms Lahore, Statistical Officer
Summary and Review Govt. College University
4. 11-1 Using Statistics
y
Slide 4
y
Lines
Planes
B
B
A
Slope: 1
C
A
x1
Intercept: 0
x
Any two points (A and B), or
an intercept and slope (0 and
1), define a line on a twodimensional surface.
x2
Any three points (A, B, and C), or an
intercept and coefficients of x1 and x2
(0 , 1, and 2), define a plane in a
three-dimensional surface.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
5. 11-2 The k-Variable Multiple
Regression Model
The population regression model of a
dependent variable, Y, on a set of k
independent variables, X1, X2,. . . , Xk is
given by:
Slide 5
x2
y
2
Y= 0 + 1X1 + 2X2 + . . . + kXk +
where 0 is the Y-intercept of the
regression surface and each i , i = 1,2,...,k
is the slope of the regression surface sometimes called the response surface with respect to Xi.
1
0
x1
y 0 1 x1 2 x 2
Model assumptions:
1. ~N(0,2), independent of other errors.
2. The variables Xi are uncorrelated with the error term.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
6. Simple and Multiple LeastSquares Regression
Slide 6
y
Y
x1
y b0 b1x
X
In a simple regression model,
the least-squares estimators
minimize the sum of squared
errors from the estimated
regression line.
x2
y b0 b1 x1 b2 x 2
In a multiple regression model,
the
least-squares
estimators
minimize the sum of squared
errors from the estimated
regression plane.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
7. The Estimated Regression
Relationship
Slide 7
The estimated regression relationship:
Y b0 b1 X 1 b2 X 2 bk X k
where is the predicted value of Y, the value lying on the
Y
estimated regression surface. The terms b0,...,k are the leastsquares estimates of the population regression parameters .
i
The actual, observed value of Y is the predicted value plus an
error:
yj = b0+ b1 x1j+ b2 x2j+. . . + bk xkj+e
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
8. Least-Squares Estimation:
The 2-Variable Normal Equations
Slide 8
Minimizing the sum of squared errors with respect to the
estimated coefficients b0, b1, and b2 yields the following
normal equations:
y nb b x b x
0
1
1
2
2
x y b x b x b x x
2
1
0
1
1
1
2
1
x y b x b x x b x
2
0
2
1
1
2
2
2
2
2
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
10. Example 11-1: Using the
Template
Slide 10
Regression results for Alka-Seltzer sales
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
11. Decomposition of the Total
Deviation in a Multiple
Regression Model
y
Total deviation: Y Y
y
Slide 11
Y Y : Error Deviation
Y Y : Regression Deviation
x1
x2
Total Deviation = Regression Deviation + Error Deviation
SST
=
SSR
+ SSE
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
12. 11-3 The F Test of a Multiple
Regression Model
Slide 12
A statistical test for the existence of a linear relationship between Y and any or
all of the independent variables X1, x2, ..., Xk:
H0: = = ...= =0
1
2
k
H1: Not all the (i=1,2,...,k) are 0
i
Source of
Variation
Sum of
Squares
Regression SSR
Error
Total
SSE
SST
Degrees of
Freedom Mean Square
k
n - (k+1)
n-1
SSR
MSR
MSE
F Ratio
k
SSE
( n ( k 1))
MST
SST
( n 1)
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
13. Using the Template: Analysis of
Variance Table (Example 11-1)
F Distribution with 2 and 7 Degrees of Freedom
f(F)
Test statistic
86.34
=0.01
F
0
Slide 13
The test statistic, F = 86.34, is greater
than the critical point of F(2, 7) for any
common level of significance
(p-value 0), so the null hypothesis is
rejected, and we might conclude that
the dependent variable is related to
one or more of the independent
variables.
F0.01=9.55
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
14. 11-4 How Good is the
Regression
y
The mean square error is an unbiased
estimator of the variance of the population
2
errors, , denoted by :
MSE
x1
x2
Slide 14
SSE
( n ( k 1))
(y y 2
)
( n ( k 1))
Standard error of estimate:
Errors: y - y
s=
MSE
2
The multiple coefficient of determination, R , measures the proportion of
the variation in the dependent variable that is explained by the combination
of the independent variables in the multiple regression model:
R2 =
SSR
SSE
=1SST
SST
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
15. Decomposition of the Sum of
Squares and the Adjusted
Coefficient of Determination
Slide 15
SST
SSR
R
SSE
2
=
SSR
SST
= 1-
SSE
SST
2
The adjusted multiple coefficient of determination , R , is the coefficient of
determination with the SSE and SST divided by their respective degrees of freedom:
SSE
R
2
= 1-
(n - (k + 1))
SST
(n - 1)
Example 11-1:
s = 1.911
R-sq = 96.1%
R-sq(adj) = 95.0%
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
16. Measures of Performance in Multiple
Regression and the ANOVA Table
Source of
Variation
Sum of
Squares
Degrees of
Freedom Mean Square
Regression SSR
(k)
MSR
Error
R
(n-(k+1))
=(n-k-1)
Total
2
SSE
SST
(n-1)
SSR
=
SSE
= 1-
SST
SST
F
R
MSE
Slide 16
F Ratio
F
SSR
k
MSR
MSE
SSE
(n ( k 1))
MST
SST
( n 1)
SSE
2
( n ( k 1))
2
(1 R )
(k )
R
2
=1-
(n - (k + 1))
SST
=
MSE
MST
(n - 1)
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
17. 11-5 Tests of the Significance of
Individual Regression Parameters
Slide 17
Hypothesis tests about individual regression slope parameters:
(1)
H0: b1= 0
H1: b1 0
(2)
H0: b2 = 0
H1: b2 0
.
.
.
(k)
H0: bk = 0
H1: bk 0
Test statistic for test i: t
b 0
s(b )
i
( n ( k 1 )
i
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
19. Example 11-1: Using the
Template
Slide 19
Regression results for Alka-Seltzer sales
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
20. Using the Template: Example
11-2
Slide 20
Regression results for Exports to Singapore
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
21. 11-6 Testing the Validity of the
Regression Model: Residual Plots
Slide 21
Residuals vs M1
It appears that the residuals are randomly distributed with no pattern and
with equal variance as M1 increases
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
22. 11-6 Testing the Validity of the
Regression Model: Residual Plots
Slide 22
Residuals vs Price
It appears that the residuals are increasing as the Price increases. The
variance of the residuals is not constant.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
23. Normal Probability Plot for the
Residuals: Example 11-2
Slide 23
Linear trend indicates residuals are normally distributed
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
24. Investigating the Validity of the
Regression: Outliers and Influential
Observations
y
Regression line
without outlier
. .
. ..
..
. .. ..
.. .
.
Regression
line with
outlier
* Outlier
x
Outliers
Slide 24
Point with a large
value of xi
y
..
...... .. .
. .. .
*
Regression line
when all data are
included
No relationship in
this cluster
x
Influential Observations
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
25. Outliers and Influential
Observations: Example 11-2
Unusual Observations
Obs.
M1
EXPORTS
St.Resid
1
5.10
2.6000
2
4.90
2.6000
25
6.20
5.5000
26
6.30
3.7000
50
8.30
4.3000
67
8.20
5.6000
Fit
Stdev.Fit
2.6420
2.6438
4.5949
4.6311
5.1317
4.9474
0.1288
0.1234
0.0676
0.0651
0.0648
0.0668
Slide 25
Residual
-0.0420
-0.0438
0.9051
-0.9311
-0.8317
0.6526
-0.14 X
-0.14 X
2.80R
-2.87R
-2.57R
2.02R
R denotes an obs. with a large st. resid.
X denotes an obs. whose X value gives it large influence.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
26. 11-7 Using the Multiple
Regression Model for Prediction
Sales
Slide 26
Estimated Regression Plane for Example 11-1
89.76
Advertising
18.00
63.42
8.00
Promotions
12
3
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
27. Prediction in Multiple
Regression
Slide 27
A (1 - a) 100% prediction interval for a value of Y given values of X :
i
yt
( ,( n ( k 1)))
2
s 2 ( y MSE
)
A (1 - a) 100% prediction interval for the conditional mean of Y given
values of X :
i
yt
( ,( n ( k 1)))
2
s[ E (Y )]
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
29. Picturing Qualitative Variables in
Regression
Slide 29
y
Y
Line for X2=1
b3
b0+b2
Line for X2=0
x1
b0
X1
A regression with one
quantitative variable (X1) and
one qualitative variable (X2):
y b bx b x
0
1
1
2
2
x2
A multiple regression with two
quantitative variables (X1 and X2)
and one qualitative variable (X3):
y b bx b x b x
0
1
1
2
2
3
3
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
30. Picturing Qualitative Variables in
Regression: Three Categories and
Two Dummy Variables
Y
Line for X = 0 and X3 = 1
Line for X2 = 1 and X3 = 0
b0+b3
Slide 30
A qualitative
variable with r
levels or categories
is represented with
(r-1) 0/1 (dummy)
variables.
Line for X2 = 0 and X3 = 0
b0+b2
b0
X1
A regression with one quantitative variable (X1) and two
qualitative variables (X2 and X2):
y b bx b x b x
0
1
1
2
2
3
Category X2 X3
Adventure 0 0
Drama
0 1
Romance 1 0
3
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
31. Using Qualitative Variables in
Regression: Example 11-4
Slide 31
Salary = 8547 + 949 Education + 1258 Experience - 3256 Gender
(SE) (32.6)
(45.1)
(78.5)
(212.4)
(t) (262.2)
(21.0)
(16.0)
(-15.3)
1 if Female
Gender
0 if Male
On average, female salaries are
$3256 below male salaries
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
32. Interactions between Quantitative and 32
Slide
Qualitative Variables: Shifting Slopes
LnorX20
Y
LnorX21
Slop 1
0
Slop 13
02
X1
A regression with interaction between a quantitative
variable (X1) and a qualitative variable (X2 ):
y b bx b x b x x
0
1
1
2
2
3
1
2
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
33. 11-9 Polynomial Regression
Slide 33
One-variable polynomial regression model:
Y=0+1 X + 2X2 + 3X3 +. . . + mXm +
where m is the degree of the polynomial - the highest power of X appearing in
the equation. The degree of the polynomial is the order of the model.
Y
Y
y b b X
0
y b b X
0
1
1
y b b X b X
0
1
2
y b b X b X b X
2
2
(b 0)
0
2
X1
1
2
3
X1
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
3
35. Polynomial Regression: Other
Variables and Cross-Product Terms
Slide 35
Variable Estimate Standard Error T-statistic
X1
2.34
0.92
2.54
X2
3.11
1.05
2.96
2
X1
4.22
1.00
4.22
X2 2
3.57
2.12
1.68
2
X1 X
2.77
2.30
1.20
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
36. 11-10 Nonlinear Models and
Transformations: Multiplicative
Model
Slide 36
The multiplicative model:
Y X X X
1
0
1
2
2
3
3
The logarithmic transformation:
log Y log log X log X log X log
0
1
1
2
2
3
3
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
37. Transformations: Exponential
Model
Slide 37
The exponential model:
Y e
1X
0
The logarithmic transformation:
log Y log
0
X log
1
1
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
38. Plots of Transformed Variables
S im ple R e g re s s io n of S ale s o n Ad ve rtis ing
R e gre s sio n of S ale s on Log(Advertising)
25
20
Y = 6 .5 9 2 7 1 + 1.19 176 X
R- S q u a r e d = 0 .8 9 5
10
SALES
30
SALES
Slide 38
15
Y = 3 .6 6 8 2 5 + 6 .78 4 X
R- Sq uared = 0 .9 78
5
0
5
10
15
0
1
ADVERT
2
3
LOGADV
R e gre s sio n of Lo g(S ale s ) o n Lo g(Adve rtising)
R e sid ual Plots: S ale s v s Lo g(Adv e rtising)
1.5
3.5
2.5
Y = 1.70 0 8 2 + 0 .5 53 13 6 X
R- S q uar ed = 0 .9 47
RESIDS
LOGSALE
0.5
-0.5
-1.5
1.5
0
1
2
LOGADV
3
2
12
22
Y-HAT
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
39. Variance Stabilizing
Transformations
•
Slide 39
Square root transformation:Y Y
Useful when the variance of the regression errors is
approximately proportional to the conditional mean of Y
•
Logarithmic transformation: Y log(Y )
Useful when the variance of regression errors is approximately
proportional to the square of the conditional mean of Y
• Reciprocal transformation:
1
Y
Useful when the variance of the regression errors is
Y
approximately proportional to the fourth power of the
conditional mean of Y
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
40. Regression with Dependent
Indicator Variables
Slide 40
The logistic function:
e ( X )
E (Y X )
1 e ( X )
0
1
0
1
Transformation to linearize the logistic function:
p
p log
1 p
y
Logistic Function
1
0
x
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
41. 11-11: Multicollinearity
Slide 41
x2
x2
x1
Orthogonal X variables provide
information from independent
sources. No multicollinearity.
x2
x1
Perfectly collinear X variables
provide identical information
content. No regression.
x2
x1
Some degree of collinearity.
Problems with regression depend
on the degree of collinearity.
x1
A high degree of negative
collinearity also causes problems
with regression.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
42. Effects of Multicollinearity
•
•
•
•
•
•
Slide 42
Variances of regression coefficients are inflated.
Magnitudes of regression coefficients may be different
from what are expected.
Signs of regression coefficients may not be as expected.
Adding or removing variables produces large changes in
coefficients.
Removing a data point may cause large changes in
coefficient estimates or signs.
In some cases, the F ratio may be significant while the t
ratios are not.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
43. Detecting the Existence of Multicollinearity: Correlation
Matrix of Independent Variables and Variance Inflation
Factors
Slide 43
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
44. Variance Inflation Factor
Slide 44
The variance inflation factor associated with X h :
1
VIF ( X h )
1 Rh2
where R 2 is the R 2 value obtained for the regression of X on
h
the other independent variables.
Relationship between VIF and Rh2
VIF100
50
0
0.0
0.5
1.0
Rh2
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
45. Variance Inflation Factor (VIF)
Slide 45
Observation: The VIF (Variance Inflation Factor) values
for both variables Lend and Price are both greater than
5. This would indicate that some degree of
multicollinearity exists with respect to these two
variables.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
46. Solutions to the
Multicollinearity Problem
Slide 46
• Drop a collinear variable from the
regression
• Change in sampling plan to include
elements outside the multicollinearity range
• Transformations of variables
• Ridge regression
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
47. 11-12 Residual Autocorrelation
and the Durbin-Watson Test
Slide 47
An autocorrelation is a correlation of the values of a variable
with values of the same variable lagged one or more periods
back. Consequences of autocorrelation include inaccurate
estimates of variances and inaccurate predictions.
Lagged Residuals
i
1
2
3
4
5
6
7
8
9
10
i
1.0
0.0
-1.0
2.0
3.0
-2.0
1.0
1.5
1.0
-2.5
i-1
*
1.0
0.0
-1.0
2.0
3.0
-2.0
1.0
1.5
1.0
i-2
*
*
1.0
0.0
-1.0
2.0
3.0
-2.0
1.0
1.5
i-3
*
*
*
1.0
0.0
-1.0
2.0
3.0
-2.0
1.0
i-4
*
*
*
*
1.0
0.0
-1.0
2.0
3.0
-2.0
The Durbin-Watson test (first-order
autocorrelation):
H 0 : r1 = 0
H1:r1 0
The Durbin-Watson test statistic:
n
2
( ei ei 1 )
d i2 n
2
ei
i 1
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
49. Using the Durbin-Watson
Statistic
Positive
Autocorrelation
0
Test is
Inconclusive
dL
dU
No
Autocorrelation
Test is
Inconclusive
4-dU
Slide 49
Negative
Autocorrelation
4-dL
4
For n = 67, k = 4: dU1.73 4-dU2.27
dL1.47 4- dL2.53 < 2.58
H0 is rejected, and we conclude there is negative first-order
autocorrelation.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
50. 11-13 Partial F Tests and
Variable
Selection Methods
Slide 50
Full model:
Y = 0 + 1 X 1 + 2 X 2 + 3 X 3 + 4 X 4 +
Reduced model:
Y = 0 + 1 X 1 + 2 X 2 +
Partial F test:
H0: 3 = 4 = 0
H1: 3 and 4 not both 0
Partial F statistic:
SSE ) / r
R
F
F
MSE
(r, (n (k 1))
F
where SSER is the sum of squared errors of the reduced model, SSEF is the sum of squared
errors of the full model; MSEF is the mean square error of the full model [MSEF =
SSEF/(n-(k+1))]; r is the number of variables dropped from the full model.
(SSE
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
51. Variable Selection Methods
•
Slide 51
All possible regressions
Run regressions with all possible
combinations of independent variables and
select best model
A p-value of 0.001 indicates
that we should reject the null
hypothesis H0: the slopes for
Lend and Exch. are zero.
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
52. Variable Selection Methods
•
Slide 52
Stepwise procedures
Forward selection
» Add one variable at a time to the model, on the basis of its
F statistic
Backward elimination
» Remove one variable at a time, on the basis of its F
statistic
Stepwise regression
» Adds variables to the model and subtracts variables from
the model, on the basis of the F statistic
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
53. Stepwise Regression
Slide 53
ComputF tttorvrlnotntmol
Itrtltonvrlwt pvlu Pn
No
Stop
Y
Entrmotnnt(mlltpvlu)vrlntomol
ClultprtlForllvrlntmol
ItrvrlwtpvluPout
Rmov
vrl
No
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
54. Stepwise Regression: Using the
Computer (MINITAB)
Slide 54
MTB > STEPWISE 'EXPORTS' PREDICTORS 'M1’ 'LEND' 'PRICE’
'EXCHANGE'
Stepwise Regression
F-to-Enter:
4.00
F-to-Remove:
4.00
Response is EXPORTS on 4 predictors, with N = 67
Step
Constant
M1
T-Ratio
1
0.9348
2
-3.4230
0.520
9.89
0.361
9.21
PRICE
T-Ratio
S
R-Sq
0.0370
9.05
0.495
60.08
0.331
82.48
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
55. Using the Computer: MINITAB
Slide 55
MTB > REGRESS 'EXPORTS’ 4 'M1’ 'LEND’ 'PRICE' 'EXCHANGE';
SUBC> vif;
SUBC> dw.
Regression Analysis
The regression equation is
EXPORTS = - 4.02 + 0.368 M1 + 0.0047 LEND + 0.0365 PRICE + 0.27 EXCHANGE
Predictor
Coef
Stdev
t-ratio
p
VIF
Constant
-4.015
2.766
-1.45
0.152
M1
0.36846
0.06385
5.77
0.000
3.2
LEND
0.00470
0.04922
0.10
0.924
5.4
PRICE
0.036511
0.009326
3.91
0.000
6.3
EXCHANGE
0.268
1.175
0.23
0.820
1.4
s = 0.3358
R-sq = 82.5%
R-sq(adj) = 81.4%
Analysis of Variance
SOURCE
Regression
DF
4
SS
32.9463
62
Error
Total
MS
F
8.2366
73.06
6.9898
0.1127
66
39.9361
p
0.000
Durbin-Watson statistic = 2.58
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
56. Using the Computer: SAS
(continued)
Slide 56
Parameter Estimates
Variable
INTERCEP
M1
LEND
PRICE
EXCHANGE
DF
1
1
1
1
1
Parameter
Error
Estimate
-4.015461
0.368456
0.004702
0.036511
0.267896
2.76640057
0.06384841
0.04922186
0.00932601
1.17544016
Variable
INTERCEP
M1
LEND
PRICE
EXCHANGE
Durbin-Watson D
(For Number of Obs.)
1st Order Autocorrelation
-1.452
5.771
0.096
3.915
0.228
Prob > |T|
0.1517
0.0001
0.9242
0.0002
0.8205
Variance
Inflation
DF
1
1
1
1
1
Standard
T for H0:
Parameter=0
0.00000000
3.20719533
5.35391367
6.28873181
1.38570639
2.583
67
-0.321
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
57. 11-15: The Matrix Approach to
Regression Analysis (1)
Slide 57
The population regression model:
y
y
y
.
.
.
y
1
2
3
k
1
1
1
.
.
.
1
x
x
x
.
.
.
x
11
21
31
n1
x
x
x
.
.
.
x
12
22
32
n2
x ... x
x ... x
x ... x
.
.
. . .
.
.
.
. .
.
.
. .
.
x ... x
13
1k
1
1
23
2k
2
2
33
3k
3
3
k
k
n3
nk
Y X
The estimated regression model:
Y = Xb + e
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
58. The Matrix Approach to
Regression Analysis (2)
Slide 58
The normal equations:
X Xb X Y
Estimators:
b ( X X )
1
X Y
Predicted values:
Y Xb X ( X )
X
1
V (b) ( X X )
s (b) MSE ( X )
X
2
2
X Y HY
1
1
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer
59. Slide 59
Name
Religion
Domicile
Contact #
E.Mail
M.Phil (Statistics)
Shakeel Nouman
Christian
Punjab (Lahore)
0332-4462527. 0321-9898767
sn_gcu@yahoo.com
sn_gcu@hotmail.com
GC University, .
(Degree awarded by GC University)
M.Sc (Statistics)
Statitical Officer
(BS-17)
(Economics & Marketing
Division)
GC University, .
(Degree awarded by GC University)
Livestock Production Research Institute
Bahadurnagar (Okara), Livestock & Dairy Development
Department, Govt. of Punjab
Multiple Regression (1) By Shakeel Nouman M.Phil Statistics Govt. College University Lahore, Statistical Officer