This document provides an overview of a presentation on Ordinary Least Squares (OLS) estimation in econometrics. OLS is introduced as a method used to estimate parameters of economic relationships from data by minimizing errors. Key points covered include: what OLS estimates, why it is used in econometrics to estimate regression parameters from a sample regression function in order to approximate the true population regression function, and details on how the OLS criterion minimizes the sum of squared residuals to obtain parameter estimates. Goodness of fit is also discussed as a measure of how well the estimated regression line fits the sample data.

1. Welcome to the Presentation
6/21/2013
1

2. Course Title: Econometrics II
Course No.: Econ 4203
6/21/2013
2

3. Ordinary Least Squares
(OLS) Method
6/21/2013 3

4. Points to be discussed
What is OLS (In a word)
Why OLS is In Econometrics
Details on OLS Concept
6/21/2013 4

5. What is OLS (In a word)
OLS is an econometric method used to derive estimates (particular
numerical value obtained by the estimator) of the parameters of
economic relationships from statistical observations. It is the
technique used to estimate a line that will minimize the error (The
difference between the predicted and the actual values of a
dependent variable). The method of Ordinary Least Squares is
attributed to Carl Friedrich Gauss, a German mathematicians.
6/21/2013 5

6. Why OLS is in Econometrics
In econometric analysis we usually try to find out the causal
relationship between two sets of variables. Among them one is
dependent and another is independent. By ‘Regression’ analysis
we can draw the nature and intensity of relationship between two
sets of variable. In regression, the dependent variable is termed as
‘explained’ or ‘regressand’ variable and independent variable(s)
is/are termed as ‘explanatory’ or ‘regressor’ variable(s).
6/21/2013 6

7. Why OLS is in Econometrics
A two variable regression function :
Where, = Dependent variable; = Independent variable; = Intercept;
= Slope; and = Stochastic disturbance term.
From the above function, it is clear that the relationship between X and Y is
mainly determined by , and the disturbance term . So it is
important to estimate the s.
There are some other methods like- Moment method, Maximum Likelihood
method. But having some desirable properties (property of linearity,
unbiasedness and minimum variance) we can apply OLS simply to
estimate the regression parameters , s.
But,Why we need to estimate s by OLS?
6/21/2013 7

8. Why we need to estimate s by OLS?
A two variable regression function explicits the relationship between the
explained and explanatory variables. In this case if we consider the whole
‘population’ then we get Population Regression Function (PRF). In PRF
there is no estimation error, because all the elements are considered.
But, in practice, we usually consider a ‘sample’ from the population to draw
a relationship between the variables. We get the form of relationship by
Sample Regression Function (SRF). By considering a portion of whole
elements, estimation errors (due to sampling fluctuation) are raised here.
6/21/2013 8

9. Why we need to estimate s OLS?
From SRF we need to approximate the PRF. But, there are some
estimation errors. OLS help to minimize (make least) the
estimation errors and help to find a close approximation of PRF
from SRF.
Due to the sampling fluctuation, the true parameters ( s of PRF)
varies significantly than the sampling parameters ( s of SRF). In
this case OLS is a simpler devise or method by which we can
estimate the least valued s to make the SRF as a best possible
mirror of PRF .
6/21/2013 9

10. Details on OLS Concept
To understand the method of Ordinary Least Squares, we first explain
the least squares principle.
We know , the two variable PRF:
However, the PRF is not directly observable (as noted earlier). We
estimate it from the SRF:
(1)
(2)
(3)
6/21/2013 10

11. Where is the estimated (of SRF) (conditional mean) value of .
But how is the SRF itself determined?
To see this, let us proceed as follows.
First, from equation (3), we get:
(4)
6/21/2013 11

12. Now given n pairs of observations on Y and X, we would like to
determine the SRF in such a manner that it is as close as possible
to the actual Y. To this end, we may adopt the following least
square criterion: Choose the SRF in such a way that the sum of
the squared residuals is as small as possible.
This criterion states that the SRF can be fixed in such a way that
is as small as possible, where are the squared residuals.
(5)
6/21/2013 12

13. It is obvious from equation (5) that
that is the sum of the squared residuals is some function of the
estimators and . For any given set of data, choosing different
values for and will give different .
Now which sets of values should we choose?
6/21/2013 13

14. Since s are the function of the squared residuals, we should choose those
s whice give us a lower (least) value of squared residuals.
And the method of Ordinary Least Squares (OLS) provide us a very shortcut
way to get the potential s to estimate the SRF as the best possible
approximation of PRF.
As an very essential additional concept, we will discuss shortly about the
‘goodness of fit’. It means that how ‘good’ the estimated least square
regression (SRF) line ( ) fits to the sample observations. That
is, what portion of change in dependent variable (X) can be explained by
the change of independent variable (Y). And, we also get known the
remained unexplained portion from it.
6/21/2013 14

15. The ‘goodness of fit’ is measured by coefficient of determination,
Higher the value of , more good to fit and vice versa.
From the following figure,
Total variation inY (TSS) = Explained variation (ESS) + Unexplained variation (RSS)
r2
Total variation
ExplainedVariation
UnexplainedVariation
Yi
0
Y
X
Estimated SRF
r2
0 r2 1
6/21/2013 15

16. If we expressed the variations by equations then we get,
Total variation inY=
Explained variation=
Unexplained variation=
And, by the function: we can compute the value of
‘goodness of fit’ of an estimated SRF; that is how good the estimated
SRF is close to the PRF.
6/21/2013 16

17. Please, recall that, is the summation of squared residuals. So
from the equation it is easily understandable that, if
the value of squared residuals is low then the value of will be
high, and the estimated SRF will be more close to PRF. And, the
OLS gives us the least value of . So, it is certainly realized that,
by following least squares method, we can find the best
approximated SRF of PRF. And, the concept of OLS has much
wider application in Econometrics.
r2
6/21/2013 17

18. References:
Gujarati,N.D.(2007) Basic Econometrics, 4th ed.,Tata McGraw-Hill
Publishing Company Limited: New Delhi, pp. 1-118.
Hansen, E.B.(2010) Econometrics, University Press, University of
Wisconsin, pp. 1-64.
Koutsoyiannis,A.(2003) Theory of Econometrics, 2nd ed., PALGRAVE:
NewYork, pp. 48-62.
Nagler, J.(January, 2001). Notes on Ordinary Least Squares Estimates.
Retrived December 14, 2010, from http:/ www.
Economics.about.com/OLS/pdf.
6/21/2013 18

19. Queries?
6/21/2013 19

20. 6/21/2013 20