SlideShare a Scribd company logo
1 of 13
VIOLATION OF CLRM ASSUMPTIONS:
MULTICOLLINEARITY
Chapter 8
1
MULTICOLLINEARITY: NATURE
 No exact linear relationship or no collinearity means that a variable,
say 𝑋2, can not be expressed as an exact linear function of another
variable, say 𝑋3.
 Thus if we can express that
𝑋2 = 4𝑋3
then the two variables and are collinear, for there is an exact linear
relationship between them.
 Consider the following 3-variable mathematical equation:
𝑌 = 𝛽1 + 𝛽2𝑋2 + 𝛽3𝑋3
or, 𝑌 = 𝛽1 + 𝛽2(4𝑋3) + 𝛽3𝑋3 = 𝛽1 + 4𝛽2𝑋3 + 𝛽3𝑋3
= 𝛽1 +𝑋3(4𝛽2 + 𝛽3)
 Now, assume 𝐴 = 4𝛽2 + 𝛽3, we get 𝑌 = 𝛽1 + A𝑋3
 Note that we have one equation with two unknowns, we need at
least two equations to get the values of two unknowns. 2
MULTICOLLINEARITY: NATURE
 In presence of multicollinearity, we can not disentangle the
individual effect of 𝑋2 and 𝑋3 on Y.
 Exact linear relationship: For a k-variable regression model
involving explanatory variables 𝑋1, 𝑋2, 𝑋3 … 𝑋𝑘 ( 𝑋1 is the
intercept) an exact linear relationship is said to exist if the
following condition is satisfied
𝛾1𝑋1 + 𝛾2𝑋2 + ⋯ + 𝛾𝑘𝑋𝑘 = 0
 Where 𝛾1, 𝛾2, … 𝛾𝑘 are constants and not all of them are zero
simultaneously.
 Near-exact linear relationship:
𝛾1𝑋1 + 𝛾2𝑋2 + ⋯ + 𝛾𝑘𝑋𝑘 + 𝑢𝑖 = 0
Where 𝛾1, 𝛾2, … 𝛾𝑘 are constants and not all of them are zero
simultaneously and 𝑢𝑖 is a random error term. 3
MULTICOLLINEARITY: NATURE
 Point to remember
 Multicollinearity as we have defined above refers only to
linear relationship among X variables of a regression
model. But Xs may be nonlinearly correlated as
 Variables and are functionally related with but
the relationship in nonlinear thus the model does not
violate the assumption of multicollinearity.
4
i
i
i
i
i u
X
X
X
Y 



 3
4
2
3
2
1 



2
i
X 3
i
X i
X
MULTICOLLINEARITY: EXAMPLE
5
𝑋2𝑖 𝑋3𝑖 𝑋4𝑖
3 9 13 (+4)
5 15 16 (+1)
6 18 21 (+3)
8 24 25 (+1)
10 30 30 (+0)
THEORETICAL CONSEQUENCES OF MULTICOLLINEARITY
 CASE 1: PERFECT LINEAR RELATIONSHIP BETWEEN VARIABLES
 If multicollinearity is perfect (exact linear relationship), the
partial regression coefficients of the X variables are
indeterminate and their standard errors are infinite.
 If 𝑋2 and 𝑋3 are perfectly collinear, there is no way that
𝑋2 can be kept constant when 𝑋3 changes. As long as we
fail to separate individual effect of 𝑋2 and 𝑋3 on Y, we can
not get a unique solution for individual regression
coefficients.
6
THEORETICAL CONSEQUENCES OF MULTICOLLINEARITY
 Consider the following 3-variable regression model:

7
THEORETICAL CONSEQUENCES OF MULTICOLLINEARITY
 CASE 2: NEAR PERFECT LINEAR RELATIONSHIP BETWEEN VARIABLES
 If multicollinearity is less than perfect, the regression
coefficients, although determinate, posses large standard
errors (in relation to the coefficients themselves), which means
that the coefficients can not be estimated with great precision.
 The effect of multicollinearity is to make it hard to obtain
estimates of coefficients with small standard error.
 However, having a small number of observations also has
similar effect (small sample size). For this reason,
multicollinearity is considered as a small sample phenomenon.
Some Economists also use separate term for multicollinearity,
like micronumerosity instead of multicollinearity.
8
PRACTICAL CONSEQUENCES OF MULTICOLLINEARITY
In case near perfect or high multicollinearity, one is likely to face following
problems:
1. Although BLUE, the OLS estimates have large variances and covariances
making precise estimation difficult.
2. Because of 1, the confidence intervals for parameter estimates become
large leading to accepting the zero 𝐻0 more readily.
3. Also because of 1, the t- statistics of one or more coefficients tends to be
statistically insignificant.
4. Although the t - statistics of one or more coefficients is statistically
insignificant, 𝑅2
, the overall measure of goodness-of-fit, can be very high.
5. The OLS estimators and their standard errors can be sensitive to small
changes in the data. 9
DETECTION OF MULTICOLLINEARITY
 High R2 but few significant t ratios.
 “Classic” symptom of multicollinearity. If R2 is high, say, in excess of 0.8, the F
test in most cases will reject the null hypothesis that the partial slope
coefficients are jointly or simultaneously equal to zero. But individual t tests
will show that none or very few partial slope coefficients are statistically
different from zero.
 Subsidiary, or auxiliary, regressions.
 One way of finding out which X variable is highly collinear with other X
variables in the model is to regress each X variable on the remaining X
variables and to compute the corresponding R2.
𝑌 = 𝛽1 + 𝛽2𝑋2 + ⋯ 𝛽7𝑋7 + 𝑢
 Consider the regression of Y on X2, X3, X4, X5, X6, and X7—six explanatory
variables.
 If this regression shows that the R2 is high but very few X coefficients are
individually statistically significant, we then look for the “culprit,” the variable(s)
that may be a perfect or near perfect linear combination of the other X’s. 10
DETECTION OF MULTICOLLINEARITY
11
𝐹 =
R2
/(k − 1)
(1 − R2)/(n − k)
𝑋2 on remaining Xs
DETECTION OF MULTICOLLINEARITY
 The variance inflation factor
𝑉𝐼𝐹 =
1
(1 − 𝑅2)
 The larger the value of VIFj, the more “troublesome” or
collinear the variable Xj.
 it may be noted that the inverse of the VIF is called tolerance
(TOL). That is,
 One could use TOLj as a measure of multicollinearity in view
of its intimate connection with VIFj. The closer is TOLj to
zero, the greater the degree of collinearity of that variable
with the other regressors. 12
REMEDIAL MEASURES
 What can be done if multicollinearity is serious? We have
two choices:
(1) do nothing or (2) follow some rules of thumb.
 Dropping a Variable(s) from the Model
 Acquiring Additional Data or a New Sample
 Rethinking the Model
 Prior Information about Some Parameters
 Transformation of Variables
 Combining time series and cross-sectional data,
factor or principal component analysis and ridge
regression. 13

More Related Content

Similar to LEC11 (1).pptx

Chapter18 econometrics-sure models
Chapter18 econometrics-sure modelsChapter18 econometrics-sure models
Chapter18 econometrics-sure modelsMilton Keynes Keynes
 
Correlation and regression
Correlation and regressionCorrelation and regression
Correlation and regressionAntony Raj
 
Correlation and regression impt
Correlation and regression imptCorrelation and regression impt
Correlation and regression imptfreelancer
 
REGRESSION ANALYSIS THEORY EXPLAINED HERE
REGRESSION ANALYSIS THEORY EXPLAINED HEREREGRESSION ANALYSIS THEORY EXPLAINED HERE
REGRESSION ANALYSIS THEORY EXPLAINED HEREShriramKargaonkar
 
Correlation and regression
Correlation and regressionCorrelation and regression
Correlation and regressionAntony Raj
 
Data Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Data Science - Part XII - Ridge Regression, LASSO, and Elastic NetsData Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Data Science - Part XII - Ridge Regression, LASSO, and Elastic NetsDerek Kane
 
Perfect and imperfect multicollinearitya) Define perfect multicol.pdf
Perfect and imperfect multicollinearitya) Define perfect multicol.pdfPerfect and imperfect multicollinearitya) Define perfect multicol.pdf
Perfect and imperfect multicollinearitya) Define perfect multicol.pdfakagencies2014
 
The management of a regional bus line thought the companys cost of .pdf
 The management of a regional bus line thought the companys cost of .pdf The management of a regional bus line thought the companys cost of .pdf
The management of a regional bus line thought the companys cost of .pdfnagaraj138348
 
Machine Learning Algorithm - Linear Regression
Machine Learning Algorithm - Linear RegressionMachine Learning Algorithm - Linear Regression
Machine Learning Algorithm - Linear RegressionKush Kulshrestha
 

Similar to LEC11 (1).pptx (20)

Chapter18 econometrics-sure models
Chapter18 econometrics-sure modelsChapter18 econometrics-sure models
Chapter18 econometrics-sure models
 
Econometrics ch11
Econometrics ch11Econometrics ch11
Econometrics ch11
 
Econometrics_ch11.ppt
Econometrics_ch11.pptEconometrics_ch11.ppt
Econometrics_ch11.ppt
 
2-20-04.ppt
2-20-04.ppt2-20-04.ppt
2-20-04.ppt
 
Correlation and regression
Correlation and regressionCorrelation and regression
Correlation and regression
 
Correlation and regression impt
Correlation and regression imptCorrelation and regression impt
Correlation and regression impt
 
REGRESSION ANALYSIS THEORY EXPLAINED HERE
REGRESSION ANALYSIS THEORY EXPLAINED HEREREGRESSION ANALYSIS THEORY EXPLAINED HERE
REGRESSION ANALYSIS THEORY EXPLAINED HERE
 
Modelo Generalizado
Modelo GeneralizadoModelo Generalizado
Modelo Generalizado
 
Multiple regression
Multiple regressionMultiple regression
Multiple regression
 
Correlation and regression
Correlation and regressionCorrelation and regression
Correlation and regression
 
Data Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Data Science - Part XII - Ridge Regression, LASSO, and Elastic NetsData Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
Data Science - Part XII - Ridge Regression, LASSO, and Elastic Nets
 
Perfect and imperfect multicollinearitya) Define perfect multicol.pdf
Perfect and imperfect multicollinearitya) Define perfect multicol.pdfPerfect and imperfect multicollinearitya) Define perfect multicol.pdf
Perfect and imperfect multicollinearitya) Define perfect multicol.pdf
 
Simple linear regressionn and Correlation
Simple linear regressionn and CorrelationSimple linear regressionn and Correlation
Simple linear regressionn and Correlation
 
CH3.pdf
CH3.pdfCH3.pdf
CH3.pdf
 
The management of a regional bus line thought the companys cost of .pdf
 The management of a regional bus line thought the companys cost of .pdf The management of a regional bus line thought the companys cost of .pdf
The management of a regional bus line thought the companys cost of .pdf
 
Machine Learning Algorithm - Linear Regression
Machine Learning Algorithm - Linear RegressionMachine Learning Algorithm - Linear Regression
Machine Learning Algorithm - Linear Regression
 
Regression
RegressionRegression
Regression
 
Correlation and Regression
Correlation and RegressionCorrelation and Regression
Correlation and Regression
 
Chapter 10
Chapter 10Chapter 10
Chapter 10
 
Chapter 10
Chapter 10Chapter 10
Chapter 10
 

Recently uploaded

/:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc...
/:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc.../:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc...
/:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc...lizamodels9
 
rishikeshgirls.in- Rishikesh call girl.pdf
rishikeshgirls.in- Rishikesh call girl.pdfrishikeshgirls.in- Rishikesh call girl.pdf
rishikeshgirls.in- Rishikesh call girl.pdfmuskan1121w
 
A DAY IN THE LIFE OF A SALESMAN / WOMAN
A DAY IN THE LIFE OF A  SALESMAN / WOMANA DAY IN THE LIFE OF A  SALESMAN / WOMAN
A DAY IN THE LIFE OF A SALESMAN / WOMANIlamathiKannappan
 
Catalogue ONG NUOC PPR DE NHAT .pdf
Catalogue ONG NUOC PPR DE NHAT      .pdfCatalogue ONG NUOC PPR DE NHAT      .pdf
Catalogue ONG NUOC PPR DE NHAT .pdfOrient Homes
 
Tech Startup Growth Hacking 101 - Basics on Growth Marketing
Tech Startup Growth Hacking 101  - Basics on Growth MarketingTech Startup Growth Hacking 101  - Basics on Growth Marketing
Tech Startup Growth Hacking 101 - Basics on Growth MarketingShawn Pang
 
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Dipal Arora
 
Vip Female Escorts Noida 9711199171 Greater Noida Escorts Service
Vip Female Escorts Noida 9711199171 Greater Noida Escorts ServiceVip Female Escorts Noida 9711199171 Greater Noida Escorts Service
Vip Female Escorts Noida 9711199171 Greater Noida Escorts Serviceankitnayak356677
 
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130  Available With RoomVIP Kolkata Call Girl Howrah 👉 8250192130  Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Roomdivyansh0kumar0
 
2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis Usage2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis UsageNeil Kimberley
 
Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...
Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...
Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...lizamodels9
 
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service JamshedpurVIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service JamshedpurSuhani Kapoor
 
Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999
Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999
Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999Tina Ji
 
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...lizamodels9
 
Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.Eni
 
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,noida100girls
 
Keppel Ltd. 1Q 2024 Business Update Presentation Slides
Keppel Ltd. 1Q 2024 Business Update  Presentation SlidesKeppel Ltd. 1Q 2024 Business Update  Presentation Slides
Keppel Ltd. 1Q 2024 Business Update Presentation SlidesKeppelCorporation
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Neil Kimberley
 
M.C Lodges -- Guest House in Jhang.
M.C Lodges --  Guest House in Jhang.M.C Lodges --  Guest House in Jhang.
M.C Lodges -- Guest House in Jhang.Aaiza Hassan
 
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...lizamodels9
 

Recently uploaded (20)

/:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc...
/:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc.../:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc...
/:Call Girls In Jaypee Siddharth - 5 Star Hotel New Delhi ➥9990211544 Top Esc...
 
rishikeshgirls.in- Rishikesh call girl.pdf
rishikeshgirls.in- Rishikesh call girl.pdfrishikeshgirls.in- Rishikesh call girl.pdf
rishikeshgirls.in- Rishikesh call girl.pdf
 
A DAY IN THE LIFE OF A SALESMAN / WOMAN
A DAY IN THE LIFE OF A  SALESMAN / WOMANA DAY IN THE LIFE OF A  SALESMAN / WOMAN
A DAY IN THE LIFE OF A SALESMAN / WOMAN
 
Catalogue ONG NUOC PPR DE NHAT .pdf
Catalogue ONG NUOC PPR DE NHAT      .pdfCatalogue ONG NUOC PPR DE NHAT      .pdf
Catalogue ONG NUOC PPR DE NHAT .pdf
 
Tech Startup Growth Hacking 101 - Basics on Growth Marketing
Tech Startup Growth Hacking 101  - Basics on Growth MarketingTech Startup Growth Hacking 101  - Basics on Growth Marketing
Tech Startup Growth Hacking 101 - Basics on Growth Marketing
 
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
Call Girls Navi Mumbai Just Call 9907093804 Top Class Call Girl Service Avail...
 
Best Practices for Implementing an External Recruiting Partnership
Best Practices for Implementing an External Recruiting PartnershipBest Practices for Implementing an External Recruiting Partnership
Best Practices for Implementing an External Recruiting Partnership
 
Vip Female Escorts Noida 9711199171 Greater Noida Escorts Service
Vip Female Escorts Noida 9711199171 Greater Noida Escorts ServiceVip Female Escorts Noida 9711199171 Greater Noida Escorts Service
Vip Female Escorts Noida 9711199171 Greater Noida Escorts Service
 
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130  Available With RoomVIP Kolkata Call Girl Howrah 👉 8250192130  Available With Room
VIP Kolkata Call Girl Howrah 👉 8250192130 Available With Room
 
2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis Usage2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis Usage
 
Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...
Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...
Lowrate Call Girls In Laxmi Nagar Delhi ❤️8860477959 Escorts 100% Genuine Ser...
 
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service JamshedpurVIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
VIP Call Girl Jamshedpur Aashi 8250192130 Independent Escort Service Jamshedpur
 
Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999
Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999
Russian Faridabad Call Girls(Badarpur) : ☎ 8168257667, @4999
 
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
Lowrate Call Girls In Sector 18 Noida ❤️8860477959 Escorts 100% Genuine Servi...
 
Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.Eni 2024 1Q Results - 24.04.24 business.
Eni 2024 1Q Results - 24.04.24 business.
 
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
 
Keppel Ltd. 1Q 2024 Business Update Presentation Slides
Keppel Ltd. 1Q 2024 Business Update  Presentation SlidesKeppel Ltd. 1Q 2024 Business Update  Presentation Slides
Keppel Ltd. 1Q 2024 Business Update Presentation Slides
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023
 
M.C Lodges -- Guest House in Jhang.
M.C Lodges --  Guest House in Jhang.M.C Lodges --  Guest House in Jhang.
M.C Lodges -- Guest House in Jhang.
 
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
Call Girls In Sikandarpur Gurgaon ❤️8860477959_Russian 100% Genuine Escorts I...
 

LEC11 (1).pptx

  • 1. VIOLATION OF CLRM ASSUMPTIONS: MULTICOLLINEARITY Chapter 8 1
  • 2. MULTICOLLINEARITY: NATURE  No exact linear relationship or no collinearity means that a variable, say 𝑋2, can not be expressed as an exact linear function of another variable, say 𝑋3.  Thus if we can express that 𝑋2 = 4𝑋3 then the two variables and are collinear, for there is an exact linear relationship between them.  Consider the following 3-variable mathematical equation: 𝑌 = 𝛽1 + 𝛽2𝑋2 + 𝛽3𝑋3 or, 𝑌 = 𝛽1 + 𝛽2(4𝑋3) + 𝛽3𝑋3 = 𝛽1 + 4𝛽2𝑋3 + 𝛽3𝑋3 = 𝛽1 +𝑋3(4𝛽2 + 𝛽3)  Now, assume 𝐴 = 4𝛽2 + 𝛽3, we get 𝑌 = 𝛽1 + A𝑋3  Note that we have one equation with two unknowns, we need at least two equations to get the values of two unknowns. 2
  • 3. MULTICOLLINEARITY: NATURE  In presence of multicollinearity, we can not disentangle the individual effect of 𝑋2 and 𝑋3 on Y.  Exact linear relationship: For a k-variable regression model involving explanatory variables 𝑋1, 𝑋2, 𝑋3 … 𝑋𝑘 ( 𝑋1 is the intercept) an exact linear relationship is said to exist if the following condition is satisfied 𝛾1𝑋1 + 𝛾2𝑋2 + ⋯ + 𝛾𝑘𝑋𝑘 = 0  Where 𝛾1, 𝛾2, … 𝛾𝑘 are constants and not all of them are zero simultaneously.  Near-exact linear relationship: 𝛾1𝑋1 + 𝛾2𝑋2 + ⋯ + 𝛾𝑘𝑋𝑘 + 𝑢𝑖 = 0 Where 𝛾1, 𝛾2, … 𝛾𝑘 are constants and not all of them are zero simultaneously and 𝑢𝑖 is a random error term. 3
  • 4. MULTICOLLINEARITY: NATURE  Point to remember  Multicollinearity as we have defined above refers only to linear relationship among X variables of a regression model. But Xs may be nonlinearly correlated as  Variables and are functionally related with but the relationship in nonlinear thus the model does not violate the assumption of multicollinearity. 4 i i i i i u X X X Y      3 4 2 3 2 1     2 i X 3 i X i X
  • 5. MULTICOLLINEARITY: EXAMPLE 5 𝑋2𝑖 𝑋3𝑖 𝑋4𝑖 3 9 13 (+4) 5 15 16 (+1) 6 18 21 (+3) 8 24 25 (+1) 10 30 30 (+0)
  • 6. THEORETICAL CONSEQUENCES OF MULTICOLLINEARITY  CASE 1: PERFECT LINEAR RELATIONSHIP BETWEEN VARIABLES  If multicollinearity is perfect (exact linear relationship), the partial regression coefficients of the X variables are indeterminate and their standard errors are infinite.  If 𝑋2 and 𝑋3 are perfectly collinear, there is no way that 𝑋2 can be kept constant when 𝑋3 changes. As long as we fail to separate individual effect of 𝑋2 and 𝑋3 on Y, we can not get a unique solution for individual regression coefficients. 6
  • 7. THEORETICAL CONSEQUENCES OF MULTICOLLINEARITY  Consider the following 3-variable regression model:  7
  • 8. THEORETICAL CONSEQUENCES OF MULTICOLLINEARITY  CASE 2: NEAR PERFECT LINEAR RELATIONSHIP BETWEEN VARIABLES  If multicollinearity is less than perfect, the regression coefficients, although determinate, posses large standard errors (in relation to the coefficients themselves), which means that the coefficients can not be estimated with great precision.  The effect of multicollinearity is to make it hard to obtain estimates of coefficients with small standard error.  However, having a small number of observations also has similar effect (small sample size). For this reason, multicollinearity is considered as a small sample phenomenon. Some Economists also use separate term for multicollinearity, like micronumerosity instead of multicollinearity. 8
  • 9. PRACTICAL CONSEQUENCES OF MULTICOLLINEARITY In case near perfect or high multicollinearity, one is likely to face following problems: 1. Although BLUE, the OLS estimates have large variances and covariances making precise estimation difficult. 2. Because of 1, the confidence intervals for parameter estimates become large leading to accepting the zero 𝐻0 more readily. 3. Also because of 1, the t- statistics of one or more coefficients tends to be statistically insignificant. 4. Although the t - statistics of one or more coefficients is statistically insignificant, 𝑅2 , the overall measure of goodness-of-fit, can be very high. 5. The OLS estimators and their standard errors can be sensitive to small changes in the data. 9
  • 10. DETECTION OF MULTICOLLINEARITY  High R2 but few significant t ratios.  “Classic” symptom of multicollinearity. If R2 is high, say, in excess of 0.8, the F test in most cases will reject the null hypothesis that the partial slope coefficients are jointly or simultaneously equal to zero. But individual t tests will show that none or very few partial slope coefficients are statistically different from zero.  Subsidiary, or auxiliary, regressions.  One way of finding out which X variable is highly collinear with other X variables in the model is to regress each X variable on the remaining X variables and to compute the corresponding R2. 𝑌 = 𝛽1 + 𝛽2𝑋2 + ⋯ 𝛽7𝑋7 + 𝑢  Consider the regression of Y on X2, X3, X4, X5, X6, and X7—six explanatory variables.  If this regression shows that the R2 is high but very few X coefficients are individually statistically significant, we then look for the “culprit,” the variable(s) that may be a perfect or near perfect linear combination of the other X’s. 10
  • 11. DETECTION OF MULTICOLLINEARITY 11 𝐹 = R2 /(k − 1) (1 − R2)/(n − k) 𝑋2 on remaining Xs
  • 12. DETECTION OF MULTICOLLINEARITY  The variance inflation factor 𝑉𝐼𝐹 = 1 (1 − 𝑅2)  The larger the value of VIFj, the more “troublesome” or collinear the variable Xj.  it may be noted that the inverse of the VIF is called tolerance (TOL). That is,  One could use TOLj as a measure of multicollinearity in view of its intimate connection with VIFj. The closer is TOLj to zero, the greater the degree of collinearity of that variable with the other regressors. 12
  • 13. REMEDIAL MEASURES  What can be done if multicollinearity is serious? We have two choices: (1) do nothing or (2) follow some rules of thumb.  Dropping a Variable(s) from the Model  Acquiring Additional Data or a New Sample  Rethinking the Model  Prior Information about Some Parameters  Transformation of Variables  Combining time series and cross-sectional data, factor or principal component analysis and ridge regression. 13