Linear and multiple Regression Notes and Basics Assumptions on Regression Analysis

1. REGRESSION ANALYSIS
05/15/2020 1
Dr. A. Michael J Leo, Assistant Professor of Education,
St. Xavier’s College of Education (Autonomous), Palayamkottai - 627002

2. The Background
• To understand the nature of relationship
between the independent variables and the
dependent variable
• To explore the forms of these relationships.
• Brief History
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education,
St. Xavier’s College of Education (Autonomous), Palayamkottai - 627002
2

3. Regression Analysis
Variable
(X)
Variable
(Y)
Independent Variable Dependent Variable
Explanatory Variable Target Variable
Predictor Variable Outcome variable
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education,
St. Xavier’s College of Education (Autonomous), Palayamkottai - 627002 3

4. Classification of Regression Analysis
Based on the Number of IV and DV
• Linear Regression and Multiple Regression
Based on the Scale of Dependent Variable
• Poisson Regression : Count
• Binary Logistic Regression : Two events/Two Groups
(Yes/No) (Pass/Fail)
• Nominal Logistic Regression : (Three groups)
• Ordinal Logistic Regression :
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
4

5. Linear Regression
• To find out the influence of one
Independent Variable (IV) on one
dependent (DV) variable.
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education,
St. Xavier’s College of Education (Autonomous), Palayamkottai - 627002
5

6. • Example: To find out the influence of study skill (IV)
on the performance in Semester Examination (DV)
of undergraduate students
• Null Hypothesis: Ho: There is no significant
influence of the study skill on the performance in
semester examination of undergraduate students.
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education,
St. Xavier’s College of Education (Autonomous), Palayamkottai - 627002
6

7. The Linear Regression Equation
The linear regression equation of Y on X is
Y = a + bX + c
Where
a - the intercept
b - the slope
c - the residual, or the error of prediction
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education,
St. Xavier’s College of Education (Autonomous), Palayamkottai - 627002
7

8. 05/15/2020 Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
8

9. Underlying assumptions of Linear Regression
• The sample is representative of the population for
the inference prediction.
• The variables need to be continuous in nature
• There should be no significant outliers.
• It must have independent observations.
• The data should show homoscedasticity
• The residuals or errors are normally distributed
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002 9

10. DEMO IN SPSS
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
10

11. Multiple Regression
• When One Dependent and More than one
independent variable. It is an extension of
linear regression.
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of
Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
11

12. The Regression Equation
E(Y) = a+bX1+cX2+dX3+……
• Y – dependent variable
• X1, X2, X3 are the independent variables
• b, c, d - co-efficeints or slopes
• a – intercept
Example: To find the influence of the study skill,
home environment and peer network on the
semester examination of undergraduate students.
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
12

13. Assumptions for Multiple Regression
• Dependent variable should be measured on a
continuous scale.
• Independent variable should be measured on
a continuous or metric or categorical
• It must have independent observations.
• The data should show homoscedasticity
• There should be no significant outliers.
• The residuals or errors are normally
distributed
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
13

14. Identification of Multicollinearity
1. The strong correlation among the
independent variables
2. To explain in simple terms, each
independent variable become depend
variables and regress againt the original
dependent variable
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
14

15. Influence of Sample Size
• Sample too small - overestimate
• P value would be large, the significance may
not be achieved
• Sample calculation is to be achieved
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
15

16. 05/15/2020
Dr. A. Michael J Leo, Assistant Professor of
Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
16

17. Sample size table for known population size
• Krejcie and Morgan, 1970
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St. Xavier’s College of Education
(Autonomous), Palayamkottai - 627002
17

19. Dr. A. Michael J Leo
Assistant Professor
St. Xavier’s College of Education(Autonomous)
Palayamkottai – 627002, Tirunelveli, Tamil Nadu, India
9994006762
amjlsxce@gmail.com
05/15/2020
Dr. A. Michael J Leo, Assistant Professor of Education, St.
Xavier’s College of Education (Autonomous), Palayamkottai -
627002
19