It includes all the basics of correlation and regression analysis. It also contains basic formulas to carry out the calculations.

1. CORRELATION
AND
REGRESSION ANALYSIS
1
Presented by: Group-E (The Anonymous)
Rabin BK (Editor)
Bikash Dhakal
Bimal Pradhan
Gagan Puri
Bikram Bhurtel

2. Contents
 Meaning of correlation
 What is partial correlation?
 Properties of partial correlation coefficient
 Co-efficient of Partial determination
 Regression and Multiple Regression equation
 Co-efficient of determination
 What is an error?
 Determining F-ratio
 Calculating Co-efficient of Multiple
 Auto Correlation
 Preferences 2

3. Meaning of correlation
 Relation or connection between two or more
things (in general sense)
 Interdependence of variable quantities
(statistically)
3

4. Order of correlation
 Zero order correlation
 First order correlation
 Second order correlation
4
𝑟12, 𝑟13, 𝑟23, ………
𝑟12.3, 𝑟13.2, 𝑟23.1, ………
𝑟12.34, 𝑟13.24, ………

5. For Zero order correlation,
And
5
𝑟12 =
𝑛 𝑋1 𝑋2− 𝑋1 𝑋2
𝑛 𝑋1
2
− 𝑋1
2 𝑛 𝑋2
2
− 𝑋2
2
𝑟13 =
𝑛 𝑋1 𝑋3− 𝑋1 𝑋3
𝑛 𝑋1
2
− 𝑋1
2 𝑛 𝑋3
2
− 𝑋3
2
𝑟23 =
𝑛 𝑋2 𝑋3− 𝑋2 𝑋3
𝑛 𝑋2
2
− 𝑋2
2 𝑛 𝑋3
2
− 𝑋3
2

6. What is partial correlation
co-efficient?
Relationship between two variables keeping the
other variable constant/fixed
• Relation between X1 and X2 keeping X3
constant is denoted by:
• Similarly, means relation between X1 and
X3 keeping X2 and so on
6
𝑟12.3
𝑟13.2

7. Calculating partial correlation
Based on zero order correlation
Formulas:
7
𝑟12.3 =
𝑟12 − 𝑟13.𝑟23
1− 𝑟13
2 1− 𝑟23
2
𝑟13.2 =
𝑟13 − 𝑟12.𝑟23
1− 𝑟12
2 1− 𝑟23
2
𝑟23.1 =
𝑟23 − 𝑟12.𝑟13
1− 𝑟12
2 1− 𝑟13
2

8. Properties of Correlation
co-efficient
 Its value lies between -1 to +1.
 𝑟12 = 𝑟21, 𝑟13 = 𝑟31 and 𝑟23 = 𝑟32
 𝑟12.3 = 𝑟21.3, 𝑟13.2 = 𝑟31.2 and 𝑟23.1 = 𝑟32.1
8

9. Co-efficient of partial determination
Square of partial correlation coefficient
Also known as the percent of variation
Used to measure variation in one variable explained by other
variable keeping next variable constant
Example: If 𝑟12.3 = 0.5, then partial determination is:
𝑟12.3
2 = 0.25 = 25%
Variation in 𝑋1, Explained by 𝑋2 , Constant = 𝑋3
9

10. Multiple correlation
Relation between three/more variables at the same time
Denoted by R
If R<1 (r<1), more consistent
If R>1 (r>1), less consistent
10

11. Calculating Multiple Correlation
 If 𝑋1, 𝑋2 and 𝑋3 are three variable then,
11
𝑅1.23 =
𝑟12
2 + 𝑟13
2 − 2𝑟13.𝑟23 𝑟12
1− 𝑟23
2
𝑅2.13 =
𝑟12
2 + 𝑟23
2 − 2𝑟13.𝑟23 𝑟12
1− 𝑟13
2
𝑅2.13 =
𝑟12
2 + 𝑟23
2 − 2𝑟13.𝑟23 𝑟12
1− 𝑟13
2

12. Regression
 A statistical tool used to find the nature of relationship
 Estimates the value of a dependent variable with the help of an
independent variable
 Types:
 Regression of y on x is, y = a + bx (a and b are constants)
 Regression of x on y is, x = a + by (a and b are constants)
12
𝒀 = 𝒏𝒂 + 𝒃 𝑿
𝑿𝒀 = 𝒂 𝑿 + 𝒃 𝑿
𝟐
𝑿 = 𝒏𝒂 + 𝒃 𝒀
𝑿𝒀 = 𝒂 𝒀 + 𝒃 𝒀
𝟐

13. Multiple Regression Analysis
An extension of simple linear regression
Estimates the value of a dependent variable with the
help of two independent variables
If 𝑋1 is dependent and 𝑋2 and 𝑋3 are independent
variables then,
13
𝑿𝟏 = 𝒂 + 𝒃𝟏 𝑿𝟐 + 𝒃𝟐 𝑿𝟑
𝑿𝟏 𝑿𝟐 = 𝒂 𝑿𝟐 + 𝒃𝟏 𝑿𝟐
𝟐
+ 𝒃𝟐 𝑿𝟐 𝑿𝟑
𝑿𝟏 𝑿𝟑 = 𝒂 𝑿𝟑 + 𝒃𝟏 𝑿𝟑
𝟐
+ 𝒃𝟏 𝑿𝟐 𝑿𝟑

14. Multiple Regression equation using
simple correlation and standard deviations
Multiple regression equation of 𝑋1 on 𝑋2 and 𝑋3 is:
Where,
Linear equation of 𝑥1 on 𝑥2 and 𝑥3 is:
14
𝑋1 − 𝑋1 = 𝑏12.3 𝑋2 − 𝑋2 + 𝑏13.2 𝑋3 − 𝑋3
𝑏12.3 =
𝜎1
𝜎2
𝑟12 − 𝑟23 𝑟13
1 − 𝑟23
2
𝑏13.2 =
𝜎1
𝜎3
𝑟13 − 𝑟23 𝑟12
1 − 𝑟23
2
𝑥1 = 𝑏12.3 𝑥2 + 𝑏13.2 𝑥3

15. Co-efficient of determination
It is the degree of explained variation
Denoted by 𝑅2
If 𝑅2
> 0.5, it is good fit
If 𝑅2
< 0.5, it is less fit
15

16. Total variation = Explained variation + Unexplained
variation
Co − efficient of determination
16
Explained variation = SSR= TSS – SSE
SSR= sum of square due to regression
TSS= total sum of square
SSE= Sum of square due to error
R =
𝑒𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑣𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛
𝑡𝑜𝑡𝑎𝑙 𝑣𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛

17. What is an error?
Difference between true value and estimated value
Degree of freedom (df) = n-1
For standard error we have,
OR
17
Standard error =
𝐸𝑟𝑟𝑜𝑟 𝑜𝑓 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒
𝐸𝑟𝑟𝑜𝑟 𝑜𝑓 𝑑𝑒𝑔𝑟𝑒𝑒 𝑜𝑓 𝑓𝑟𝑒𝑒𝑑𝑜𝑚
Standard error =
𝑈𝑛𝑒𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑣𝑎𝑟𝑖𝑎𝑡𝑖𝑜𝑛
𝑛 − 3

18. Determining F-ratio
1.
2.
18
F − ratio =
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑚𝑒𝑎𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒 𝑜𝑓 1 𝑠𝑡
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑚𝑒𝑎𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒 𝑜𝑓 2 𝑛𝑑
1 𝑠𝑡 𝑀𝑒𝑎𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒 =
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑑𝑓
2 𝑛𝑑 𝑀𝑒𝑎𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒 =
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑑𝑓

19. Calculating Co-efficient of Multiple
determination in terms of Multiple regression
19
𝑅1.23
2
=
𝑎 𝑋1 + 𝑏1 𝑋1 𝑋2 + 𝑏2 𝑋1 𝑋3 − 𝑛 𝑋1
2
𝑋1
2
− 𝑛 𝑋1
2
𝑅2.13
2
=
𝑎 𝑋2 + 𝑏1 𝑋1 𝑋2 + 𝑏2 𝑋2 𝑋3 − 𝑛 𝑋2
2
𝑋2
2
− 𝑛 𝑋2
2
𝑅3.12
2
=
𝑎 𝑋3 + 𝑏1 𝑋1 𝑋3 + 𝑏2 𝑋2 𝑋3 − 𝑛 𝑋3
2
𝑋3
2
− 𝑛 𝑋3
2

20. Auto Correlation
• Error terms are assumed independent in regression
• Difference between the observed value and estimated value is
known as error term
• Error terms are correlated instead of being independent is known
as auto correlation
• Formula:

 
 2
22
t
tt
e
ee


 
 2
1
t
tt
e
ee


21. Reference:
Statistics-I (Vikash Raj Satyal)
Teachers note
21

22. Queries
22