This document provides an overview of simple linear regression analysis and correlation coefficients. It will cover fitting a straight line to bivariate data, calculating and interpreting Pearson's correlation coefficient using the formula, and calculating and interpreting Spearman's correlation coefficient for ordinal data. Examples are provided to demonstrate calculating the coefficients and interpreting their sign, strength, and what they indicate about the relationship between variables.

1. Topic 7:
Regression Analysis 1

2. This topic will cover:
◦ Simple linear regression
◦ Pearson’s (product moment) correlation coefficient
◦ Spearman’s (rank order) correlation coefficient

3. By the end of this topic students will be able
to:
◦ Understand a straight line fit to bivariate data
◦ Calculate and interpret Pearson’s correlation
coefficient
◦ Calculate and interpret Spearman’s correlation
coefficient

4. ◦ The search for, and strength of, predictors
 What is a good predictor of future job
performance?
 What is this product’s price-demand curve?
 Which process factors affect production
yield?
 How does a particular share price move with
the market average?

5. income
saving
s
x
y
xy x
y
y = mx +
c
•Assuming interval or ratio
data.

6. ◦ yi = mxi + c + ei
◦ The least squared line
is the line that
minimizes the sum of
square errors
 e1
2
+ e2
2
+ ⋯ + en
2
◦ y = mxi + c
x
y
e5
e1
e2
e3
e4
yi = mxi + c
+ ei

7. ◦ For the least SSE
straight line, y = mxi +
c
◦ m =
xi − x yi − y
xi − x
2
◦ c = y − mx
x
y
e5
e1
e2
e3
e4
yi = mxi + c
+ ei

8. ◦ For the least SSE
straight line, y = mxi +
c
◦ m =
xi − x yi − y
xi − x
2
◦ c = y − mx
• For the least SSE straight
line, y = mxi + c
• m =
n xiyi − xi yi
n xi
2 − xi
2
• c = y − mx

9. A company has been carrying out experiments with the
position of a button on its website.
Page
Position
(% vertical)
Click
Throughs
(%)
25 3.07
50 5.64
75 9.63
100 10.26
Independent
Variable
(X axis)
Dependent
Variable
(Y axis)
X
Y

10. ◦ A company has been carrying out experiments with the
position of a button on its website.
x y xy x2
25 3.07
50 5.64
75 9.63
100 10.26
total
mean
m = {(4 × 2107) + (250 ×28.6)}
{(4 × 18750) -(250 ×250)}
m = 0.10224,c = y − mx
c = 7.15 – (0.10224 ×62.5) = 0.76
m =
n xy − x y
n x2− x 2
x y xy x2
25 3.07 76.75 625
50 5.64 282.00 2500
75 9.63 722.25 5625
100 10.26 1026.00 10000
total 250 28.6 2107 18750
mean 62.5 7.15

11. ◦ A company has been carrying out experiments with the
position of a button on its website.
Page
Position
(% vertical)
Click
Throughs
(%)
25 3.07
50 5.64
75 9.63
100 10.26

12. ◦ A company has been carrying out experiments with the
position of a button on its website.
x y xy x2
25 1.44
50 5.58
75 14.64
100 6.94
total
mean
m =
n xy − x y
n x2− x 2
c = y − mx
x y xy x2
25 1.44 36 625
50 5.58 279 2500
75 14.64 1098 5625
100 6.94 694 10000
total 250 28.6 2107 18750
mean 62.5 7.15
= 0.10224
= 0.76

13. x y
25 1.44
50 5.58
75 14.64
100 6.94
x y
25 3.07
50 5.64
75 9.63
100 10.26

14. explained
unexplained
y
• Total variation in two parts
y − y 2 = y
x

15. ◦ R2 or r2 is called the coefficient of determination
◦ 0 ≤r2 ≤ 1
◦ r is called the Pearson correlation coefficient
◦ -1 ≤ r ≤ 1
◦ Following rearrangement
R = r =
n xiyi − xi yi
n xi
2
− xi
2
n yi
2
− yi
2

16. 9.425
31.810
92.737
105.268
239.239
R = r =
n xiyi − xi yi
n xi
2
− xi
2
n yi
2
− yi
2
x y xy x2 y2
25 3.07 76.75 625
50 5.64 282.00 2500
75 9.63 722.25 5625
100 10.26
1026.0
0
1000
0
250 28.6 2107 18750

17. x y
25 1.44
50 5.58
75
14.6
4
100 6.94
x y
25 3.07
50 5.64
75 9.63
100
10.2
6

18. ◦ Sign
 r > 0 positive linear relationship
 r = 0 no linear relationship
 r < 0 negative linear relationship

19. ◦ Strength
 Physical sciences / engineering R2 > 0.6 often found
 Social sciences / policy R2 > 0.25 sometimes useful
 Business and management – includes science &
social science!
 But you will see language like R2 > 0.8 strong, R2 >
0.5 moderate, R2 > 0.25 weak relationship
 Be careful; context and numbers often more
informative than descriptive word, but words help to
communicate.

20. What is r?
A B C
D E F
r = 1 r = 0.9 r = 0.7
r = 0 r = -0.7 r = -0.9

21. ◦ Interpolation - Estimates between values already
known
◦ Extrapolation - Estimates outside known values

22. ◦ Plot scatter graph to intuit whether straight line is
reasonable
◦ Look at r2 = R2 for strength of relationship
◦ Look at sign of r for direction
 Check agrees with graph
◦ Look at m for gradient of relationship
◦ Use straight line equation to interpolate (with care)
◦ Use straight line equation to extrapolate (with caution)

23. ◦ Sometimes we only have ordinal data
 two interviewers rank candidates
◦ Can we still define a correlation function? Yes
◦ 𝑟𝑠 = 1 −
6 𝑑2
n n2
−1
 where d is the difference between ranking.

24. ◦ Two interviewers individually rank prospective job
candidates. What is the Spearman correlation
coefficient?
Candidat
e
Interviewer
1
Interviewer 2
Hidayat 3 E
Elisa 2 A
Nouman 1 B
Bernie 4 C
Li Ren 5 D
Ahere 6 F

25. ◦ Two interviewers individually rank prospective job
candidates. What is the Spearman correlation coefficient?
◦ 𝑟𝑠 = 1 −
6 𝑑2
n n2
−1
= 1 −
8
35
= 0.771
Candidat
e
Interviewer
1
Interviewer 2 d d2
Hidayat 3 E 5 -2 4
Elisa 2 A 1 1 1
Nouman 1 B 2 -1 1
Bernie 4 C 3 1 1
Li Ren 5 D 4 1 1
Ahere 6 F 6 0 0

26. ◦ For tied ranks use mean rank, then
◦ Use formula for Pearson correlation
Candidat
e
Interviewer
1
Interviewer 2
Hidayat 3
Elisa 2
Nouman 1
Bernie 4
Li Ren 5
Ahere 6
4
1
2
4
4
6
OK
Excellent
Good
OK
OK
Poor

27. By the end of this topic students will be able
to:
◦ Understand a straight line fit to bivariate data
◦ Calculate and interpret Pearson’s correlation
coefficient
◦ Calculate and interpret Spearman’s correlation
coefficient

28. ◦ Burton, G., Carrol, G. and Wall, S. Quantitative
Methods for Business and Economics. Longman.
◦ Buglear, J. Quantitative Methods for Business.
Elsevier Butterworth Heinemann
◦ Hinton, PR. Statistics Explained. Routledge

29. Any Questions?