This document defines and explains covariance and correlation. Covariance is a measure of how two random variables change together, and can be positive if they move in the same direction, negative if opposite. Correlation scales covariance between -1 and 1 to allow comparison between variables with different units or variances. It also provides formulas for calculating covariance and correlation, and properties such as how they are affected by transformations of the random variables.

1. Covariance & Correlation
Jerrell T.Stracener – Ph.D.
1

2. Covariance – Cov(X,Y)
Covariance between X and Y is a measure
of the association between two random
variables, X & Y
If positive, then both move up or down
together
If negative, then if X is high, Y is low, vice
versa
[
σ XY = Cov( X , Y ) = E ( X − µ X )( Y − µY )
]

3. Correlation Between X and Y
Covariance is dependent upon the units of
X & Y [Cov(aX,bY)=abCov(X,Y)]
Correlation, Corr(X,Y), scales covariance
by the standard deviations of X & Y so that
it lies between 1 & –1
ρ XY
σ XY
Cov ( X , Y )
=
=
1
σ X σ Y [Var ( X )Var (Y )] 2

4. More Correlation & Covariance
If σX,Y =0 (or equivalently ρX,Y =0) then X
and Y are linearly unrelated
If ρX,Y = 1 then X and Y are said to be
perfectly positively correlated
If ρX,Y = – 1 then X and Y are said to be
perfectly negatively correlated
Corr(aX,bY) = Corr(X,Y) if ab>0
Corr(aX,bY) = –Corr(X,Y) if ab<0

5. Properties of Expectations
E(a)=a, Var(a)=0
E(µX)=µX, i.e. E(E(X))=E(X)
E(aX+b)=aE(X)+b
E(X+Y)=E(X)+E(Y)
E(X-Y)=E(X)-E(Y)
E(X- µX)=0 or E(X-E(X))=0
E((aX)2)=a2E(X2)

6. More Properties
Var(X) = E(X2) – µx2
Var(aX+b) = a2Var(X)
Var(X+Y) = Var(X) +Var(Y) +2Cov(X,Y)
Var(X-Y) = Var(X) +Var(Y) - 2Cov(X,Y)
Cov(X,Y) = E(XY)-µxµy
If (and only if) X,Y independent, then

Var(X+Y)=Var(X)+Var(Y), E(XY)=E(X)E(Y)

7. Covariance of X and Y
Let X and Y be random variables with joint mass function
p(x,y) if X & Y are discrete random variables or with joint
probability density function f(x, y) if X & Y are continuous
random variables. The covariance of X and Y is
σ XY = E [ ( X − µ X )( Y − µ Y ) ] = ∑ ∑ ( x − µ x ) ( y − µ y ) p( x, y )
x
y
if X and Y are discrete, and
σ XY = E [ ( X − µ X )( Y − µ Y ) ] =
∞ ∞
∫ ∫ ( x − µ ) ( y − µ ) f ( x, y ) dxdy
x
− ∞− ∞
if X and Y are continuous.
Jerrell T.Stracener – Ph.D.
7
y

8. Covariance of X and Y
The covariance of two random variables X and Y with means
µX and µY , respectively is given by
σ XY = E ( XY ) − µ X µ Y
Jerrell T.Stracener – Ph.D.
8

9. Correlation Coefficient
Let X and Y be random variables with covariance σXY and
standard deviation σX and σY , respectively. The correlation
coefficient of X and Y is
ρ XY
σ XY
=
σ Xσ Y
Jerrell T.Stracener – Ph.D.
9

10. Theorem
If X and Y are random variables with joint probability
distribution f(x, y), then
σ
2
aX + bY
= a σ + b σ + 2abσ XY
2
2
X
Jerrell T.Stracener – Ph.D.
10
2
2
Y

11. Theorem
If X and Y are independent random variables, then
σ
2
aX + bY
= a σ +b σ
2
2
X
Jerrell T.Stracener – Ph.D.
11
2
2
Y

12. Correlation Analysis
A statistical analysis used to obtain a quantitative measure of
the strength of the linear relationship between a dependent
variable and one or more independent variables
Jerrell T.Stracener – Ph.D.
12

13. Correlation – Scatter Diagram
Visual Relationship Between X and Y
Jerrell T.Stracener – Ph.D.
13