Properties of bi-variate Gaussian pdf
Properties of conditional Gaussian pdf
Effect of correlation on bi-variate and conditional Gaussian pdf
Analytic expressions of bivariate and conditional Gaussian pdfs
3-D and 2-D contour plots of Gaussian pdfs
Conditional mean and variance
Matlab code of density functions plots
2. Bivariate Gaussian PDF
• Joint (bivariate) PDF of two jointly Gaussian
random variables x and y is
1
,
1 1
( , ) exp
22
C
C
x
x y x y
y
x
f x y x y
y
2
2
Covariance matrix of and
Deter
C
C
C
x
y
x x xy
x y
y xy y
E x Expectation of x
E y Expectation of y
x
x
E x y
y
y
minant of C
3. 0
1 0
0 1
C
x y
σy = σx
Joint PDF 3-D plot
4. 0
0.25 0
0 1
C
x y
σy > σx
Joint PDF 3-D plot
5. 0
1 0
0 0.25
C
x y
σx > σy
Joint PDF 3-D plot
6. Contour
• Contour of a 3-D plot is 2-D plot showing
relationship between x and y when fx,y(x,y) =
constant
• Set f (x,y) = constant Gives an equation of• Set fx,y(x,y) = constant Gives an equation of
x and y Plotting this equation (y versus x)
gives the so-called contour
• As constant varies, we get different contours
• Let’s plot contours of previous figures
7. Contour
,
22
, 2 2
22
( , )
1
( , ) exp Constant
2 2
:Get Contou
12ln
r equation of wit
2
h 0
C
yx
x y
yx
x
y
xy
y
x
E f x y
y
T
y
xam
x
l
f
e
x y
p
x
2 2
12ln
This is an equation f
2
o Ellips
yx
x yx y
yx
T
22 2
2
2
2 2
12 l
centered @ ( , ) = ,
If ==> It becomes
This is equation of centered @ ( , ) = , .Its radius depend
n
2
s on
e
Circle
x y
x
y
x
x
y
x
x
x y
x
y
y
T
T
8. 0
1 0
0 1
C
x y
σy = σx
1
2
3
0.7
0.8
0.9
1
,
Plot of versus
when
( , ) 0.3x y
y x
f x y
Contour is
circle
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
0.1
0.2
0.3
0.4
0.5
0.6
,
Plot of y versus x
when
( , ) 0.9x yf x y
9. 1
2
3
0.7
0.8
0.9
0
0.25 0
0 1
C
x y
σy > σx
Contour is
Ellipse with
major axis on
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
0.1
0.2
0.3
0.4
0.5
0.6major axis on
y-axis and
minor axis on
x-axis
because
σy > σx
10. Contour is
Ellipse with
major axis on
0
1 0
0 0.25
C
x y
σx > σy
1
2
3
0.7
0.8
0.9
1
major axis on
x-axis and
minor axis on
y-axis
because
σx > σy
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
0.1
0.2
0.3
0.4
0.5
0.6
11. Effect of Correlation
• Now, we saw PDF and contour when σxy = 0,
i.e., when x and y are uncorrelated
• How would contour look like when x and y are
correlated, i.e., σxy ≠ 0 ?
• Correlation coefficient ρ = σ / (σ σ )• Correlation coefficient ρ = σxy / (σx σy )
-1 < ρ < 1
• ρ > 0 x and y are positively correlated, i.e.,
as x increases , y increases
• ρ < 0 x and y are negatively correlated, i.e.,
as x increases, y decreases
12. 0.50,
1 0.5
0.5 1
C
x y
ρ = 0.5 Contour is
Rotated Ellipse
x y x y
1
2
3
0.7
0.8
0.9
Major axis
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Major axis
has positive
slop as ρ > 0
13. 0.50,
1 0.5
0.5 1
C
x y
ρ = - 0.5 Contour is
Rotated Ellipse
Major axis
x y x y
1
2
3
0.7
0.8
0.9
1
Major axis
has negative
slop as ρ < 0
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
14. Effect of Correlation
• As correlation ρ increases, knowing one
variable gives more information about the
other
• For large ρ Given any value of x, variance of
y decreases [because more information abouty decreases [because more information about
y is available]
• This means that y will become more
consternated around its mean at any given
value of x
• See next slide for Contour @ ρ = 0.98
15. 0.980,
1 0.98
0.98 1
C
x y
ρ = 0.98 Contour is
Rotated Ellipse
1
2
3
0.5
0.6
0.7
Compare with
Contour for
ρ = 0.5 in
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
0.1
0.2
0.3
0.4
0.5
ρ = 0.5 in
slide ρ=0.5
where y has
larger variance
around its
mean
for any given
value of x
y has small variance
around its mean
At any given value
of x
16. Effect of Correlation
• For ρ>0, increasing x makes average level of y
(mean of y) increases
• For ρ<0, increasing x makes average level of y• For ρ<0, increasing x makes average level of y
(mean of y) decreases
• For ρ=0, increasing/decreasing x does not affect
average level of y (mean of y)
17. Effect of Correlation
y-axis
0
1
2
0.4
0.5
0.6
E(y|x=1) = Mean of y when x = 1
ρ = 0.98
x-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0.1
0.2
0.3
E(y|x=0) = Mean of y when x = 0
E(y|x=1) > E(y|x=0) As x increases, E(y|x) increases ρ>0
18. Effect of Correlationρ = 0
y-axis
0
1
2
3
0.6
0.7
0.8
0.9
E(y|x=0) =
Mean of y
when x = 0
E(y|x=1) = E(y|x=0) As x changes, E(y|x) doesn’t change ρ=0
x-axis
y-axis
-3 -2 -1 0 1 2 3
-3
-2
-1
0
0.1
0.2
0.3
0.4
0.5
E(y|x=1) =
Mean of y
when x = 1
19. Effect of Correlation
• For ρ>0, E(y|x) increases as x increases
• For ρ<0, E(y|x) decreases as x increases
• For ρ=0, E(y|x) doesn’t change as x changes
E(y|x) not function of x
20. Conditional PDF
• So, we have seen that correlation ρ
determines how E(y|x) changes as function of
x See slide_ ρ _0.98 and slide_ ρ_0
• We also saw how magnitude of ρ affects
variance of y around its mean E(y|x) at anyvariance of y around its mean E(y|x) at any
given x See Slide_var
• Let’s develop these relationship analytically
and further verifies it through graphs
• We will get fy(y|x) and observe its mean E(y|x)
and variance var(y|x)
21. Conditional PDF
, 0
0
0 , 0
0 Bayes' Rule
,
|
,
x
x x y
y
x y
y
f x
f x f x
f y x x
f x y
y dy
,
,
0
0
0
0
0
0
,
Just a scalar to make 1
is a scaled version of
|
| ,
, Cross section , @of =x
y
x
y
x y
x y y
y
y
f
f y x x
f y x
x y x x
x f x y
f x d
f x
y
y
22. Conditional PDF
, 00
0 , 0
is a scaled version of
To plot we just plot
Since | ,
| , ,
y x y
y yx
f x y
f
f y x x
f y x x x y
0
0
, 0
, 0
To plot we just plot
We plot Scaled version of
against for different
|
|
, ,
, [ ]
y y
y y
x
x
ff y x x x y
f x y
y
f y x x
23. Conditional PDF ρ = zero
0.14
0.16
x
= y
= 1, = 0, x
= 0, y
= 0
xo
= 0
xo
= 0.5
x = 1ρ = 0
, 0
, 0 , 0
0Vertical axis ==> Scaled version o, [ ]
,
f
==> Cross , @section of =
|
x y
x y
x y
yf y x x
f
f x y
f x y x y x x
-3 -2 -1 0 1 2 3
0
0.02
0.04
0.06
0.08
0.1
0.12
y-axis
fy
(y|x=xo)scalar
xo
= 1
xo
= 1.5
ρ = 0
24. Conditional PDF ρ = zero
• From previous plot of Conditional PDF when
ρ=0, we observe:
A. fy(y|x=xo) is Gaussian
B. Location of maximum of fy(y|x=xo), i.e., its
Mean, i.e. E(y|x=x ), is fixed regardless of xMean, i.e. E(y|x=xo), is fixed regardless of xo
Cross section of fx,y(x=xo,y) is centered
around same point regardless of position of
cross section
C. Variance of fy(y|x=xo), i.e., var(y|x=xo) does
not depend on xo
25. 0.16
0.18
0.2
x
= 1, y
= 1, = 0.5, x
= 0, y
= 0
xo
= 0
xo
= 0.5
Conditional PDF ρ = 0.5
, 0
, 0 , 0
0Vertical axis ==> Scaled version o, [ ]
,
f
==> Cross , @section of =
|
x y
x y
x y
yf y x x
f
f x y
f x y x y x x
-3 -2 -1 0 1 2 3
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
y-axis
fy(y|x=xo)scalar
xo
= 1
xo
= 1.5
ρ = 0.5
y=0= 0 x ρ
y=0.25=0.5 ρ
y=0.5=1 x ρ
y=0.75=1.5 ρ
26. Conditional PDF ρ = 0.5
• From previous plot of Conditional PDF when
ρ=0.5, we observe:
A. fy(y|x=xo) has a Gaussian shape ==> Gaussian
B. Location of maximum of fy(y|x=xo), i.e., its
Mean, i.e. E(y|x=xo), increases as xo increases
Cross section of fx,y(x=xo,y) @x=xo is Cross section of fx,y(x=xo,y) @x=xo is
centered at different positions of the cross
section
C. Location of maximum of fy(y|x=xo), i.e.,
E(y|x=xo) = ρ xo
D. Variance of fy(y|x=xo), i.e., var(y|x=xo) does not
depend on xo
27. 10
12
x
= 1, y
= 1, = -0.9999, x
= 0, y
= 0
xo
= 0
xo
= 0.5
xo
= 1
x = 1.5
Conditional PDF ρ ≈ -1
, 0
, 0 , 0
0Vertical axis ==> Scaled version o, [ ]
,
f
==> Cross , @section of =
|
x y
x y
x y
yf y x x
f
f x y
f x y x y x x
-3 -2 -1 0 1 2 3
0
2
4
6
8
y
fy
(y|x=xo)scalar
xo
= 1.5
ρ ≈ -1y=0= 0 x ρ
y=-0.5=0.5 ρ
y=-1=1 x ρ
y=-1.5=1.5 ρ
28. Conditional PDF ρ ≈ -1
• From previous plot of Conditional PDF when ρ ≈ -1,
we observe:
A. fy(y|x=xo) has a Gaussian shape ==> Gaussian
B. Location of maximum of fy(y|x=xo), i.e., its Mean,
i.e. E(y|x=xo), increases as xo increases Cross
section of fx,y(x=xo,y) @x=xo is centered atsection of fx,y(x=xo,y) @x=xo is centered at
different positions of the cross section
C. Location of maximum of fy(y|x=xo), i.e., E(y|x=xo)
= ρ xo
D. Variance of fy(y|x=xo), i.e., var(y|x=xo) does not
depend on xo
E. var(y|x=xo) is smaller than case of ρ = 0.5
29. Conclusion on Conditional PDF
1) If , are jointly Gaussian
2) with coefficient
( | ) when 0,
( | ) is also
( | ) is in
Gaussian
LINEAR al
x y
E y x x
f y x
E y x x
( | ) when 0,
3)
4)var( | )
var( | ) is function of
Asdepends on ,
NOT
x y x yE y x x
y x
y x x
var( | )y x
30. Analytical expression of fy|x(y|x)
2
|,
22
||
| 2
, 1
( | ) exp
22
|
y xx y
x y xy x
xy x
y x y x y y
x x
yf x y
f y x
f x
x
E y x x
2
2 2 2 2
|
| 2
function ofvar 1|
x x
xy
y x y y
x
x
y x y x
y
x
xy x
As var |y x
31. Analytical expression of fy|x(y|x)
• We see that analytical expressions are inline with
our graphical observations:
– E(y|x) is linear in x
– var(y|x) does not depend on x
– var(y|x) decreases as |ρ| increases
• If ρ = 0, we have
– E(y|x) = μy Not function of x
– var(y|x) = var(y)