Module iii sp

Module III
Stochastic Processes
1

Vector Random Variable
• Assigning a vector of real numbers to each outcome of
Sample Space of a random experiment.
• Example: Random Experiment consists of selecting student’s
name from an urn based on the Height, Weight and Age of
the student.
• - Height of student in inches• - Height of student in inches
• - Age of student in years
• - Weight of student in Kg
• Then the vector is the vector random
variable.
)(H
)(A
)(W
       WAH ,,
2Vijaya Laxmi, Dept. of EEE

• We have
y
y2
   221 yYxXx 
x1 x2 x
x
y1
y2
y
x2x1
   2121 yYyxXx 

y
y2
y1
   211 yYyxX 
x1 y2
y
y2
y1
y2x1-x1
   211 yYyxX 

MULTIPLE RANDOM VARIABLESMULTIPLE RANDOM VARIABLES

Joint Distribution
• If there are two RVs X and Y and the sets and are
events with probabilities
consisting of all outcomes ξ such that
is also an event, then
 xX   yY 
   
    yYxXyYxX
producttheandyFyYPandxFxXP YX


,
)()(
yYandxX  )()( 
is also an event, then
is called the Joint Distribution of the RVs X and Y.
• It is given by
•
 yYxXP  ,
 yYxXPyxFXY  ,),(

Discrete Random Variable
• For discrete RV,
 


iyxfWhere
yxfyYxXP
).....(..........0),(,
),(,
 

x y
iiyxf
iyxfWhere
).....(..........1),(
).....(..........0),(,

• Let X be a RV which assumes any one value of
• And, Y be a RV which assumes any one of
mxxx .,..........,, 21
nyyy .,..........,, 21
• Then, the probability of an event that X=xj and Y=yk
is given by
n21
  ),(, kjkj yxfyYxXP 

X Y y1 y2 . . yn Total
f(x1,y1) f(x1,y2) . . f(x1,yn) f1(x1)
x2 f(x2,y1) f(x2,y2) . . f(x2,yn) f1(x2)
x3 f(x3,y1) f(x3,y2) . . f(x3,yn) f1(x3)
x1
. . . . . . .
. . . . . . .
xm f(xm,y1) f(xm,y2) . . f(xm,yn) f1(xm)
Total f2(y1) f2(y2) . . f2(yn) 1
Grand Total

• The probability that X=xj is obtained by adding all
entries in the row corresponding to xj is given by
• The probability that Y=yk is obtained by adding all
   

n
k
kjjj yxfxfxXP
1
1 ,)(
• The probability that Y=yk is obtained by adding all
entries in the row corresponding to yk is given by
   

m
j
kjkk yxfyfyYP
1
2 ,)(

• f1(xj) and f2(yk) or simply f1(x) and f2(y) are obtained from the
margins of table, hence called Marginal Probability function of
X and Y
• Or,
which, can be written as
    11
1 1
21   
m
j
n
k
kj yfandxf
  1,  
m n
yxf
which, can be written as
• i.e., Total probability of all entries is 1.
• Joint distribution function of X and Y is given by
• This is the sum of all entries for which
  1,
1 1
  j k
kj yxf
      

xu yv
vufyYxXPyxF ),(,,
yyandxx kj 

Continuous Random Variables
• The properties of joint PDF for the continuous RVs
are
  )..(....................0, iyxf   )..(....................0, iyxf 
12
)........(1),( iidxdyyxf 





Vijaya Laxmi, Dept. of EEE

Joint Distribution function for
Continuous RV
• The joint distribution function of two RVs X
and Y is given by
 
 
dudvvufyYxXPyxF
x
u
y
v
 

),(,),(
  FunctionDensityyxf
yx
yxF



),(
,2

Marginal Distribution Function
• The marginal distribution function of X is given by
• The marginal distribution function of Y is given by
   



x
u v
dudvvufxFxXP ),()(1
• The marginal distribution function of Y is given by
   

 

u
y
v
dudvvufyFyYP ),()(2

Marginal Density Function
• The marginal density function of RV X is given by
• The marginal density function of RV Y is given by
  



v
dvvxfxF
dx
d
xf ),()(11
• The marginal density function of RV Y is given by
  



u
duyufxF
dy
d
yf ),()(22

• Or, If for all x and y, f(x,y) is the product of a function
of x alone and a function of y alone (which are the
marginal probability of X and Y), then, X and Y are
independent.
• If, f(x,y) cannot be expressed as function of x and y,
then, X and Y are dependent.

Independence
• If X and Y are two independent random variables, events that
involve only X should be independent of the events that
involve only Y.
• If A1 is any event that involve X only and A2 is any event that
involve only Y, then for discrete RVs,
     , yYPxXPyYxXP 
• In general, n random variables are independent,
when
• Knowledge about probabilities of RVs in isolation is sufficient to specify
the probabilities of joint events.
     
)()(),(,
,
21 yfxfyxfor
yYPxXPyYxXP


nXXX ,......, 21
       nnnn xXPxXPxXPxXxXxXP  ............,,, 22112211

• X and Y (continuous RVs)
• If, the events are independent events for
all x and y,
   yYandxX 
     , yYPxXPyYxXP 
)()(),(
),()(),(
21
21
yfxfyxf
andyFxFyxF



Properties of Joint CDF
• The joint CDF is non-decreasing in the
‘northeast’ direction, i.e.,
),(),( yyandxxifyxFyxF  21212211 ),(),( yyandxxifyxFyxF XYXY 
(x1,y1)
(x2,y2)
x
y

x
y
x1
  YxXPxFX ,)( 11

x
y
y1
 11 ,)( yYXPyFY 

• It is impossible for either X or Y to assume a value less than -
∞, therefore
• It is certain that X and Y will assume values less than infinity,
therefore
    0,, ,,  xFyF YXYX
therefore
• If, one of the variables approach infinity while keeping the
other fixed, marginal cumulative distribution functions are
obtained as
1),(, YXF
   
   yYPyYXPyxFyF
and
xXPYxXPyxFxF
YXY
YXX


,),()(
,),()(
,
,

• The joint CDF is continuous from the north and from
the east, i.e.,


ax
YXYX
and
yaFyxF ),(),(lim ,,


by
YXYX bxFyxF
and
),(),(lim ,,

Problem
• If X and Y are two discrete RVs, whose joint PDF
is given by
 
0
3020,2
),(


 

otherwise
yandxwhereyxc
yxf
 
 2,1)(
1,2)(
)(
0



YXPc
YXPb
caFind
otherwise

Solution: (a)
• We haveX Y 0 1 2 3 Total
0 0 C 2c 3c 6c
1 2c 3c 4c 5c 14c
2 4c 5c 6c 7c 22c2 4c 5c 6c 7c 22c
Total 6c 9c 12c 15c 42c
C=1/42

• (b)
• (c)
 
42
5
51,2  cYXP
  ),(2,1
1 2

  
yxfYXP
X Y
7
4
42
24
24
)654()432(


c
cccccc

Problem
• Find the marginal probability of (a) X and (b) Y

Solution
• Marginal Prob. function of X,
 













2
21
11
22
1
3
1
14
0
7
1
6
)()(1
xforc
xforc
xforc
xfxfxXP X
• Marginal Prob. function of Y,
 
















3
14
5
15
2
7
2
12
1
14
3
9
0
7
1
6
)()(2
yforc
yforc
yforc
yforc
yfyfyYP Y

Problem
• Show that the RVs X and Y are dependent

Solution
• If x and y are independent, then for all x and y
     yYPxXPyYxXP  ,
 
42
5
1,2  YXP
30
   
14
3
1,
21
11
2,  YPandXPBut
dependentareYandX

14
3
.
21
11
42
5

Problem
• If X and Y are two cont. RV, whose joint density
function is given by
0
51,40
),(


 

otherwise
yxforcxy
yxf
 
 2,3)(
32,21)(
)(


YXPc
YXPb
caFind

Solution
• (a)
• (b) 96
1
96
1),(
4
0
5
1
4
0
5
1











  
 
  




cc
dxxydyccxydxdy
dxdyyxf
x yx y
• (c)
 
128
5
96
32,21
2
1
3
2

   x y
dxdy
xy
YXP
 
128
7
96
2,3
4
3
2
1

   x y
dxdy
xy
YXP

Problem
• Find the Marginal distribution function of X
and Y

Solution
• Marginal Distribution function of X
 
1696
1
96
),()(
2
0
5
10
5
1
x
duuvdvdudv
uv
dudvvufxXPxF
x
u v
x
u v
x
u v
X

















  
 
  



34
.0)(,0
1)(,4
40,



xFx
xFxFor
xBecause
X
X











41
40
16
00
)(
2
xfor
xfor
x
xfor
xFX

• Marginal Prob. function of Y,
   

 


u
y
v
Y
y
dudvvufyYPyF
24
1
),()(
2
51,  yBecause
35
.0)(,1
1)(,5


yFy
yFyFor
Y
Y












51
51
24
1
10
)(
2
yfor
yfor
y
yfor
yFY

Change of Variables (Discrete RVs)
• Theorem 1: If X is a discrete RV having Prob. Function
f(x) and another RV U is defined by U=ø(X)
• For each value of X there corresponds one and only
one value of U,one value of U,
• Proof:
 )()(,..,
)(
ufugUoffnprobThen
UX




   
    UfuXP
uXPuUPug




)(
)()(

• Theorem 2: If X and Y is discrete RVs having joint
prob. Function f(x,y)
• Another RV U and V defined by
   YXVandYXU ,, 21  
• For each pair of values of X and Y there corresponds
one and only one pair of values of U & V
• Then, joint Prob. Function of U & V is given by
   VUYandVUX ,, 21  
    vuvufvug ,,,),( 21 

• Proof:
      
    
    vuvuf
vuYvuXP
vYXuYXPvVuUPvug
,,,
,,,
,,,,),(
21
21
21







Change of Variables (Continuous RVs)
• Theorem 1: If X is a continuous RV having Prob.
Function f(x) and another RV U is defined by U=ø(X),
where
• Then, prob. Density function of U is given by g(u),
)(UX 
Then, prob. Density function of U is given by g(u),
where
   uuf
du
dx
xfug
dxxfduug
'
)()()(
)()(



• Proof: If u is an increasing function i.e., if x increases
then u increases.
u
u2
u1
   
)()(
2 2
2121


  dxxfduug
xXxPuUuP
u
u
v
v
x1 x2
u1
x
   
    0)('')()(
)()(
2
1
2
1
1 1
'

 
 
uhereuufug
duuufduug
u
u
u
u
u v


This can be proved for 0)('0)('  uoru 

• Theorem 2:
• The joint density function g(u,v) of U and V is given
by
   
   VUYVUXwhere
YXVYXUDefine
,,,,
,,,
21
21




by
• Where,
    Jvuvufvug
dxdyyxfdudvvug
,,,),(
),(),(
21 

 
 
v
y
u
y
v
x
u
x
vu
yx
J












,
,

• Proof:
• If x and y increases then, u and v also increases
   21212121 ,, yYyxXxPvVvuUuP 
2 22 2 x yu v
This can be proved also for 0J
42
    
 


2
1
2
1
2
1
2
1
2
1
2
1
,,,
),(),(
21
u
u
v
v
x
x
y
y
u
u
v
v
Jdudvvuvuf
dxdyyxfdudvvug

     0,,,),( 21  JhereJvuvufvug 

Problem
• If X is a discrete RV with prob. Function
• Find the prob. Function of RV


 


otherwise
xfor
xf
x
0
....3,2,12
)(
14
XU
• Find the prob. Function of RV

Solution


uwhereuxgives
xu
.......,82,17,2,1
,1
4
4



 



otherwise
u
ug
uwhereuxgives
u
0
.....,82,17,2,2
)(
.......,82,17,2,1
4
1
4

Problem
• If X is a Cont. RV with density function
• Find the prob. Density Function for


 

otherwise
xforx
xf
0
6381/
)(
2
 XU  12
3
1
• Find the prob. Density Function for 3

Solution
 
2,6
,5,3
3'



ux
anduxFor
dx
du
u 
uxux
xu
312312
12
3
1


Check:
 
 

5
2
2
.1
27
312
du
u
46
 








otherwise
u
u
ug
0
52,
27
312
)(
2

Problem
• If X and Y are two cont. RVs whose joint density
function is given by






otherwise
yandx
xy
yxf
0
5140
96),(
• Find the density function of U=X+2Y
 otherwise0

Solution
 
102,4051,40
2
1
,
)(2



vuvyxrangethe
vuyvx
chosenyarbitrarilxvandyxu
v
u-v=2
u
u-v=2
u-v=10
v=4
v=0 102
I
II
III
2
1
2
1
2
1
10












v
y
u
y
v
x
u
x
J
 
otherwise
vvuvuvvug
0
40,102384/),( 

• Marginal density function of U is given by
ufordv
vuv
ufordv
vuv
ug
v
u
v
106
384
)(
62
384
)(
)(
4
0
2
0
1










 




   
 
 
otherwise
u
uu
u
u
u
uu
ufordv
vuv
uv
v
,0
1410,
2304
2128348
106,
144
83
62,
2304
42
1410
384
)(
384
3
2
4
10
0







 











Expected Value of function of random variables
• The expected value of Z=g(X,Y) can be obtained by
 



 
 
continuouslyjoYXdxdyyxfyxg
ZE
YX int,,),(
)(
,
   






  
RVsdiscreteYXyxpyxg
ZE
i
niYX
n
ni ,,,
)(
,

Sum of Random Variables
• Let Z=X+Y, Find E(Z)
    






''','''
)()(
, dydxyxfyx
YXEZE
YX
     








 ''',''''','' ,, dydxyxfydydxyxfx YXYX
The expected value of the sum of two random variables is equal to
the sum of individual expected values.
X and Y need not be independent.
51
     
  








  













 ''',''''','' ,, dydxyxfydxdyyxfx YXYX
   
)()(
''''''
YEXE
dyyfydxxfx YX

 





• The expected values of sum of n random variables is
equal to the sum of the expected values.
       nn XEXEXEXXXE  ........ 2121       nn XEXEXEXXXE  ........ 2121

Sum of Discrete RVs
• If X and Y are two discrete RVs
   
yxyfyxxf
yxfyxYXE
x y




),(),(
),(
   YEXE
yxyfyxxf
x yx y

  ),(),(

Expected value of Product of Two RVs
• If X and Y are two independent RVs
• Proof: (continuous RV)
     YEXEXYE 
   
 
   
 
   
   YEXE
dyyfydxxfx
dydxyfxfyx
dydxyxfyxXYE
YX
YX
YX

















 
 












''''''
'''()'''
''',''' ,

Product of two discrete RVs
• If X and Y are two independent RVs
)()(),( 21 yfxfyxf 
 
 
 


x y
yfxxyf
yxxyfXYE
)()(
),(
 
x y
yfxxyf )()( 21
55
  
   YEXE
YExxf
yyfxxf
x
x y










 
)(
)()(
1
21

Product of functions of Random variables
• If X and Y are two independent RVs and g(X,Y)=g1(X)g2(Y)
• Find
• Solution:
             dydxyfxfygxgYgXgE YX2121 ''''''  
 
       YgEXgEYXgE 21),( 
            
       
     YgEXgE
dyyfygdxxfxg
dydxyfxfygxgYgXgE
YX
YX
21
21
2121
''''''
''''''
















 




 

• In general, for n independent RVs,
nXXX ,........, 21
               nnnn XgEXgEXgEXgXgXgE ................. 22112211                nnnn 22112211

Conditional Probability Distribution
• For P(A)>0, Probability of event B given that A has
occurred
        )(|
)(
| APABPBAPor
AP
BAP
ABP 


• If X and Y are discrete RVs with events (A:X=x) and
(B:Y=y)
  Xofprobinalmtheisxfwhere
xf
yxf
xyf .arg)(,
)(
),(
| 1
1


• Conditional prob. function of Y given X is given by
• Conditional prob. function of X given Y is given by
 
)(
),(
|(
xf
yxf
xyf
X

• Conditional prob. function of X given Y is given by
 
)(
),(
|
yf
yxf
yxf
Y


Conditional Probability Distribution for
Cont. RV
• If X and Y are two cont. RVs, the conditional prob.
Density function of Y given X is given by
 
)(
),(
|
xf
yxf
xyf 
)( xf X

Problem
• The joint density function of two cont. RVs X and Y
are given by






otherwise
yxxy
yxf
0
10,10,
4
3
),3(
• Find (a)
• (b)
 otherwise0
 xyf






 dxXYP
2
1
2
1
2
1

Solution
• (a) For 0<x<1
  











 
10,
43
),(
4
23
24
3
4
3
)(
1
0
1
y
xy
yxf
xx
dyxyxf
• (b)
 





)(0
10,
23
)(
),(
1
definednototherwise
y
x
xf
yxf
xyf
16
9
4
23
2
1
2
1
2
1
2
1
2/1
2/1




















dy
y
dyyfdxXYP

Variance of Sum of Two RVs
• For two independent RVs X and Y,
Var(X+Y)=Var(X)+Var(Y)
• Proof:
     
     
)(
2
YXEYXVar YX  
     
       
         
           
     
)()(
2
2
2
22
22
22
22
2
YVarXVar
YEXE
YEYEXEXE
YEYXEXE
YYXXE
YXE
YX
YYXX
YYXX
YYXX
YX












• Var(X-Y)=Var(X)+Var(Y)
• Proof:
     
     
)(
2
2
YXE
YXEYXVar
YX
YX




     
       
         
     
)()(
2
2
22
22
22
YVarXVar
YEXE
YEYEXEXE
YYXXE
YXE
YX
YYXX
YYXX
YX










Variance of Sum of Two RVs
• In general,
XYYXYXor
YXCovYVarXVarYXVar
 2,
),(2)()()(
222




Correlation and Covariance of Two RVs
• The jkth joint moment of two RVs X and Y is defined
as
   
 
 









kj
YX
kjKj
ContinuouslyjoYandXFordxdyyxfyxYXE int,,
 
i n
niYX
k
n
j
i RVsdiscreteYandXyxpyx ,,
If j=0, moments of Y are obtained
If k=0, moments of X are obtained
If j=k=1, E[XY] gives the correlation of X and Y
IF E[XY]=0, X and Y are orthogonal

Jkth central moment about the mean
• The jkth central moment of X and Y about the mean is
given by
    k
Y
j
X YXE  
If j=2, k=0, variance of X is obtained
If j=0, k=2, variance of Y is obtained
If j=k=1, Covariance of X and Y is obtained
    YandXofianceCoYXCovYXE YX var),(  

Covariance of Two RVs
   
 
     
         YEXEYEXEXYE
YEXEXYE
YXXYE
YXEYXCov
YXXY
YXXY
YX




2
),(



         
     YEXEXYE
YEXEYEXEXYE

 2
Hence,
Cov(X,Y)=E[XY], if either of the RVs has mean value equal to zero

Problem
• If X and Y are two independent RVs each having
density function


 


otherwise
ufore
uf
u
0
02
)(
2
• Find
     XYEYXEYXE ,, 22


Solution
     
1
2.2.
0
2
0
2








dyeydxex
YEXEYXE
yx
  1
4 22
  
 

dxdyexyeXYE yx
Check E[X+Y]=E[X]+E[Y]
E[XY]=E[X]E[Y]
70
 
4
4
0 0
22
  

dxdyexyeXYE yx
  122
0
22
0
2222
 




dyeydxexYXE yx

Problem
• If X and Y are two discrete independent RVs







4/3.2
3/2.0
3/1.1
probwith
Y
probwith
probwith
X
• Find





4/1.3 probwith
Y
       YXEXYEYXEYXE 222
,,2,23 

Solution
     
3
1
0
3
2
1
3
1
XE
     
4
3
3
4
1
2
4
3
YE
72
     
2
5
4
3
2
3
1
3
2323













 YEXEYXE

     22
 YEXEYXE
     
4
1
4
3
.
3
1

 YEXEXYE
     
4
1
4
3
.
3
1

 YEXEYXE

     
3
1
0
3
2
1
3
1 22
XE
     
4
21
4
9
33
4
1
2
4
3 222
YE
74
     
12
55
4
21
3
2
4
21
3
1
2
22 2222













 YEXEYXE

Problem
• If X and Y are two independent RVs, find the Covariance
• Solution:
   
     
0
),(



YX
YX
YEXE
YXEYXCov


0
For the pairs of independent RVs, Covariance is zero.

Correlation Coefficient of X and Y
• The correlation coefficient between the two RVs, X and Y is
given by
     YEXEXYEYXCov
YXYX
YX
),(
,






deviationdardSYVarandXVarwhere YX tan)()(,  
11 ,  YX
X and Y are uncorrelated, if 0, YX
If X and Y are independent, Cov(X,Y)=0, 0,  YX
If X and Y are independent, they are uncorrelated.
-------From Theorem 4

Theorems on Covariance
• Theorem 1:
• Theorem 2: If X and Y are independent RVs
• Theorem 3:
     YEXEXYEXY 
0),(  YXCovXY
• Theorem 3:
• Theorem 4:
 
XYYXYXor
YXCovYVarXVarYXVar
 2,
),(2)()(
222



YXXY  
If X and Y are independent, Theorem 3 reduces to Theorem 4.
The converse of Theorem 3 is not necessarily true.

Problem
• If X and Y are two discrete RVs, whose joint density function is
given by
• Find E[X], E[Y], E[XY], E[X2], E[Y2], Var(X), Var(Y), Cov(X,Y) and
 
otherwise
yxforyxcyxf
0
30,202),( 
• Find E[X], E[Y], E[XY], E[X2], E[Y2], Var(X), Var(Y), Cov(X,Y) and
correlation coeff.

Solution
• We have
X Y 0 1 2 3 Total
0 0 C 2c 3c 6c
1 2c 3c 4c 5c 14c1 2c 3c 4c 5c 14c
2 4c 5c 6c 7c 22c
Total 6c 9c 12c 15c 42c

   
cccc
xfxyxfxyxxfXE
x y xx y
5822.214.16.0
)(,),(







   
      yyfyxfyyxyfYE
x y yy x
)(,, 





   
80
ccccc
x y yy x
7815.312.29.16.0 

   
c
ccc
yxxyfXYE
x y
102
........3.3.02.2.0.1.00.0.0
,


 

     
                ccccc
yxfyyxfyYE
x y y x
1921531229160
,,
2222
222







   
     
            cccc
yxfxyxfxXE
x y x y
10222214160
,,
222
222







   
81
      
      2222
2222
78192)(
58102)(
ccYEYEYVar
ccXEXEXVar
Y
X




     
YX
YX
YXCov
ccc
YEXEXYEYXCov



),(
78.58102
),(,




Problem
• If X and Y are two Cont. RVs
• Find (a) c
• (b) E[X]
 
 


 

otherwise
yxforyxc
yxf
0
50,622
,
• (c) E[Y]
• (d) E[XY]
• (e) E[Y2]
• (f) Var(X)
• (g) Var(Y)
• (h) Cov(X,Y)
• (i) Correlation Coeff.

Solution
 
 
210
1
12
1,sin
6
2
5
0



 
 
 




c
dxdyyxc
dxdyyxfgU
x y
    2681
6 5
 
83
   
63
268
2
210
1
6
2
5
0
   x y
dxdyyxxXE
   
7
80
2
210
1
  




dxdyyxxyXYE
   
63
170
2
210
1
  




dxdyyxyYE

  63
12202
XE
    
3969
5036
)(
222
 XEXEXVarX
7938
16225
)(2
 YVarY
  126
11752
YE
84
     
3969
200
),(  YEXEXYEYXCov
03129.0
YX
XY



7938
Y

Problem
• Let Ɵ be uniformly distributed RV in the interval
(0,2π) and X and Y are two RVs defined as X=Cos Ɵ,
Y=SinƟ.
• Show that X and Y are uncorrelated.Show that X and Y are uncorrelated.

Solution
• The marginal PDF of X and Y are arcsine functions which are
nonzero in the interval
• So, if X and Y were independent the point (X,Y) would assume
all values in the square.
• But, this is not the case, hence X and Y are dependent.
1111  yandx
• But, this is not the case, hence X and Y are dependent.
(CosƟ, SinƟ)
Ɵ
y
x
1
-1
-1
1

   
02
4
1
2
1
2
0
2
0











dSin
dCosSinCosSinEXYE
Since E[X]=E[Y]=0, X and Y are uncorrelated
Uncorrelated but dependent Random Variables

Conditional Expectation, Variance and
Moments
• The conditional expectation of a RV Y given X is expressed as
• Properties:
 If X and Y are independent,
    dxxXxmeansxXwheredyxyyfxXYE  


,
 If X and Y are independent,

   YExXYE 
        XYEEdxxfxXYEYE  


1

• Hence,
         YoffunctionaisYhwhereXYhEEYhE ,
    
and
thKKK
,

89
     origintheaboutmomentktheisYwhereXYEEYE thKKK
,

Theorem
    XYEEYE 
Proof:
      
   






dxxfXYE
dxxfXYEXYEERHS
X
:
   
 dxxfXYE X
90
   
 
 
  
 










dxdyxf
xf
yxf
y
dxxfdyxyyf
XY
XY
,
 
   YEdyyyf
dxdyyxfy



 






,

Problem
• If X and Y are two RVs whose density function is
given by
   
otherwise
yxforyxcyxf
0
30,202, 
• Find E[Y|X=2]

Solution
• The marginal density function of RV X is given by
• And, marginal density function of Y is given by
 









222
114
06
1
xforc
xforc
xforc
xf
• And, marginal density function of Y is given by
 












315
212
19
06
2
yforc
yforc
yforc
yforc
yf

• The conditional density function of Y given X=2 is
obtained as
   
 
 
22
4
42/22
42/2,
2
1
yyx
xf
yxf
yf




93
     




 

yy
y
yyyfXYE
22
4
22
       
11
19
22
7
3
22
6
2
22
5
1
22
4
0 

























Problem
• The average time of travel from city ‘A’ to city ‘B’ is ‘c’
hours by car and ‘b’ hours by bus. A man cannot
decide whether to drive the car or to take bus, so he
tosses a coin. What is the expected time of travel?

Solution
• X is RV, which is the outcome of toss
• Y is travel time 
1
0


XifY
XifYY car
The RVs X and Y are independent, because Ycar and Ybus
are independent of X
• Using Property 1,
• Using Property 2, (for discrete RVs)
•
1XifYbus
     
      bYEXYEXYE
cYEXYEXYE
busbus
carcar


11
00
         
2
1100
bc
XPXYEXPXYEYE




Conditional Variance
• The conditional variance of Y given X is defined
as
• The rth conditional moment of Y about any value
a is given as
      
 xXYEwhere
ydxyfyxXYE

 


2
2
2
2
2


a is given as
      


 dyxyfayxXaYE
rr
The usual theorems for variance and moments extend to conditional
variance and moments.

Gaussian PDF and CDF
 
 
2
2
2
2
1
: 

mx
X exfXofPDF


 
 



x mx
dxexXPCDF '
2
1
:
2
2
2
'


FX(x) is the CDF of Gaussian RV with zero mean and unit variance.
97
 

mxtLet  '
 
 





 

 







mx
dtexFThen
mx
t
X
2
2
2
1
,
  


x
t
dtexwhere 2
2
2
1



Gaussian PDF
• Show that Gaussian PDF integrates to one.
• Solution:
• Take Square of PDF of Gaussian RV











dyedxedxe
yxx
22
2
2
222
11

 
 












dxdye
dyedxedxe
yx
2
22
2
1
22


98
1
2
1
,
0
2
0
2
0
2
22



 




drrerdrde
rSinyrCosxLet
rr





Joint Characteristic Function
• The Joint Ch. Fn of n RVs, is given by
• And, the joint ch. fn. of two RVs X and Y is given by
   
 nn
n
XXXj
nXXX eE 
 
 ...
21.....
2211
21
,...,,
nXXX ,...,, 21
   
 
 eE YXj
YX
21
, 21,


   
 





 dxdyeyxf yxj
YX
21
,,

Joint Characteristic function is the 2-dimensional Fourier
transform of joint PDF of X and Y
     
 





 2121,2,
21
,
4
1
, 


ddeyxf yxj
YXYX

Marginal Characteristic Function
   
   

,0
0,
XYY
XYX


If X and Y are independent
   
   
       21
21
21
2121
,




YX
YjXj
YjXjYXj
XY
eEeE
eeEeE

 

   
   
    


 













21
0
2
0
1
21
!!
, 21
ki
ki
k
k
i
i
YjXj
XY
jj
YXE
k
Yj
i
Xj
E
eeE


 
    
 

0 0
21
!!i k
ki
k
j
i
j
YXE

101
    0,021
21
21
|,
1
,



 

XYki
ki
ki
ki
j
YXE
And

Problem
• If Z=aX+bY, find the ch. Fn. Of Z
• Solution:
   
   
 
 
 
ba
eEeE
XY
bYaXjbYaXj
Z
,
 
• If X and Y are independent
        baba YXXYZ  ,

Problem
• If U and V are independent zero mean, unit
variance Gaussian RV, and
• X=U+V and Y=2U+V
• Find the joint ch. Fn of X and Y and find E[XY]• Find the joint ch. Fn of X and Y and find E[XY]

Solution
• The joint Ch. Fn. Of X and Y is given by
• The joint Ch. Fn. Of U and V is given by
   
      
 
   
 )2(
2
21
2121
2121
,
VUj
VUVUjYXj
XY
eE
eEeE







   
   
 2
, 
 
 eEeE VjUj
   
   
 
   
   2
21
2
21
2121
2
1
2
2
1
2121
2
21
2
,









ee
eEeE
VU
VjUj
XY
  2/22

 
 jm
X e
104
   
3
|,
1
0,021
21
2
2 21



  
 XY
j
XYE
We have

Jointly Gaussian RV
• The RVs X and Y are said to be jointly Gaussian, if
their joint PDF has the form
 
 
2
2
2
2
2
2
1
1
,
2
1
1
2
,
12
2
12
1
exp
,
YX
YX
XY
mymymxmx
yxf


























 





 





 





 



• for
•
2
,21 12 YX 
 yandx

Problem
• The PDF for jointly Gaussian RVs is given by
 








16168
3
8
3
16
3
4
2
1
,
22
2
1
,
yx
xy
yx
YX eyxf

• Find E[X], E[Y], Var(X), Var(Y) and Cov(X,Y)

Problem
• The joint PDF of X and Y is given by
• Find the marginal PDF.
      

 
yxeyxf yxyx
YX ,,
12
1
,
222
12/2
2,


• Find the marginal PDF.

Solution
• The marginal PDF of X is obtained by integrating f(x,y) over y as
 
 
   
 
     






2,
12
12
2222212/
2
12/
12/2
2
12/
2222
22
22
22
xxy
x
xyy
x
X
xxxyyHeredye
e
dye
e
xf














   
 





2
122
2,
12
2
2
12/2
2
2
22
2
x
xy
x
e
dy
ee
xxxyyHeredye














• Hence
 The last integral equals one because its integrand is a Gaussian
PDF with mean and variance
 The marginal PDF of X is a one-dimensional Gaussian PDF with mean
x 2
1 
 The marginal PDF of X is a one-dimensional Gaussian PDF with mean
0 and variance 1. Form symmetry, the marginal PDF of Y will also be a
Gaussian PDF with mean 0 and variance 1.

N jointly Gaussian RV
• The RVs are said to be jointly Gaussian RVs
if their joint PDF is given by
•
nXXX ,...,, 21
 
   
 
 









n
T
nXXXX
bydefinedvectorscolumnaremandXwhere
K
mXKmX
xxxfXf n
,
2
2
1
exp
,...,,)(
2
1
2
1
21,...,, 21

•
 
 
 
     
     
      



































































nnn
n
n
nnn
XVarXXCovXXCov
XXCovXVarXXCov
XXCovXXCovXVar
Kand
XE
XE
XE
m
m
m
m
x
x
x
X
..,,
.....
.....
,..,
,..,
,
.
.
.
.,
.
.
21
2212
1211
2
1
2
1
2
1
Where, K is called the Covariance Matrix 110Vijaya Laxmi, Dept. of EEE

Module iii sp

More Related Content

What's hot

Similar to Module iii sp

More from Vijaya79

Recently uploaded

Module iii sp