Module v sp

Module 5
Stochastic Processes
1

Stochastic Processes
• The outcome of a random experiment is a function of time or
space
• Examples:
 Speech recognition system (based on voltage waveforms)
 Image processing system (intensity of pixels is a function of Image processing system (intensity of pixels is a function of
space)
 Queuing system (no. of customers varies as a function of time)
 Based on temperature demand of electricity varies
2Vijaya Laxmi, Dept. of EEE

• In electrical engineering, voltage or current are used
for collecting, transmitting and processing
information as well as for controlling and providing
power to a variety of devices.

Signal
• These are functions of time and belongs to two
classes:
 Deterministic: These are described by functions in
the mathematical sense with time ‘t’ as independentthe mathematical sense with time ‘t’ as independent
variable
 Random signal

Random signal
 This always has some element of uncertainty associated in it
and hence it is not possible to determine exact value at the
given point of time.
 Example: Audio waveform transmitted over a telephone
channel.channel.
i.e. , we cannot precisely specify the value of random signal in
advance. However, may describe its average properties such
as average power, spectral distribution, probability that the
signal amplitude exceeds a certain value etc.

• The probabilistic model used for characterizing a
random signal is called random process or stochastic
process.
• This deals with time varying waveforms that have
some element of chance or randomness associatedsome element of chance or randomness associated
with them.
• Example: data communication system in which a
number of terminals are sending information in
binary format over noisy transmission links to a
central computer.

Observation
• By observing the waveform of x1(t) for [t1,t2], we cannot with
certainty predict the value of xi(t) for any other value of
• The knowledge of one member function xi(t) will not enable
us to know the value of another member function xj(t).
 21,ttt 

• We should use a probabilistic model to describe or
characterize the ensemble of waveforms so that to answer
a) what are the spectral properties of ensemble of waveforms?
b) how does the noise affects the system performance as
measured by the receivers ability to recover the transmittedmeasured by the receivers ability to recover the transmitted
data correctly?
c) what is the optimum processing algorithm, the receiver
should use?

Example
• Tossing of N coins simultaneously and repeating N
tossings once every T seconds.
• Draw the waveforms.

Random Variable vs Random Process
• A random variable maps the outcome of random
experiment to a set of real numbers, similarly
• A random process maps the outcome of a random
experiment to a set of waveforms or functions ofexperiment to a set of waveforms or functions of
time.

• Suppose there is a large number of people, each flipping a fair
coin every minute. If we assign the value 1 to a head and the
value 0 to a tail.

Tossing of coins

Example: Tossing of die
• For tossing of a die
22
4224)(
654321
tttX 

• The set of waveforms
is called an ensemble.
 )(.....,),(),( 621 txtxtx

• For specific value of time, t0 , is collection of
numerical values of various member function at t=t0.,
where t is time and represents an outcome in
sample space S
),( 0 tX


           
   
           
0
0020100
21
,....,,|,,.3
mindet,.2
,....,,|,,.1
ttatfunctionmemberofvaluesnumericalofcollection
txtxtxStXtX
timeoffunctionisticertxtX
timeoffunctionsofCollection
txtxtxStXtX
nii
ii
nii







    000
0
,.4 ttatfunctionmemberitheofvaluenumericaltxtX
ttatfunctionmemberofvaluesnumericalofcollection
th
ii 



Problem
• Tossing of die
• Find      
 0)0(|2)4(
2)4(,0)0(,0)4(,2)4(


XXP
XXPXPXP

Solution
• Let A be the set of outcomes such that
• (a)
Ai 
   
   
3
1
6
2
2)4(
5,22,4


APXP
AX i
• (b)
• (c)
• (d)
36
 
2
1
6
3
)(0)4(  APXP
   
6
1
,5  BPB
   
  2
1
6
2
6
1
0)0(
0)0(,2)4(
0)0(|2)4( 



XP
XXP
XXP

Classification of Random Process
X(t) t
Continuous Discrete
Continuous Continuous Random
Process
Continuous Random
Sequence
Discrete Discrete Random Discrete RandomDiscrete Discrete Random
Process
Discrete Random
Sequence

• Stationary Random Process:
The probability distribution function or averages do not
depend upon time ‘t’.
• Non Stationary Random Process:• Non Stationary Random Process:
The prob. distribution function or averages depend on time
‘t’.

Based on observation of past values
• Predictable,
and
• Unpredictable• Unpredictable

• Real Valued RP
• Complex valued RP
If a RP Z(t) is given by
 
RPsvaluedrealthearetandtAfrequencycarriertheisfwhere
ttfCostAtZ c
)()(,
,)(2)()(

 
• Z(t) =real part of
= real part of
Complex envelope
• Here, W(t) is complex valued RP and X(t), Y(t) and Z(t) are Real valued
RPs.
RPsvaluedrealthearetandtAfrequencycarriertheisfwhere c )()(, 
    tfjtjtA c 2exp)(exp)(
  tfjtw c2exp)(
)()(
)()()()()(
tjYtX
tSintjAtCostAtw

 

Definition of Random Process
• A real valued RP X(t), is a measurable
function on that maps onto R .
Tt
S
.var
,
1 linerealRsetinvalueswithiablet
andSpaceSampleS


function on that maps onto R1.
• If is a set of one or more intervals on the real
line, X(t) is called Random Process.
• If is a subset of integers, X(t) is called
Random Sequence.
S S



• A real valued RP X(t) is described by nth order
distribution function
        

 nnnXXX
tttandnallfor
xtXxtXxtXPxxxF n
,...,,
,...,,,...,, 221121....21
• It satisfy all requirements of joint probability
distribution function (CDF).
ntttandnallfor ,...,, 21

Methods of Description
1. Joint Distribution:
• First order dist. Fn. is given by , which
gives the idea about instantaneous amplitude
distribution of the process.
  11 atXP 
distribution of the process.
• Second order dist. Fn. is given by
which gives the information about the
structure of the signal in time domain.
    2211 , atXatXP 

Problem : Tossing of die
22
4224)(
654321
tttX 

The outcomes and the corresponding waveforms are given by
Find
    60 XandXPJoint probability
Find
Marginal probabilities P[X(0)] and P[X(6)]

Values
of X(0)
Values of X(6) Total
-4 -3 -2 2 3 4
-4 1/6 0 0 0 0 0 1/6
-2 0 0 1/6 0 0 0 1/6
0 0 1/6 0 0 1/6 0 2/6
MarginalProb.OfX(0)
2 0 0 0 1/6 0 0 1/6
4 0 0 0 0 0 1/6 1/6
Total 1/6 1/6 1/6 1/6 1/6 1/6 1
Grand Total
Marginal Probability of X(6)
MarginalProb.OfX(0)

2. Analytical description of RP using Random
Variables
• A RP Y(t) is expressed as
 
iablesrandomareandAWhere
tCosAtY
var
,10)( 8



3. Average Values
• Mean,
• Autocorrelation,
• Autocovariance,
    tXEtX 
      
XofconjugatetheisXwhere
tXtXEttRXX
*
21
*
21 , 
       21
*
2121 ,, ttttRttC XXXXXX • Autocovariance,
• Correlation Coefficient,
       212121 ,, ttttRttC XXXXXX 
   
   
   111
2211
21
21
varvar,
,,
,
,
tXiablerandomofiancetheisttCwhere
ttCttC
ttC
ttr
XX
XXXX
XX
XX 

Problem
• Tossing of die
• Find      212121 ,,,,),( ttrandttCttRt XXXXXXX

Solution
• We have
       0
6
1 6
1
 i
iX txtXEt
           
6
212121
6
1
, txtxtXtXEttR iiXX
X is real, hence conjugate is omitted
31
          
 













 
21
2121
1
212121
2
1
40
6
1
4
1
4
1
164416
6
1
6
,
tt
tttt
txtxtXtXEttR
i
iiXX
Vijaya Laxmi, Dept. of EEE

• The autocovariance and correlation coefficient is
given by
   2121 ,, ttRttC XXXX 
32
 
2
2
2
1
21
21
2
1
40
2
1
40
2
1
40
,,
tt
tt
ttrand XX




Problem
• A RP X(t) is given by
 
RVstindependenareandA
inddistributeuniformlyis
ianceandmeanwithRVisAwhere
tACostX



],[
1var0
100)(


• Find
RVstindependenareandA 
    11,ttRandt XXX

Solution
• We have
       0100   tCosEAEtX
      
    100100100
,,
2
2121
A
tACostACosE
ttandttwheretXtXEttRXX





It is a function of τ and periodic in τ
If a process has a periodic component, its autocorrelation
function will also have a periodic component with same period.
34
    
]sec[100
2
1
2100200100
2
2
zerotoequalispartondasCos
tCosCos
A
E











4. Two or more RPs
• If X(t) and Y(t) are two RPs
• Cross Correlation Function
            
functionondistributiJocalledis
ytYytYytYxtXxtXxtXP nnnn
int
,...,,,,....,, '
2
'
21
'
12211 
      *
, tYtXEttR • Cross Correlation Function
• Cross Covariance Function
• Correlation Coefficient
      21
*
21, tYtXEttRXY 
       21
*
2121 ,, ttttRttC YXXYXY 
   
   2121
21
21
,,
,
,
ttCttC
ttC
ttr
YYXX
XY
XY 

• Equality: If their respective member functions are identical
for each outcome , the two RPs are called equal.
• Uncorrelated:
• Orthogonal:
• Independent:
   2121 ,,0, ttforttC XY
   2121 ,,0, ttforttRXY
• Independent:
        
         



''
2
'
121
'
1
'
111
'
1
'
111
,...,,,,..,,,
,...,,......,
,...,;,......,
mn
mmnn
mmnn
ttttttandmnallfor
ytYytYPxtXxtXP
ytYytYxtXxtXP
Independent implies uncorrelated but converse is not true.

Problem
• If Ɵ is uniformly distributed RV in [0,2π]
and X=CosƟ , Y=SinƟ
• Show that X and Y are uncorrelated.

Solution
• We have,
   
2
1
2







dCosSin
CosSinEXYE
    0 YEXE
02
4
1
2
2
0
0









dSin
dCosSin
Hence, X and Y are uncorrelated.

Problem
• Suppose X(t) is a RP with
  212.0
21 49,
,3)(
tt
ettR
t



• Find the mean, variance and covariance of RVs, Z and
W, If Z=X(5) and W=X(8).

Solution
   
    38
35




WE
ZE    
    138,8
135,5
2
2


RWE
RZE
         
          4949888,88,8
4949555,55,5




RCWVar
RCZVar
40
    195.1149498,5 6.032.0
 
eeRZWE
       
195.243349
858,58,5
6.06.0



ee
RC 

Stationarity
• It describes the time invariance of certain properties of a RP,
whereas individual member functions of a RP may fluctuate
rapidly as a function of time, the ensemble averaged values
such as, mean of the process might remain constant with
respect to time.respect to time.
• A process is said to be stationary, if its distribution function or
certain expected values are invariant with respect to a
translation of the time axis.
• Example: Signal from radar used for both searching and
tracking is non stationary.
Considering only searching time is stationary RP.

Types of stationarity
• Strict sense stationarity
• Wide sense stationarity

Strict sense stationarity
• A RP X(t) is called Strict sense stationary (SSS) or stationary in the strict
sense, if all of the distribution functions describing the process are
invariant under a translation of time.
i.e., for
nn tttttt   ,...,,,,...,, 2121
• If the above equation holds for all kth order distribution function
k=1,2,…,N, but not necessarily for k>N, the process is called Nth order
stationary.
      
      kk
kk
nn
xtXxtXxtXP
xtXxtXxtXP
kalland



 ,...,,
,...,,
...,2,1
2211
2211
2121

First and second order dist. Fn.
• First order distribution
• Second order distribution
       anyforxtXPxtXP 
The first order distribution is independent of t
…(i)
Second order distribution
• From (i) and (ii),
           anyforxtXxtXPxtXxtXP 22112211 ,, 
The second order distribution is strictly a function of time difference (t2-t1)
…(ii)
  
      1221
*
tan
ttRtXtXE
tConstXE
XX
X

 

• Two real valued RPs X(t) and Y(t) are jointly SSS, if
their joint distribution are invariant under a
translation of time.
• A complex RP, Z(t)=X(t)+jY(t) is SSS, if the processesA complex RP, Z(t)=X(t)+jY(t) is SSS, if the processes
X(t) and Y(t) are jointly stationary in strict sense.

Wide sense stationarity
• A RP X(t) is said to be stationary in wide sense (WSS or weakly
stationary), if its mean is a constant and the autocorrelation
function depends only upon the time difference.
  
      

XX
X
RtXtXE
tXE


*
• Two processes X(t) and Y(t) are called jointly WSS, if
• For random sequences,
       XXRtXtXE 
       XYRtYtXE *
  
      kRknXnXE
kXE
XX
X


*

SSS implies WSS, but the converse is not true.

Problem
• If
• Check for stationarity.
            .5,3,1,1,3,5 654321  tXtXtXtXtXtX

Solution
   0tXE
      
 
6
70
25911925
6
1
, 2121

 ttRtXtXE XX
Stationary in strict sense, since the translation of time axis
does not result in any change in member function.

Problem
• If
• Check for stationarity.
     
      6,3,3
3,3,6
654
321


tytCostytCosty
tSintytSintyty

Solution
• We have
   0tYE
         
 
122121 1872
6
1
, ttCosttRtYtYE YY


Stationary in wide sense.
50
 12 ttRYY 

Problem
• Establish the necessary and sufficient condition for
stationarity of the process
tbSintaCostX  )(

Problem
• If X(t) is a RP given by
 
   
   BEAE
BEAE
tSinBtCosAtX
ii
ii
n
i
iiii
,0
)(
222
1


 


• Discuss about stationarity.
   
ondistributiGaussianjohave
andeduncorrelatareniforBandA
BEAE
ii
ii
int
,...,2,1
 

• A RP X(t) is asymptotically stationary, if distribution function
of does not depend on τ, when τ is large.
• A RP X(t) is stationary in an interval, if
       ntXtXtX ....,,, 21
         ,....,,..., 1111
whichforallfor
xtXxtXPxtXxtXP kkkk

 
• A RP X(t) is said to have independent increments, if its
increments form a stationary process for every
τ. Example: Poisson and Wiener Process.
• A RP is cyclostationary or periodically stationary, if it is
stationary under a shift of the time origin by integer multiple
of a constant T0 (which is the period of the process).
.int,...,, 21 ervalaninliettt
whichforallfor
k 


)()()( tXtXtY  

Time Averaging and Ergodicity
• It is a common practice in laboratory to obtain multiple
measurements of a variable and average them to reduce
measurement errors.
• If value of a variable being measured is constant, and the
errors are due to noise or due to the instability of the
measuring instrument, then averaging is indeed a valid andmeasuring instrument, then averaging is indeed a valid and
useful technique.
• Time averaging is used to reduce the variance associated with
the estimation of the value of a random signal or the
parameters of a random process.
• Ergodicity is a relationship between statistical average and
sample average.

Ergodicity
• It is a property of stationary RP with the assumption that time
average over one member function of a RP is equivalent to an
average over the ensemble of functions.
• After a sufficient length of time, the effect of initial conditions
is negligible.is negligible.
• If one function of the ensemble is inspected over a long
period of time, all salient characteristics of the ensemble of
functions will be observed.
• Any one function can be used to represent the whole
ensemble.

• It can be interpreted in terms of prob. dist. fn.
• If
• Then, 1st prob. dist. fn. p(y)is the probability that at
any given time t1, any function fj(t) lies between y
      ensembleoffunctionsmembertftftf n ,...,, 21
any given time t1, any function fj(t) lies between y
and y+dy.
• p(y) is the probability that any function fj(t) at any
time lies between y and y+dy called ergodic property.

• Statistical characteristics of an ensemble can be
determined by consideration of any one function.
• The ensemble average is the average taken at a fixed
time for a large number of member functions of thetime for a large number of member functions of the
ensemble.
• The time average is the average taken with large
number of widely separated choices of initial time
and using only one function of the ensemble.

Special types of random processes
• Poisson process
• Random Walk process
• Wiener process
• Markov process• Markov process

Poisson process
• It is a continuous time, discrete amplitude random
process.
• It is used to model phenomena such as emission of
photons from a light emitting diode, arrival ofphotons from a light emitting diode, arrival of
telephone calls, occurance of failures etc.
• A counting function Q(t) is defined for
is the number of events that have occurred during
time period.
),0[ t

• Q(t) is an integer valued RP and said to be Poisson process if
the following assumptions hold:
• For any time number of events Q(t2)-
Q(t1) that occur in the interval t1 to t2 is Poisson distribution
as:
1221, ttandtt 
as:
 Number of events that occur in any interval of time is
independent of number of events that occur in other non-
overlapping time intervals.
           ...2,1,0,exp
!
12
12
12 

 kfortt
k
tt
ktQtQP
k



• We have,
• The autocorrelation of Q(t) is obtained as
      
      ttQVarttQE
kt
k
t
ktQP
k





,
...,2,1,0,exp
!
      , tQtQEttR       
         
          
    
 
  





212121
2
1221
121
2
1
2
1
1211
2
121211
2121
,,min.
1
,
ttallfortttt
ttfortt
ttttt
tQtQEtQEtQE
ttfortQtQtQtQE
tQtQEttRQQ



    
    222
222
XEXE
XEXE




It is not a martingale, since the mean is time varying.

Process with independent increments
• A RP X(t), is said to have independent increments, if for
all times the RVs,
are mutually
independent.
t
...,4,3....21  kandttt k
           12312 ....,,  kk tXtXandtXtXtXtX

Markov Process
• A RP X(t), is called a first order Markov process,
if for all sequences of times
t
...,2,1,0.....21  kandttt k
           111 ..........   kkkkkk tXxtXPtXtXxtXP
i.e., conditional probability distribution of X(tk) given for all past
values of X(t1)=x1,…..,X(tk-1)=xk-1 depends upon the most recent
value of X(tk-1)=xk-1.

Martingale
• A RP X(t), is called a Martingale, ift
  
       2112112 ; ttallfortXtttXtXE
andtallfortXE


i.e., Constant mean
It plays an important role in prediction of future values of random
processes based on past observation.

Random Walk Process
• It is a discrete version of Wiener process used to model the
random motion of a particle.
• Assumptions:
 A particle is moving along a horizontal line, until it collides
with another particle.with another particle.
 Each collision causes the particle to move ‘up’ or ‘down’ from
its previous path by a distance ‘d’.
 Collision takes place once every T seconds and movement
after the collision is independent of its position.
It is analogous to tossing a coin once every T seconds and taking a step
‘up’ if head show and ‘down’ if tail show, called Random Walk.

Sample function of Random walk process
d
X(n)
2d
d
-d
-2d
-3d
t/T=n

• Position of particle at t=nT is a random sequence
X(n).
• Assume X(0)=0 and jump of appears instantly
after each toss.
d
after each toss.

• If k heads show up in the first n tosses, then the position of
the particle at t=nT is given by
  
 
nnkwheremd
dnk
dknkdnX
,1...,,2,1,0,
2
)(




• If, the sequence of jumps is denoted by a sequence of random
variables {Ji}, then X(n) can be expressed as
• The RVs Ji, i=1,2,3,…,n are independent and have identical
distribution functions with
nnnnnm ,2...,,4,2, 
  nJJJnX  ...21
   
    2
22
2
2
,0
2
1
,
2
1
d
dd
JEJE
dJPdJP
ii
ii





• We have,
• The number of heads in n tosses has Binomial distribution,
hence
   
2
,)(
nm
ktossesninheadskPmdnXP


hence
  
  
     
2
2
21
2
...
0
2;...,,2,1,0,
2
1
2
1
2
1
nd
JJJEnXE
nXE
and
nkmnk
k
n
k
n
mdnXP
n
nknk




































• The autocorrelation function of random walk sequence is given by
• If n2>n1, X(n1) and X(n2)-X(n1) are independent RVs, hence the number of
heads from the 1st to n th tossing is independent of the number of heads
      
         
          121
2
1
1211
2121 ,
nXnXnXEnXE
nXnXnXnXE
nXnXEnnRXX



2 1 2 1
heads from the 1st to n1
th tossing is independent of the number of heads
from (n+1)th tossing to the n2
th tossing. Hence,
• If n1>n2,
• Hence,
            
  
2
1
2
1
121
2
121,
dn
nXE
nXnXEnXEnXEnnRXX



  2
221, dnnnRXX 
    2
2121 ,min, dnnnnRXX 
Random Walk is a Markov sequence and a Martingale.

Gaussian Process
• A RP X(t), is called a Gaussian process, if all its
nth order distribution function
are n-variate Gaussian distributions
t
 nXXX xxxF n
,...,, 21.....21
are n-variate Gaussian distributions
• If the Gaussian process is also a Markov process, it is
called a Gaussian-Markov process.
  iin tXXandttt ,......,, 21

Wiener process
• Define Y(t) as continuous random process for from
the random sequence X(n) as
),0[ t






,....2,1)1(,)(
0,0
)(
nfornTtTnnX
t
tY
 
nTtAt


Y(t) is Broken line of sample function of random walk process.
• Wiener process is obtained from Y(t) by letting time (T)
between jumps and the step size (d) approach zero with
constraint d2 =αT to assume that variance will remain finite
and nonzero for finite values of t.
72
 
   2
2
2
,
,0)(,
nd
T
td
tYEVariance
andtYEMean



Properties of Wiener process
• W(t) is a constant amplitude, continuous time, independent
increment process.
• It has
• It has Gaussian distribution
  






t
w
t
wfW
 2
exp
2
1 2
      ttWEandtWE  2
0
• For any value of t’, ,the increment w(t)-w(t’) has a
Gaussian PDF with zero mean and variance
• Autocorrelation of W(t) is
tt  '0
 'tt 
  




tt
wfW
 2
exp
2
   2121 ,min, ttttRXX 
Wiener process is a nonstationary Markov process and a Martingale.

Correlation Function
• The core of statistical design theory is mean square
error criterion.
• The synthesis should aim towards minimization of
mean square error between actual output andmean square error between actual output and
desired output.
• The input is assumed as stationary time series
existing over all time.

• The mean square error is expressed as
• To express it in terms of system characteristic and
    

T
T
d
T
dttftf
T
e
2
0
2
2
1
lim
• To express it in terms of system characteristic and
input signal, f0(t) is replaced by fi(t) and g(t), the unit
impulse response.
• The convolution theorem states that
      ,0  dtfgtf i  


75
 
 
 sG
sF
sF
i
0

• The mean square error is then expressed as
     
2
2
2
1
lim  










T
T
di
T
tfdtfgdt
T
e 
                  














T
T
didii
T
tfdtfgtfdtfgdtfgdt
T
2
2
2
1
lim 
76
       
       
 












T
T
d
T
id
T
T
T
ii
T
T
T
dttf
T
dtfgtfdt
T
dtfgdtfgdt
T
2
2
1
lim
2
1
lim2
2
1
lim


       
       
 














T
T
d
T
d
T
T
i
T
T
T
ii
T
dttf
T
dttftf
T
dg
dttftf
T
dgdg
2
2
1
lim
2
1
lim2
2
1
lim



• The fi(t) and fd(t) are in the form of an averaging of the
product of two time functions.
• If
•
     dttftf
T
T
T
ba
T
ab 

 
2
1
lim
           02
2
ddidii dgdgdge  

           02 ddidii dgdgdge   

Correlation function of statistics
Auto correlation function of input signal fi(t)
Auto correlation function of desired output
Cross correlation function between input signal and
desired output
77
 
 
 
 







id
dd
ii
Where

Measurement of Autocorrelation function
• The meter will read the autocorrelation function for one
particular value of τ, since wattmeter performs multiplication
and averaging.
• For the plot of autocorrelation function, delay of line can be
varied and a number of discrete readings are taken.varied and a number of discrete readings are taken.
• It is qualitatively a measure of regularity of the function.
• If there is no DC component in the signal, the autocorrelation
function will be small if the argument τ is taken larger than
the interval over which values of the function are strongly
dependent.

• Autocorrelation function for any argument τ is the average of
the product of e1 and e2 values of the function τ seconds apart
and average is given by:
    




 21212111 , dedeeepee 
• The expected or average value is summation or integration of
all products multiplied by their respective probabilities.
 
is the probability of any given product having 1st term between e1
and e1+de1 and 2nd term between e2 and e2+de2
79
  2121 , dedeeepwhere 

Properties of Autocorrelation Function
• It is an even function of τ, i.e., .
Since the functions are averaged over a doubly infinite
interval, the averaged product is independent of the direction
of the shift.
         dttftf
T
dttftf
T
T
T
T
T
T
T  



 111111
2
1
lim
2
1
lim 
     1111
of the shift.
• is autocorrelation function with zero argument, i.e.,
average power of time function.
If function f1(t) represents the voltage across a 1 Ω
resistor or the current through a 1 Ω resistor, autocorrelation
function is the power consumed by the 1 Ω resistor .
 011

   01111  
o The Maximum value of autocorrelation function appears, when
function is multiplied by itself without shifting
               tftftftftftf 2
1
2
1
2
1111
2
1
,sidesbothontakenAverage
81
           00
2
1
,
1111
2
1111   tftf
sidesbothontakenAverage
      0
2
1
11
2
11   tftf
        
   0
2
1
0
1111
2
111111



 tftf

• If the signal contains periodic components (or DC value), the
autocorrelation function contain components of the same
periods (or a DC component), i.e., a periodic wave shifted by
one period is indistinguishable from the unshifted wave.
• If the input signal contains only random components (no• If the input signal contains only random components (no
periodic components), the autocorrelation function tends to
zero as τ tends to infinity. As the shift of time function
becomes very large, the two functions f1(t) and f1(t+τ)
becomes essentially independent.

• Aurocorrelation function is equal to the sum of the
autocorrelation functions of the individual frequency
components, since the multiplication of components of
different frequency results in zero average value, i.e., voltage
and current of different frequencies result in zero averageand current of different frequencies result in zero average
value.
• A given autocorrelation function may correspond to an
infinite number of different time functions.
• Autocorrelation function of the derivative of f(t) can be
expressed in terms of autocorrelation function of f(t) as,
     


T
T
T
tftf
T
 '
1
'
1
2
1
lim
83
    ''
11

Power Spectral Density (PSD)
• If input is stationary time series and minimization of mean
square error is used as design criterion, signals are described
by correlation functions.
• The correlation functions are sufficient data for synthesis of a
minimum mean square error system.minimum mean square error system.
• It is convenient to describe the input signal in frequency
domain characteristics.
• If the autocorrelation function defines the system adequately,
then frequency domain function must carry information
contained in the autocorrelation functions.

• A function satisfying such requirements in Laplace
transform of autocorrelation function,
• To determine what characteristics of random input
     
des s


 1111
• To determine what characteristics of random input
signal are measured by the frequency function,
consideration of Laplace transform is significant.
• If fp(t) is periodic,
  



n
tjn
np eatf 
85
 
 






Tt
t
tjn
pn
Tt
t
p
dtetf
T
a
dttf
T
a
0
0
0
0
1
1
0


Amplitude spectrum
• The energy in signal is concentrated at isolated frequencies.
|an|
n
Indicates actual amplitude of signal component
at the corresponding frequency
• The energy in signal is concentrated at isolated frequencies.
• But, in aperiodic signals conversion from Fourier series to
Fourier transform of Laplace integral transformation involves
a limiting process and spectrum is called Amplitude density
spectrum.

f1(t)
w
t
|F1(iw)|
   
  221
11
1
1
0
00











 
a
jF
as
sF
tfore
tfor
tf at
• If f1(t) is the voltage across 1 ohm resistor, plot of |F1(jw)|2 vs
w is called the Energy density spectrum, i.e., direct indication
of energy dissipated in 1ohm resistor as a function of
frequency.
• The area under the curve between w1 and w2 is proportional
to the total energy at all frequencies within these limits.

• Total energy is proportional to the area under the entire
curve, i.e.,
 










a
d
a
djF
2
11
2
1
2
1
22
2
1





• Thus, spectrum of periodic wave is called the amplitude
spectrum and the spectrum of aperiodic waves is called the
amplitude density spectra.
• But, random time functions, can be characterized by Laplace
transform of autocorrelation function.

 aa 22 

• Hence,
• Consider f (t) to exist over the time interval –T to T instead of
   
   
   dttftfde
T
dttftf
T
de
des
T
T
s
T
T
T
T
s
s

























11
11
1111
2
1
lim
2
1
lim
• Consider f11(t) to exist over the time interval –T to T instead of
over all time,
• Laplace transform of f11(t) exists, the order of integration is
interchanged
     dttftfd
T
s
T
T
T  

  
1111
T
T
s-
11 e
2
1
lim
      




T
T
s
T
T
T
detfdttf
T
s  
111111
2
1
lim
89
TandTTandTthatelsoisT
xtLet




arg

• So, we have
       
   











T
T
sx
T
T
st
T
T
T
txs
T
T
T
dxexfdtetf
T
dxexfdttf
T
s
1111
111111
2
1
lim
2
1
lim
,expint ressionsconjugateseparateareegralsTwo
• Now, the consideration is reverted to the original function
f1(t), which is equal to f11(t) in the interval –T to T, an interval
which is allowed to become infinite after integral of above
equation is evaluated.
90
   
2
1111
2
1
lim
,expint




T
T
st
T
dtetf
T
s
ressionsconjugateseparateareegralsTwo


• Random time functions involve infinite energy, hence it is
necessary to convert by the averaging process with respect to
time, to a consideration of power.
• is the power spectral density.
• The significance of PSD is that the power between the
frequencies w1 and w2 is 1/2π times the integral of ø11(jw)
from w1 and w2.
  j11
  

djPtotal 


 11
2
1
Ø11(0) is total power, if f1(t) is the voltage across 1 ohm resistor. The
integral yields the power dissipated in the resistor by all signal component
with frequency lying within the range w1 and w2.
91
   
    




 
djwith
dej
givestiontransformaInverse
j








1111
1111
2
1
0,0
2
1

Measurement of PSD
• The wattmeter measures the power dissipated in the 1 ohm
resistor. The filter output voltage across 1 ohm resistor
R=1
f1(t)
-wc wc w
Gain
f1(t) is electrical voltage, passed through an ideal low pass filter
resistor. The filter output voltage across 1 ohm resistor
contains all frequencies of f1(t) below wc, with no distortion
introduced, but none of the frequencies above wc.
• The wattmeter reads the power dissipated in the resistor by
all frequency components of f1(t) from –wc to wc.
•
  



djadingWattmeter
c
c

 11
2
1
Re

Characteristics of Power spectral density
• It measures the power spectral density rather than the amplitude or
phase spectra of a signal, i.e., relative phase of the various frequency
components is lost.
• As a result of discarding the phase information, a given power density
spectrum may correspond to a large number of different time functions.
• It is purely real, i.e., time average power dissipated in a pure resistance is• It is purely real, i.e., time average power dissipated in a pure resistance is
being measured.
• It is an even function of frequency, i.e.,
   
   
   
  functionEvendCos
functionoddanisitaszeroistermonddSinjdCos
dej
jj
j





















11
1111
1111
1111
][sec

• It is nonnegative at all frequencies. Negative values in any
frequency band indicates that the power is being taken from
the passive 1 ohm resistor.
• If the input signal contains a periodic component such that
the Fourier series for this component contains terms
representing frequencies w , w ,…w , PSD will containrepresenting frequencies w1, w2,…wn, PSD will contain
impulses at w1,-w1, w2, -w2,….., wn, -wn.
If f1(t) contains periodic component of
frequency w1, PSD will contain a term of form a1Cosw1τ

Problem
• Let X(t) is a RP defined as,
where Ɵ is a uniformly distributed RV in the interval
(-π, π).
   tfaCostX 02)(
(-π, π).
• Find PSD of X(t).

Solution
• Mean of the process is zero.
• Autocorrelation and autocovariance is
  02 f
       0
2
1
 





dtCostCosEtX
        



21
2
2121
0
,,
2
tCostCosaEttRttC
f
XXXX
96
       



dttCosttCos
a

 2
22
1
2121
2
     Cos
a
ttCos
a
22
2
21
2


• The autocorrelation function is obtained as,
• The PSD is thus,
     0
2
2
2
fCos
a
Ror XXXX 
      0
2
2
2
fCos
a
jorfS XXXX 
• The signal has average power Rx(0)=a2/2.
• All the power is concentrated at the frequencies f0,-f0,
• So, the power density at these frequencies is infinite.
97
   0
2
0
2
44
ff
a
ff
a
 

Response of Linear Systems to Random Inputs
• Regardless of whether or not the system is linear, for each
member function x(t) of the input process X(t), the system
produces an output y(t) and an ensemble of output functions
form a random process Y(t), which is the response of the
system to the random input signal X(t).system to the random input signal X(t).
• Given the description of input process X(t) and that of system,
obtain the properties of Y(t) such as mean, autocorrelation
function or lower order probability distribution of Y(t).

Classification of Systems
• A system is functional relationship between input x(t) and
output y(t).
• Lumped : A dynamic system is called lumped if it can be
modeled by a set of ordinary differential or difference
 ,);()( 0  ttxfty
modeled by a set of ordinary differential or difference
equation.
• Linear
• Time Invariant
• Causal

Response of LTIVC Continuous time
systems
• Input-output relationship of linear, time invariant and causal
system driven by deterministic input signal x(t) can be
represented by convolution integral as
   
   
 dthxty 





)(
where h(t) is the impulse response of the system and
zero initial conditions are assumed.
• For a stable system
     dtxh  


 
  0,0 





hand
dh

• In frequency domain, input output relationship is
expressed as
• Y(t) is obtained by taking the inverse Fourier
     fXfHfY FF 
• Y(t) is obtained by taking the inverse Fourier
transform of YF(f).
• The forward and inverse transforms are defined as
 
    









dfefYfYty
dtetyfY
ftj
FF
ftj
F


21
2
)(
)(

• When the input to the system is a RP X(t), the resulting output
process Y(t) is given by
   
    

dthX
dhtXtY








)(
• The above equation implies that each member function of
X(t) produces a member function of Y(t).
• In case of discrete time inputs, distribution function of the
process Y(t) are very difficult to obtain except for the Gaussian
case in which Y(t) is Gaussian, if X(t) is Gaussian.


Mean and Autocorrelation function
• Assuming that h(t) and X(t) are real valued and that the expectation and
integration can be interchanged because integration is a linear operator,
mean and autocorrelation function of the output is calculated as
       





 


 dhtXEtYE

103
      2121 , tYtYEttR YY 
    
    

dht
dhtXE
X







       
      21221121
21222111
, 

ddttRhh
ddhtXhtXE
XX 
 

















Stationarity of the output
• We have,
• If the processes X(t) and X(t+ε) have the same distribution
(i.e., X(t) is SSS) then the same is true for Y(t) and Y(t+ε) and
     dhtXtY 


)(        dhtXtY
and




(i.e., X(t) is SSS) then the same is true for Y(t) and Y(t+ε) and
hence Y(t) is SSS.
• If X(t) is WSS, then mean does not depend upon t and we
have
• Thus, the mean of the output does not depend on time.
     dhtYE X


)(
104
  )0(Hdh XX   



• The autocorrelation function of the output is given
by
           2112122121,  ddttRhhttR XXYY   




Since, the integral only depends on the time difference t2-t1,
RYY(t1,t2) will also be a function of the difference t2-t1. This is coupled
with the fact that the output process Y(t) is WSS, if the input process X(t)
is WSS.

PSD of the output
• When X(t) is WSS, it can be shown that
     
     

hRR
hRR
XXXY
XXYX
*
*

       
     
   














hR
tXdthxE
tXtYER
XX
YX
*     
     



hhR
hRR
XX
YXYY
**
*
where * denotes convolution
• Taking Fourier transform of both sides, PSD of the output is obtained as
     2
fHfSfS XXYY 
The input spectral component at frequency f is modified according
to |H(f)|2 ,hence is sometimes called power transfer function.
106
       hhRXX **

Mean square value of the output
• The mean square value of the output, which is a
measure of the average value of the output power.
• It is given by
      

 dffSRtYE YYYY
2
0
• Assumption that Rxx(τ) can be expressed as a sum of
complex exponentials [i.e., Sxx(f) is a rational function
of f] simplifies the integral.
      
   






dffHfS
dffSRtYE
XX
YYYY
2
0
Except in some cases, evaluation of preceding integral in difficult.

• The transformation s=2πjf is used and we have
where a(s)/b(s) has all of its poles and zeros in the
LHS and a(-s)/b(-s) has all its roots in the RHS.
   
   
,
2 sbsb
sasa
j
s
SXX









LHS and a(-s)/b(-s) has all its roots in the RHS.
• Therefore,
  
     
)()(
)()(
|*,
)()(
)()(
2
1
2/
2
sdsd
scsc
fHfHfSwhere
ds
sdsd
scsc
j
tYE
jsfXX
j
j













Problem
• X(t) is the input voltage to the system shown in
Figure is a stationary RP with
• Find the mean, PSD and autocorrelation of the
     exp0 XXX Rand
• Find the mean, PSD and autocorrelation of the
output.

Solution
• We have  
fLjR
R
fH
2

 22
2
2
f



           dfjdfjfS
Also
XX 2expexp2expexp
,
0
0
 


 
   22
2
22
22
2
,
fLR
R
f
fSSo YY











110
 2 f 
 
 
 
 
 
  















 





L
R
L
R
L
R
L
R
L
R
R
FTinverseTaking
YY 2
22
2
2
exp
,
0, YhaveWe 

Problem
• The input to a RC lowpass filter with
is a zero mean stationary RP with
 
 1000/1
1
fj
fH


  HzwattfSXX /10 12

• Find the mean square value of output Y(t).

Solution
• We have  











 
1000
1
1
.
1000
1
1
10 12
fjfj
fSYY
  
   
10101 66  

j
jfs 2
112
  
   
 


100010
2000
1
10
2000
1
10
2
1
12
66
2






  ds
ssj
tYE
j

Module v sp

More Related Content

What's hot

Similar to Module v sp

More from Vijaya79

Recently uploaded

Module v sp