2. Stochastic Processes
• The outcome of a random experiment is a function of time or
space
• Examples:
Speech recognition system (based on voltage waveforms)
Image processing system (intensity of pixels is a function of Image processing system (intensity of pixels is a function of
space)
Queuing system (no. of customers varies as a function of time)
Based on temperature demand of electricity varies
2Vijaya Laxmi, Dept. of EEE
3. • In electrical engineering, voltage or current are used
for collecting, transmitting and processing
information as well as for controlling and providing
power to a variety of devices.
3Vijaya Laxmi, Dept. of EEE
4. Signal
• These are functions of time and belongs to two
classes:
Deterministic: These are described by functions in
the mathematical sense with time ‘t’ as independentthe mathematical sense with time ‘t’ as independent
variable
Random signal
4Vijaya Laxmi, Dept. of EEE
5. Random signal
This always has some element of uncertainty associated in it
and hence it is not possible to determine exact value at the
given point of time.
Example: Audio waveform transmitted over a telephone
channel.channel.
i.e. , we cannot precisely specify the value of random signal in
advance. However, may describe its average properties such
as average power, spectral distribution, probability that the
signal amplitude exceeds a certain value etc.
5Vijaya Laxmi, Dept. of EEE
6. • The probabilistic model used for characterizing a
random signal is called random process or stochastic
process.
• This deals with time varying waveforms that have
some element of chance or randomness associatedsome element of chance or randomness associated
with them.
• Example: data communication system in which a
number of terminals are sending information in
binary format over noisy transmission links to a
central computer.
6Vijaya Laxmi, Dept. of EEE
7. Observation
• By observing the waveform of x1(t) for [t1,t2], we cannot with
certainty predict the value of xi(t) for any other value of
• The knowledge of one member function xi(t) will not enable
us to know the value of another member function xj(t).
21,ttt
7Vijaya Laxmi, Dept. of EEE
8. • We should use a probabilistic model to describe or
characterize the ensemble of waveforms so that to answer
a) what are the spectral properties of ensemble of waveforms?
b) how does the noise affects the system performance as
measured by the receivers ability to recover the transmittedmeasured by the receivers ability to recover the transmitted
data correctly?
c) what is the optimum processing algorithm, the receiver
should use?
8Vijaya Laxmi, Dept. of EEE
9. Example
• Tossing of N coins simultaneously and repeating N
tossings once every T seconds.
• Draw the waveforms.
9Vijaya Laxmi, Dept. of EEE
10. Random Variable vs Random Process
• A random variable maps the outcome of random
experiment to a set of real numbers, similarly
• A random process maps the outcome of a random
experiment to a set of waveforms or functions ofexperiment to a set of waveforms or functions of
time.
10Vijaya Laxmi, Dept. of EEE
12. • Suppose there is a large number of people, each flipping a fair
coin every minute. If we assign the value 1 to a head and the
value 0 to a tail.
12Vijaya Laxmi, Dept. of EEE
14. Example: Tossing of die
• For tossing of a die
22
4224)(
654321
tttX
• The set of waveforms
is called an ensemble.
)(.....,),(),( 621 txtxtx
14Vijaya Laxmi, Dept. of EEE
15. • For specific value of time, t0 , is collection of
numerical values of various member function at t=t0.,
where t is time and represents an outcome in
sample space S
),( 0 tX
15Vijaya Laxmi, Dept. of EEE
17. Problem
• Tossing of die
• Find
0)0(|2)4(
2)4(,0)0(,0)4(,2)4(
XXP
XXPXPXP
17Vijaya Laxmi, Dept. of EEE
18. Solution
• Let A be the set of outcomes such that
• (a)
Ai
3
1
6
2
2)4(
5,22,4
APXP
AX i
• (b)
• (c)
• (d)
36
2
1
6
3
)(0)4( APXP
6
1
,5 BPB
2
1
6
2
6
1
0)0(
0)0(,2)4(
0)0(|2)4(
XP
XXP
XXP
18Vijaya Laxmi, Dept. of EEE
19. Classification of Random Process
X(t) t
Continuous Discrete
Continuous Continuous Random
Process
Continuous Random
Sequence
Discrete Discrete Random Discrete RandomDiscrete Discrete Random
Process
Discrete Random
Sequence
19Vijaya Laxmi, Dept. of EEE
20. Classification of Random Process
• Stationary Random Process:
The probability distribution function or averages do not
depend upon time ‘t’.
• Non Stationary Random Process:• Non Stationary Random Process:
The prob. distribution function or averages depend on time
‘t’.
20Vijaya Laxmi, Dept. of EEE
21. Classification of Random Process
Based on observation of past values
• Predictable,
and
• Unpredictable• Unpredictable
21Vijaya Laxmi, Dept. of EEE
22. Classification of Random Process
• Real Valued RP
• Complex valued RP
If a RP Z(t) is given by
RPsvaluedrealthearetandtAfrequencycarriertheisfwhere
ttfCostAtZ c
)()(,
,)(2)()(
• Z(t) =real part of
= real part of
Complex envelope
• Here, W(t) is complex valued RP and X(t), Y(t) and Z(t) are Real valued
RPs.
RPsvaluedrealthearetandtAfrequencycarriertheisfwhere c )()(,
tfjtjtA c 2exp)(exp)(
tfjtw c2exp)(
)()(
)()()()()(
tjYtX
tSintjAtCostAtw
22Vijaya Laxmi, Dept. of EEE
23. Definition of Random Process
• A real valued RP X(t), is a measurable
function on that maps onto R .
Tt
S
.var
,
1 linerealRsetinvalueswithiablet
andSpaceSampleS
function on that maps onto R1.
• If is a set of one or more intervals on the real
line, X(t) is called Random Process.
• If is a subset of integers, X(t) is called
Random Sequence.
S S
23Vijaya Laxmi, Dept. of EEE
24. • A real valued RP X(t) is described by nth order
distribution function
nnnXXX
tttandnallfor
xtXxtXxtXPxxxF n
,...,,
,...,,,...,, 221121....21
• It satisfy all requirements of joint probability
distribution function (CDF).
ntttandnallfor ,...,, 21
24Vijaya Laxmi, Dept. of EEE
25. Methods of Description
1. Joint Distribution:
• First order dist. Fn. is given by , which
gives the idea about instantaneous amplitude
distribution of the process.
11 atXP
distribution of the process.
• Second order dist. Fn. is given by
which gives the information about the
structure of the signal in time domain.
2211 , atXatXP
25Vijaya Laxmi, Dept. of EEE
26. Problem : Tossing of die
22
4224)(
654321
tttX
The outcomes and the corresponding waveforms are given by
Find
60 XandXPJoint probability
Find
Marginal probabilities P[X(0)] and P[X(6)]
26Vijaya Laxmi, Dept. of EEE
27. Values
of X(0)
Values of X(6) Total
-4 -3 -2 2 3 4
-4 1/6 0 0 0 0 0 1/6
-2 0 0 1/6 0 0 0 1/6
0 0 1/6 0 0 1/6 0 2/6
MarginalProb.OfX(0)
2 0 0 0 1/6 0 0 1/6
4 0 0 0 0 0 1/6 1/6
Total 1/6 1/6 1/6 1/6 1/6 1/6 1
Grand Total
Marginal Probability of X(6)
MarginalProb.OfX(0)
27Vijaya Laxmi, Dept. of EEE
28. 2. Analytical description of RP using Random
Variables
• A RP Y(t) is expressed as
iablesrandomareandAWhere
tCosAtY
var
,10)( 8
28Vijaya Laxmi, Dept. of EEE
32. • The autocovariance and correlation coefficient is
given by
2121 ,, ttRttC XXXX
32
2
2
2
1
21
21
2
1
40
2
1
40
2
1
40
,,
tt
tt
ttrand XX
Vijaya Laxmi, Dept. of EEE
33. Problem
• A RP X(t) is given by
RVstindependenareandA
inddistributeuniformlyis
ianceandmeanwithRVisAwhere
tACostX
],[
1var0
100)(
• Find
RVstindependenareandA
11,ttRandt XXX
33Vijaya Laxmi, Dept. of EEE
34. Solution
• We have
0100 tCosEAEtX
100100100
,,
2
2121
A
tACostACosE
ttandttwheretXtXEttRXX
It is a function of τ and periodic in τ
If a process has a periodic component, its autocorrelation
function will also have a periodic component with same period.
34
]sec[100
2
1
2100200100
2
2
zerotoequalispartondasCos
tCosCos
A
E
Vijaya Laxmi, Dept. of EEE
35. 4. Two or more RPs
• If X(t) and Y(t) are two RPs
• Cross Correlation Function
functionondistributiJocalledis
ytYytYytYxtXxtXxtXP nnnn
int
,...,,,,....,, '
2
'
21
'
12211
*
, tYtXEttR • Cross Correlation Function
• Cross Covariance Function
• Correlation Coefficient
21
*
21, tYtXEttRXY
21
*
2121 ,, ttttRttC YXXYXY
2121
21
21
,,
,
,
ttCttC
ttC
ttr
YYXX
XY
XY
35Vijaya Laxmi, Dept. of EEE
36. • Equality: If their respective member functions are identical
for each outcome , the two RPs are called equal.
• Uncorrelated:
• Orthogonal:
• Independent:
2121 ,,0, ttforttC XY
2121 ,,0, ttforttRXY
• Independent:
''
2
'
121
'
1
'
111
'
1
'
111
,...,,,,..,,,
,...,,......,
,...,;,......,
mn
mmnn
mmnn
ttttttandmnallfor
ytYytYPxtXxtXP
ytYytYxtXxtXP
Independent implies uncorrelated but converse is not true.
36Vijaya Laxmi, Dept. of EEE
37. Problem
• If Ɵ is uniformly distributed RV in [0,2π]
and X=CosƟ , Y=SinƟ
• Show that X and Y are uncorrelated.
37Vijaya Laxmi, Dept. of EEE
38. Solution
• We have,
2
1
2
dCosSin
CosSinEXYE
0 YEXE
02
4
1
2
2
0
0
dSin
dCosSin
Hence, X and Y are uncorrelated.
38Vijaya Laxmi, Dept. of EEE
39. Problem
• Suppose X(t) is a RP with
212.0
21 49,
,3)(
tt
ettR
t
• Find the mean, variance and covariance of RVs, Z and
W, If Z=X(5) and W=X(8).
39Vijaya Laxmi, Dept. of EEE
41. Stationarity
• It describes the time invariance of certain properties of a RP,
whereas individual member functions of a RP may fluctuate
rapidly as a function of time, the ensemble averaged values
such as, mean of the process might remain constant with
respect to time.respect to time.
• A process is said to be stationary, if its distribution function or
certain expected values are invariant with respect to a
translation of the time axis.
• Example: Signal from radar used for both searching and
tracking is non stationary.
Considering only searching time is stationary RP.
41Vijaya Laxmi, Dept. of EEE
42. Types of stationarity
• Strict sense stationarity
• Wide sense stationarity
42Vijaya Laxmi, Dept. of EEE
43. Strict sense stationarity
• A RP X(t) is called Strict sense stationary (SSS) or stationary in the strict
sense, if all of the distribution functions describing the process are
invariant under a translation of time.
i.e., for
nn tttttt ,...,,,,...,, 2121
• If the above equation holds for all kth order distribution function
k=1,2,…,N, but not necessarily for k>N, the process is called Nth order
stationary.
kk
kk
nn
xtXxtXxtXP
xtXxtXxtXP
kalland
,...,,
,...,,
...,2,1
2211
2211
2121
43Vijaya Laxmi, Dept. of EEE
44. First and second order dist. Fn.
• First order distribution
• Second order distribution
anyforxtXPxtXP
The first order distribution is independent of t
…(i)
Second order distribution
• From (i) and (ii),
anyforxtXxtXPxtXxtXP 22112211 ,,
The second order distribution is strictly a function of time difference (t2-t1)
…(ii)
1221
*
tan
ttRtXtXE
tConstXE
XX
X
44Vijaya Laxmi, Dept. of EEE
45. • Two real valued RPs X(t) and Y(t) are jointly SSS, if
their joint distribution are invariant under a
translation of time.
• A complex RP, Z(t)=X(t)+jY(t) is SSS, if the processesA complex RP, Z(t)=X(t)+jY(t) is SSS, if the processes
X(t) and Y(t) are jointly stationary in strict sense.
45Vijaya Laxmi, Dept. of EEE
46. Wide sense stationarity
• A RP X(t) is said to be stationary in wide sense (WSS or weakly
stationary), if its mean is a constant and the autocorrelation
function depends only upon the time difference.
XX
X
RtXtXE
tXE
*
• Two processes X(t) and Y(t) are called jointly WSS, if
• For random sequences,
XXRtXtXE
XYRtYtXE *
kRknXnXE
kXE
XX
X
*
SSS implies WSS, but the converse is not true.
46Vijaya Laxmi, Dept. of EEE
47. Problem
• If
• Check for stationarity.
.5,3,1,1,3,5 654321 tXtXtXtXtXtX
47Vijaya Laxmi, Dept. of EEE
48. Solution
0tXE
6
70
25911925
6
1
, 2121
ttRtXtXE XX
Stationary in strict sense, since the translation of time axis
does not result in any change in member function.
48Vijaya Laxmi, Dept. of EEE
49. Problem
• If
• Check for stationarity.
6,3,3
3,3,6
654
321
tytCostytCosty
tSintytSintyty
49Vijaya Laxmi, Dept. of EEE
51. Problem
• Establish the necessary and sufficient condition for
stationarity of the process
tbSintaCostX )(
51Vijaya Laxmi, Dept. of EEE
52. Problem
• If X(t) is a RP given by
BEAE
BEAE
tSinBtCosAtX
ii
ii
n
i
iiii
,0
)(
222
1
• Discuss about stationarity.
ondistributiGaussianjohave
andeduncorrelatareniforBandA
BEAE
ii
ii
int
,...,2,1
52Vijaya Laxmi, Dept. of EEE
53. • A RP X(t) is asymptotically stationary, if distribution function
of does not depend on τ, when τ is large.
• A RP X(t) is stationary in an interval, if
ntXtXtX ....,,, 21
,....,,..., 1111
whichforallfor
xtXxtXPxtXxtXP kkkk
• A RP X(t) is said to have independent increments, if its
increments form a stationary process for every
τ. Example: Poisson and Wiener Process.
• A RP is cyclostationary or periodically stationary, if it is
stationary under a shift of the time origin by integer multiple
of a constant T0 (which is the period of the process).
.int,...,, 21 ervalaninliettt
whichforallfor
k
)()()( tXtXtY
53Vijaya Laxmi, Dept. of EEE
54. Time Averaging and Ergodicity
• It is a common practice in laboratory to obtain multiple
measurements of a variable and average them to reduce
measurement errors.
• If value of a variable being measured is constant, and the
errors are due to noise or due to the instability of the
measuring instrument, then averaging is indeed a valid andmeasuring instrument, then averaging is indeed a valid and
useful technique.
• Time averaging is used to reduce the variance associated with
the estimation of the value of a random signal or the
parameters of a random process.
• Ergodicity is a relationship between statistical average and
sample average.
54Vijaya Laxmi, Dept. of EEE
55. Ergodicity
• It is a property of stationary RP with the assumption that time
average over one member function of a RP is equivalent to an
average over the ensemble of functions.
• After a sufficient length of time, the effect of initial conditions
is negligible.is negligible.
• If one function of the ensemble is inspected over a long
period of time, all salient characteristics of the ensemble of
functions will be observed.
• Any one function can be used to represent the whole
ensemble.
55Vijaya Laxmi, Dept. of EEE
56. • It can be interpreted in terms of prob. dist. fn.
• If
• Then, 1st prob. dist. fn. p(y)is the probability that at
any given time t1, any function fj(t) lies between y
ensembleoffunctionsmembertftftf n ,...,, 21
any given time t1, any function fj(t) lies between y
and y+dy.
• p(y) is the probability that any function fj(t) at any
time lies between y and y+dy called ergodic property.
56Vijaya Laxmi, Dept. of EEE
57. • Statistical characteristics of an ensemble can be
determined by consideration of any one function.
• The ensemble average is the average taken at a fixed
time for a large number of member functions of thetime for a large number of member functions of the
ensemble.
• The time average is the average taken with large
number of widely separated choices of initial time
and using only one function of the ensemble.
57Vijaya Laxmi, Dept. of EEE
58. Special types of random processes
• Poisson process
• Random Walk process
• Wiener process
• Markov process• Markov process
58Vijaya Laxmi, Dept. of EEE
59. Poisson process
• It is a continuous time, discrete amplitude random
process.
• It is used to model phenomena such as emission of
photons from a light emitting diode, arrival ofphotons from a light emitting diode, arrival of
telephone calls, occurance of failures etc.
• A counting function Q(t) is defined for
is the number of events that have occurred during
time period.
),0[ t
59Vijaya Laxmi, Dept. of EEE
60. • Q(t) is an integer valued RP and said to be Poisson process if
the following assumptions hold:
• For any time number of events Q(t2)-
Q(t1) that occur in the interval t1 to t2 is Poisson distribution
as:
1221, ttandtt
as:
Number of events that occur in any interval of time is
independent of number of events that occur in other non-
overlapping time intervals.
...2,1,0,exp
!
12
12
12
kfortt
k
tt
ktQtQP
k
60Vijaya Laxmi, Dept. of EEE
61. • We have,
• The autocorrelation of Q(t) is obtained as
ttQVarttQE
kt
k
t
ktQP
k
,
...,2,1,0,exp
!
, tQtQEttR
212121
2
1221
121
2
1
2
1
1211
2
121211
2121
,,min.
1
,
ttallfortttt
ttfortt
ttttt
tQtQEtQEtQE
ttfortQtQtQtQE
tQtQEttRQQ
222
222
XEXE
XEXE
It is not a martingale, since the mean is time varying.
61Vijaya Laxmi, Dept. of EEE
62. Process with independent increments
• A RP X(t), is said to have independent increments, if for
all times the RVs,
are mutually
independent.
t
...,4,3....21 kandttt k
12312 ....,, kk tXtXandtXtXtXtX
62Vijaya Laxmi, Dept. of EEE
63. Markov Process
• A RP X(t), is called a first order Markov process,
if for all sequences of times
t
...,2,1,0.....21 kandttt k
111 .......... kkkkkk tXxtXPtXtXxtXP
i.e., conditional probability distribution of X(tk) given for all past
values of X(t1)=x1,…..,X(tk-1)=xk-1 depends upon the most recent
value of X(tk-1)=xk-1.
63Vijaya Laxmi, Dept. of EEE
64. Martingale
• A RP X(t), is called a Martingale, ift
2112112 ; ttallfortXtttXtXE
andtallfortXE
i.e., Constant mean
It plays an important role in prediction of future values of random
processes based on past observation.
64Vijaya Laxmi, Dept. of EEE
65. Random Walk Process
• It is a discrete version of Wiener process used to model the
random motion of a particle.
• Assumptions:
A particle is moving along a horizontal line, until it collides
with another particle.with another particle.
Each collision causes the particle to move ‘up’ or ‘down’ from
its previous path by a distance ‘d’.
Collision takes place once every T seconds and movement
after the collision is independent of its position.
It is analogous to tossing a coin once every T seconds and taking a step
‘up’ if head show and ‘down’ if tail show, called Random Walk.
65Vijaya Laxmi, Dept. of EEE
66. Sample function of Random walk process
d
X(n)
2d
d
-d
-2d
-3d
t/T=n
66Vijaya Laxmi, Dept. of EEE
67. • Position of particle at t=nT is a random sequence
X(n).
• Assume X(0)=0 and jump of appears instantly
after each toss.
d
after each toss.
67Vijaya Laxmi, Dept. of EEE
68. • If k heads show up in the first n tosses, then the position of
the particle at t=nT is given by
nnkwheremd
dnk
dknkdnX
,1...,,2,1,0,
2
)(
• If, the sequence of jumps is denoted by a sequence of random
variables {Ji}, then X(n) can be expressed as
• The RVs Ji, i=1,2,3,…,n are independent and have identical
distribution functions with
nnnnnm ,2...,,4,2,
nJJJnX ...21
2
22
2
2
,0
2
1
,
2
1
d
dd
JEJE
dJPdJP
ii
ii
68Vijaya Laxmi, Dept. of EEE
69. • We have,
• The number of heads in n tosses has Binomial distribution,
hence
2
,)(
nm
ktossesninheadskPmdnXP
hence
2
2
21
2
...
0
2;...,,2,1,0,
2
1
2
1
2
1
nd
JJJEnXE
nXE
and
nkmnk
k
n
k
n
mdnXP
n
nknk
69Vijaya Laxmi, Dept. of EEE
70. • The autocorrelation function of random walk sequence is given by
• If n2>n1, X(n1) and X(n2)-X(n1) are independent RVs, hence the number of
heads from the 1st to n th tossing is independent of the number of heads
121
2
1
1211
2121 ,
nXnXnXEnXE
nXnXnXnXE
nXnXEnnRXX
2 1 2 1
heads from the 1st to n1
th tossing is independent of the number of heads
from (n+1)th tossing to the n2
th tossing. Hence,
• If n1>n2,
• Hence,
2
1
2
1
121
2
121,
dn
nXE
nXnXEnXEnXEnnRXX
2
221, dnnnRXX
2
2121 ,min, dnnnnRXX
Random Walk is a Markov sequence and a Martingale.
70Vijaya Laxmi, Dept. of EEE
71. Gaussian Process
• A RP X(t), is called a Gaussian process, if all its
nth order distribution function
are n-variate Gaussian distributions
t
nXXX xxxF n
,...,, 21.....21
are n-variate Gaussian distributions
• If the Gaussian process is also a Markov process, it is
called a Gaussian-Markov process.
iin tXXandttt ,......,, 21
71Vijaya Laxmi, Dept. of EEE
72. Wiener process
• Define Y(t) as continuous random process for from
the random sequence X(n) as
),0[ t
,....2,1)1(,)(
0,0
)(
nfornTtTnnX
t
tY
nTtAt
Y(t) is Broken line of sample function of random walk process.
• Wiener process is obtained from Y(t) by letting time (T)
between jumps and the step size (d) approach zero with
constraint d2 =αT to assume that variance will remain finite
and nonzero for finite values of t.
72
2
2
2
,
,0)(,
nd
T
td
tYEVariance
andtYEMean
Vijaya Laxmi, Dept. of EEE
73. Properties of Wiener process
• W(t) is a constant amplitude, continuous time, independent
increment process.
• It has
• It has Gaussian distribution
t
w
t
wfW
2
exp
2
1 2
ttWEandtWE 2
0
• For any value of t’, ,the increment w(t)-w(t’) has a
Gaussian PDF with zero mean and variance
• Autocorrelation of W(t) is
tt '0
'tt
tt
wfW
2
exp
2
2121 ,min, ttttRXX
Wiener process is a nonstationary Markov process and a Martingale.
73Vijaya Laxmi, Dept. of EEE
74. Correlation Function
• The core of statistical design theory is mean square
error criterion.
• The synthesis should aim towards minimization of
mean square error between actual output andmean square error between actual output and
desired output.
• The input is assumed as stationary time series
existing over all time.
74Vijaya Laxmi, Dept. of EEE
75. • The mean square error is expressed as
• To express it in terms of system characteristic and
T
T
d
T
dttftf
T
e
2
0
2
2
1
lim
• To express it in terms of system characteristic and
input signal, f0(t) is replaced by fi(t) and g(t), the unit
impulse response.
• The convolution theorem states that
,0 dtfgtf i
75
sG
sF
sF
i
0
Vijaya Laxmi, Dept. of EEE
76. • The mean square error is then expressed as
2
2
2
1
lim
T
T
di
T
tfdtfgdt
T
e
T
T
didii
T
tfdtfgtfdtfgdtfgdt
T
2
2
2
1
lim
76
T
T
d
T
id
T
T
T
ii
T
T
T
dttf
T
dtfgtfdt
T
dtfgdtfgdt
T
2
2
1
lim
2
1
lim2
2
1
lim
T
T
d
T
d
T
T
i
T
T
T
ii
T
dttf
T
dttftf
T
dg
dttftf
T
dgdg
2
2
1
lim
2
1
lim2
2
1
lim
Vijaya Laxmi, Dept. of EEE
77. • The fi(t) and fd(t) are in the form of an averaging of the
product of two time functions.
• If
•
dttftf
T
T
T
ba
T
ab
2
1
lim
02
2
ddidii dgdgdge
02 ddidii dgdgdge
Correlation function of statistics
Auto correlation function of input signal fi(t)
Auto correlation function of desired output
Cross correlation function between input signal and
desired output
77
id
dd
ii
Where
Vijaya Laxmi, Dept. of EEE
78. Measurement of Autocorrelation function
• The meter will read the autocorrelation function for one
particular value of τ, since wattmeter performs multiplication
and averaging.
• For the plot of autocorrelation function, delay of line can be
varied and a number of discrete readings are taken.varied and a number of discrete readings are taken.
• It is qualitatively a measure of regularity of the function.
• If there is no DC component in the signal, the autocorrelation
function will be small if the argument τ is taken larger than
the interval over which values of the function are strongly
dependent.
78Vijaya Laxmi, Dept. of EEE
79. • Autocorrelation function for any argument τ is the average of
the product of e1 and e2 values of the function τ seconds apart
and average is given by:
21212111 , dedeeepee
• The expected or average value is summation or integration of
all products multiplied by their respective probabilities.
is the probability of any given product having 1st term between e1
and e1+de1 and 2nd term between e2 and e2+de2
79
2121 , dedeeepwhere
Vijaya Laxmi, Dept. of EEE
80. Properties of Autocorrelation Function
• It is an even function of τ, i.e., .
Since the functions are averaged over a doubly infinite
interval, the averaged product is independent of the direction
of the shift.
dttftf
T
dttftf
T
T
T
T
T
T
T
111111
2
1
lim
2
1
lim
1111
of the shift.
• is autocorrelation function with zero argument, i.e.,
average power of time function.
If function f1(t) represents the voltage across a 1 Ω
resistor or the current through a 1 Ω resistor, autocorrelation
function is the power consumed by the 1 Ω resistor .
011
80Vijaya Laxmi, Dept. of EEE
81. 01111
o The Maximum value of autocorrelation function appears, when
function is multiplied by itself without shifting
tftftftftftf 2
1
2
1
2
1111
2
1
,sidesbothontakenAverage
81
00
2
1
,
1111
2
1111 tftf
sidesbothontakenAverage
0
2
1
11
2
11 tftf
0
2
1
0
1111
2
111111
tftf
Vijaya Laxmi, Dept. of EEE
82. • If the signal contains periodic components (or DC value), the
autocorrelation function contain components of the same
periods (or a DC component), i.e., a periodic wave shifted by
one period is indistinguishable from the unshifted wave.
• If the input signal contains only random components (no• If the input signal contains only random components (no
periodic components), the autocorrelation function tends to
zero as τ tends to infinity. As the shift of time function
becomes very large, the two functions f1(t) and f1(t+τ)
becomes essentially independent.
82Vijaya Laxmi, Dept. of EEE
83. • Aurocorrelation function is equal to the sum of the
autocorrelation functions of the individual frequency
components, since the multiplication of components of
different frequency results in zero average value, i.e., voltage
and current of different frequencies result in zero averageand current of different frequencies result in zero average
value.
• A given autocorrelation function may correspond to an
infinite number of different time functions.
• Autocorrelation function of the derivative of f(t) can be
expressed in terms of autocorrelation function of f(t) as,
T
T
T
tftf
T
'
1
'
1
2
1
lim
83
''
11
Vijaya Laxmi, Dept. of EEE
84. Power Spectral Density (PSD)
• If input is stationary time series and minimization of mean
square error is used as design criterion, signals are described
by correlation functions.
• The correlation functions are sufficient data for synthesis of a
minimum mean square error system.minimum mean square error system.
• It is convenient to describe the input signal in frequency
domain characteristics.
• If the autocorrelation function defines the system adequately,
then frequency domain function must carry information
contained in the autocorrelation functions.
84Vijaya Laxmi, Dept. of EEE
85. • A function satisfying such requirements in Laplace
transform of autocorrelation function,
• To determine what characteristics of random input
des s
1111
• To determine what characteristics of random input
signal are measured by the frequency function,
consideration of Laplace transform is significant.
• If fp(t) is periodic,
n
tjn
np eatf
85
Tt
t
tjn
pn
Tt
t
p
dtetf
T
a
dttf
T
a
0
0
0
0
1
1
0
Vijaya Laxmi, Dept. of EEE
86. Amplitude spectrum
• The energy in signal is concentrated at isolated frequencies.
|an|
n
Indicates actual amplitude of signal component
at the corresponding frequency
• The energy in signal is concentrated at isolated frequencies.
• But, in aperiodic signals conversion from Fourier series to
Fourier transform of Laplace integral transformation involves
a limiting process and spectrum is called Amplitude density
spectrum.
86Vijaya Laxmi, Dept. of EEE
87. f1(t)
w
t
|F1(iw)|
221
11
1
1
0
00
a
jF
as
sF
tfore
tfor
tf at
• If f1(t) is the voltage across 1 ohm resistor, plot of |F1(jw)|2 vs
w is called the Energy density spectrum, i.e., direct indication
of energy dissipated in 1ohm resistor as a function of
frequency.
• The area under the curve between w1 and w2 is proportional
to the total energy at all frequencies within these limits.
87Vijaya Laxmi, Dept. of EEE
88. • Total energy is proportional to the area under the entire
curve, i.e.,
a
d
a
djF
2
11
2
1
2
1
22
2
1
• Thus, spectrum of periodic wave is called the amplitude
spectrum and the spectrum of aperiodic waves is called the
amplitude density spectra.
• But, random time functions, can be characterized by Laplace
transform of autocorrelation function.
aa 22
88Vijaya Laxmi, Dept. of EEE
89. • Hence,
• Consider f (t) to exist over the time interval –T to T instead of
dttftfde
T
dttftf
T
de
des
T
T
s
T
T
T
T
s
s
11
11
1111
2
1
lim
2
1
lim
• Consider f11(t) to exist over the time interval –T to T instead of
over all time,
• Laplace transform of f11(t) exists, the order of integration is
interchanged
dttftfd
T
s
T
T
T
1111
T
T
s-
11 e
2
1
lim
T
T
s
T
T
T
detfdttf
T
s
111111
2
1
lim
89
TandTTandTthatelsoisT
xtLet
arg
Vijaya Laxmi, Dept. of EEE
90. • So, we have
T
T
sx
T
T
st
T
T
T
txs
T
T
T
dxexfdtetf
T
dxexfdttf
T
s
1111
111111
2
1
lim
2
1
lim
,expint ressionsconjugateseparateareegralsTwo
• Now, the consideration is reverted to the original function
f1(t), which is equal to f11(t) in the interval –T to T, an interval
which is allowed to become infinite after integral of above
equation is evaluated.
90
2
1111
2
1
lim
,expint
T
T
st
T
dtetf
T
s
ressionsconjugateseparateareegralsTwo
Vijaya Laxmi, Dept. of EEE
91. • Random time functions involve infinite energy, hence it is
necessary to convert by the averaging process with respect to
time, to a consideration of power.
• is the power spectral density.
• The significance of PSD is that the power between the
frequencies w1 and w2 is 1/2π times the integral of ø11(jw)
from w1 and w2.
j11
djPtotal
11
2
1
Ø11(0) is total power, if f1(t) is the voltage across 1 ohm resistor. The
integral yields the power dissipated in the resistor by all signal component
with frequency lying within the range w1 and w2.
91
djwith
dej
givestiontransformaInverse
j
1111
1111
2
1
0,0
2
1
Vijaya Laxmi, Dept. of EEE
92. Measurement of PSD
• The wattmeter measures the power dissipated in the 1 ohm
resistor. The filter output voltage across 1 ohm resistor
R=1
f1(t)
-wc wc w
Gain
f1(t) is electrical voltage, passed through an ideal low pass filter
resistor. The filter output voltage across 1 ohm resistor
contains all frequencies of f1(t) below wc, with no distortion
introduced, but none of the frequencies above wc.
• The wattmeter reads the power dissipated in the resistor by
all frequency components of f1(t) from –wc to wc.
•
djadingWattmeter
c
c
11
2
1
Re
92Vijaya Laxmi, Dept. of EEE
93. Characteristics of Power spectral density
• It measures the power spectral density rather than the amplitude or
phase spectra of a signal, i.e., relative phase of the various frequency
components is lost.
• As a result of discarding the phase information, a given power density
spectrum may correspond to a large number of different time functions.
• It is purely real, i.e., time average power dissipated in a pure resistance is• It is purely real, i.e., time average power dissipated in a pure resistance is
being measured.
• It is an even function of frequency, i.e.,
functionEvendCos
functionoddanisitaszeroistermonddSinjdCos
dej
jj
j
11
1111
1111
1111
][sec
93Vijaya Laxmi, Dept. of EEE
94. • It is nonnegative at all frequencies. Negative values in any
frequency band indicates that the power is being taken from
the passive 1 ohm resistor.
• If the input signal contains a periodic component such that
the Fourier series for this component contains terms
representing frequencies w , w ,…w , PSD will containrepresenting frequencies w1, w2,…wn, PSD will contain
impulses at w1,-w1, w2, -w2,….., wn, -wn.
If f1(t) contains periodic component of
frequency w1, PSD will contain a term of form a1Cosw1τ
94Vijaya Laxmi, Dept. of EEE
95. Problem
• Let X(t) is a RP defined as,
where Ɵ is a uniformly distributed RV in the interval
(-π, π).
tfaCostX 02)(
(-π, π).
• Find PSD of X(t).
95Vijaya Laxmi, Dept. of EEE
96. Solution
• Mean of the process is zero.
• Autocorrelation and autocovariance is
02 f
0
2
1
dtCostCosEtX
21
2
2121
0
,,
2
tCostCosaEttRttC
f
XXXX
96
dttCosttCos
a
2
22
1
2121
2
Cos
a
ttCos
a
22
2
21
2
Vijaya Laxmi, Dept. of EEE
97. • The autocorrelation function is obtained as,
• The PSD is thus,
0
2
2
2
fCos
a
Ror XXXX
0
2
2
2
fCos
a
jorfS XXXX
• The signal has average power Rx(0)=a2/2.
• All the power is concentrated at the frequencies f0,-f0,
• So, the power density at these frequencies is infinite.
97
0
2
0
2
44
ff
a
ff
a
Vijaya Laxmi, Dept. of EEE
98. Response of Linear Systems to Random Inputs
• Regardless of whether or not the system is linear, for each
member function x(t) of the input process X(t), the system
produces an output y(t) and an ensemble of output functions
form a random process Y(t), which is the response of the
system to the random input signal X(t).system to the random input signal X(t).
• Given the description of input process X(t) and that of system,
obtain the properties of Y(t) such as mean, autocorrelation
function or lower order probability distribution of Y(t).
98Vijaya Laxmi, Dept. of EEE
99. Classification of Systems
• A system is functional relationship between input x(t) and
output y(t).
• Lumped : A dynamic system is called lumped if it can be
modeled by a set of ordinary differential or difference
,);()( 0 ttxfty
modeled by a set of ordinary differential or difference
equation.
• Linear
• Time Invariant
• Causal
99Vijaya Laxmi, Dept. of EEE
100. Response of LTIVC Continuous time
systems
• Input-output relationship of linear, time invariant and causal
system driven by deterministic input signal x(t) can be
represented by convolution integral as
dthxty
)(
where h(t) is the impulse response of the system and
zero initial conditions are assumed.
• For a stable system
dtxh
0,0
hand
dh
100Vijaya Laxmi, Dept. of EEE
101. • In frequency domain, input output relationship is
expressed as
• Y(t) is obtained by taking the inverse Fourier
fXfHfY FF
• Y(t) is obtained by taking the inverse Fourier
transform of YF(f).
• The forward and inverse transforms are defined as
dfefYfYty
dtetyfY
ftj
FF
ftj
F
21
2
)(
)(
101Vijaya Laxmi, Dept. of EEE
102. • When the input to the system is a RP X(t), the resulting output
process Y(t) is given by
dthX
dhtXtY
)(
• The above equation implies that each member function of
X(t) produces a member function of Y(t).
• In case of discrete time inputs, distribution function of the
process Y(t) are very difficult to obtain except for the Gaussian
case in which Y(t) is Gaussian, if X(t) is Gaussian.
102Vijaya Laxmi, Dept. of EEE
103. Mean and Autocorrelation function
• Assuming that h(t) and X(t) are real valued and that the expectation and
integration can be interchanged because integration is a linear operator,
mean and autocorrelation function of the output is calculated as
dhtXEtYE
103
2121 , tYtYEttR YY
dht
dhtXE
X
21221121
21222111
,
ddttRhh
ddhtXhtXE
XX
Vijaya Laxmi, Dept. of EEE
104. Stationarity of the output
• We have,
• If the processes X(t) and X(t+ε) have the same distribution
(i.e., X(t) is SSS) then the same is true for Y(t) and Y(t+ε) and
dhtXtY
)( dhtXtY
and
(i.e., X(t) is SSS) then the same is true for Y(t) and Y(t+ε) and
hence Y(t) is SSS.
• If X(t) is WSS, then mean does not depend upon t and we
have
• Thus, the mean of the output does not depend on time.
dhtYE X
)(
104
)0(Hdh XX
Vijaya Laxmi, Dept. of EEE
105. • The autocorrelation function of the output is given
by
2112122121, ddttRhhttR XXYY
Since, the integral only depends on the time difference t2-t1,
RYY(t1,t2) will also be a function of the difference t2-t1. This is coupled
with the fact that the output process Y(t) is WSS, if the input process X(t)
is WSS.
105Vijaya Laxmi, Dept. of EEE
106. PSD of the output
• When X(t) is WSS, it can be shown that
hRR
hRR
XXXY
XXYX
*
*
hR
tXdthxE
tXtYER
XX
YX
*
hhR
hRR
XX
YXYY
**
*
where * denotes convolution
• Taking Fourier transform of both sides, PSD of the output is obtained as
2
fHfSfS XXYY
The input spectral component at frequency f is modified according
to |H(f)|2 ,hence is sometimes called power transfer function.
106
hhRXX **
Vijaya Laxmi, Dept. of EEE
107. Mean square value of the output
• The mean square value of the output, which is a
measure of the average value of the output power.
• It is given by
dffSRtYE YYYY
2
0
• Assumption that Rxx(τ) can be expressed as a sum of
complex exponentials [i.e., Sxx(f) is a rational function
of f] simplifies the integral.
dffHfS
dffSRtYE
XX
YYYY
2
0
Except in some cases, evaluation of preceding integral in difficult.
107Vijaya Laxmi, Dept. of EEE
108. • The transformation s=2πjf is used and we have
where a(s)/b(s) has all of its poles and zeros in the
LHS and a(-s)/b(-s) has all its roots in the RHS.
,
2 sbsb
sasa
j
s
SXX
LHS and a(-s)/b(-s) has all its roots in the RHS.
• Therefore,
)()(
)()(
|*,
)()(
)()(
2
1
2/
2
sdsd
scsc
fHfHfSwhere
ds
sdsd
scsc
j
tYE
jsfXX
j
j
108Vijaya Laxmi, Dept. of EEE
109. Problem
• X(t) is the input voltage to the system shown in
Figure is a stationary RP with
• Find the mean, PSD and autocorrelation of the
exp0 XXX Rand
• Find the mean, PSD and autocorrelation of the
output.
109Vijaya Laxmi, Dept. of EEE
110. Solution
• We have
fLjR
R
fH
2
22
2
2
f
dfjdfjfS
Also
XX 2expexp2expexp
,
0
0
22
2
22
22
2
,
fLR
R
f
fSSo YY
110
2 f
L
R
L
R
L
R
L
R
L
R
R
FTinverseTaking
YY 2
22
2
2
exp
,
0, YhaveWe
Vijaya Laxmi, Dept. of EEE
111. Problem
• The input to a RC lowpass filter with
is a zero mean stationary RP with
1000/1
1
fj
fH
HzwattfSXX /10 12
• Find the mean square value of output Y(t).
111Vijaya Laxmi, Dept. of EEE