0
Stochastic Processes Elements of Stochastic Processes By Mahdi Malaki
Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of ...
Basic Definitions <ul><li>Suppose a set of random variables indexed by a parameter </li></ul><ul><li>Tracking these variab...
Basic Definitions  (cont’d) <ul><li>In a random process, we will have a family of functions called an  ensemble of functio...
Basic Definitions  (cont’d) <ul><li>With fixed “beta”, we will have a  “time”  function called  sample path . </li></ul><u...
Basic Definitions  (cont’d) <ul><li>With fixed “t”, we will have a random variable. </li></ul><ul><li>With fixed “t” and “...
Basic Definitions  (cont’d) <ul><li>Example I </li></ul><ul><ul><li>Voltage of a generator with fixed frequency </li></ul>...
Basic Definitions  (cont’d) <ul><li>Equality </li></ul><ul><ul><li>Ensembles should be equal for each “beta” and “t” </li>...
Basic Definitions  (cont’d) <ul><li>First-Order CDF of a random process </li></ul><ul><li>First-Order PDF of a random proc...
Basic Definitions  (cont’d) <ul><li>Second-Order CDF of a random process </li></ul><ul><li>Second-Order PDF of a random pr...
Basic Definitions  (cont’d) <ul><li>n th  order can be defined. (How?) </li></ul><ul><li>Relation between first-order and ...
Basic Definitions  (cont’d) <ul><li>Mean  of a random process </li></ul><ul><li>Autocorrelation  of a random process </li>...
Basic Definitions  (cont’d) <ul><li>Autocovariance  of a random process </li></ul><ul><li>Correlation Coefficient </li></u...
Basic Definitions  (cont’d) <ul><li>Example </li></ul><ul><ul><li>Poisson Process </li></ul></ul><ul><ul><li>Mean </li></u...
Basic Definitions  (cont’d) <ul><li>Complex process </li></ul><ul><ul><li>Definition </li></ul></ul><ul><ul><li>Specified ...
Basic Definitions  (cont’d) <ul><li>Cross-Correlation </li></ul><ul><ul><li>Orthogonal Processes </li></ul></ul><ul><li>Cr...
Basic Definitions  (cont’d) <ul><li>a -dependent processes </li></ul><ul><li>White Noise </li></ul><ul><li>Variance of Sto...
Basic Definitions  (cont’d) <ul><li>Existence Theorem </li></ul><ul><ul><li>For an arbitrary mean function  </li></ul></ul...
Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of ...
Stationary/Ergodic Processes <ul><li>Strict Sense Stationary (SSS) </li></ul><ul><ul><li>Statistical properties are invari...
Stationary/Ergodic Processes  (cont’d) <ul><li>Wide Sense Stationary (WSS) </li></ul><ul><ul><li>Mean is constant </li></u...
Stationary/Ergodic Processes  (cont’d) <ul><li>Autocovariance of a WSS process </li></ul><ul><li>Correlation Coefficient <...
Stationary/Ergodic Processes  (cont’d) <ul><li>White Noise </li></ul><ul><ul><li>If white noise is an stationary process, ...
Stationary/Ergodic Processes  (cont’d) <ul><li>Example </li></ul><ul><ul><li>SSS </li></ul></ul><ul><ul><ul><li>Suppose  a...
Stationary/Ergodic Processes  (cont’d) <ul><li>Example </li></ul><ul><ul><li>Suppose for a WSS process </li></ul></ul><ul>...
Stationary/Ergodic Processes  (cont’d) <ul><li>Ergodic Process </li></ul><ul><ul><li>Equality of  time  properties and  st...
Stationary/Ergodic Processes  (cont’d) <ul><li>Slutsky’s Theorem </li></ul><ul><ul><li>A process X(t) is  mean-ergodic  if...
Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of ...
Stochastic Analysis of Systems <ul><li>Linear Systems </li></ul><ul><li>Time-Invariant Systems </li></ul><ul><li>Linear Ti...
Stochastic Analysis of Systems  (cont’d) <ul><li>Memoryless Systems </li></ul><ul><li>Causal Systems </li></ul><ul><ul><li...
Stochastic Analysis of Systems  (cont’d) <ul><li>Linear time-invariant systems </li></ul><ul><ul><li>Mean </li></ul></ul><...
Stochastic Analysis of Systems  (cont’d) <ul><li>Example I  </li></ul><ul><ul><li>System:  </li></ul></ul><ul><ul><li>Impu...
Stochastic Analysis of Systems  (cont’d) <ul><li>Example II </li></ul><ul><ul><li>System:  </li></ul></ul><ul><ul><li>Impu...
Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of ...
Power Spectrum <ul><li>Definition </li></ul><ul><ul><li>WSS process </li></ul></ul><ul><ul><li>Autocorrelation </li></ul><...
Power Spectrum  (cont’d) <ul><li>Inverse trnasform </li></ul><ul><li>For real processes </li></ul>
Power Spectrum  (cont’d) <ul><li>For a linear time invariant system </li></ul><ul><li>Fact (Why?) </li></ul>
Power Spectrum  (cont’d) <ul><li>Example I (Moving Average) </li></ul><ul><ul><li>System </li></ul></ul><ul><ul><li>Impuls...
Power Spectrum  (cont’d) <ul><li>Example II </li></ul><ul><ul><li>System </li></ul></ul><ul><ul><li>Impulse Response </li>...
<ul><li>Question? </li></ul>
Upcoming SlideShare
Loading in...5
×

Elements Of Stochastic Processes

2,984

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,984
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
139
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Elements Of Stochastic Processes"

  1. 1. Stochastic Processes Elements of Stochastic Processes By Mahdi Malaki
  2. 2. Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of Systems </li></ul><ul><li>Power Spectrum </li></ul>
  3. 3. Basic Definitions <ul><li>Suppose a set of random variables indexed by a parameter </li></ul><ul><li>Tracking these variables with respect to the parameter constructs a process that is called Stochastic Process . </li></ul><ul><li>i.e. The mapping of outcomes to the real (complex) numbers changes with respect to index. </li></ul>
  4. 4. Basic Definitions (cont’d) <ul><li>In a random process, we will have a family of functions called an ensemble of functions </li></ul>
  5. 5. Basic Definitions (cont’d) <ul><li>With fixed “beta”, we will have a “time” function called sample path . </li></ul><ul><li>Sometimes stochastic properties of a random process can be extracted just from a single sample path. (When?) </li></ul>
  6. 6. Basic Definitions (cont’d) <ul><li>With fixed “t”, we will have a random variable. </li></ul><ul><li>With fixed “t” and “beta”, we will have a real (complex) number. </li></ul>
  7. 7. Basic Definitions (cont’d) <ul><li>Example I </li></ul><ul><ul><li>Voltage of a generator with fixed frequency </li></ul></ul><ul><ul><ul><li>Amplitude is a random variables </li></ul></ul></ul>
  8. 8. Basic Definitions (cont’d) <ul><li>Equality </li></ul><ul><ul><li>Ensembles should be equal for each “beta” and “t” </li></ul></ul><ul><li>Equality (Mean Square Sense) </li></ul><ul><ul><li>If the following equality holds </li></ul></ul><ul><ul><li>Sufficient in many applications </li></ul></ul>
  9. 9. Basic Definitions (cont’d) <ul><li>First-Order CDF of a random process </li></ul><ul><li>First-Order PDF of a random process </li></ul>
  10. 10. Basic Definitions (cont’d) <ul><li>Second-Order CDF of a random process </li></ul><ul><li>Second-Order PDF of a random process </li></ul>
  11. 11. Basic Definitions (cont’d) <ul><li>n th order can be defined. (How?) </li></ul><ul><li>Relation between first-order and second-order can be presented as </li></ul><ul><li>Relation between different orders can be obtained easily. (How?) </li></ul>
  12. 12. Basic Definitions (cont’d) <ul><li>Mean of a random process </li></ul><ul><li>Autocorrelation of a random process </li></ul><ul><li>Fact: (Why?) </li></ul>
  13. 13. Basic Definitions (cont’d) <ul><li>Autocovariance of a random process </li></ul><ul><li>Correlation Coefficient </li></ul><ul><li>Example </li></ul>
  14. 14. Basic Definitions (cont’d) <ul><li>Example </li></ul><ul><ul><li>Poisson Process </li></ul></ul><ul><ul><li>Mean </li></ul></ul><ul><ul><li>Autocorrelation </li></ul></ul><ul><ul><li>Autocovariance </li></ul></ul>
  15. 15. Basic Definitions (cont’d) <ul><li>Complex process </li></ul><ul><ul><li>Definition </li></ul></ul><ul><ul><li>Specified in terms of the joint statistics of two real processes and </li></ul></ul><ul><li>Vector Process </li></ul><ul><ul><li>A family of some stochastic processes </li></ul></ul>
  16. 16. Basic Definitions (cont’d) <ul><li>Cross-Correlation </li></ul><ul><ul><li>Orthogonal Processes </li></ul></ul><ul><li>Cross-Covariance </li></ul><ul><ul><li>Uncorrelated Processes </li></ul></ul>
  17. 17. Basic Definitions (cont’d) <ul><li>a -dependent processes </li></ul><ul><li>White Noise </li></ul><ul><li>Variance of Stochastic Process </li></ul>
  18. 18. Basic Definitions (cont’d) <ul><li>Existence Theorem </li></ul><ul><ul><li>For an arbitrary mean function </li></ul></ul><ul><ul><li>For an arbitrary covariance function </li></ul></ul><ul><ul><li>There exist a normal random process that its mean is and its covariance is </li></ul></ul>
  19. 19. Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of Systems </li></ul><ul><li>Power Spectrum </li></ul>
  20. 20. Stationary/Ergodic Processes <ul><li>Strict Sense Stationary (SSS) </li></ul><ul><ul><li>Statistical properties are invariant to shift of time origin </li></ul></ul><ul><ul><li>First order properties should be independent of “t” or </li></ul></ul><ul><ul><li>Second order properties should depends only on difference of times or </li></ul></ul><ul><ul><li>… </li></ul></ul>
  21. 21. Stationary/Ergodic Processes (cont’d) <ul><li>Wide Sense Stationary (WSS) </li></ul><ul><ul><li>Mean is constant </li></ul></ul><ul><ul><li>Autocorrelation depends on the difference of times </li></ul></ul><ul><li>First and Second order statistics are usually enough in applications. </li></ul>
  22. 22. Stationary/Ergodic Processes (cont’d) <ul><li>Autocovariance of a WSS process </li></ul><ul><li>Correlation Coefficient </li></ul>
  23. 23. Stationary/Ergodic Processes (cont’d) <ul><li>White Noise </li></ul><ul><ul><li>If white noise is an stationary process, why do we call it “noise”? (maybe it is not stationary !?) </li></ul></ul><ul><li>a -dependent Process </li></ul><ul><ul><li>a is called “Correlation Time” </li></ul></ul>
  24. 24. Stationary/Ergodic Processes (cont’d) <ul><li>Example </li></ul><ul><ul><li>SSS </li></ul></ul><ul><ul><ul><li>Suppose a and b are normal random variables with zero mean. </li></ul></ul></ul><ul><ul><li>WSS </li></ul></ul><ul><ul><ul><li>Suppose “ ” has a uniform distribution in the interval </li></ul></ul></ul>
  25. 25. Stationary/Ergodic Processes (cont’d) <ul><li>Example </li></ul><ul><ul><li>Suppose for a WSS process </li></ul></ul><ul><ul><li>X(8) and X(5) are random variables </li></ul></ul>
  26. 26. Stationary/Ergodic Processes (cont’d) <ul><li>Ergodic Process </li></ul><ul><ul><li>Equality of time properties and statistic properties. </li></ul></ul><ul><li>First-Order Time average </li></ul><ul><ul><li>Defined as </li></ul></ul><ul><li>Mean Ergodic Process </li></ul><ul><li>Mean Ergodic Process in Mean Square Sense </li></ul>
  27. 27. Stationary/Ergodic Processes (cont’d) <ul><li>Slutsky’s Theorem </li></ul><ul><ul><li>A process X(t) is mean-ergodic iff </li></ul></ul><ul><ul><li>Sufficient Conditions </li></ul></ul><ul><ul><ul><li>a) </li></ul></ul></ul><ul><ul><ul><li>b) </li></ul></ul></ul>
  28. 28. Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of Systems </li></ul><ul><li>Power Spectrum </li></ul>
  29. 29. Stochastic Analysis of Systems <ul><li>Linear Systems </li></ul><ul><li>Time-Invariant Systems </li></ul><ul><li>Linear Time-Invariant Systems </li></ul><ul><ul><li>Where h(t) is called impulse response of the system </li></ul></ul>
  30. 30. Stochastic Analysis of Systems (cont’d) <ul><li>Memoryless Systems </li></ul><ul><li>Causal Systems </li></ul><ul><ul><li>Only causal systems can be realized. (Why?) </li></ul></ul>
  31. 31. Stochastic Analysis of Systems (cont’d) <ul><li>Linear time-invariant systems </li></ul><ul><ul><li>Mean </li></ul></ul><ul><ul><li>Autocorrelation </li></ul></ul>
  32. 32. Stochastic Analysis of Systems (cont’d) <ul><li>Example I </li></ul><ul><ul><li>System: </li></ul></ul><ul><ul><li>Impulse response: </li></ul></ul><ul><ul><li>Output Mean: </li></ul></ul><ul><ul><li>Output Autocovariance: </li></ul></ul>
  33. 33. Stochastic Analysis of Systems (cont’d) <ul><li>Example II </li></ul><ul><ul><li>System: </li></ul></ul><ul><ul><li>Impulse response: </li></ul></ul><ul><ul><li>Output Mean: </li></ul></ul><ul><ul><li>Output Autocovariance: </li></ul></ul>
  34. 34. Outline <ul><li>Basic Definitions </li></ul><ul><li>Stationary/Ergodic Processes </li></ul><ul><li>Stochastic Analysis of Systems </li></ul><ul><li>Power Spectrum </li></ul>
  35. 35. Power Spectrum <ul><li>Definition </li></ul><ul><ul><li>WSS process </li></ul></ul><ul><ul><li>Autocorrelation </li></ul></ul><ul><ul><li>Fourier Transform of autocorrelation </li></ul></ul>
  36. 36. Power Spectrum (cont’d) <ul><li>Inverse trnasform </li></ul><ul><li>For real processes </li></ul>
  37. 37. Power Spectrum (cont’d) <ul><li>For a linear time invariant system </li></ul><ul><li>Fact (Why?) </li></ul>
  38. 38. Power Spectrum (cont’d) <ul><li>Example I (Moving Average) </li></ul><ul><ul><li>System </li></ul></ul><ul><ul><li>Impulse Response </li></ul></ul><ul><ul><li>Power Spectrum </li></ul></ul><ul><ul><li>Autocorrelation </li></ul></ul>
  39. 39. Power Spectrum (cont’d) <ul><li>Example II </li></ul><ul><ul><li>System </li></ul></ul><ul><ul><li>Impulse Response </li></ul></ul><ul><ul><li>Power Spectrum </li></ul></ul>
  40. 40. <ul><li>Question? </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×