Optimal State Estimation , Sensor Fusion Study - Ch2. Probability Theory [Stella]

1. Probability theory
Stella Seo Yeon Yang
AI Robotics KR Sensor Fusion Study

2. Index
-2.1 Probability
-2.2 Random Variables
-2.3 Transformations of Random Variables
-2.4 Multiple Random Variables
2.4.1 Statistical independence
2.4.2 Multivariate statistics
- 2.5 Stochastic process
- 2.6 White noise and colored noise
- 2.7 Simulating correlated noise

3. 2.1 PROBABILITY
Relative Frequency Definition
R exp, Repeated R exp, Not Repeated Not Repeated, Ordered Not Repeated, Not Ordered

4. 2.1 PROBABILITY
Conditional Probability

5. 2.1 PROBABILITY
Bayes’ Rule
Independent

6. 2.2 RANDOM VARIABLES
Random Variable (RV) : a functional mapping from a set of experimental
outcomes (the domain) to a set of real numbers (the range)

7. 2.2 RANDOM VARIABLES

8. 2.2 RANDOM VARIABLES
Probability distribution function (pdf)

9. 2.2 RANDOM VARIABLES
Probability density function (PDF)
Q- function

10. 2.2 RANDOM VARIABLES
Conditional probability distribution function (PDF)
Chapman-Kolmogorov equation

11. 2.2 RANDOM VARIABLES
Expectation(기댓값) : The expected value of an RV X is defined as its average
value over a large number of experiments. This can also be called the
expectation, the mean, or the average of the RV

12. 2.2 RANDOM VARIABLES
i th moment / central moment:

13. 2.2 RANDOM VARIABLES
https://www.kdnuggets.com/2020/02
/probability-distributions-data-scienc
e.html
Distributions

14. 2.2 RANDOM VARIABLES
Gaussian Distributions

15. 2.2 RANDOM VARIABLES
So for odd i, the i th moment in Equation is zero. We see that all of the odd moments of a
zero-mean random variable with a symmetric pdf are equal to 0 .
For Odd i,
i th Momentum

16. 2.3 TRANSFORMATIONS OF RANDOM VARIABLES
RVs, X and Y, related to one another by
the monotonic^2 functions g() and h()
RVs, X and Y, related to one another by
the nonmonotonic function, g() and h()

17. 2.3 TRANSFORMATIONS OF RANDOM VARIABLES
EXAMPLE 2.4 :

18. 2.4 MULTIPLE RANDOM VARIABLES
Joint Distribution Function

19. 2.4 MULTIPLE RANDOM VARIABLES
Joint Probability Density Function
Expected Value of a function g( )

20. 2.4.1 Statistical independence
Extend the independence definition to joint PDF
Central limit theorem
Sum of
independent RVs

21. 2.4.1 Statistical independence
Covariance of RVs X and Y
Correlation coefficient of RVs X and Y

22. 2.4.1 Statistical independence
Correlation of two scalar RVs X and Y
uncorrelated
Independence vs Correlation
orthogonal
http://jaejunyoo.blogspot.com/2018/08/what-is-relationship-between-orthogonal.html
uncorrelated independent
uncorrelated independent : Both Gaussian

23. 2.4.2 Multivariate statistics
n-element RV X , m-element RV Y,
Correlation
Covariance
Autocorrelation

24. 2.4.2 Multivariate statistics
n-element column vector z, with autocorrelation
positive semidefinite.
n-element column vector z, with autocovariance
positive semidefinite.

25. 2.4.2 Multivariate statistics

26. 2.5 STOCHASTIC PROCESSES
-If the RV at each time is continuous and time is continuous, then X(t) is a continuous random process.
EX. the temperature at each moment of the day is a continuous random process because both temperature and time are
continuous.
- If the RV at each time is discrete and time is continuous, then X(t) is a discrete random process.
EX. the number of people in a given building at each moment of the day is a discrete random process because the number
of people is a discrete variable and time is continuous.
- If the RV at each time is continuous and time is discrete, then X(t) is a continuous random sequence.
EX. the high temperature each day is a continuous random sequence because temperature is continuous but time is
discrete (day one, day two, etc.).
If the RV at each time is discrete and time is discrete, then X(t) is a discrete random sequence.
EX. the highest number of people in a given building each day is a discrete random sequence because the number of
people is a discrete variable and time is also discrete.

27. 2.5 STOCHASTIC PROCESSES
a stochastic process is an RV that changes with time, it has a distribution and density function that are functions of time

28. 2.5 STOCHASTIC PROCESSES
joint distribution, joint density functions
autocorrelation
autocovariance

29. 2.5 STOCHASTIC PROCESSES
strict-sense stationary (SSS) : For some stochastic processes, the pdf does not change with time. In this case, the
stochastic process is called strict-sense stationary (SSS)
just stationary for short. In this case, the mean of the stochastic process is constant with respect to time, and
wide-sense stationary (WSS) :
For some stochastic processes, these two conditions are true even though the pdf does change with time.
Stochastic processes for which these two conditions are true are called wide-sense stationary (WSS).

30. 2.5 STOCHASTIC PROCESSES

31. 2.5 STOCHASTIC PROCESSES
Stationary vs WSS
Stationary Wide sense
stationary
Time Average
Auto correlation
These quantities are defined for continuous-time random processes as

32. 2.5 STOCHASTIC PROCESSES
Ergodic Process
1. Suppose each unit of an electrical instrument is manufactured with a small random bias. Is the noise of the instrumentation
ergodic? If we measure the noise of one instrument then we measure its bias, which is equal to its mean. However, if
we measure the noise of another instrument it might have a different mean because it has a different bias. In other
words, we cannot obtain the mean of the stochastic process by simply investigating one instrument (i.e., one realization of the
stochastic process). Therefore, the stochastic process is not ergodic.
2. Suppose each unit of an electrical instrument is manufactured identically, each with zero-mean stationary Gaussian
noise. Is the noise ergodic? In this case we could measure the mean of the process by measuring the noise of many
separate instruments at one instant of time, or by measuring the noise of one instrument over an extended period of
time. Either experiment would correctly inform us that the mean of the stochastic process is zero. We could find the
statistics of the stochastic process using all the instruments at a single time, or using a single instrument at many different
times. Therefore, the stochastic process is ergodic.

33. 2.5 STOCHASTIC PROCESSES

34. 2.6 WHITE NOISE AND COLORED NOISE
Power Spectrum
Autocorrelation
Power of
WSS stochastic process
Wiener-Khintchine Relation

35. 2.6 WHITE NOISE AND COLORED NOISE
Cross Power Spectrum
Discrete-time random processes

36. 2.6 WHITE NOISE AND COLORED NOISE
Discrete-time
white noise
Continuous-time
white noise

37. 2.6 WHITE NOISE AND COLORED NOISE

38. 2.7 SIMULATING CORRELATED NOISE

39. 2.7 SIMULATING CORRELATED NOISE

40. 2.7 SIMULATING CORRELATED NOISE
Correlated noise simulation

41. 감사합니다!
Q & A