Turning from discrete to continuous distributions, in this section we discuss the normal distribution. This is the most important continuous distribution because in applications many random variables are normal random variables (that is, they have a normal distribution) or they are approximately normal or can be transformed into normal random variables in a relatively simple fashion. Furthermore, the normal distribution is a useful approximation of more complicated distributions, and it also occurs in the proofs of various statistical tests.
Normal Distribution, also called Gaussian Distribution, is one of the widely used continuous distributions existing which is used to model a number of scenarios such as marks of students, heights of people, salaries of working people etc.
Each binomial distribution is defined by n, the number of trials and p, the probability of success in any one trial.
Each Poisson distribution is defined by its mean.
In the same way, each Normal distribution is identified by two defining characteristics or parameters: its mean and standard deviation.
The Normal distribution has three distinguishing features:
• It is unimodal, in other words there is a single peak.
• It is symmetrical, one side is the mirror image of the other.
• It is asymptotic, that is, it tails off very gradually on each side but the line representing the distribution never quite meets the horizontal axis
Turning from discrete to continuous distributions, in this section we discuss the normal distribution. This is the most important continuous distribution because in applications many random variables are normal random variables (that is, they have a normal distribution) or they are approximately normal or can be transformed into normal random variables in a relatively simple fashion. Furthermore, the normal distribution is a useful approximation of more complicated distributions, and it also occurs in the proofs of various statistical tests.
Normal Distribution, also called Gaussian Distribution, is one of the widely used continuous distributions existing which is used to model a number of scenarios such as marks of students, heights of people, salaries of working people etc.
Each binomial distribution is defined by n, the number of trials and p, the probability of success in any one trial.
Each Poisson distribution is defined by its mean.
In the same way, each Normal distribution is identified by two defining characteristics or parameters: its mean and standard deviation.
The Normal distribution has three distinguishing features:
• It is unimodal, in other words there is a single peak.
• It is symmetrical, one side is the mirror image of the other.
• It is asymptotic, that is, it tails off very gradually on each side but the line representing the distribution never quite meets the horizontal axis
Numerical Analysis and Its application to Boundary Value ProblemsGobinda Debnath
here is a presentation that I have presented on a webinar. in this presentation, I mainly focused on the application of numerical analysis to Boundary Value problems. I described one of the most useful traditional methods called the finite difference method. those who are interested to do research in applied mathematics, CFD, Numerical analysis may go through it for the basic ideas.
Markov chains are a very common model for systems that change probablistically over time. We show a few fun examples, define the objects, state the main theorems, and show how to find the steady-state vector.
"Random variables", "stochastic Processes" graduate course.
Lecture notes of Prof. H.Amindavar.
Professor of Electrical engineering at Amirkabir university of technology.
Different kind of distance and Statistical DistanceKhulna University
A short brief of distance and statistical distance which is core of multivariate analysis.................you will get here some more simple conception about distances and statistical distance.
Numerical Analysis and Its application to Boundary Value ProblemsGobinda Debnath
here is a presentation that I have presented on a webinar. in this presentation, I mainly focused on the application of numerical analysis to Boundary Value problems. I described one of the most useful traditional methods called the finite difference method. those who are interested to do research in applied mathematics, CFD, Numerical analysis may go through it for the basic ideas.
Markov chains are a very common model for systems that change probablistically over time. We show a few fun examples, define the objects, state the main theorems, and show how to find the steady-state vector.
"Random variables", "stochastic Processes" graduate course.
Lecture notes of Prof. H.Amindavar.
Professor of Electrical engineering at Amirkabir university of technology.
Different kind of distance and Statistical DistanceKhulna University
A short brief of distance and statistical distance which is core of multivariate analysis.................you will get here some more simple conception about distances and statistical distance.
https://utilitasmathematica.com/index.php/Index
Utilitas Mathematica journal that publishes original research. This journal publishes mainly in areas of pure and applied mathematics, statistics and others like algebra, analysis, geometry, topology, number theory, diffrential equations, operations research, mathematical physics, computer science,mathematical economics.And it is official publication of Utilitas Mathematica Academy, Canada.
On Twisted Paraproducts and some other Multilinear Singular IntegralsVjekoslavKovac1
Presentation.
9th International Conference on Harmonic Analysis and Partial Differential Equations, El Escorial, June 12, 2012.
The 24th International Conference on Operator Theory, Timisoara, July 3, 2012.
Existance Theory for First Order Nonlinear Random Dfferential Equartioninventionjournals
In this paper, the existence of a solution of nonlinear random differential equation of first order is proved under Caratheodory condition by using suitable fixed point theorem. 2000 Mathematics Subject Classification: 34F05, 47H10, 47H4
In this talk, we discuss some recent advances in probabilistic schemes for high-dimensional PIDEs. It is known that traditional PDE solvers, e.g., finite element, finite difference methods, do not scale well with the increase of dimension. The idea of probabilistic schemes is to link a wide class of nonlinear parabolic PIDEs to stochastic Levy processes based on nonlinear version of the Feynman-Kac theory. As such, the solution of the PIDE can be represented by a conditional expectation (i.e., a high-dimensional integral) with respect to a stochastic dynamical system driven by Levy processes. In other words, we can solve the PIDEs by performing high-dimensional numerical integration. A variety of quadrature methods could be applied, including MC, QMC, sparse grids, etc. The probabilistic schemes have been used in many application problems, e.g., particle transport in plasmas (e.g., Vlasov-Fokker-Planck equations), nonlinear filtering (e.g., Zakai equations), and option pricing, etc.
Seminar Talk: Multilevel Hybrid Split Step Implicit Tau-Leap for Stochastic R...Chiheb Ben Hammouda
In biochemically reactive systems with small copy numbers of one or more reactant molecules, the dynamics are dominated by stochastic effects. To approximate those systems, discrete state-space and stochastic simulation approaches have been shown to be more relevant than continuous state-space and deterministic ones. These stochastic models constitute the theory of Stochastic Reaction Networks (SRNs). In systems characterized by having simultaneously fast and slow timescales, existing discrete space-state stochastic path simulation methods, such as the stochastic simulation algorithm (SSA) and the explicit tau-leap (explicit-TL) method, can be very slow. In this talk, we propose a novel implicit scheme, split-step implicit tau-leap (SSI-TL), to improve numerical stability and provide efficient simulation algorithms for those systems. Furthermore, to estimate statistical quantities related to SRNs, we propose a novel hybrid Multilevel Monte Carlo (MLMC) estimator in the spirit of the work by Anderson and Higham (SIAM Multiscal Model. Simul. 10(1), 2012). This estimator uses the SSI-TL scheme at levels where the explicit-TL method is not applicable due to numerical stability issues, and then, starting from a certain interface level, it switches to the explicit scheme. We present numerical examples that illustrate the achieved gains of our proposed approach in this context.
We apply tensor train (TT) data format to solve an elliptic PDE with uncertain coefficients. We reduce complexity and storage from exponential to linear. Post-processing in TT format is also provided.
"Digital communications", undergraduate course,Fall 2018.
Prof. H.Amindavar, Electrical engineering,Amirkabir university of technology.
written by S.Ghasemi.
"Wavelet Signal Processing",graduate course.
Lecture notes of Prof. H. Amindavar.
Professor of Electrical engineering, Amirkabir university of technology
"Detection & Estimation Theory" graduate course.
Lecture notes of Prof. H.Amindavar.
Professor of Electrical engineering at Amirkabir university of technology.
A brief discussion of cyclostationary processes.
Prof. H.Amindavar complementary notes for the first session of "Advanced communications theory" course, Spring 2021
A brief discussion of Multivariate Gaussin, Rayleigh & Rician distributions
Prof. H.Amindavar complementary notes for the first session of "Advanced communications theory" course, Spring 2021
A brief discussion of Introduction to communication systems.
Prof. H.Amindavar complementary notes for the first session of "Advanced communications theory" course, Spring 2021
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
This presentation was provided by Steph Pollock of The American Psychological Association’s Journals Program, and Damita Snow, of The American Society of Civil Engineers (ASCE), for the initial session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session One: 'Setting Expectations: a DEIA Primer,' was held June 6, 2024.
Thinking of getting a dog? Be aware that breeds like Pit Bulls, Rottweilers, and German Shepherds can be loyal and dangerous. Proper training and socialization are crucial to preventing aggressive behaviors. Ensure safety by understanding their needs and always supervising interactions. Stay safe, and enjoy your furry friends!
This presentation includes basic of PCOS their pathology and treatment and also Ayurveda correlation of PCOS and Ayurvedic line of treatment mentioned in classics.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
বাংলাদেশের অর্থনৈতিক সমীক্ষা ২০২৪ [Bangladesh Economic Review 2024 Bangla.pdf] কম্পিউটার , ট্যাব ও স্মার্ট ফোন ভার্সন সহ সম্পূর্ণ বাংলা ই-বুক বা pdf বই " সুচিপত্র ...বুকমার্ক মেনু 🔖 ও হাইপার লিংক মেনু 📝👆 যুক্ত ..
আমাদের সবার জন্য খুব খুব গুরুত্বপূর্ণ একটি বই ..বিসিএস, ব্যাংক, ইউনিভার্সিটি ভর্তি ও যে কোন প্রতিযোগিতা মূলক পরীক্ষার জন্য এর খুব ইম্পরট্যান্ট একটি বিষয় ...তাছাড়া বাংলাদেশের সাম্প্রতিক যে কোন ডাটা বা তথ্য এই বইতে পাবেন ...
তাই একজন নাগরিক হিসাবে এই তথ্য গুলো আপনার জানা প্রয়োজন ...।
বিসিএস ও ব্যাংক এর লিখিত পরীক্ষা ...+এছাড়া মাধ্যমিক ও উচ্চমাধ্যমিকের স্টুডেন্টদের জন্য অনেক কাজে আসবে ...
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Dr. Vinod Kumar Kanvaria
Exploiting Artificial Intelligence for Empowering Researchers and Faculty,
International FDP on Fundamentals of Research in Social Sciences
at Integral University, Lucknow, 06.06.2024
By Dr. Vinod Kumar Kanvaria
How to Fix the Import Error in the Odoo 17Celine George
An import error occurs when a program fails to import a module or library, disrupting its execution. In languages like Python, this issue arises when the specified module cannot be found or accessed, hindering the program's functionality. Resolving import errors is crucial for maintaining smooth software operation and uninterrupted development processes.
Film vocab for eal 3 students: Australia the movie
Stochastic Processes - part 1
1. Let ξ denote the random outcome of an experiment. To ev-
ery such outcome suppose a waveform X(t, ξ) is assigned.
The collection of such waveforms form a stochastic process.
The set of {ξk} and the time index t can be continuous or
discrete (countably infinite or finite) as well. For fixed ξk ∈ S
(the set of all experimental outcomes), X(t, ξ) is a specific
time function. For fixed t, X1 = X(t1, ξi) is a random vari-
able.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.1/105
2. The ensemble of all such realizations X(t, ξ) over time
represents the stochastic process X(t). For example
X(t) = a cos(ω0t + φ), φ ∼ U(0, 2π)
Stochastic processes are everywhere: Brownian motion,
stock market fluctuations, various queuing systems all
represent stochastic phenomena.
If X(t) is a stochastic process, then for fixed t, X(t) repre-
sents a random variable.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.2/105
3. Its distribution function is given by
FX(x, t) = P{X(t) ≤ x}
Notice that FX(x, t) depends on t, since for a different t, we
obtain a different random variable. Further
fX(x, t) =
dFX(x, t)
dx
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.3/105
4. FX(x1, x2, t1, t2) = P{X(t1) ≤ x1, X(t2) ≤ x2}
fX(x1, x2, t1, t2) =
∂2
FX(x1, x2, t1, t2)
∂x1∂x2
represents the 2nd order density function of the process
X(t).
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.4/105
5. Similarly fX(x1, x2, · · · xn, t1, t2 · · · , tn)represents the nth or-
der density function of the process X(t). Complete specifica-
tion of the stochastic process X(t) requires the knowledge
of fX(x1, x2, · · · xn, t1, t2 · · · , tn) for all ti, i = 1, 2, · · · , n
and for all n, an almost impossible task in reality.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.5/105
6. µ(t) = E{X(t)} =
Z +∞
−∞
xfX(x, t)dx
In general, the mean of a process can depend on the time
index t.
RXX(t1, t2)=E{X(t1)X∗
(t2)}=
Z Z
x1x∗
2fX(x1, x2, t1, t2)dx1dx2
It represents the interrelationship between the random vari-
ables X1 = X(t1) and X2 = X(t2) generated from the pro-
cess X(t).
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.6/105
8. Example:
z =
Z T
−T
X(t)dt
E[|z|2
] =
Z T
−T
Z T
−T
E{X(t1)X∗
(t2)}dt1dt2
=
Z T
−T
Z T
−T
RXX(t1, t2)dt1dt2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.8/105
10. Stationary Stochastic Processes
Stationary processes exhibit statistical properties that are
invariant to shift in the time index. Thus, for example,
second-order stationarity implies that the statistical proper-
ties of the pairs {X(t1), X(t2)} and {X(t1 +c), X(t2 +c)} are
the same for any c. Similarly first-order stationarity implies
that the statistical properties of X(ti) and X(ti + c) are the
same for any c.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.10/105
11. Strict-Sense Stationary (S.S.S)
if
fX(x1, · · · xn, t1, · · · , tn) ≡ fX(x1, · · · xn, t1 +c, · · · , tn +c) (1)
for any c, where the left side represents the joint density
function of the random variables
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.11/105
12. X1 = X(t1), X2 = X(t2), · · · , Xn = X(tn) and the right side
corresponds to the joint density function of the random
variables
X′
1 = X(t1 + c), X′
2 = X(t2 + c), · · · , X′
n = X(tn + c).
A process X(t) is said to be strict-sense stationary if (1) is
true for all ti, i = 1, 2, · · · , n, n = 1, 2, · · · , for any c.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.12/105
13. For a first-order strict sense stationary process,
fX(x, t) ≡ fX(x, t + c)
for c = −t,
fX(x, t) = fX(x) (2)
the first-order density of X(t) is independent of t. In that
case
E[X(t)] =
Z +∞
−∞
xf(x)dx = µ, a constant.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.13/105
14. Similarly, for a 2nd-order strict-sense stationary process
fX(x1, x2, t1, t2) ≡ fX(x1, x2, t1 + c, t2 + c)
For c = −t2:
fX(x1, x2, t1, t2) ≡ fX(x1, x2, t1 − t2) (3)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.14/105
15. the second order density function of a strict sense
stationary process depends only on the difference of the
time indices t2 − t2 = τ. In that case the autocorrelation
function is given by
E{X(t1)X∗
(t2)} =
Z Z
x1x∗
2fX(x1, x2, τ = t1 − t2)dx1dx2
= RXX(t1 − t2) = RXX(τ) = R∗
XX(−τ),
the E{X(t1)X∗
(t2)} of a 2nd SSS depends only on τ.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.15/105
16. The conditions for 1st and 2nd order SSS are in equations
(2), (3), these are usually hard to verify. For this reason we
resort to wide-sense stationarity WSS.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.16/105
17. WSS:
1. E{X(t)} = µ
2. E{X(t1)X∗
(t2)} = RXX(|t2 − t1|)
SSS always implies WSS, but the converse is not true, but,
the only exception are Gaussian processes.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.17/105
18. This follows, since if X(t) is a Gaussian process, then by
definition
X1 = X(t1), X2 = X(t2), · · · , Xn = X(tn)
are jointly Gaussian random variables for any t1, t2 · · · , tn
whose joint characteristic function is given by
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.18/105
19. φX(ω1, ω2, · · · , ωn) = e
j
n
P
k=1
µ(tk)ωk−0.5
n
P
l=1
n
P
k=1
CXX (tl,tk)ωlωk
CXX(ti, tk) is the element of covariance matrix, for WSS
process CXX(ti, tk) = CXX(ti − tk)
φX(ω1, ω2, · · · , ωn) = e
j
n
P
k=1
µωk−0.5
n
P
l=1
n
P
k=1
CXX (tl−tk)ωlωk
(4)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.19/105
20. and hence if the set of time indices are shifted by a
constant c to generate a new set of jointly Gaussian
random variables X′
2 = X(t2 + c), · · · ,X′
n = X(tn + c) then
their joint characteristic function is identical to (4). Thus the
set of random variables {Xi}n
i=1 and {X′
i}n
i=1 have the
same JPDF for all n and all c, establishing the SSS of
Gaussian processes from it WSS.
For Gaussian processes: WSS⇒SSS
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.20/105
26. C = V DV H
, V = [V1 V2]
C =
0.5 −.15
−0.15 0.2
V =
0.31625 −.9487
−0.9487 0.3162
D =
0.05 0
0 0.55
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.26/105
32. RY Y (t1, t2) = E{Y (t1)Y ∗
(t2)}
= E{
Z +∞
−∞
X(t1 − β)h(β)dβY ∗
(t2)}
=
Z +∞
−∞
E{X(t1 − β)Y ∗
(t2)}h(β)dβ
=
Z +∞
−∞
RXY (t1 − β, t2)h(β)dβ
= RXY (t1, t2) ∗ h(t1),
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.32/105
33. RY Y (t1, t2) = RXX(t1, t2) ∗ h∗
(t2) ∗ h(t1).
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.33/105
34. If X(t) is WSS:
µY (t) = µX
Z +∞
−∞
h(τ)dτ = µXc, a constant .
RXY (t1, t2) =
Z +∞
−∞
RXX(t1 − t2 + α)h∗
(α)dα
= RXX(τ) ∗ h∗
(−τ) = RXY (τ), τ = t1 − t2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.34/105
35. RY Y (t1, t2) =
Z +∞
−∞
RXY (t1 − β − t2)h(β)dβ, τ = t1 − t2
= RXY (τ) ∗ h(τ) = RY Y (τ)
RY Y (τ) = RXX(τ) ∗ h∗
(−τ) ∗ h(τ).
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.35/105
36. Example: If X(t) is WSS:
z =
Z T
−T
X(t)dt.
Y (t) = X(t) ∗ h(t), h(t) = u(t + T) − u(t − T)
RY Y (τ) = RXX(τ) ∗ h(τ) ∗ h(−τ)
h(τ) ∗ h(−τ) = (2T − |τ|), |τ| ≤ 2T
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.36/105
37. Y (t) = g{X(t)}
SSS input SSS output
Memoryless
system
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.37/105
38. Y (t) = g{X(t)}
WSS input Can’t say about
stationarity
in any sense
Memoryless
system
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.38/105
39. Y (t) = g{X(t)}
Gaussian input Not Gaussian
Memoryless
system
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.39/105
40. If X(t) is a zero mean stationary Gaussian process, and
Y (t) = g{X(t)}, where g(·) represents a nonlinear
memoryless device, then
RXY (τ) = ηRXX(τ), η = E{g′
(X)}.
Proof:
RXY (τ) = E{X(t)Y (t − τ)} = E[X(t)g{X(t − τ)}]
=
Z Z
x1g(x2)fX1X2 (x1, x2)dx1dx2, (5)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.40/105
41. where X1 = X(t) and X2 = X(t − τ) are jointly Gaussian
RVs.
fX1X2 (x1, x2) = 1
2π
√
|A|
e−x∗A−1x/2
X = (X1, X2)T
, x = (x1, x2)T
A = E{X X∗
} =
RXX(0) RXX(τ)
RXX(τ) RXX(0)
= LL∗
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.41/105
42. where L is an upper triangular factor matrix with positive
diagonal entries.
L =
ℓ11 ℓ12
0 ℓ22
.
Consider the transformation:
Z = L−1
X = (Z1, Z2)T
, z = L−1
x = (z1, z2)T
so that
E{Z Z∗
} = L−1
E{XX∗
}L∗−1
= L−1
AL∗−1
= I
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.42/105
43. and hence Z1, Z2 are zero mean independent Gaussian
random variables. Also
x = Lz ⇒ x1 = ℓ11z1 + ℓ12z2, x2 = ℓ22z2
and hence,
x∗
A−1
x = z∗
L∗
A−1
Lz = z∗
z = z2
1 + z2
2
then the Jacobian of transformation
|J| = |L−1
| = |A|−1/2
.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.43/105
44. RXY (τ) =
Z +∞
−∞
Z +∞
−∞
(ℓ11z1 + ℓ12z2)g(ℓ22z2) 1
|J|
1
2π|A|1/2 e−z2
1/2
e−z2
2/2
= ℓ11
Z +∞
−∞
Z +∞
−∞
z1g(ℓ22z2)fz1 (z1)fz2 (z2)dz1dz2
+ ℓ12
Z +∞
−∞
Z +∞
−∞
z2g(ℓ22z2)fz1 (z1)fz2 (z2)dz1dz2
= ℓ11
Z +∞
−∞
z1fz1 (z1)dz1
Z +∞
−∞
g(ℓ22z2)fz2 (z2)dz2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.44/105
45. + ℓ12
Z +∞
−∞
z2g(ℓ22z2) fz2 (z2)
| {z }
1
√
2π
e−z2
2/2
dz2
= ℓ12
ℓ2
22
Z +∞
−∞
ug(u) 1
√
2π
e−u2/2ℓ2
22 du
where u = ℓ22z2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.45/105
46. This gives
RXY (τ) = ℓ12ℓ22
R +∞
−∞
g(u) u
ℓ2
22
fu(u)
z }| {
1
√
2πℓ2
22
e−u2/2ℓ2
22
| {z }
−
dfu(u)
du
=−f′
u(u)
du
= −RXX(τ)
R +∞
−∞
g(u)f′
u(u)du,
Since A = LL∗
gives ℓ12ℓ22 = RXX(τ). Hence,
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.46/105
47. RXY (τ) = RXX(τ){−g(u)fu(u)|+∞
−∞ +
R +∞
−∞
g′
(u)fu(u)du}
= RXX(τ)E{g′
(X)} = ηRXX(τ),
This is the desired result, where η = E{g′
(X)}. Thus if the
input to a memoryless device is stationary Gaussian, the
cross correlation function between the input and the output
is proportional to the input autocorrelation function.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.47/105
51. White noise W(t) is said to be a white noise process if
RWW (t1, t2) = q(t1)δ(t2 − t1)
W(t) is said to be WSS white noise if E{W(t)} = constant,
and
RWW (t1, t2) = qδ(t1 − t2) = qδ(τ).
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.51/105
52. If W(t) is also a Gaussian process (white Gaussian pro-
cess), then all of its samples are independent random vari-
ables(Why?)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.52/105
53. White noise Colored noise
W(t) N(t) = h(t) ∗ W(t)
LTI system
h(t)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.53/105
55. Thus the output of a white noise process through an LTI
system represents a (colored) noise process.
Note: White noise need not be Gaussian.
“White” and “Gaussian” are two different concepts!
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.55/105
57. Upcrossings and Downcrossings of a stationary Gaussian pro-
cess: Consider a zero mean stationary Gaussian process
X(t) with autocorrelation function RXX(τ). An upcrossing
over the mean value occurs whenever the realization X(t)
passes through zero with positive slope. Let ρ∆t represent
the probability of such an upcrossing in the interval (t, t+∆t)
We wish to determine ρ.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.57/105
58. Since X(t) is a stationary Gaussian process, its derivative
process X′
(t) is also zero mean stationary Gaussian with
autocorrelation function
RXX(τ) =
Z ∞
−∞
SXX(ω)ejωτ dω
2π
R
′′
XX(τ) =
Z ∞
−∞
(jω)2
SXX(ω)ejωτ dω
2π
SX′X′ (ω) = |jω|2
SXX(ω) ⇒
RX′X′ (τ) = −R
′′
XX(τ)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.58/105
59. Further X(t) and X′
(t) and are jointly Gaussian stationary
processes, and since
RXX′ (τ) = −
dRXX(τ)
dτ
,
RXX′ (−τ) = −
dRXX(−τ)
d(−τ)
=
dRXX(τ)
dτ
= −RXX′ (τ)
which for τ = 0 gives
RXX′ (0) = 0 ⇒ E[X(t)X′
(t)] = 0
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.59/105
60. the jointly Gaussian zero mean random variables
X1 = X(t), X2 = X′
(t)
are uncorrelated hence independent with variances
σ2
1 = RXX(0) and σ2
2 = RX′X′ (0) = −R′′
XX(0) 0
fX1X2 (x1, x2) = fX(x1)fX(x2) =
1
2πσ1σ2
e
−
x2
1
2σ2
1
+
x2
1
2σ2
2
!
.
To determine ρ the probability of upcrossing rate,
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.60/105
61. we argue as follows: In an interval (t, t + ∆t) the realization
moves from X(t) = X1 to
X(t + ∆t) = X(t) + X′
(t)∆t = X1 + X2∆t, and hence the
realization intersects with the zero level somewhere in that
interval if
X1 0, X2 0, and X(t + ∆t) = X1 + X2∆t 0
i.e., X1 −X2∆t Hence the probability of upcrossing in
(t, t + ∆t) is given by
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.61/105
62. ρ∆t =
Z ∞
x2=0
Z 0
x1=−x2∆t
fX1X2 (x1, x2)dx1dx2
=
Z ∞
0
fX2 (x2)dx2
Z ∞
−x2∆t
fX1 (x1)dx1. (6)
Differentiating both sides of 6 with respect to ∆t we get
ρ =
Z ∞
0
fX2 (x2)x2fX1 (−x2∆t)dx2
and letting ∆t → 0, we have
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.62/105
63. c
ρ =
Z ∞
0
x2fX(x2)fX(0)dx2 =
1
p
2πRXX(0)
Z ∞
0
x2fX(x2)dx2
=
1
p
2πRXX(0)
1
2
(σ2
p
2/π) =
1
2π
s
−R′′
XX(0)
RXX(0)
There is an equal probability for downcrossings, and hence
the total probability for crossing the zero line in an interval
(t, t + ∆t) equals ρ0∆t where
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.63/105
65. A discrete time stochastic process Xn = X(nT) is a
sequence of random variables. The mean, autocorrelation
and auto-covariance functions of a discrete-time process
are gives by
µn = E{X(nT)}
R(n1, n2) = E{X(n1T)X∗
(n2T)}
C(n1, n2) = R(n1, n2) − µn1 µ∗
n2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.65/105
67. The positive-definite property of the autocorrelation
sequence can be expressed in terms of certain
Hermitian-Toeplitz matrices as follows:
Theorem: A sequence {rn} |∞
−∞ forms an autocorrelation se-
quence of a wide sense stationary stochastic process if and
only if every Hermitian-Toeplitz matrix Tn given by
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.67/105
83. the transfer function H(z) is rational with p poles and q zeros
that determine the model order of the underlying system.
X(n) undergoes regression over p of its previous values and
at the same time a moving average based on W(n), W(n −
1), · · · , W(n − q) of the input over (q + 1) values is added
to it, thus generating an Auto Regressive Moving Average
(ARMA (p, q)) process X(n).
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.73/105
84. Generally the input {W(n)} represents a sequence of
uncorrelated random variables of zero mean and constant
variance σ2
W so that
RWW (n) = σ2
W δ(n).
If in addition, {W(n)} is normally distributed then the
output {X(n)} also represents a strict-sense stationary
normal process.
If q = 0, then we have an AR(p) process (all-pole process),
and if p = 0, then an MA(q)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.74/105
85. AR(1) process: An AR(1) process has the form
X(n) = aX(n − 1) + W(n)
H(z) =
1
1 − az−1
=
∞
X
n=0
an
z−n
h(n) = an
, |a| 1
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.75/105
86. RXX(n) = σ2
W δ(n) ∗ {a−n
} ∗ {an
} = σ2
W
∞
X
k=0
a|n|+k
ak
= σ2
W
a|n|
1 − a2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.76/105
88. It is instructive to compare an AR(1) model discussed
above by superimposing a random component to it, which
may be an error term associated with observing a first
order AR process X(n). Thus
Y (n) = X(n) + V (n), X(n) ∼ AR(1), V (n) ∼ White noise (8)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.78/105
89. RY Y (n) = RXX(n) + RV V (n) = RXX(n) + σ2
V δ(n)
= σ2
W
a|n|
1 − a2
+ σ2
V δ(n)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.79/105
90. ρY (n) =
RY Y (n)
RY Y (0)
=
1, n = 0
ca|n|
, n = ±1, ±2, · · ·
(9)
c =
σ2
W
σ2
W + σ2
V (1 − a2)
1.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.80/105
91. Eqs. (7) and (8) demonstrate the effect of superimposing
an error sequence on an AR(1) model. For non-zero lags,
the autocorrelation of the observed sequence {Y (n)} is re-
duced by a constant factor compared to the original process
{X(n)}. From (8), the superimposed error sequence V (n)
only affects the corresponding term in Y (n) (term by term).
However, a particular term in the “input sequence” W(n)
affects X(n) and Y (n) as well as all subsequent observa-
tions.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.81/105
92. AR(2) Process: An AR(2) process has the form, W(n) is a
zero-mean white process
X(n) = a1X(n − 1) + a2X(n − 2) + W(n)
H(z) =
∞
X
n=0
h(n)z−n
=
1
1 − a1z−1 − a2z−2
=
b1
1 − λ1z−1
+
b2
1 − λ2z−1
h(0) = 1, h(1) = a1, h(n) = a1h(n − 1) + a2h(n − 2), n ≥ 2
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.82/105
96. The normalized autocorrelation function:
ρX(n) =
RXX(n)
RXX(0)
= c1λ∗n
1 + c2λ∗n
2
where c1 and c2 are the appropriate constant.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.86/105
97. ARMA(p, q) system has only p + q + 1 independent
coefficients, and hence its impulse response sequence
{hk} also must exhibit a similar dependence among them.
An old result due to Kronecker (1881) states that the nec-
essary and sufficient condition for H(z) =
P∞
k=0 h(n)z−n
to
represent a rational system (ARMA) is that
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.87/105
98. det Hn = 0, n ≥ N (for all sufficiently large n),
where
Hn =
h0 h1 h2 · · · hn
h1 h2 h3 · · · hn+1
.
.
.
hn hn+1 hn+2 · · · h2n
. (10)
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.88/105
99. In the case of rational systems for all sufficiently large n, the
Hankel matrices Hn in (10) all have the same rank.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.89/105
100. The necessary part easily follows from
∞
X
k=0
h(n)z−n
=
Pq
k=0 bkz−k
Pp
k=0 akz−k
by cross multiplying and equating coefficients of like
powers of z−n
for n = 0, 1, 2, · · · .
We have
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.90/105
101. b0 = h0
b1 = h0a1 + h1
.
.
.
bq = h0aq + h1aq−1 + · · · + hm
0 = h0aq+i + h1aq+i−1 + · · · + hq+i−1a1 + hq+i, i ≥ 1.
For systems with q ≤ p−1, letting i = p−q, p−q+1, · · · , 2p−q
in the last equation we get
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.91/105
102. h0ap + h1ap−1 + · · · + hp−1a1 + hp = 0
.
.
.
hpap + hp+1ap−1 + · · · + h2p−1a1 + h2p = 0
which gives det(Hp) = 0, and similarly for i = p − q + 1, · · · .
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.92/105
104. the ARMA (p, q) model, the input white noise process W(n)
is uncorrelated with its own past sample values as well as
the past values of the system output. This gives
E{W(n)W∗
(n − k)} = 0, k ≥ 1
E{W(n)X∗
(n − k)} = 0, k ≥ 1.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.94/105
105. ri = E{x(n)x∗
(n − i)}
= −
p
X
k=1
ak{x(n − k)x∗
(n − i)} +
q
X
k=0
bk{w(n − k)w∗
(n − i)}
= −
p
X
k=1
akri−k +
q
X
k=0
bk{w(n − k)x∗
(n − i)}
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.95/105
106. p
X
k=1
akri−k + ri 6= 0, i ≤ q
p
X
k=1
akri−k + ri = 0, i ≥ q + 1. ‡
We note that the above equation (‡) is the same as (†) with
{hk} replaced by {rk}. Hence, the Kronecker conditions for
rational systems can be expressed in terms of its output
autocorrelations as well.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.96/105
107. X(n) ∼ ARMA (p, q) represents a wide sense stationary
stochastic process, then its output autocorrelation
sequence {rk} satisfies
rankDp−1 = rankDp+k = p, k ≥ 0,
where
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.97/105
108. Dk =
r0 r1 r2 · · · rk
r1 r2 r3 · · · rk+1
.
.
.
rk rk+1 rk+2 · · · r2k
represents the (k + 1) × (k + 1) Hankel matrix generated
from r0, r1, · · · , r2k. It follows that for ARMA (p, q) systems,
we have
det Dn = 0, for all sufficiently large n.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.98/105
109. Cyclostationarity:
In the determinitic case, a signal is called periodic if it
repeats after a period of time. In this section we will look at
the statistical equivalent of periodicity. A random process
X(t) is said to be nth-order cyclostationary if the nth-order
PDF is periodic:
fX(~
x;~
t) = fX(~
x;~
t + ~
T
(n)
o
~
1); ~
T
(n)
o ∈ R
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.99/105
110. The periodicity of the statistics, ~
T
(n)
o is often referred to as
the cyclostationarity parameter. As in the case of
stationarity, n = 1 cyclostationarity implies that
µX(t) = µX(t + T(1)
o )
σX(t) = σX(t + T(1)
o )
Second-order cyclostationarity specifically implies that
RXX(t1, t2) = RXX(t1 + T(2)
o , t2 + T(2)
o )
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.100/105
111. Since the ACF of the process is dependent on both t1 t2,
these processes are non-stationary. Furthermore the
ensemble ACF is periodic and as a consequence it has a
Fourier series expansion of the form:
RXX(t, τ) = E{X(t)X∗
(t − τ)} =
∞
X
n=−∞
R̃
(n)
XX(τ) exp
j
2πnt
T
(2)
o
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.101/105
112. The Fourier series coefficients R
(n)
XX(τ) are referred to as
the cyclic autocorrelation function. The DC Fourier
coefficient in particular is called the time averaged ACF
and is determined via:
R̃
(0)
XX(τ) =
1
T
(2)
o
Z T
(2)
o /2
−T
(2)
o /2
R
(0)
XX(t, t − τ) dt
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.102/105
113. The corresponding time-averaged power spectrum is
defined via the Fourier transform:
P
(0)
XX(Ω) =
Z ∞
−∞
R
(0)
XX(τ) exp(−jΩτ)dτ
Using the cyclic autocorrelation function we can define a
corresponding cyclic power-spectrum P
(n)
XX(Ω) via:
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.103/105
114. P
(n)
XX(Ω) =
Z ∞
−∞
R
(n)
XX(τ) exp(−jΩτ)dτ
R
(n)
XX(τ) =
1
2π
Z ∞
−∞
P
(n)
XX(Ω) exp(jΩτ)dΩ
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.104/105
115. The motivation behind the definition of the time-averaged
ACF is to average out and remove the dependence of the
ACF on the t variable so that we do not have to deal with
two dimensional Fourier transforms.
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.105/105