00400_Systems.pdf

4: Linear Time Invariant Systems
4: Linear Time Invariant
Systems
• LTI Systems
• Convolution Properties
• BIBO Stability
• Frequency Response
• Causality +
• Convolution Complexity
• Circular Convolution
• Frequency-domain
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
DSP and Digital Filters (2017-10159) LTI Systems: 4 – 1 / 13

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Linear Time-invariant (LTI) systems have two properties:

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Linear: H (αu[n] + βv[n]) = αH (u[n]) + βH (v[n])

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Time Invariant: y[n] = H (x[n]) ⇒ y[n − r] = H (x[n − r]) ∀r

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
The behaviour of an LTI system is completely defined by its impulse
response: h[n] = H (δ[n])

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Proof:
We can always write x[n] =
P∞
r=−∞ x[r]δ[n − r]

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Proof:
P∞
r=−∞ x[r]δ[n − r]
Hence H (x[n]) = H
P∞
r=−∞ x[r]δ[n − r]

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Proof:
P∞
r=−∞ x[r]δ[n − r]
Hence H (x[n]) = H
P∞
r=−∞ x[r]δ[n − r]

=
P∞
r=−∞ x[r]H (δ[n − r])

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Proof:
P∞
r=−∞ x[r]δ[n − r]
Hence H (x[n]) = H
P∞
r=−∞ x[r]δ[n − r]

=
P∞
r=−∞ x[r]H (δ[n − r])
=
P∞
r=−∞ x[r]h[n − r]

LTI Systems
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Proof:
P∞
r=−∞ x[r]δ[n − r]
Hence H (x[n]) = H
P∞
r=−∞ x[r]δ[n − r]

=
P∞
r=−∞ x[r]H (δ[n − r])
=
P∞
r=−∞ x[r]h[n − r]
= x[n] ∗ h[n]

Convolution Properties
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Convolution: x[n] ∗ v[n] =
P∞
r=−∞ x[r]v[n − r]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Convolution obeys normal arithmetic rules for multiplication:

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Commutative: x[n] ∗ v[n] = v[n] ∗ x[n]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
(i) substitute p = n − r

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
Associative: x[n] ∗ (v[n] ∗ w[n]) = (x[n] ∗ v[n]) ∗ w[n]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
⇒ x[n] ∗ v[n] ∗ w[n] is unambiguous

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
Proof:
P
r,s x[n − r]v[r − s]w[s]
(i)
=
P
p,q x[p]v[q − p]w[n − q]
(i) substitute p = n − r, q = n − s

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
Proof:
P
(i)
=
P
Distributive over +:
x[n] ∗ (αv[n] + βw[n]) = (x[n] ∗ αv[n]) + (x[n] ∗ βw[n])

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
Proof:
P
(i)
=
P
Proof:
P
r x[n − r] (αv[r] + βw[r]) =
α
P
r x[n − r]v[r] + β
P
r x[n − r]w[r]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
Proof:
P
(i)
=
P
Proof:
P
r x[n − r] (αv[r] + βw[r]) =
α
P
r x[n − r]v[r] + β
P
r x[n − r]w[r]
Identity: x[n] ∗ δ[n] = x[n]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
r=−∞ x[r]v[n − r]
Proof:
P
r x[r]v[n − r]
(i)
=
P
p x[n − p]v[p]
Proof:
P
(i)
=
P
Proof:
P
r x[n − r] (αv[r] + βw[r]) =
α
P
r x[n − r]v[r] + β
P
r x[n − r]w[r]
Identity: x[n] ∗ δ[n] = x[n]
Proof:
P
r δ[r]x[n − r]
(i)
= x[n] (i) all terms zero except r = 0.

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
BIBO Stability: Bounded Input, x[n] ⇒ Bounded Output, y[n]

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
The following are equivalent:
(1) An LTI system is BIBO stable

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(2) h[n] is absolutely summable, i.e.
P∞
n=−∞ |h[n]| ∞

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
(3) H(z) region of absolute convergence includes |z| = 1.

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
Proof (1) ⇒ (2):
Define x[n] =
(
1 h[−n] ≥ 0
−1 h[−n] 0
then y[0] =
P
x[0 − n]h[n] =
P
|h[n]|.

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
Proof (1) ⇒ (2):
Define x[n] =
(
1 h[−n] ≥ 0
−1 h[−n] 0
then y[0] =
P
x[0 − n]h[n] =
P
|h[n]|.
But |x[n]| ≤ 1∀n so BIBO ⇒ y[0] =
P
|h[n]| ∞.

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
Proof (1) ⇒ (2):
Define x[n] =
(
1 h[−n] ≥ 0
−1 h[−n] 0
then y[0] =
P
x[0 − n]h[n] =
P
|h[n]|.
But |x[n]| ≤ 1∀n so BIBO ⇒ y[0] =
P
|h[n]| ∞.
Proof (2) ⇒ (1):
Suppose
P
|h[n]| = S ∞ and |x[n]| ≤ B is bounded.

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
Proof (1) ⇒ (2):
Define x[n] =
(
1 h[−n] ≥ 0
−1 h[−n] 0
then y[0] =
P
x[0 − n]h[n] =
P
|h[n]|.
But |x[n]| ≤ 1∀n so BIBO ⇒ y[0] =
P
|h[n]| ∞.
Proof (2) ⇒ (1):
Suppose
P
Then |y[n]| =
P∞
r=−∞ x[n − r]h[r]

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
Proof (1) ⇒ (2):
Define x[n] =
(
1 h[−n] ≥ 0
−1 h[−n] 0
then y[0] =
P
x[0 − n]h[n] =
P
|h[n]|.
But |x[n]| ≤ 1∀n so BIBO ⇒ y[0] =
P
|h[n]| ∞.
Proof (2) ⇒ (1):
Suppose
P
Then |y[n]| =
P∞
r=−∞ x[n − r]h[r]
≤
P∞
r=−∞ |x[n − r]| |h[r]|

BIBO Stability
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
P∞
n=−∞ |h[n]| ∞
Proof (1) ⇒ (2):
Define x[n] =
(
1 h[−n] ≥ 0
−1 h[−n] 0
then y[0] =
P
x[0 − n]h[n] =
P
|h[n]|.
But |x[n]| ≤ 1∀n so BIBO ⇒ y[0] =
P
|h[n]| ∞.
Proof (2) ⇒ (1):
Suppose
P
Then |y[n]| =
P∞
r=−∞ x[n − r]h[r]
≤
P∞
r=−∞ |x[n − r]| |h[r]|
≤ B
P∞
r=−∞ |h[r]|≤ BS ∞

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
For a BIBO stable system Y (ejω
) = X(ejω
)H(ejω
)
where H(ejω
)is the DTFT of h[n] i.e. H(z) evaluated at z = ejω
.

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

0

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω 0

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
0

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
0

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
0
0 1 2 3
0
1
2
3
ω

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
∠H(ejω
) = −ω + π 1−sgn(1+2 cos ω)
2
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
∠H(ejω
) = −ω + π 1−sgn(1+2 cos ω)
2
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω
0 1 2 3
-2
-1
0
1
ω
∠

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
∠H(ejω
) = −ω + π 1−sgn(1+2 cos ω)
2
Sign change in (1 + 2 cos ω) at ω = 2.1 gives
(a) gradient discontinuity in |H(ejω
)|
(b) an abrupt phase change of ±π.
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω
0 1 2 3
-2
-1
0
1
ω
∠

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
∠H(ejω
) = −ω + π 1−sgn(1+2 cos ω)
2
)|
Group delay is − d
dω ∠H(ejω
) : gives delay of the
modulation envelope at each ω.
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω
0 1 2 3
-2
-1
0
1
ω
∠

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
∠H(ejω
) = −ω + π 1−sgn(1+2 cos ω)
2
)|
dω ∠H(ejω
Normally varies with ω but for a symmetric filter it is
constant: in this case +1 samples.
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω
0 1 2 3
-2
-1
0
1
ω
∠

Frequency Response
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
) = X(ejω
)H(ejω
)
where H(ejω
.
Example: h[n] =

1 1 1

H(ejω
) = 1 + e−jω
+ e−j2ω
= e−jω
(1 + 2 cos ω)
H(ejω
) = |1 + 2 cos ω|
∠H(ejω
) = −ω + π 1−sgn(1+2 cos ω)
2
)|
dω ∠H(ejω
Normally varies with ω but for a symmetric filter it is
constant: in this case +1 samples.
Discontinuities of ±kπ do not affect group delay.
0
0 1 2 3
0
1
2
3
ω
0 1 2 3
-10
-5
0
5
10
ω
0 1 2 3
-2
-1
0
1
ω
∠

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Causal System: cannot see into the future
i.e. output at time n depends only on inputs up to time n.

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Formal definition:
If v[n] = x[n] for n ≤ n0 then H (v[n]) = H (x[n]) for n ≤ n0.

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Formal definition:
(1) An LTI system is causal

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Formal definition:
(2) h[n] is causal ⇔ h[n] = 0 for n 0

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Formal definition:
(3) H(z) converges for z = ∞

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Formal definition:
Any right-sided sequence can be made causal by adding a delay.

Causality +
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Formal definition:
Any right-sided sequence can be made causal by adding a delay.
All the systems we will deal with are causal.

Convolution Complexity
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
y[n] = x[n] ∗ h[n]: convolve x[0 : N − 1] with h[0 : M − 1]
x
∗

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
y[n] is only non-zero in the range
0 ≤ n ≤ M + N − 2
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
0 ≤ n ≤ M + N − 2
Thus y[n] has only
M + N − 1 non-zero values
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
0 ≤ n ≤ M + N − 2
Thus y[n] has only
Algebraically:
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10
x[n − r] 6= 0 ⇒ 0 ≤ n − r ≤ N − 1

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
0 ≤ n ≤ M + N − 2
Thus y[n] has only
Algebraically:
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10
x[n − r] 6= 0 ⇒ 0 ≤ n − r ≤ N − 1
⇒ n + 1 − N ≤ r ≤ n

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
0 ≤ n ≤ M + N − 2
Thus y[n] has only
Algebraically:
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10
x[n − r] 6= 0 ⇒ 0 ≤ n − r ≤ N − 1
⇒ n + 1 − N ≤ r ≤ n
Hence: y[n] =
Pmin(M−1,n))
r=max(0,n+1−N) h[r]x[n − r]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
∗ →
Convolution sum:
y[n] =
PM−1
r=0 h[r]x[n − r]
0 ≤ n ≤ M + N − 2
Thus y[n] has only
Algebraically:
y[0] y[9]
N = 8, M = 3
M + N − 1 = 10
x[n − r] 6= 0 ⇒ 0 ≤ n − r ≤ N − 1
⇒ n + 1 − N ≤ r ≤ n
Hence: y[n] =
Pmin(M−1,n))
r=max(0,n+1−N) h[r]x[n − r]
We must multiply each h[n] by each x[n] and add them to a total
⇒ total arithmetic complexity (× or + operations) ≈ 2MN

Circular Convolution
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
y⊛[n] = x[n] ⊛N h[n]: circ convolve x[0 : N − 1] with h[0 : M − 1]
x
⊛N

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N
Convolution sum:
y⊛N
[n] =
PM−1
r=0 h[r]x[(n − r)mod N ]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y[0] y[7]
N = 8, M = 3

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y⊛N
[n] has period N
y[0] y[7]
N = 8, M = 3

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y⊛N
[n] has period N
⇒ y⊛N [n] has N distinct values
y[0] y[7]
N = 8, M = 3

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y⊛N
[n] has period N
y[0] y[7]
N = 8, M = 3
• Only the first M − 1 values are affected by the circular repetition:

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y⊛N
[n] has period N
y[0] y[7]
N = 8, M = 3
y⊛N
[n] = y[n] for M − 1 ≤ n ≤ N − 1

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y⊛N
[n] has period N
y[0] y[7]
N = 8, M = 3
y⊛N
[n] = y[n] for M − 1 ≤ n ≤ N − 1
• If we append M − 1 zeros (or more) onto x[n], then the circular
repetition has no effect at all and:
y⊛N+M−1 [n] = y[n] for 0 ≤ n ≤ N + M − 2

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
x
⊛N →
Convolution sum:
y⊛N [n] =
PM−1
y⊛N
[n] has period N
y[0] y[7]
N = 8, M = 3
y⊛N
[n] = y[n] for M − 1 ≤ n ≤ N − 1
• If we append M − 1 zeros (or more) onto x[n], then the circular
repetition has no effect at all and:
y⊛N+M−1 [n] = y[n] for 0 ≤ n ≤ N + M − 2
Circular convolution is a necessary evil in exchange for using the DFT

Frequency-domain convolution
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Idea: Use DFT to perform circular convolution - less computation

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(1) Choose L ≥ M + N − 1 (normally round up to a power of 2)

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(2) Zero pad x[n] and h[n] to give sequences of length L: x̃[n] and h̃[n]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Arithmetic Complexity:
DFT or IDFT take 4L log2 L operations if L is a power of 2
(or 16L log2 L if not).

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Total operations: ≈ 12L log2 L ≈ 12 (M + N) log2 (M + N)

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Beneficial if both M and N are ∼ 70 .

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Example: M = 103
, N = 104
:

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Example: M = 103
, N = 104
:
Direct: 2MN = 2 × 107

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Example: M = 103
, N = 104
:
Direct: 2MN = 2 × 107
with DFT: = 1.8 × 106
,

Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
(3) Use DFT: ỹ[n] = F−1
(X̃[k]H̃[k]) = x̃[n] ⊛L h̃[n]
(4) y[n] = ỹ[n] for 0 ≤ n ≤ M + N − 2.
Example: M = 103
, N = 104
:
Direct: 2MN = 2 × 107
with DFT: = 1.8 × 106
,
But: (a) DFT may be very long if N is large
(b) No outputs until all x[n] has been input.

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
(1) chop x[n] into N
K chunks of length K

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
(2) convolve each chunk with h[n]

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Each output chunk is of length K + M − 1 and overlaps the next chunk

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
(3) add up the results

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Computational saving if ≈ 100 M ≪ K ≪ N

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Example: M = 500, K = 104
, N = 107
Direct: 2MN = 1010

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Example: M = 500, K = 104
, N = 107
Direct: 2MN = 1010
single DFT: 12 (M + N) log2 (M + N) = 2.8 × 109

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Example: M = 500, K = 104
, N = 107
Direct: 2MN = 1010
overlap-add: N
K × 8 (M + K) log2 (M + K) = 1.1 × 109
,

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Example: M = 500, K = 104
, N = 107
Direct: 2MN = 1010
overlap-add: N
K × 8 (M + K) log2 (M + K) = 1.1 × 109
,
Other advantages:
(a) Shorter DFT

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Example: M = 500, K = 104
, N = 107
Direct: 2MN = 1010
overlap-add: N
K × 8 (M + K) log2 (M + K) = 1.1 × 109
,
Other advantages:
(a) Shorter DFT
(b) Can cope with N = ∞

Overlap Add
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
If N is very large:
Operations: ≈ N
K × 8 (M + K) log2 (M + K)
Example: M = 500, K = 104
, N = 107
Direct: 2MN = 1010
overlap-add: N
K × 8 (M + K) log2 (M + K) = 1.1 × 109
,
Other advantages:
(a) Shorter DFT
(b) Can cope with N = ∞
(c) Can calculate y[0] as soon as x[K − 1] has been read:
algorithmic delay = K − 1 samples

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
chunks of length K + M − 1

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
(2) ⊛K+M−1 each chunk with h[n]

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
The first M − 1 points of each output chunk are invalid

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
(3) discard first M − 1 from each chunk

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
(4) concatenate to make y[n]

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
Operations: slightly less than overlap-add because no addition needed to
create y[n]

Overlap Save
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
Alternative method:
K overlapping
Operations: slightly less than overlap-add because no addition needed to
create y[n]
Advantages: same as overlap add
Strangely, rather less popular than overlap-add

Summary
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
• LTI systems: impulse response, frequency response, group delay

Summary
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
• BIBO stable, Causal, Passive, Lossless systems

Summary
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
• Convolution and circular convolution properties

Summary
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
• Efficient methods for convolution
◦ single DFT
◦ overlap-add and overlap-save

Summary
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
• Efficient methods for convolution
◦ single DFT
◦ overlap-add and overlap-save
For further details see Mitra: 4 5.

MATLAB routines
Systems
• LTI Systems
• BIBO Stability
• Causality +
convolution
• Overlap Add
• Overlap Save
• Summary
• MATLAB routines
fftfilt Convolution using overlap add
x[n]⊛y[n] real(ifft(fft(x).*fft(y)))

00400_Systems.pdf

Recommended

Recommended

More Related Content

Similar to 00400_Systems.pdf

Similar to 00400_Systems.pdf (20)

Recently uploaded

Recently uploaded (20)

00400_Systems.pdf