Chap 7. 01-38.pdf

Nonlinear Signal Processing
ELEG 833
Gonzalo R. Arce
Department of Electrical and Computer Engineering
University of Delaware
arce@ee.udel.edu
Fall 2008
Gonzalo R. Arce (Department of Electrical and Computer Engineering University of Delaware arce@ee.udel.edu) Fall 2008 1 / 75

Outline
1 Linear Combination of Order Statistics
L-Estimates of Location
L-Smoothers
Lℓ-Filters
Design and Optimization
Hybrid Median/Linear FIR Filters
Linear Combination of Weighted Medians
LCWM Filters
Design of LCWM filters
Symmetric LCWM Filters

Linear Combination of Order Statistics
Linear Combination of Order Statistics
Given the ordered set X(1), X(2), · · · , X(N), the linear combination
Y =
N
X
i=1
Wi X(i) (1)
is known as L-statistic or L-estimate. If Z is a linear transformation of X,
Z = α + γX, then Z(i) = α + γX(i), and Y (Z)
= α
P
Wi + γY (X)
. Thus,
Y (Z)
= α + γY (X)
if
P
Wi = 1.

Linear Combination of Order Statistics L-Estimates of Location
Outline
L-Smoothers
Lℓ-Filters
LCWM Filters

In the location estimation problem, we have Xi = β + Zi , where β is the constant
location parameter to be estimated, and where Zi is a zero mean sequence of iid
noise samples with variance σ2
. For simplicity, we assume that the noise is
symmetrically distributed. The L-estimate of location is then
β̂ =
N
X
i=1
Wi X(i) (2)
where the estimate is required to be unbiased, E{β̂} = β,

E{β̂} =
N
X
i=1
Wi E{X(i)}
=
N
X
i=1
Wi E{β + Z(i)} (3)
= β
N
X
i=1
Wi +
N
X
i=1
Wi E{Z(i)}.
Assuming the noise samples are i.i.d. and symmetrically distributed, then
E[Z(i)] = −E[Z(N−i+1)]. Using this fact in (3), the estimate is unbiased if the
weights are symmetric (WN−i+1 = Wi ) and if
PN
i=1 Wi = 1.

In vector notation
β̂ = WT
(β e + ZL) (4)
where e = [1, 1, . . . , 1]T
, and
ZL = [Z(1), Z(2), · · · , Z(N)]T
. (5)
The mean-square estimation error is
J(W) = E
h
| β − β̂ |2
i
= E

| WT
ZL |2

(6)
= WT
RLW,

where the unbiasedness constraint WT
e = 1 was utilized, and where
RL =





EZ2
(1) EZ(1)Z(2) · · · EZ(1)Z(N)
EZ(2)Z(1) EZ2
(2) EZ(2)Z(N)
.
.
.
.
.
.
...
.
.
.
EZ(N)Z(1) EZ(N)Z(2) · · · EZ2
(N)





(7)
where EZ(i)Z(j) is the correlation moment of the ith and jth noise order statistics.

The Lagrangian function of the constrained MSE cost function J(W) is
F(λ, W) = WT
RLW + λ(WT
e − 1). (8)
Taking derivative with respect to W and setting it to zero
2RLW + λe = 0. (9)
The equation above is multiplied by eT
R−1
L obtaining
λ =
−2
eT R−1
L e
(10)
which is then used in (9) to obtain the optimal L-estimate weights
Wo =
R−1
L e
eT R−1
L e
, (11)
with the corresponding mean square error
J(Wo) = Jmin =
1
eT R−1
L e
. (12)

There is no assurance that other estimators, such as ML estimators are more
efficient.
Since the sample mean is included, optimal L-estimates will always do better,
or at least equal, than the linear estimate.
The L-filter estimate will perform better than the sample mean whenever the
row sums (or column sums) of the correlation matrix E{ZLZT
L } are not equal
to one.
The optimal weights of the L-estimate will have markedly different
characteristics depending on the parent distribution of the noise samples Zi .

Table: Optimal weights for the L-estimate of location N = 9
Distribution Weights
W1 W2 W3 W4 W5 W6 W7 W8 W9
Uniform 0.5 0 0 0 0 0 0 0 0.5
Gaussian 0.11 0.11 0.11 0.11 0.11 0.11 0.11 0.11 0.11
Laplacian -0.018 0.029 0.069 0.238 0.364 0.238 0.069 0.029 -0.018

Linear Combination of Order Statistics L-Smoothers
Outline
L-Smoothers
Lℓ-Filters
LCWM Filters

L-Smoothers
L-smoothers are obtained when L-estimates are computed at each location of a
running window.
Definition (Running L-smoothers)
Given a set of N real valued weights W1, W2, · · · , WN assigned to the
order-statistics X(1), X(2), · · · , X(N), in the running window
X(n) = [X1(n), X2(n), · · · , XN (n)]T
, the L-smoother output is
Y (n) =
N
X
i=1
Wi X(i). (13)

If the weights are chosen as Wi = 1/N, the L-smoother reduces to the running
mean.
The simplest L-smoother is found by zero-weighting all order statistics except for
one, leading to the rank-smoother
Y (n) = rth Largest Sample of [X1(n), X2(n), · · · , XN (n)],
with the median smoother as a special case.

70
80
90
100
110
120
130
140
150
160
0 10 20 30 40 50 60 70 80 90 100
Detected signal
Original signal
Original signal
Figure: Rank-order AM demodulation. The window size is 9, and the output is the 8th largest
in the window. Baseband signal is at 5kHz with a carrier of 31kHz. The sampling frequency is
250kHz. (a) noiseless reception. (b) noisy reception with impulsive noise.

70
80
90
100
110
120
130
140
150
160
0 10 20 30 40 50 60 70 80 90 100
Noisy signal
Detected signal
Signal corrupted with impulsive noise
Figure: Rank-order AM demodulation. The window size is 9, and the output is the 8th largest
in the window. Baseband signal is at 5kHz with a carrier of 31kHz. The sampling frequency is
250kHz. (a) noiseless reception. (b) noisy reception with impulsive noise.

Linear Combination of Order Statistics Lℓ-Filters
Outline
L-Smoothers
Lℓ-Filters
LCWM Filters

Lℓ-Filters
L-smoothers exhibit “low-pass” characteristics even though positive and
negative valued weights are allowed.
In this case, L-smoother weights cannot exploit the time ordering relationship
of time-series.
Lℓ-Filters overcome this limitations.

Definition (Lℓ-filters)
Given the observation vector at time n, X(n) = [X1(n), X2(n), · · · , XN (n)]T
,
where Xi (n) = X (n + i − (K + 1)) with N = 2K + 1, and the ranks Ri for each
of the samples Xi , i = 1, 2, · · · , N, the Lℓ-filter output is defined as
Y (n) =
N
X
i=1
Wi,Ri
Xi (n) (14)
where the weight given to the ith sample Xi , Wi,Ri
, depends on the sample’s rank
Ri .
The Lℓ-filter requires N2
weights.

Let X = [X1, X2, · · · , XN ]T
and XL = [X(1), X(2), · · · , X(N)]T
, the N2
-long vector
XLℓ that combines the rank and temporal ordering is
XT
Lℓ = [X1(1), X1(2), · · · , X1(N)| · · · , Xi(j), · · · |XN(1), XN(2), · · · , XN(N)], (15)
where
Xi(j) =

Xi if Xi ←→ X(j)
0 else
(16)
and where Xi ←→ X(j) denotes the event that the ith element in X is the jth
smallest in the sample set. Thus, the ith input sample is mapped into the bin of
samples Xi(1), Xi(2), . . . , Xi(N) of which N − 1 are zero and where only one is
non-zero having the same value as Xi . For example, the ranks of the elements in
the observation vector X = [ 3, 5, 2 ]T
are R1 = 2, R2 = 3, and R3 = 1 leading
to the Lℓ vector
XLℓ = [ 0, 3, 0 | 0, 0, 5 | 2, 0, 0 ]T
. (17)

The decomposition X ∈ RN
←→ XLℓ ∈ RN2
is a one-to-one nonlinear mapping
where X can be reconstructed from XLℓ as
X =

IN ⊗ eT
N

XLℓ, (18)
where IN is an N × N identity matrix, eN is an N × 1 one-valued vector, and ⊗ is
the matrix Kronecker product. Since the XLℓ vector contains both, time and rank
ordering information, it is not surprising that we can also obtain XL from XLℓ as
XL =

eT
N ⊗ IN

XLℓ. (19)

The cost function can be expressed as
J(W) = E[(D(n) − WT
XLℓ(n))(DT
(n) − XT
Lℓ(n)W)]
= σ2
d − 2pT
LℓW + WT
RLℓW, (20)
where pLℓ = E{D(n) XLℓ} and where RLℓ is the N2
× N2
symmetric correlation
matrix RLℓ = E{XLℓXT
Lℓ}. J(W) is quadratic in W.

The global minimum is found through
∇ =
∂(J(W))
∂W
=

∂(J(W))
∂W1(1)
,
∂(J(W))
∂W1(2)
, · · · ,
∂(J(W))
∂WN(N)
T
= 2RLℓW − 2pLℓ.
Setting the gradient to zero yields the optimal weights
Wo = R−1
Lℓ pLℓ. (21)

Note that
XLℓ = [X1(1), X1(2), · · · , X1(N)|X2(1), · · · , X2(N)| · · · |XN(), · · · , XN(N)]
= [XT
1 , XT
2 , · · · , XT
N ]T
(22)
where XT
i = [Xi(1), Xi(2), · · · , Xi(N)], the Lℓ correlation matrix can be written as
RLℓ = E{XLℓXT
Lℓ} (23)
=





R11 R12 · · · R1N
R21 R22 · · · R2N
.
.
.
.
.
.
...
.
.
.
RN1 RN2 . . . RNN





(24)
in which Ruv = E{XuXv }

RLℓ =





R11 R12 · · · R1N
R21 R22 · · · R2N
.
.
.
.
.
.
...
.
.
.
RN1 RN2 . . . RNN





(25)





















X2
(1)1
0 0 · · · 0 X(1)1X(3)2 X(1)1X(3)3
0 X2
(1)2
0 · · · X(1)2X(3)1 0 X(1)2X(3)3
0 0 X2
(1)3
· · · X(1)3X(3)1 X(1)3X(3)2 0
0 X(2)1X(1)2 X(2)1X(1)3 · · · 0 X(2)1X(3)2 X(2)1X(3)3
X(2)2X(1)1 0 X(2)2X(1)3 · · · X(2)2X(3)1 0 X(2)2X(3)3
X(2)3X(1)1 X(2)3X(1)2 0 · · · X(2)3X(3)1 X(2)3X(3)2 0
0 X(3)1X(1)2 X(3)3X(1)3 · · · X2
(3)1
0 0
X(3)2X(1)1 0 X(3)2X(1)3 · · · 0 X2
(3)2
0
X(3)2X(1)1 X(3)3X(1)2 0 · · · 0 0 X2
(3)3






















Figure 3. Reconstructed Signals Using Wavelet Shrinkage
Figure 2. Signals with Gaussian Noise
Figure 1. Clean Signals, Blocks and Doppler
−10
0
10
20
30
Reconstructed Blocks
−20
−10
0
10
20
Reconstructed Doppler
−10
0
10
20
30
Gaussian Blocks
−20
−10
0
10
20
Gaussian Doppler
−10
0
10
20
30
Clean Blocks
−20
−10
0
10
20
Clean Doppler
Figure: Wavelet denoising of “block” and “doppler” signals corrupted by Gaussian
noise.

Figure 5. Reconstructed Signals Using Wavelet Shrinkage
Figure 4. Signals with Contaminated Gaussian Noise
Figure 1. Clean Signals, Blocks and Doppler
−10
0
10
20
30
Clean Blocks
−20
−10
0
10
20
Clean Doppler
−10
0
10
20
30
Contaminated Gaussian Blocks
−10
0
10
20
30
Reconstructed Blocks
−20
−10
0
10
20
Contaminated Gaussian Doppler
−20
−10
0
10
20
Reconstructed Doppler
Figure: Wavelet denoising of “block” and “doppler” signals corrupted by contaminated
Gaussian noise.

Figure 13. Reconstructed Signals Using Ll−Filter Decomposition and Wavelet Shrinkage
Figure 12. Reconstructed Signals Using Ll−Filter Decomposition
Figure 4. Signals with Contaminated−Gaussian Noise
−10
0
10
20
30
Ll−Filtered Blocks
−10
0
10
20
30
Ll−Shrinkage Blocks
−15
−10
−5
0
5
10
15
Ll−Filtered Doppler
−15
−10
−5
0
5
10
15
Ll−Shrinkage Doppler
−10
0
10
20
30
Contaminated−Gaussian Blocks
−20
−10
0
10
20
Contaminated−Gaussian Doppler
Figure: Lℓ filtering and Robust wavelet denoising of “block” and “doppler” signals
corrupted by contaminated Gaussian noise.

Linear Combination of Order Statistics Hybrid Median/Linear FIR Filters
Outline
L-Smoothers
Lℓ-Filters
LCWM Filters

Weighted Median Affine Filters
Weighted median affine filters use a weighted median as the trimming reference
point, the trimming is soft rather than hard, and the samples are weighted
averaged according to their temporal ordering.
Definition
Given the set of N observations {X1, X2, . . . , XN } in an observation window, a set
of N real-valued affinity weights {C1, C2, · · · , CN }, and a set of N filter weights
{W1, W2, · · · , WN }, the trimming reference µ(n) is defined as the weighted
median
µ(n) = MEDIAN(|C1| ⋄ sgn(C1)X1, · · · , |CN | ⋄ sgn(CN )XN ) (26)
where |C| ⋄ X = X, X, · · · , X
| {z }
|C| times
.

Definition
The (normalized) WM affine FIR filter:
Yγ(n) = K(n)
N
X
i=1
g

Xi − µ(n)
γ

Wi Xi (27)
where K(n) is the normalization constant K(n) = [
PN
i=1 |Wi |g

Xi −µ(n)
γ

]−1
. The
function g(·) measures the affinity of the ith
observation sample with respect to
the weighted median reference µ(n). The dispersion parameter γ is user defined.

ι
γ 1
1
γ 2
γ
2
g( )
x-µ
___
γ
γ
x x
low affinity
high affinity
µ (n)
(n)
(n)
x−µ
γ
g( )
Figure: The affinity function assigns a low or high affinity to the sample Xi depending
on the location and dispersion parameters µ(n) and γ(n).

The affinity function can take on many forms. The exponential distance
g

Xi − µ(n)
γ

= exp

−
(Xi − µ(n))2
γ2

(28)
By varying the dispersion parameter γ certain properties of the WM affine filter
can be stressed:
Large values of γ emphasize the linear properties of the filter whereas
small values of γ put more weight on its order-statistics properties.

For γ → ∞:
lim
γ→∞
Yγ(n) =
PN
i=1 Wi Xi
PN
i=1 |Wi |
(29)
and the WM affine estimator reduces to a normalized linear FIR filter.
For γ → 0, the estimate is equal to the weighted median µ(n), i.e.
lim
γ→0
Yγ(n) = µ(n). (30)
The WM affine filter assumes a particularly simple form when the reference point
is equal to the sample median.

Table: Summary of the Median Affine Adaptive Optimization Algorithm
Parameters: N = number of taps
νW = positive weight adaptation constant
νγ = positive dispersion adaptation constant
Initial
Conditions: Wi (n) = 0, i = 1, 2, · · · , N; γ set to a large value
Data
(a) Given: The N observation samples Xi at location n and
the desired response at time n, D(n)
(b) Compute: Wi (n + 1) = estimate of tap weight at time n + 1, i = 1, · · · , N
Computation: n = 0, 1, · · ·
e(n) = D(n) − Yγ(n)
Wi (n + 1) = Wi (n) + νW e(n)

gi
PN
k=1 Wk gk (sgn(Wk )Xi − tanh(Wi )Xk )

γ(n + 1) = γ(n) + νγ e(n)
PN
i=1(Xi − Yγ (n))Wi
(Xi −µ(n))2
γ2 e−(Xi −µ(n))2/γ
!
where gi stands for the abbreviated affinity function in (28).

FIR Affine L-Filters
Definition
Consider the observations X1, X2, . . . , XN and their corresponding order-statistics
X(1), X(2), · · · , X(N). Given a set of N affinity weights {C1, C2, · · · , CN }, and a set
of N filter weights {W1, W2, · · · , WN }, the trimming reference µ(n) is the FIR
filter output µ(n) =
PN
i=1 Ci Xi . The (normalized) FIR affine L-filter is then
defined as:
Yγ(n) = K(n)
N
X
i=1
g

X(i) − µ(n)
γ

Wi X(i) (31)
where K(n) is the normalization constant K(n) = [
PN
i=1 |Wi |g

X(i)−µ(n)
γ

]−1
.
The function g(·) measures the affinity of the ith
order-statistic X(i) with respect
to the FIR filter output reference µ(n). The dispersion parameter γ is user defined.

The affinity function can take on many forms. The exponential distance
g

X(i) − µ(n)
γ

= exp

−
(X(i) − µ(n))2
γ

(32)
is commonly used.
Note that:
lim
γ→0
Yγ(n) =
N
X
i=1
Ci Xi (33)
and
lim
γ→∞
Yγ(n) =
PN
i=1 Wi X(i)
PN
i=1 |Wi |
. (34)

Example
Elimination of Interference of the DWD
The Wigner distribution (WD) is defines as:
WDx (t, f ) =
Z
τ
x

t +
τ
2

x∗

t −
τ
2

e−j2πf τ
dτ, (35)
Its use has been limited by the presence of cross terms. The Wiener distribution of
the sum of two signals x(t) + y(t)
WDx+y (t, f ) = WDx (t, f ) + 2Re (WDx,y (t, f )) + WDy (t, f ) (36)
includes the cross term 2Re (WDx,y (t, f )) where WDx,y is defined as:
WDx,y (t, f ) =
Z ∞
−∞
x

t +
τ
2

y∗

t −
τ
2

e−j2πf τ
dτ. (37)

Chap 7. 01-38.pdf

Recommended

Recommended

More Related Content

Similar to Chap 7. 01-38.pdf

Similar to Chap 7. 01-38.pdf (20)

Recently uploaded

Recently uploaded (20)

Chap 7. 01-38.pdf