1. Wavelet Packets
Shortcomings of standard orthogonal
(bi-orthogonal) multi-resolution
structure of DWT,
Insufficient flexibility for the analysis
of signal under various application
requirements
2. Wavelet Packets
Signal components with wide range of
frequencies, are packed into details
D1
Desire to focus on signal components
with spectral bandwidths that are
sufficiently narrow at high and low
frequency ranges.
3. Wavelet Packets
Adaptive Decomposition
There are application areas such as
compression or pattern recognition/
classification problems where it is highly
desirable to identify wavelets with
certain properties suitable for the
requirements of a given applications
including the following
4. Application Requirements
Demand for High Resolution both in Time
and in Frequency
Signal decomposition and approximation
by a few number of large amplitude
coefficients for high compression,
Discriminatory separation of different
classes in pattern recognition and
classifications problems,
High efficiency of computations,
6. Dictionary Generation
Redundancy and “Best Basis”
To use redundancy in wavelet packet to
generate a dictionary that is composed of
large number of basis functions ( basis
functions that are not necessarily
orthogonal) and as such to have a higher
degree of freedom for selecting wavelets
suitable for a given application
10. Wavelet Packet Structure
Extended QMF structure
Decomposition of signal i.e. splitting
of signal into high and low resolution,
is applied to high resolution details as
shown below.
This is followed similarly during the
synthesis stage
12. Wavelet Packets
High Pass
Low Pass
2
2
2
Low pass
High pass
Wavelet Packets -Analysis Stages
2
High freq details
High pass
Low pass 2
2
V space of the signal
19. Redundancy in Wavelet Packet
Decomposition
Wavelet packet decomposition of a
signal, results in a considerable
redundancy and an increase in the
number of wavelet bases in which the
size of the library will grow rapidly
when the number of scale levels is
increased. Redundancy results in a
substantial increase of both
computational and storage costs
20. Basis Selection Algorithm
A pruning algorithm is needed for
selecting a subset of nodes for signal
representation considered suitable for
a given application and reducing
computational costs. These nodes
should provide a sufficiently accurate
approximation to a given signal
21. Basis Selection Algorithm
a selection criteria is needed in
which information cost of each
node in the expansion tree is
utilized as the basis for retaining
or discarding the node. Different
cost functions may result in different
approximations ( see below).
22. Basis Designation Indices
Parent node of a pair of children
nodes
A library of bases functions that are
indexed by three parameters (instead
of two parameters of scale and
translation as used in standard
multiresolution structure) as follows.
scale index j,
spatial location (translation) index k
frequency bands indexed by n.
23. Alternative Decomposition, Orthogonal or
non-orthogonal
A signal can be decomposed using orthogonal bases
functions that reside at children nodes at a given
scale.
However this may not be a ‘best representation for a
given application though an orthogonal
representation..
Computational cost of signal decomposition by
wavelet packet is of order O(Nlog(N)) as compared
with standard wavelet decomposition which is of order
of N ( i.e linear cost).
Frequency bandwidths of the details and pproximation
at a given scale are equal in size, similar to FFT
having an order of computational complexity of
Nlog(N).
24. Dictionary Construction, Best Basis
Selection
Cost Functions as Criteria for Best
Basis Selection
Cost functions are defined in accordance
with the requirements of a particular
application.
They are often described in terms of
concentration of information contained in
the coefficients
An Example: entropy-based cot function
25. Properties of Cost Functions
‘Additive property’
Under additive property, total cost is
given by the sum of costs of
individual nodes in a tree. It relates
information cost of a node to
information costs of individual
coefficients at that node.
C({xi}= C{∑ (xi)}=∑ C(xi)
C(0)=0
26. Different Criteria and Cost Function
Entropy Measure
Entropy is a measure of uncertainty in
predicting a particular outcome of an
experiment
Pdf of random variable x
Low Entropy High Entropy
x x
27. Entropy Measure
We define Shannon-Weaver entropy
of a sequence of x={xj, j=1,2,…n} as
following.
x={xj}, H(x)= -∑ pj log pj
pj=||xj||2/ ||x||2 and p.log p=0 if
p=0
Coefficients at a node are used as xj.
28. Entropy Measure
Under an entropy criteria for best
bases selection, wavelet packet nodes
with the coefficients that are
distributed across a narrow dynamic
range are selected, leading to high
compression rate for signal
representation
Comparison with Karhunen-Loeve
transform of PCA
29. Symmetric Entropy Measure
Symmetric Entropy and Discriminatory
Classification.
x={xi}, H(x)= -[∑ pj log pj/ qj +∑ qj log
qj/ pj ]
pj=||xj||2/ ||x||2 and p.log p=0 if p=0
qj=||yj||2/ ||y||2 and q.log q=0 if q=0
coefficients at wavelet packet nodes are
used for
x={xi} and y={yj}
30. Log of Energy Criteria
M(x) =∑ log|xj|2,
log0=0
Log of energy may be interpreted as
the entropy of so called Gauss-
Markov process, a stochastic process
composed of N uncorrelated Gaussian
random variables with variances as
σ21 =|x1|2, …..σ2N =|xN |2
31. Log of Energy Criteria
Minimizing log energy, finds a best
basis and best approximation for
signal representation using minimum
variance criteria similar to principal
component analysis (PCA) and
Karhunen-Lo’ve Transform (KLT)
32. Concentration in l p norm, p<2.
For this norm, we choose p<2 and set
M(x)=||{x}||p
Maximization of this norm results in
selecting a few large amplitude coefficients
with high concentration of signal energy in
few coefficients. That is, in a binary tree of
WP structure, nodes are selected that have
a few large amplitude coefficients leading
to a peaky pdf
33. Secondary Feature of WT and l p norm
In signal expansion by wavelets, it is also
observed that often large amplitude coefficients
are large across several neighboring scales as
well as neighboring coefficients at the given
scale.
This occurs at points of sudden changes and
singularities where clustering of coefficients is
observed both at within scale and cross-scales.
This property of wavelets, is often referred to as
a secondary feature of wavelet analysis.
34. Other Cost Function, Selection Rules.
“Informative Wavelets”,
“Dictionary Projection Pursue”’
“Local Discriminatory Basis” selection
criteria
“Matching Pursue”
On items 1 and 3 , papers have been
included in ‘Archives of Project
Reports’ posted on course web site
35. Best Bases and Best Tree
Best tree’ is a tree in the binary array of
the signal decomposition that corresponds
to minimal cost function.
And ‘best basis’ are the basis set that
belong to the nodes of the ‘best tree’. For
example in entropy-based basis selection,
best basis are wavelets(and scaling
functions) of those nodes having minimal
entropy of the coefficients at the node.Best
Tree: Refer to Matlab Wavemenu for
Illustrations
38. On the
Illustrative Examples of WP Basis
Note Db 45 (of longer length) generates
higher oscillatory wavelets (higher # of
oscillations) as compared with those of Db
5 (of short length). Also note equi-
frequency bandwidth of different wavelets
of the same scale.
Translation of wavelets are also seen easily
for different wavelets indicating the shift in
the location of center of the wavelets.
Note time span of illustrations are all equal
to the length of wavelet at lowest scale
