Wavelet packets provide an adaptive decomposition that overcomes limitations of the discrete wavelet transform (DWT). In wavelet packets, signal decomposition using high-pass and low-pass filters is applied recursively to both low-pass and high-pass outputs, allowing more flexible time-frequency analysis. This results in a redundant dictionary with increased flexibility but also higher computational costs. Pruning algorithms are used to select an optimal subset of bases for a given application based on cost functions related to properties like sparsity, entropy, or energy concentration.

1. Wavelet Packets
 Shortcomings of standard orthogonal
(bi-orthogonal) multi-resolution
structure of DWT,
 Insufficient flexibility for the analysis
of signal under various application
requirements

2. Wavelet Packets
 Signal components with wide range of
frequencies, are packed into details
D1
 Desire to focus on signal components
with spectral bandwidths that are
sufficiently narrow at high and low
frequency ranges.

3. Wavelet Packets
 Adaptive Decomposition
 There are application areas such as
compression or pattern recognition/
classification problems where it is highly
desirable to identify wavelets with
certain properties suitable for the
requirements of a given applications
including the following

4. Application Requirements
 Demand for High Resolution both in Time
and in Frequency
 Signal decomposition and approximation
by a few number of large amplitude
coefficients for high compression,
 Discriminatory separation of different
classes in pattern recognition and
classifications problems,
 High efficiency of computations,

5. Beyond Orthogonality
Compression Requirements
Slow rate of decay Fast rate of decay
Coeffs 2Coeffs 1
Orthogonal Expansion Non-orthogonal Expansion

6. Dictionary Generation
Redundancy and “Best Basis”
 To use redundancy in wavelet packet to
generate a dictionary that is composed of
large number of basis functions ( basis
functions that are not necessarily
orthogonal) and as such to have a higher
degree of freedom for selecting wavelets
suitable for a given application

7. Time-Frequency Partitioning
in Standard DWT
Constant Q factor
Fc ∆F Fc / ∆F = constant
Phase-plane Diagram of standard DWT and Frequency Partitioning

8. Perfect Reconstruction QMF Filterbank
High Pass
Low Pass Low pass
High pass
Perfect Reconstrution Filter bank
Wavelet
Subspace W

9. Standard QMF Structure of DWT

10. Wavelet Packet Structure
 Extended QMF structure
 Decomposition of signal i.e. splitting
of signal into high and low resolution,
is applied to high resolution details as
shown below.
 This is followed similarly during the
synthesis stage

11. Subspace Illustration
of Two Channel Wavelet Packet
Wj-1,0
Vj-1
Vj
Wj-1,1

12. Wavelet Packets
High Pass
Low Pass
2
2
2
Low pass
High pass
Wavelet Packets -Analysis Stages
2
High freq details
High pass
Low pass 2
2
V space of the signal

13. Standard and Wavelet Packet Structure

14. Frequency Partitioning

15. Wavelet Packets Tree

16. Binary Tree of DWT in Wavelet Packet

17. Wavelet, scaling Basis at WP Cells

18. Spectrum of Basis inWP

19. Redundancy in Wavelet Packet
Decomposition
 Wavelet packet decomposition of a
signal, results in a considerable
redundancy and an increase in the
number of wavelet bases in which the
size of the library will grow rapidly
when the number of scale levels is
increased. Redundancy results in a
substantial increase of both
computational and storage costs

20. Basis Selection Algorithm
 A pruning algorithm is needed for
selecting a subset of nodes for signal
representation considered suitable for
a given application and reducing
computational costs. These nodes
should provide a sufficiently accurate
approximation to a given signal

21. Basis Selection Algorithm
 a selection criteria is needed in
which information cost of each
node in the expansion tree is
utilized as the basis for retaining
or discarding the node. Different
cost functions may result in different
approximations ( see below).

22. Basis Designation Indices
 Parent node of a pair of children
nodes
 A library of bases functions that are
indexed by three parameters (instead
of two parameters of scale and
translation as used in standard
multiresolution structure) as follows.
 scale index j,
 spatial location (translation) index k
 frequency bands indexed by n.

23. Alternative Decomposition, Orthogonal or
non-orthogonal
 A signal can be decomposed using orthogonal bases
functions that reside at children nodes at a given
scale.
 However this may not be a ‘best representation for a
given application though an orthogonal
representation..
 Computational cost of signal decomposition by
wavelet packet is of order O(Nlog(N)) as compared
with standard wavelet decomposition which is of order
of N ( i.e linear cost).
 Frequency bandwidths of the details and pproximation
at a given scale are equal in size, similar to FFT
having an order of computational complexity of
Nlog(N).

24. Dictionary Construction, Best Basis
Selection
 Cost Functions as Criteria for Best
Basis Selection
 Cost functions are defined in accordance
with the requirements of a particular
application.
 They are often described in terms of
concentration of information contained in
the coefficients
 An Example: entropy-based cot function

25. Properties of Cost Functions
 ‘Additive property’
 Under additive property, total cost is
given by the sum of costs of
individual nodes in a tree. It relates
information cost of a node to
information costs of individual
coefficients at that node.
 C({xi}= C{∑ (xi)}=∑ C(xi)
 C(0)=0

26. Different Criteria and Cost Function
 Entropy Measure
 Entropy is a measure of uncertainty in
predicting a particular outcome of an
experiment
 Pdf of random variable x
Low Entropy High Entropy
x x

27. Entropy Measure
 We define Shannon-Weaver entropy
of a sequence of x={xj, j=1,2,…n} as
following.
 x={xj}, H(x)= -∑ pj log pj
 pj=||xj||2/ ||x||2 and p.log p=0 if
p=0
 Coefficients at a node are used as xj.

28. Entropy Measure
 Under an entropy criteria for best
bases selection, wavelet packet nodes
with the coefficients that are
distributed across a narrow dynamic
range are selected, leading to high
compression rate for signal
representation
 Comparison with Karhunen-Loeve
transform of PCA

29. Symmetric Entropy Measure
 Symmetric Entropy and Discriminatory
Classification.
 x={xi}, H(x)= -[∑ pj log pj/ qj +∑ qj log
qj/ pj ]
 pj=||xj||2/ ||x||2 and p.log p=0 if p=0
 qj=||yj||2/ ||y||2 and q.log q=0 if q=0
 coefficients at wavelet packet nodes are
used for
 x={xi} and y={yj}

30. Log of Energy Criteria
 M(x) =∑ log|xj|2,
 log0=0
 Log of energy may be interpreted as
the entropy of so called Gauss-
Markov process, a stochastic process
composed of N uncorrelated Gaussian
random variables with variances as
σ21 =|x1|2, …..σ2N =|xN |2

31. Log of Energy Criteria
 Minimizing log energy, finds a best
basis and best approximation for
signal representation using minimum
variance criteria similar to principal
component analysis (PCA) and
Karhunen-Lo’ve Transform (KLT)

32. Concentration in l p norm, p<2.
 For this norm, we choose p<2 and set
M(x)=||{x}||p
 Maximization of this norm results in
selecting a few large amplitude coefficients
with high concentration of signal energy in
few coefficients. That is, in a binary tree of
WP structure, nodes are selected that have
a few large amplitude coefficients leading
to a peaky pdf

33. Secondary Feature of WT and l p norm
 In signal expansion by wavelets, it is also
observed that often large amplitude coefficients
are large across several neighboring scales as
well as neighboring coefficients at the given
scale.
 This occurs at points of sudden changes and
singularities where clustering of coefficients is
observed both at within scale and cross-scales.
 This property of wavelets, is often referred to as
a secondary feature of wavelet analysis.

34. Other Cost Function, Selection Rules.
 “Informative Wavelets”,
 “Dictionary Projection Pursue”’
 “Local Discriminatory Basis” selection
criteria
 “Matching Pursue”
On items 1 and 3 , papers have been
included in ‘Archives of Project
Reports’ posted on course web site

35. Best Bases and Best Tree
 Best tree’ is a tree in the binary array of
the signal decomposition that corresponds
to minimal cost function.
 And ‘best basis’ are the basis set that
belong to the nodes of the ‘best tree’. For
example in entropy-based basis selection,
best basis are wavelets(and scaling
functions) of those nodes having minimal
entropy of the coefficients at the node.Best
Tree: Refer to Matlab Wavemenu for
Illustrations

36. Illustrative Examples of basis
0 50 100 150 200 250 300
-2
-1
0
1
2
wp(1:5)
Wavelet Packet wavelets generated by wpfun("db5",100,5)
0 50 100 150 200 250 300
0
10
20
30
40
fft(wp(1:10)
0 50 100 150 200 250 300
0
10
20
30
fft(wp(11:20)

37. Illustrative Examples of WP Basis
0 500 1000 1500 2000 2500 3000
-10
-5
0
5
10
wp(1:5)
Wavelet Packet wavelets generated by wpfun("db45",100,5)
0 500 1000 1500 2000 2500 3000
0
500
1000
fft(wp(1:10)
0 500 1000 1500 2000 2500 3000
0
500
1000
1500
2000
fft(wp(11:20)

38. On the
Illustrative Examples of WP Basis
 Note Db 45 (of longer length) generates
higher oscillatory wavelets (higher # of
oscillations) as compared with those of Db
5 (of short length). Also note equi-
frequency bandwidth of different wavelets
of the same scale.
 Translation of wavelets are also seen easily
for different wavelets indicating the shift in
the location of center of the wavelets.
 Note time span of illustrations are all equal
to the length of wavelet at lowest scale