This document provides an overview and table of contents for the book "Insight into Wavelets - From Theory to Practice, Third Edition". The book covers topics such as the history and origins of wavelets, continuous and discrete wavelet transforms, designing orthogonal wavelet systems, computing and plotting wavelet functions, biorthogonal wavelets, m-band wavelets, and multiwavelets. Each chapter introduces key concepts and includes exercises. The book is intended to provide both theoretical foundations and practical applications of wavelet analysis.
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
1. , ,, / - ,
. (~. Third Edition ..
11e'"1I
Insight into
~
stem
conomy
dition
FROM THEORYID PRACTICE
KP. SOMAN •KI. RAMACHANDRAN •N.G. RESMI
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
2. R... 375.00
INSIGHT INTO WAVEl£Ts--f'rom Theory to PracUce, 3rd MII. (wilt! CO-ROM)
K.P. Soman, K.I. A.amachandran, N.G. Rasmi
0 2010 by PHI Learning Private Umiled. New 0eIII. AI rights .--ved. No pari 0I1hIs book may be
reproduCad in any klnn. by mimeog...llln Of any 0Ihef ~. -..iIholA pemIisaion in writing from ".
publishef.
-----...,-..........- ...----.....-..~--_ ...... co--.n. .......... _ _ ....... _ ..... _ ... _ .. _ • 0l0I _ _ ..
_. , -. .. -.g ... COI. ... ~ ""..oo.=. ... _ .. lOo E_ _•
ISBN-978-81-203-40S3-4
The .~potI rights 01 IhII book Ire 11951ed IoIIIy w;., !he publisher.
S!Jc1t! Prlntl"1l (ThIn! Edition) M..-dI,2010
P~bli5h.d by A50h K. Gh-osh, PHI leamlng Privati llmitad. 1.4-97, Connaught Circus.
New [)aH.ll000t and Printed by Mudrak. 3O-A. Patparganj. Delhi-l10091 .
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
3. Contents
Prrjoce
PrrjllCe /0 the Fir" Edition
A.ckno...led~mellU
1. The Age of Wavelets
inlmdMction--.l
1.1 The OrigiN of Waveleta-A~ They Fundllmcntally New? I
1.2 Wavelets and Other Rcality TrwlSfonns 3
1.3 Managing HeisenberJ', Uncenai!!!y: Gho6t j
1.4 Histor}' of Wavelel: from Morld 10 Daubechies via Mall&! 6
L4.LDiffcmrlLCommunitiu..oLWaveM:1L..9
1.4.L...Diffcreol EamiliCLoLWavcieuJ'jthin W'l'Ciet Communitics----.lO
.4.3 IlIIClt:5ting R«ent Devclopmcnb: 11
1.5 W.vt:lets in thr; Furun: 12
1.6 What llistOf}' of Wavelet T-=hu UI 15
S...........ry lj
SilIlUlM F,mJter RMding~ 15
1. Fourier Series and Geometry
lfllrOducliOfl /6
2.1 ¥«tot Space 17
2.U-----.S.uu.----.l7
2.1.2 Onhonormality 17
2. J.3 Proja;tion 17
22 Functions and Function S~ 18
2.2.1 Orthogon.l Fu.netions /8
=_OrtboooonalEullCtion5~9
2.23 Fwx:tion Spac:eI 19
2.2.4 0rtb0g0aa.L Basis Fl,lno;tions 22
2.2.5 OrthooormaIil)' and the Method of Finding the Coefficients 22
'"
Id'
16-32
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
4. viii • Contents
2.2.6 Comp~x Foorier Series 16
2.2.7 Orthogonality of Complex Exponential Bases 17
Sum/lUl_fl'...- 29
Ezerr.isu_ 30
SU8llt slt d Furthu Rtading~
3. Continuous Wavelet and Short TIme Fourier Transform
/fltroducliOll_ 33
3.LWavdeLJnnsform--A..Eim.L:.vdJnlrodLII:linn---.J3
3,2~athemalic;tl&eliminarie$-EQuri« TnlIl$form JJ
3.2.1 of ~gnal$ 39
3.2.2 The Founc! Transforml 40
3.2.3 The tli 41
3.3 Propenics of Wavelets In 45
3.4 Conlinuow; versus Discrete Wavelet Transform 45
SWllmQry 47
E:arr:isLL 48
Suggt~!td F...~r Rtadi~&s 49
33-4'
A. DisClocte Wankt...Transronn 50-74
/mroductio~ 50
4.1 Haar Scaling Functions and Function Spaces 50
4.1.1 Translation and &alinS of ;(1) 51
4.1.2 Orthosonal"l)' of Translates~O 51
4. 1.3 Function S~ V. 53
4.1.4 Finer Hair ScaliRJ Functions 55
4.2 Nested Spaces 56
43 Haar Wavdd Function 57
43.1 Scaled Haar Wavekt-Func6oos 59
4.4 OrtItogooality of ;(1) and 11(1) 64
45.......Nonnalizalion_oUiaaLBasa..ILDiffcrenLScak.s_ 65
4.6 Standardizing the Notations 67
4.7 Refinement Relation with Res~t to Normaliwl Base.< 67
4.8 Support of & Wavelet System 68
4.8.1 Triangle Scaling Function 69
4.9 Daubtthics Wavelets 70
4.10 Stocing tilt Hidden-PloI:ting tilt Daubtoch.iu Wa~It" 7J
Summary 73
Extrr.Ut~ 7J
Suggtutd FUr/htr R",di"ls 74
S. Designing OrthogonaJ Wavelet Systems-A Direct Approach 75-93
ImrodUClioll_ "
5.1 Refinement Relation for OrtItogo;>!!a1 Wavelet S)'stcms 75
5.2......Restrictionu.n-Filtcr Coefficients 76
5.2.1 Condition 1: Unit Area Undor Sc.a1inll Funetion 76
5.2.2 Coodition 2: Orthonorma.lity of Translates of Scaling Funetions 76
5.2.3 Condition 3: Orthooormality of ScalinS and Wavelet Functions 79
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
5. Coments • ix
5.24 Condition 4: Approximation Condition5 (Smoolhne55 Conditions) 81
5.1.5 Condition 5;j!edundant) OrthonormaIity of Tl'IIl$lates of Wavelet
Fu.nctions 82
5.2.6 Condit~UIIdant) Orthooonnal'ty of ,(0and Translates of ttC!:) 82
5.3 Designing Daubt:chics Orthogonal Wavdet bstem Coefficients 83
5.3.1 ColIStnIints fow D.ubcchies· 6-1ap Scaling FLJ.Ilf;'tion 84
5.4 Design of Coiflet Wavelets 85
5.s StmlelS 86
5.6 An Intri,guin~y of Ortbogonal Scaling FWICtion 86
S.6..1~ SlImm·'ion.Form~88
5.6.2 Proof of Partition of Unity of Scaling Func:tion 90
5.6.3 Ne<:cni!), of Partition of Unity of Scaling Function 91
SIUftIIIQf)'_ "
E2~j!el-----.92
Sugg~sled Funher R~odill" 93
6.---»i.Krete WaYdet.Tnms!orm and Relation to f'Ute..-.lJank5 9+----.ll0
11llrOdJ.djOlt~
6.1 Signal Decom~ition (Analy~ 94
6.2 Relation with Filter Banks 97
6.3 F~ Response 102
6.4 Sig~1 Reconstruction: Synthesis from Coarse Scale to Fine Scale /03
6.4.1 Upsampling and filleTing 1()4
6.5 Perfect Match.i!!8 Filten 107
6.6 Computing Initial 'itt Coefficients 107
6.7 Vanishing Moments of Wavelet Function and Filter Properties 107
S"'""""ry---.JP9
Suglu/ed Funhu Rtadill8£ 110
7. Computing and PlottJng Scaling and Wavelet Functions 111-126
..
IItlmduClimt----.lH
1.1 Daubcchies-Lagatias Algorithm 112
1.1.1 Discrete Dilation EqlWion lJ4
1.1.2 Swcmcnt of Daubcchies-l...a8!!ias AI~thm /15
7.1.3 Generating Binary Equivalent of a Decimil Number JJ6
7.1.4 Implementation in Microsoft Excel (or Any Spreadsheet Pacuge) JJ6
7.1.5 CoIllR!!Un.LWavelet FuocUon 118
7.2 Sub:1ivis.ion..Scbem.L...l18
7.3 S""",,"';ve Appro.imation 121
8.1 Bionho&onali!J: in Vector S~ 127
8.2 Biorthogonal Wavelet Systems 129
S.3 Sigru.l~ UsinS_BiortlKlgonal Wavelet S)'Slem H2
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
6. " • Contents
8.4 Bioooogonal Analy~lJZ
8.S Coarse Scale to Fine Scale JJ.4
8.6 Wa""let~aum& 1.15
Summary-HJ
E:arru£L-L42
,o••1,"'••,., S)'~", or Cohen-Daubcchies-Fca.uveau
117
Suggesled FlIftMr Readings 141
9, Designing Wavekts-FrequelK)' Domain Approach 143-UI3
InJrrxluC/ioIL ...1j3
9.1 Basic Proptrtics of Filter Cocf("t<.:icntli 143
9.2 Fillcr Properties in Terms of H..l(li) 144
9.3 Filter Properties in TCITll5 of H,(z) /45
9.4 F~uelK)'_ Domain Characteril.lllion of Filter Coefficients 146
9.S Choice er Wavelcl Function Coeflkicnts Ig(.)) 150
9.6 Vanishing Moment CondiliO/lll in Fourier Domain 153
9.l.......DerivatiQn nLDaube:c.h.ic.LWavdcts.......15.4
9.7.1 Steps Involved in DerivaliOfl of Daubechies Wavelets 1S4
9.7.2 Daubechies Wavelets with I Vanishing Moment 1S7
9.7.3 Daubechies Wavelets with 2 Vanishing Moments 158
9.8 Parametric Design <.If O"~onal and Biortoollooat Wavelets 159
9.8.1 Polyphase Factorizatioo Approach 1<1 Onhogonal Wavelet Design 159
9.8.2 Bionoollonal Wavelet Desig.........A Factor Multiplication ~h 166
9.8.3 Exte",,""" to Orthog<llllll Wavelets 179
9.8.4 Applicatioos of Panunetcriud W.""lcts /81
10, Groebner Basis (or Wavelet Design 184-192
InJrrx/ucliOll--.-l84
lo..LGrocbner...Basu.--.l1tl
10.2 Deriving Grocbner Basis for Daubechiel Wavclcl Syaum 185
10.2. Daubedties' 6-tap W.ve!ct with 3 Vanishing Moments 187
10.3 Deriving Grocbnet Basis for Coiflet /88
10.3.1 ~rties of the Moments of Scaling Functions 189
10.3.2 Des'lln of 6-tap Coiflet 190
10.3.3 Gcnenolitcd Coif1c:t S)'stem 19/
Summ<l~ /92
11. WaveletPacket Ana1ysis
btl~IimL....JJ)J
1l.L ...HuLWaveIeLl'aculL ...194
~];:!:::':B~U:i'~SeIecIiOO f()!" SiS.nal CII" Image Compassion 100
S"""""ry 202
193-202
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
7. 12. M-Band Wavelc:ts
/nlmdJJctian 203
l21---.Motivation----.203
12.2 Multi·resolution Formulation of M·B~nd Wavelet Symm 206
12.3 Derivation of tbc ProJ1C'rties of M-Band Filter Coefficients 209
12.4 SupportS of SQIIing FUrK:lion lJld Wavelets Zf4
Conknts • xi
203-228
12.S Dcaign of 4-Band Symmetric Or1hogonal Wavelet Filtu Banks Based on Il!oOO.i 215
12.S.1 Groebnc:r Based Design Example with Length = 12 and Regularity =2 216
12.6 Parametric: Da.ign of M-Band Wave1c1S 219
12.6.1 Down~g and IVlynomial Represcnt:>l.ion 219
Summary 228
13. Introduction to lfultiwaveiets
1.3.1 Refrnable Function Vcctor 229
13.I.I Fuoction Sp3Ce VI 231
13.1.2 Su~ of, 234
13.1.3 Symbol of, 234
13. 1.4 FourierTransfocm of f: 235
13.U OrthononnaI.ity of, 236
13..6 Orthonormality of , in ternlS of H. 237
IJ.2......Multiwavc:let.Eunctioo----.2J10
13.2.1 Symbol of tt(o) 240
1l.2.2_Fourier Transform of tl!!-240
13.2.3 Orthogonality..&J! and ~'l 241
13.L ..Momcnts_ 241
13.3.1 DiKrek Moment of; 24J
13.3.2 DUcrete Moment of ~,) 241
13.3.3 Relation Bd_n M. and H(CtI) 241
13.3.4 Relation Bdween N,{ij and G(I) (IV) 143
13.3.S Continuous Moment of;' 143
13.3.6 Conlinuous Moment of "I') 143
13.3.7 Relation Between Jl> and Fourier Transform of, 243
13.3.8 Relation Between vi') and Fourier Transform of ttI') 244
13.3.9 Relation Between J!t and M. 244
13.4 Approximation Order 246
I1..L...MuUiwavc..lo::l_Eilta..BaDk 149
13.S.1 Analysis and 149
13.S.2 250
13.'.3 Signal ProeC5&ing) 250
13.6
13.7 TImc-varyinLMultiwavelet Filter Bank 252
13.8 Balancing Condil;ons 256
1,3~.8~.~1>lB~"~~~;";8~O~f~""";~'~I.~';'~O~:25"13.8.2 J; _ I 258
13.9 259
162
229-277
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
8. xii • Contents
13.9.3 Constn>elion of InterJ1Qlating Mu.ltiwavdet 175
SU/MIOI)' 177
~8!sled Further Rwdl'!8s In
14. U fting Scheme 278-313
IIIJrD<luction--.l.18
]4.] Wavelct Transform Using Poly£1a$e M.m~ Factori~on 179
14.1.1 Invenc Lifting 183
14.].2 Example: Forward Wavelet Tran~form 184
14.2 Geometrical Found;Wons of Lifting Scheme 186
14.2.1
14.2.2
14.2.3 Using Lifting 291
[4.2.4 Higher Order Wavelet Transfonn 295
[4.3 Lifting Scheme in the Z-domain 296
14.3.1 Desi,n Example I 301
]4.3.2 Example 2: Lifting Haar Wavelet 305
14.4 Mathematical Pttliminaries for Polyphase Factorization 307
14.4.1 Laurent Polynomial 307
[4.4.2 The Eoclidcan AI,orithm 308
14.4.3 Factoring Wavelet Transform into Liftin~A Z-domain Al!Pf"OICh 314
]4.5 Dealinl_wilh Signal Bolmdir)'_ 1i9
1"-'..1 CircI,laLCooYOlutiOll_ 319
14.5.2 Padding Policies 310
14-'.3_ lteraLioo_Beh.aviour------111
SU/MIOr)' 31 J
Exercuu 311
15. Image Compression
lntrD<luctit11L-314
15.] Overvicwof
15.J. l
15.2 Wavelet Transform
]5.3 Quantization 331
15.3. 1 Uniform Quantjution 331
15.3.2 Subband Uniform Quanti~ion 332
15.3.3 Uniform Dead-zone Quantization 333
15,3.4 Non·uniform Quantization 333
]5.4 Entropy Encoding----.l34
15.4.1 Huffman Encodi"B 334
15.4.2 Run Length Encoding 3]5
15.5 ElW Coding (Embedded Zero-tree Wavelet Coding) 336
15.5.] EZW Pcrformar>ec 345
15.6 SPUIT (Set Partitioning in Hicran:hical Tree) 346
15.7 £Bear (Embedded Block Coding with Optimized Tl1.lICation) 346
SIUll/lW~ 346
Webk_ (Wtb Based Prob/e/llS) 347
Suggesftd FUrlM' ReMillgs 347
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
9. Contents • xiii
16. Denolslng 349-365
IntrrJ<iJu:tion 349
16. A Sim~1c E;o;~lanaJjon and a 1-0 Eumpk....}49
16.2 DenoisinJl U,ing W~1d Shrinkagc-Stati51ical Modelling and Estimation 3JO
16.3....NoiJc.Es.tima!iIHL JJl
16.4 Shrinkage FUllClions JjJ
16. ~ Shrinkage Rules Jjj
16.~.1 Universal Jjj
16.~.2 Minimizing the False Discovery Rate JJJ
16.~.3 T~ J56
l6.M S.IU:C_ JJ6
16.~.S Translation Invariant Thresllolding J57
16.5.6 BayesShrink 357
16.6 Denoisina Images with MATI.AB 358
16.7 MAl1.A8 Programs for Denoising J60
16.8 Simulalion for Finding Effectiveness of Thresholding Method 361
SWMlClry 36J
EMn:ises 364
Su.ggulni Fu.rtMr R,.,Jillgs 365
17. Ber.ond Wavelets: The Ridgelets and Curvcleb
In1rrJ<iJu:lion 366
.8.
17.1 AWlOlIimation Rates 367
17.2 Why 1 and Curvclets.? J68
17.3 The Transform 369
17.3.3 Finite Radon Transform J7I
17.3.4 Applications of Rid~1et Transforms 371
17.4 The Digilal Curvclel Transform 377
17.4.1 Propenics of Curvclct Tmnsform 378
17.4.2 Analysis: The Curvc:lel Decomposition 379
17.4.3 Curvclel Dcc<>mposition Algorilhm 379
17.4.4 Sy_nlhcsis: Reoonslruct.ioo from !he CtmIelet Transform J80
17.4.5 Digit.aJ Implementation or the Curvclet Transform 38()
17.5 S«ond Gc~ralion ~lets. 382
17.~.1 Tight Frame of Curveleu 384
1.7.5L Cw:vekt Constructioo.....J8.S
17.~.J
l7.6
S~l
'"N_1i~ Fnme of Curvcletl 388
18.1.1 Splinc eurve. and,:"u.r{aces 395
18.1.2 Cubic Spli~ Interpolation Methods J99
18.1.3 Hennitc Spli~ Interpolation 399
366-391
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
10. xiv • Co~tents
18.1.4 CUbic SJllines #JI
S.1.5 Splines in Signal and Ima~ ProccssinS_ #J3
18. 1.6 Belier a.n'Ves and Surfaces 404
IS.1.7 Pr~rties of Buier CUrves 406
18.I.S Quadratic and Cubic Bezier Curves 407
18.1.9 Paramwic Cubic Surfaces 409
18..10 Buiu Su.rfaccs 4./1
IS.1.11 B·splincCu1'Ves 4/2
IS.1.12 Cubic Periodic B·splincs 414
IS. .1) Co,w"rsion Between .Icnnit". Be"i". and B·splinc Representations
IS.1.14 Non·uniform B·sjl:lines 416
IS.1.5 Relation Iktwo:en SJlline, BWe. and B·~linc ~s 418
IS.I.16 B.splinc Surfaces 419
IS. I.l1 Beta.splincs aoo Rational Spli""" 419
IS.2 Multiresolution Methods and Wavelet A~ysis 420
IS.1...Jhe_Filtu-.B~411
8.4 Ortbo.l!onaJ and Semi-ilrtboJOOal Wa~lets 413
IS.S Splinc Wa,,,,lcts 414
IS.6 ~rties of Splinc Wavelets 427
18.7 Advantages of B·splinc Based Wavelets in Signal Processing Applications
18.8 B·spliroe Filter Bank 430
IS./LMuIliresolutiOlLCun>a and Facc.s_ '"
18.10 Wavelet Based CUrve and Su.rfacc Ediling 431
18.11 Variational Modelling and the Finite Element Method 431
18.12 Ada~i..... Variational Modo!llina U£ina W.""lets 43S
SUnurt<l1)' 436
Sugge.ted Funhe, RetuJingJ 436
439-443
445-447
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
11. Preface
Wavelet theory has matured and has entered into ils second pllase of development and evolution
in which practitioners an: finding newer applications in ever-widening scientific domains such
115 bio-informatics, computational drug discovery and nano-material simulation. Parallelly. the
theory of wavelel$ got more and more demystified and has become an e~~ryday tool for signal
and image processing. Postgraduate courses in mathematics and physics now ioclude a subject
on wavdet theory either as a separate eJoctive or as part of other related subjects. In many
technical universities, w3sclel has been introduced cven at the undergraduate level. In this third
edition of the book we have taken into account this increasing popularity and the needs of the
relatively 'young' readers from such wide range of backgrounds.
One of the main additions in this third edition is that we have shown how the ubiquitous
electronic spreadsbcct can be utilized for wavelet ba.o;cd signal and image processing. The theory
behind the algorithm for computing e)lact values of wavelet and scaling function is simplified
and implemented in Microsoft E)lcel as a workshcct function. Onc can now draw the fUlICtion
by writing and dragging an e)lcd formula in a cell. Many of the intriguing properties of wavelet
and scaling functions such as orthogonality of integer translates, partition of unity and
refinement relation of scaling functions can be easily visualized in spreadsheets. The
accompanying C D contains several worksheets that demonstrate the power of spreadshcct
packages as a computational and visualization 1001.
Recent years have secn heightened interest in 'parametric wavelet filter design' which
allows the tuning of wavelet fiheB for various applications. Theory of its design procedures arc
added in respt(;tive chapteB with several e:o;amples.
Another new feature is that parametric and !"IOn-parametric biorthogonal wavelet design
are e:o;plained in more detail.
M-band wavelets are finding increasing applications in Communication Engineering as a
tool for multirate signal procciSing and as signal modulators. So the chapter on M-band wavelet
is eJlpanded to include the more recent and simplified design procedures.
A scparate and elaborate chapter on Multiwavelet theory is added. Muhiwavelet represents
the highest level of generali:.tation in wavelet theory and provides short filtcrs with most Of all
of the desirable properties that a filtcr should possess. 1beory of baI:ma:d and interpolating
multiwavelets are discussed in detail.
"
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
12. xvi • Preface
We earnestly hope that this edition will meet the: needs of readen of different academic
backgrounds for their undcrgl1lduate, poslgraduate and research le'el sludies.
Finally, we acknowledge OUT heanfelt Ihanks here to our ex-students Ms, K. Hemalalha
and T. Ar.nhi who prepared all Ihe worksheets given in the accomp311ying CD.
K.P. SOMAN
K,I, RAMACHANDRAN
N.G. RESMI
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
13. Preface to the First Edition
In the past few years, the study of wa'c1ets and the exploration of the principles governing their
behaviour have brought about sweeping changes in the disciplines of pure and applied
mathemaTics and sciences. One of the most significant development is the realization thai., in
addition 10 the canonical tool of representing a furn:tion by ils Fourier series, then:: is a different
representation more adapted 10 certain problems in data compression. noise removal, pauem
classification and fast scientific computation.
Many books are available on wavelets but most of them are wriuen at such a level Ihat
only research mathematicians can avail them. The purpose of this book is to make wavelets
accessible 10 anyone (for example. graduate and undergraduate students) wilh a modest
background in basic linear algebra and to serve as an introduction for the non-specialist. lbe
level of the applications and the format of this book are such as to make this suitable as a
textbook for an introductory course on wavelets.
Chapter I begins with a brief note on the origin of wavelets. mentioning the main early
contributors who laid the foundations of the theory and on the recent developments and
applications. Cbapter 2 introduces the basic concepts in FotJrier series and orients thc reader to
look at everything found in the Fourier kingdom from a geometrical point of view. In
Chapter 3, the focus is on the continuous wavelet transform and its relation with soort time
Fourier transform. Readers who have oot had much exposure to Foorier transforms carlicr may
skip this chapler. which is included only for the purpose of completeness.
Chapter 4 places the wavelet theory in a concrete selling usinS Ille Haar scaling and
wavelet function. lbe rcst of the book builds on Illis material. To urKlerstand the conceptS in Ihis
chapter fully, the reader need to have only an understanding of the basic concepts in linear
algebra: addition and multiplication of vectors by scallUl;, linear independence and dependence,
orthogonal bases, basis set. vector spaces and fUllClion spaces and projection of vectorffunction
on to the bases. TIle chapter introduces the concept of nested spaces. which is Ille comer stone
of mulliresolUlion analysis. Dauhechies' wavelets are also introduced. The chapter concludes
wilh a note on the fact that moSt of the wavelets art fractal in nature and that iterative methods
are required to display wavelets.
Designing wavelets is traditionally camed out in the Fourier domain. Readers who are IlOl
experts in Fourier analysis usually find the theoretical arguments and terminology used quite
ntl
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
14. xviii • Pref:>C(: to the l-irst Edition
baffling and tOlally out of Ihe world. This book. the~fore. adopts a lime domain approach 10
designing. lltc: orthogonality and smoothness/regularity constraints are di~ctly mapped on to
constraints on the scaling and wavelet filter coefficients, which can then be solved using solvers
available in Microsoft Excel-a spreadst.eet package or a scientific computillion package like
MAn.AB or MOlhematica.
Engineers onen view signal processing in terms of filtering by appropriate filters. Thus.
Chapter 6 is devoted to eSlablish the relationship between 'signal expansion in terms of wavelet
bas.c:s· and the ·filter bank· approach to signal analysis/synthesis.
Chapter 7 discuss.c:s tile theory behind parametric wa,·elets in an intuitive way I1Ither than
by using rigorous mathematical approach. The chapter also discusses various methods of
plotting scaling and wavelet functions. The focus of Chapter 8 is 011 biortoogonal wavelets
which is relatively a new concept. To drive oome this concept to the readers, biorthogooality
is explained using linear algebl1l. The chapter then goes on to discuss the design of elementary
S-spline biorthogonaJ wavelets.
Chapter 9 addresses orthogonal wavelet design using the Fourier domain approach.
Chapter 10 is devoted to the lifting scheme which provides a simple means to design wavelets
with the desil1lble propetties. lltc: chapter also shows how lhe lifting scheme allows faster
implementation of wavelet decomposition/reconstruction.
Chapters 11 to 13 describe applications of wavelets in Image Compression, Signal
Denoising and Computer Graphics. The notations used in Chapter 13 are that used by
~searchers in this patticular area and could be slightly different from those in the rest of the
chapters.
To make the book more useful 10 the readers. we propose 10 post lhe teaching malerial
(mainly PowerPoinl slides for each chapter, IlIld MATLABlExcel demonstralion programs) at
the companion webs-ile of the book: www.umritG.tdultrnlpublictdionslwu.vrkts.
We earnestly hope Ihat this book will initiate several persons 10 this exciting and
vigorously growing area. Though we have spared no pains 10 make this book free from
mislakes, some errors may slill have survived our scrutiny. We gladly welcome all com:ctions.
recommeOOatioos, suggestions aOO constructive criticism from our readers.
K.P. SOMAN
K.I. RAMACHANDRAN
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
15. Acknowledgements
First and foremost. we would like 10 express our gratitude 10 Brahmachari Abhayamrita
Chaitanya. who persuaded us to lake this project. and never ceased to lend his encouragement
and support. We thank Dr. P. Venkal Rangan. Vice Chancellor of the university and
Dr. K.B.M. Nambudiripad. Dean (Research}-our guiding Slars--who c,mlinunlly showed us
what perfection means and demanded perfection in everything Ihal we did. We would like to
thank, especially, Dr. P. Murali Krishna, a scientist at NPOL, Cochin, for his endearing support
during the summer school on 'Wavelets Fractals and Chaos' thal we conducted in 1998. It was
then Ihal we learned wavelets seriously. We take this opportunity 10 thank our research students
C.R. Nitya. Shyam Divakar, Ajilh Peter, Sal1lhana Krishnan and V. Ajay for their help in
simplifying the concepts. G. Sreenivasan and S. Soornj. who helped liS in drnwing the various
figllres in the textbook, dcserve a special thanks. Finally. we express our sincere gl3titude to the
editors of PHI Learning.
K.P.SOMAN
K.I. RAMACIiANDRAN
N.G. RESMI
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
16. The Age of Wavelets
INTRODUCTION
Wavelct analysis is a new development in the area of applied mathematics. 'Illey were firsl
introduced in seislTIQlogy to provide a time dimension to seismic analysis that Fourier analysis
lacked. Fourier analysis is ideal for studying stationary data (data whose statistical propcnics
are invariant over time) bul is nol well suited for studying data with transient events that cann<ll
be statistically predicted from the data's past. Wavelets were: designed with such non-stationary
data in mind, and with their generality and strong results have quickly become useful 10 a
number of discipl incs.
I.I THE ORIGINS OF WAVELETS-ARE THEY
FUNDAMENTALLY NEW?
Research can be thought of as a continuous growing fractal (see Figure 1.1) which often folds
bad: onlo itself. This folding back definitely occurred several times in tne wavelet field. Even
though as an organized research topic wavelets is less than two decades old, it arises from a
"ollstellat;on of related eonc:t:pts developed o""r 11. period of ne;orly two ""nturies, repell.ledly
redisrovered by scientists wbo wanted to solve techni"al problems in their various disciplines.
Signal processors were seeking a way to transmit clear messages o~r telephone wires. Oil
prospectors wanted a better way to interpret seismic traces. Yet "wa~lets" did 001 become a
oousehold word among scientists until the theOf)' was liberated from the di~rse appliCalions in
which it arose and was synthesiud into a purely mathenwical thCQf)'. This synthesis, in turn,
opened scientists' eyes to new .ppliCalions. Today, for example, w.""lets are !K)( only the
workhorse in computer imaging and animation; they also are used by the FBI to encode its data
base of 30 million fingerprints (see Figure 1.2). in the future. scientists may put w.~kt analysis
for diagnosing breast cancer, looking for heart aboonnalities (look at Figure 1.3) or predicting
the weather.
•
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
17. 2 • Insighl inlo Wavelets-From Theory 10 Praclice
FIGURE 1.1 Fnoctal. Wt fold Net on i~lf.
FIGURE 1.2 An FBI-digilized left thumb fingerprint (1lIc: image on the left i. the original; the one on the
right is reronsuuccr:d from a 16:1 """'pI"'ssion.)
,.,
LO
-<
0.'
'" 0.' - Hcallhy
- H~an failun:
0.'
0.0 0.' 0.' 0 3 0.'
,
FIGURE I.J MuhifractaJ 'pec(fum of hcar1 heat oscillations. h is a ""ph of singularity mcasun: versus flllCl&l
dimension. Thi'i spWrum clplum I di ff~n[ kind of iofoonalion Ihill CIIlOOl. be cljlluml by
a '1nl"''''')' spectrum. Wn-.:1c1S an: used for muhifraclal spectrum estimation.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
18. The Age of Wavelets • 3
1.2 WAVELETS AND OTHER REALITY TRANSFORMS
Wavelet analysis allows researchers to isolate and manipulate specific types of p<lttems hidden
in masses of data, in much the same way our eyes can pick out the trees in a forest, or our ears
can pick out the flute in a symphony. Otle approach to understanding how wavelets do this is
to stan with the difference between two kinds of sounds-a luning fork and lhe human voice
(see Figures 1.4 and 1.5). Strike a luning fork and you get a pure tone that lasts for a 'ery long
time. [n mathematical theory, such a tone is said to be "localized" in frequency, that is. il
consists of a single note with no higher-frequency overtol1oe.'l. A spoken word. by cOntl1lSt, laslll
for only a second. and thus is "localized'" in lime. It is not localized in frequency because the
word is not a single tone: but a combination of many different frequencies.
f f f
V V
I'TGURE lA Graphs of the sound WllveS prodoced by • tuning fQr< (top) and thr spol.C1 wool "grea.ly"
(bottom) iliu!MItc the diffm:l"ICc bet'.ieen • tone I<x:alized in freqUfflC)' and one I<x:ali«<i in
ti..... "The tuning fork produces a .imp'" "si"" wa>·c".
1/
FIGURE 1.5 A w."" and a _velcL
Graphs of the sound waves produced by the tuning fork and human voice highlight the
difference, as illUlitrated here. 1be vibrations of the tuning fork trace out what mathematicians
call a sine wa~. a smoothly undulating curve that, ill theory, could repeal forever. In contrast,
the graph of the word "greasy" contains a series of sharp spikes: there are nO oscillations.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
19. 4 • Insight into Wavelets-From Theory to Practice
In the nineteenlh century, mathematicians perfected what might be called the tuning fork
version of reality, a theory known as Fourier u1Ullysis. lean Baptiste loseph Fourier, a French
mathematician, claimed in 1807 that any repeating waveform (or periodic function), like the
luning fork sound wave, can be cllpressed as an infinite sum of sine waves and cosine waves
of various frequencies. (A cosine wave is a sine wave shifted forward a quarter cycle.)
A familiar demonstration of Fourie(s thcory occurs in music. When a musician plays a
note, he or she creates an irregularly shaped sound wave. n.e same shape repeats itself for as
long as the musician holds the note. Therefore, according to Fourier, the note can be separa~d
into a sum of sine and cosine waves. The lowest frequency wave is called the fundamental
frequency of the note. and the higher fn::quency ones are called o,·ertoncs. For example, the
note A. played on a violin or a nute, has a fundamental frequency of 440 cycles per second and
ovenones with frequencies of S8O. 1320. and so on. Even if a violin and a flute arc playing the
same note, they will sound different because their ovenones have different strengths or
"amplitudes". As music synthesizers demonstrated in the 1960s, a very convincing imitation of
a violin or a nute can be obtained by recombining pure sine waves with the appropriate
amplitudes. That, of course. is ellactly what Fourier predicled back in IS07.
Mathematicians later elltendcd Fourier's idea to non-periodic [unctions (or waves) that
change over time, rather than repeating in the same shape forever. Mosl real-world waves arc
of this type: say, the sound of a m(){or that speeds up. slows down. and hiccups now and then.
In images, too. the distinction between repeating and non-repeating patterns is imponant.
A repeatins pattern may be seen as a texture or background while a non-repeating one is
picked out by the eye as an object. Periodic or repeating waves composed of a discrete series
of overtonc:s can be used to repres.c:nt repeating (background) pauerns in an image. Non-periodic
features can be resolved into a much more eomplu spectrum of frequencies, called the I'-ourier
transform, just as sunlight can be separated into a spectrum of colours. The Fourier transform
portrays the structure of a periodic wave in a much more revealing and conccntraled form than
a traditional graph of a wave would. For elllmple, a rallle in a motor will show up as a peak
at an unusual frequency in the Fourier tnulsform.
Fourier transforms have been a hit. During the nineteenth century they solved many
problems in physics and engineering. This promirtence led scientists and engineers 10 think of
them as the preferred way to analyze phenomena of all kinds. This ubiquity forced a close
examination of the method. As a n::SUII, througOOut the twentieth century, mathematicians,
physicists. and engineers came to realize a drawback of the Fourier transform: they have trouble
reproducing transient signals or signals with abrupt changes, such as the spoken word or the rap
of a snare drum. Music synthesizers. as good as they are, still do not match the sound of eoncert
violinists because the playing of a violinist contains transient features--such as the contact of
the bow on the string-that arc poorly imitated by representations based on sine waves.
The principle underlying this problem can be illustrated by what is known as the
Heisenberg Indeterminacy Principle. In 1927, the physicist Werner Heisenberg stated that the
position and the velocity of an object cannot be measured CIlactly at the SIlIl"Ie time even in
theory. In signal processing terms, this means it is impossible to know simultaneously the exact
frequency and the exact time of occurrence of this frequellCy in a signal. In order to know its
frequency, the signal must be spread in time or vicc versa. In musical terms. the trade-off means
that any signal with a short duration must have a complicated frequency spectrum made of a
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
20. The Age of Wavelets • 5
rich variety of sine waves whereas any signal made from a simple combination of a few sine
waves must have a complicated appear:J.nce in the time domain. Thus, we can't e;t;pect to
reproduce the sound of a drum with an orchestra of tuning forks.
1.3 MANAGING HEISENBERG'S UNCERTAINTY GHOST
Over the course of the twentieth century, scientists in different fields struggled to get around
these limitations, in order to allow representations of the data to adapt to the nature of the
infomullion. In essence, they wanted 10 capture both Ihe low-resolution forest- the repealing
background sigr"lal- aoo the high-resolution trees-the individual, localized variations in the
background. Although the scielllislS wcre trying to solve the problems panicular to their
respective fields. they began to arrh'e at the same conclusion- namely. Ihat Fourier InlIIsforms
themselves were 10 blame. They also arrived at essentially the same solution. Perhaps by
spliuing a signal into components that were not pure sine waves. it would be possible to
condense the information in bolh the time and frequerocy domains. This is the idea that would
ultimotely be known as wa"l~lcts.
Wavelet transforms allow timc·frcQuency localisation
The first entrant in the wavelet derby was 0 Hungarian mathematician nanw:d Alfred Haar,
who introduced in 1909 the furoctions that are now called Haar ",..ave1ets. These functions
consist si mply of a short positive pulse followed hy a short negative pulse. Although the short
pulses of Haar wavelets are e;t;cellent for teaching wavelet theory, they are less uscful for moSt
applkations because they yield jagged lincs instead of smooth curves. For example. an image
reconstrtlcted with Haar wavelcts looks like a cheap calculator display and a Haar wavelet
reconslrtiction of the sound of a i1me is too homh.
From time to time over the ne~t several decades, other precursors of wavelet theory arose.
In the 1930s, the English mathematicians 10hn Linlewood and R.E.A.C. Paley developed a
method of grouping frequencies by octaves thereby creating a signal that is well localized in
frequency (il$ spectrum lies within one octave) and also relatively well localized in time. In
1946, Dennis Gabor, a British-Hungarian physicist. introduced the Gabor transform. analogous
to the Fourier transform. which separatcs a w""e into "time-frequency packets" or "coherent
states" (see Figure 1.6) that have the gre.1test possible simultaneous localization in both time and
frequency.
And in the 1970s and 19805. the signal processing and image processing communities
introduced their ()wn versions of wavelet analysis. going by such names as "subband coding:'
"quadrature mirror filters" and the "pyramidal algorithm".
While not precisely identical, all of these techniques had similar features. They
decomposed or transformed signals into pieces that could be localized to any time interval and
could also be dilated or contracted to analyze the signal at different scales of rcsolulion. These
prC<:urso(S of wavelets had one other thing in common: no one knew about them beyond
individual specialized communities. But in 1984, wavelet theory finally came into its own.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
21. 6 • Insighl into Wa~lelS-From Theory 10 Pnctice
••
•
•
•
•
•
•
•
•
••
•- -~I't------- --,,------
FIGURE 1.6 Oocomposing lignal inlo t;"",·f~l>e1lCy alom~. BOllom of the pictun: ~ twOtime fn:qumcy
atom•. The signal and the time·f""lllCnc:y map i. """"'n "'->ve that.
1.4 HISlURY OF WAVELET FROM MORLET TO DAUBECHIES
VIA MALLAT
lean Morle[ didn't plan to start a scientific revolution. He was merely trying [0 give geologists
a beller way 10 search for oil.
Petroleum geologists usually locale underground oil deposits by making lood noises.
Because sound waves lnIvel through different materials at differenl speeill;. geologisl.'l can infer
whal kind of malerial lies under the surface by sending seismic waves inlO the ground and
measuring how quickly they rehourKl. If the waves propagate especially quickly through onc:
layer. it may be it salt dome. which can trap a layer of oil underneath.
Figuring oot just how the geology translates into a sound wave (or vice versa) is a tricky
mathematical problem. and one Ihat engineers traditionally solve with Fourier analysis.
Unfortunately. seismic signals contain lots of transienl.'l-llbrupt changes in the wave as it passes
from one rock layer to another. Foorier analysis spreads thal spatial information oot all over the
place.
Morlet. an engineer for Elf-Aquitaine. developed his own way of analyzing the seismic
signals to creale componenl.'l that were localized in space. which he called Wllvdets of constant
shape. Later. they WQUld be known as Monet wllvelets. Whether the components are dilated.
compressed or shifted in time. they maintain the same shape. Other families of wavelets can be
huilt by Illking a different shape. called a motm-r WIIvelel. and dilating. compressing or shifting
it in time. Researchers woold find that the exact shape of the mother wavelet strongly affeclS
the accuracy and compression properties of the approximation. Many of the differences between
earlier versions of wavelets simply amounted to different choices for the mother wavelet.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
22. The Age of Wavelets • 7
Morlel's method wasn't in the books but it seemed to work. On his personal computer. he
could separate a wave into its wavelet components and then reassemble them into the original
wave. But he wasn't satisfied with this empirical proof and began asking other scientists if the
method was mathematically sound.
Morlct found the answer he wanted from Alex Grossmann. a physicist at the Centre de
Pltysique ThOOrique in MlIIOeilles. Grossmann worked with Morlet for a year to confirm that
waves could be re.constructed from their wavelet decompositions. In fact. wavelet transforms
turned out to work beller than Fourier transforms because they are much less sensitive to small
errorn in the computation. An error or an unwise truncation of the Fourier coefficients can turn
a smOOlh signal into a jumpy onc or vice versa: wavelets avoid such disastrous consequences.
Morlet and Grossmann's paper. the first to use the word "wavelet", was published in 1984.
Yvc:s Meyer. currently at the Ecole Normale Suptrieure de Cachan. widely acknowledged as
one of the founders of wavelet theory. heard about lheir work in the fall o f the same year. He
was the first to realize the connection between Morlet's wavelets and earlier mathematical
wavelets. such as those in the work of Littlewood and Palcy. (Indeed. Mt!yer has counted 16
separate rediscoveries of the wavelet concept before Morlet and Grossmann'S paper.)
Meyer went on 10 discover a new kind of wavelet. with a mathematical property called
orthogonality that made the wavelet lTansform as easy to work with and manipulate as a Fourier
transform. C·Qrthogonality" means that the information captured by one wavelet is completely
independent of the information captured by aoother.) Perhaps most importantly. he became the
IlCXUS of the emerging wavelet community.
In 1986. St6phane Mallat (see Figure 1.7). a former student of Meyer's who was working
on a doctorate in computer vision. linked the theory of wavelets 10 the existing literature on
nGURE 1.7 Sephane Malla! (CMAP. Ecoic PoIyIcchniq.... 911211 Palais.eau Cedeo.. Fr'anc:e).
sUbband coding and quadrature mirror filters which are the image processing community's
versions of wavelets. The idea of multiresolution ana1ysis--that is. looking at signals at different
scales of resolution-was already familiar to experts in image processing. MaUat, collaborating
with Meyer. showed that wavelets are implicit in the process of multiresolution analysis.
Thanks to Mallat's work. wavelets became much easier. One could now do a wavelet
analysis without knowing the formula for a mother wavelet. The process was reduced to simple
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
23. 8 • Insight into Wawlets-From 'Theory to Practice
operations of avel'llging groups of pixels IOgether and taking their differences, over and over.
'The language of wavelets also became more comfortable to e l~trical engineers, who embraced
familiar teons such as "'filters"', "'high frequencies" and "Iow frequencies".
The final great salvo in the wavelet revolution was fired in 1987, when Ingrid Daubechies
(see Figure 1.8), while visiting the Courant I nstitUl~ at New York University and later during
tlGURE 1.8 Ingrid Daub«hits (Prof...,.,.-, Ikpartment 01 M3lhema.tiC$. Princo:ton University).
her appointment al AT&T Bell Laboratories, discovered a woole IICW class of wavelets [sec
Figure l.9(b)] which were not only otthogonal (like Meyer's) but which could be implemented
using simple digital filtering ideas, in fact, using shon digital filters. The new wavelets were
almost as simple to program and use as Haar wavelets but they were smooth, without the jumps
of Haar wavelets. Signal processors flOW had a dream tool: a way to breair;: up digital data into
contributions of various scales. Combining Daubechies and Mal1at's ideas. there was a simple.
orthogonal transform that could be rapidly computed on modem digital computers.
,, /~,) ,
I o.,
,n 0
"-, -0.,
1
,
o , 3
(a) Haar wavelet (b) Daulxdlic. · 4-...p Muna( WlI>"ek(
FIGURE 1.9 (Cont.).
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
24. 11< Ag' 0 rw "''"
, • 9
0'
0.'
0.2
0
/-0.2
- 0.4
- 0.6
-5-4 - 3 - 2 - 10 , , , .(c) GaIIsi... ".""lel
n GUR£ 1.9 GrapJu of .....·era! different types of wa."lea
The Daubechics wa'eleLS have surprising features-such as intimate connections with the
theory of fractals. If their graph is viewed under magnification, characteristic jagged wiggles
can be seen, no matter bow strong Ihe magnification is. This exquisite !;omplexity of detail
means, there is no simple formula for these wavelets. They arc ungainly and asymmeuic;
nineletnth-eemury mathematicians would have recoiled from them in hOllOr. But like the
Model-T Ford. they are beautiful because they work. The Daubechies wavelets turn the theory
into a practical tool that can be easily programmed and used by any scientist with a minimum
of mathematical training.
1.4.1 Different Communities of Wavelets
There are several instances of functions (see Figures 1.9 and 1.10) that can be used for
muhiresolution analysis of data. All of them are referred to as wavelets. Some of these instances
=,
• Dyadic lrans/arts and di/mes of Olle funC/ion: These are classical wavelets.
• Wavelet fHlclcets: This is an extension of the dassi,al wavelets whi,h yields basis
fullO;lions with bener frequency localizalion at the ,ost of slightly more expensive
tnmsfonn.
• Local trigonometric basu: The main idea is to work with cosines and sines defined on
finite intervals combined with a simple but very powerful way 10 smoothly join Ihe
basis functions at the end points.
• Mulliwavtlets: Instead of using one fixed fun"ion 10 lranslate and dilate for making
basis fun,lions. we use a finite number of wavelet functions.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
25. 10 • Insighl inlo Waveku From Theory 10 Prac:ti~
,.•
"
,1----"1 I1
-<l.' L_~__-'-'-'---_~__---'
-0.8 -0.4 0 0.4 0.8
(a) Model _velet
,.•
,.,
,
" I
-<l.,
-. , , •(b) Mexican Hal .....""lel
FIGURE 1.10 Wavelets uoed in conlinuoul wavelet tranJ(O,TU.
• Second gene,flIion wlll.'elefS; Here: one entirely abandons the idea of translation and
dilation, This gives elClra flelCibility which can be used 10 construct wavelets adapted to
im:gular samples.
1.4.2 Different Families of Wavelets within Wavelet
Communities
Like humans, wavelets also live in families. Each member of a family has cenain common
features that distinguish each member of a family. Some wavelets aJ"e fQr CQntinuQUs wavelet
transform and OIhe1; are: for discre:te wavelet transform. Some of the families that belong to
'classical' community of wavelets are (see Figures 1.10 and 1.11):
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
26. g. • •The A of W vcLets 11
2
,
0
I
-,
o 2 3
• ,
nGURE 1.11 Coinel wavelet.
• Wavelets for continuous wavelet transfoml (Gaussian. Morlel. Mexican Hat)
• Daubechies Maxllat wavelets
• Symleu
• Coillcts
• Biorthogonal splille wavelets
• Complex wavelets
1.4.3 Interesting Recent Developments
• Wavelet based denoising has opened up other fields and important tcchniques such as
dictionaries and non-linear approximation. smoothing and reduction to small
optimizalion problems are real achievements.
• Wavelets have had a big psychological impact. People from many different Ilreas
be!;ame interested in time-frequency and time-scale transfonns. There has beCII a
revolution in signal processing. There is less specialization and the subjcct is now
opened to new problems. More than just a simple tool, wavelet ideas prompt new points
of view. Some of the best ideas aren't writtcn down. The big diffcrence will come from
new gencnuion rescarchers flOw growing up amidlll wavelet ideas.
• Wavelets have advan~d our urKIelltanding of singularities. The singularity spectrum
completely characterizes the complcxity of the data. Now We must go to an
undelltanding of the underlying phenomenon to get an equation from the solution.
Wavelets don't give all the ansWCIl but lhey force us to ask right questions.
• Wavelets can be used to distinguish coherent VefSUS incoherent pans of turbulence in
fluid flow. They give some information but don't entirely solve the problem.
• The results on regularity, approximation power and wavelet design techniques have led
10 significant developments in signal and image processing.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
27. 12 • Insigh.t into Wavelets-From Theory to Practice
1.5 WAVELETS IN THE FUTURE
Wavelet ar.alysis is unquestionably one of the beSt achieveme:nts of matllematics in the twentieth
century. Its initial applications were mainly in sparse signal representation and denoising of
signal and images. Probably, it was due to the fact that the theory was hard to understand at
that time. In recent years wavelets have spread in many fundame:ntal sciences other than
mathematics. such as me:dicine, biology, geophysics. physics, mechanics. economics, etc. This
is e,·ident from the title of the books that are appearing.
At present there are more Ihan hundred books on wavelets most of which are on
mathematical aspects of wavelet theory. Slowly but steadily many books are appearing in
specific application domains. Following are some: of them.
Wavelets in medicine and biology (by Akram Aldroubi and Michael Unser,
CRC Press, 1996)
This book explores application of wavelets in me:dical imaging and tomography biomedical
signal processing. wavelet based modeling of problems in biology.
Wavelets in physics (edited by J.C. van den Berg, 1999)
This book details the application of wavelets in many branches of pnysics such as acoustics,
spectroscopy. geophysics. astrophysics, fluid mechanics (turbulence), medical imagery, alomic
physics (laser-atom interaction), solid state physics (structure calculation).
Wavelets in chemistry (editor Beata Walczak, Elsevier, 2000)
This book is an introduction to wavelets intended for chemiSts. It covers all impor1ant aspects
of wavelet theol)' and presents wavelet applications in chemistry and in medical engineering.
It is addressed 0 analytical crn:mists. dealing wilh any type of spectral data, organic chemists,
in"Olved in combinatorial chemistry, chemists involved in chemome:trics and engineers involved
in process control.
ChcmomClrics: From basics to wavelet transfonn (by, Foo-Tim Chau, Vi-
Zeng Liang, Junbin Cao, Xue-Cuang Shao, WHey, 2004)
This book explores the use of wavelets in ·chemome:trics based signal processing'.
Wavelets for sensing technologies (by K. Chall, Cheng Peng. Arlech House
Publishers, 2003)
This reference book focuses on the processing of signals from Synthetic Aperture Radar (SAR).
Specific remOle sensing applications presented in the book include noise and clutter reduction
in SAR images. SAR image compression. texture and boundary enhancement in SAR images,
direction·al noise removal and general image processing.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
28. 'TlIe Age of Wavelets • 13
Wavelet and wave analysis as applied to materials with micro or nano-
structure (by C. Cattani and J. Rushchitsky, World Scientific Publishing Co..
200?)
This book explores physical wavelets as solution of certain POEs anslng in solitary wave
propagation in elastic dispersive media. Three different types of physical wavelets. namely.
Kaiser physical (optical and acoustic) wavelets. Newland harmonic wavelets and elastic
wavelets are discussed. Optical wavelets own this name be<:ause they satisfy Ihe linear wave
equations of optics in the simplified form of Maxwcll electromagnetic equations. The acoustic
wavelets were proposed as those wavelets satisfying tlK: linear wave equations in acoustics.
Harmonic wavelets were suggested by Newland. The Newland hannonic wavelets can be
referred to physical family of wavelets for many reasons, btu mainly be<:ause they are especially
proposed for the analysis of physical problcms on oscillations.
Ultra-low biomedical signal processing: An analog wavelet filler approach
(or pacemakers (by Haddad and Serdijn. Springer, 2008)
In ultl1il low-power applications such as biomedical implantable devices. u IS not suilllble to
implement the wavelet transfonn by means of digital circuitry due to Ihe relatively high power
consumption associated with the required NO converter. Low-power analog realization of the
wal"elet transfoml enables its application in vivo, e.g.. in pacemakers. where the wavelet
transform provides a means to extremely reliable cardiac signal detection. The methodology
presented in this book focuses on the development of ultra low-power analog integrated circuits
that implement the required signal prucessing. taking into account the limitations imposed by
an implantable device.
Wavelets in wireless communication
Though there is no exclusive book on applicalion of wavelets in communication. there are
hundreds of research papers available on the topic. Wavelets give a new dimension to the
capacity of wireless communication. It provides "Waveform diversity'". to the physical
diversities cum:ntly exploited. namely. space (muhi-antenna wireless communication system),
frequency and time-diversity. Signa.! diversity which is similar to spread spectrum systems.
could be: exploited in a cellular communication system, where adjacent cells can be designated
different wavelets in onJer to minimize inter-cell interference. In addition wavelets provide the
following:
(a) Sc",i-u,birrury divisi<m of ,I.e sigll"l spuc" Ulld ",,,/,i"'lt: S1sr""'11: Wavetet
transform can create subcarriel'S of different bandwidth and symbol length. Since
each subcarrier has the same time-frequency plane area. an increase (or decrease) of
bandwidth is bound to a decrease (or increase) of subcamer symbol length. Such
characteristics of the wavelets can be: exploited to Create a muhirate system. From
11 communication perspecti·e. such a feature is favourable for systems that must
support multiple data streams with different transport delay requirements.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
29. 14 • Insight into Wa~le!s-From Theory to Practice
(b) Flexibility ...irh ,ime-frequency ,i/ing: Another advantage of wavelets lies in their
ability to arrange the time-frequency tiling in a manller that minimizes the channel
disturbances. By nexibly aligning the time.frequency tiling. the effect of noise and
interference on the signal can be minimiZo!d. Wavelet based systems are capable of
ovt'rcoming kTlOwn channel disturbances at the transmitter. rather than waiting to
deal with them at the receiver. Thus. they can enhance the quality of service (QoS)
of wireless systems.
(c) Semi,i"iry ro chmmei ~ffecl5; The performance of a modulation scheme depends
On the set of waveforms that·the carriers use. 11le wavelet scheme. therefore. holds
the promise of reducing the sensitivity of the system 10 harmful channel effects like
inter-symbol interference (ISO and inter-carrier interference (ICI).
(d) Vltra widdxmd llpplicmiolls: Impulse radio. or ultra wideband (UWB) radio. is a
promising new technology for wireless communicruions. Rather than modulating the
information on a carrier. the data is transmiued using a coded series of very narrow
pulses. carrying information in the time and the frequency domain. Wavelet bases are
good candidates for these pulses.
Figure 1.12 shows communication engineering areas where wavelets are fiBding new
applications.
ChanllCI clumlct.erizluion
I. ChanllCl modeting
2. Electrorroagnctic compuuuions and "Jlltnna desis.n
3. SpeW estimation
COlnitive rmio
Inl~11i~nt co;>mmunkation
syst~ms
Uhra widcband communication
l. Impulse: radio
2. Multil>and OFDM
Muhiple access communical;on
t. COMA
2. SCDMA
3. MC-COMA
[nlt"l'f=e
1. Signal denoi.inl
2. O'lta amval estimation
3. Intcrf"= mitigation
4. ISI. ICI mitil'llion
Modulation aod rnuhipte.in,
l. Wa,~ 5ILapin,
2. Siolle carrier modulalion
3. MUhi-carrier modulation
4. FllIC1al modulation
~ . Multiplex;n,
Networking
1. Power con..........tion
2. Traffic proiction
). Net"'-ort< tnlffit modeling
4. Data rccon<IJUCtioo
5. Disuibull'(] data processinl
FIGURE 1.12 WaV(:iets in wireles~ communication.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
30. TIw: Age of Wavelet. • IS
In summary, we may safely bet that wavelets are here to stay and they have a bright future.
Of course wavelet do nOI solve every difficulty, and must be continually developed and
enriched. We can expecl proliferation of specialized wavelets each dedicated to a panicular type
of problem and an increasingly divcrse spectrum of applications.
1.6 WHAT HISTORY OF WAVELET TEACHES US
When asked 10 justify the value of mathematics, mathematicians often point out that ideas
developed 10 solve a pure mathematical problem can lead to unexpected applications years later.
BUI the story of wavelets paints a more complicated and somewhat more interesting picture. In
(his case, specific applicd research led to a new theoretical synthesis, which in lurn opened
sciemists' eyes to new applications. Perhaps the broader lesson of wavelets is that we should
not view basic and applied sciences as scparate endeavours. Good science requires us to see
both the theoretical forest and the practical trees.
SUMMARY
Though wavelet is an organized research IOpic. it is only two decades old, and has been in use
for a long time in various disciplines under different names. Morlet and Grossman were the first
10 use the word 'wavelet'. Stephan Mallat brought out the relation between wavelet
methodology used by Morlel and filter bank theOI)' used in image processing applications. TIle
greatest contribution came from Ingrid Daubechies who put tile whole theory on a strong
matherruuical foundation. Wavelets are now emerging as one of the fastest growing field with
application ranging from seismology to astrophysics.
Suggested Further Readings
Amara's wavelet page htlp:llwww.amara.com/currmll....ol·e/u/uml.
An Introduction 10 wavelets: hllp:llwww.amara.comll£££K"(JI.tII£££....al.eiel.hlm/f
Dilubel:lies, I.. Where do wavelet come from? A perwnal point of view, Proceedings of liltl
1£££ Specia/Issue on W'lI'e/eIS, S4 (4), pp. SIO-SJ3. April 1996.
Sweldons. W.. Wavelets what neltt? Procudill8s of 'he IEEE. Vol. 84. no. 4 April 1996.
Wa'elets; Seeing the Forests and the Trees, hltp:llw"'l'o'.M)"om/discol'ery.orgl.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
31. Fourier Series and Geometry
INTRODUCTION
Deemed one of the crowning achievements of the 20th Century, the Fourier series has
applications Ilia! are far n:aching in varioLls fields of science and mathematics. The Discrete
Fou';". "ansfurm i. one p"I1icular tool widely u&ed in today'~ age of compulers and solid state
electronics. From graphic equalizers in stereos 10 the moSI advanced scientific sampling
software, the userullless of this mathematical feal is ISlolJllding. Most of the readers of this book
might already know this fact but many of them may nor be knowing tha! Fourier series. as a
mathematical tooJ in analy:ting signals, h:J.S strong connection wilh geometry. In this chapter our
aim is to understand the theory of Fourier series from a geometrical viewpoillL
We assume that you are familial" with vector algebra and co-ordinate geometry. If you are
really comfonablc in using Ihc!iC topics, you can very well understand what Fourier series is,
computation and interprelalion of Fourier series coefficients, Fourier transform. discrete Fourier
transform. fast Fourier transform. etc. l1iere exists a strong analogy between what you do in
vector algebra and what you do in signal processing. This analogy will help you to visualize
and give interpretation to the processes and output of the processes that you do on signals.
Development of tllis geometrical mental picture about signal processing is the main aim of this
chapter. Einstein once said "Imagination is more imponant than knowledge". Signal processing,
which many students consider as dry and non·intuitive. demands imagination and some form
of abstract thinking from the pan of students. In fact. it requires only that. Once )'OIl develop
a conceptual link between geometry (vector algebra) and signal processing, you need nO(
remember any fonnula and every formula that you come across will become transparent to your
mind. Here we will refresh concepts from "Ulor spoce. The detailed exposition of vector space
is given in Appendix A. In this imroductory chapter, we won't try 10 be 100% mathematically
precise in our staternems regarding vector space. Instead our aim is to relate geometry and
Fourier series in an intuitive way.
..
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
32. Fourier Serie$ and Geometry • 17
2.1 VECTOR SPACE
Any vector in 3-dimensional ,;pace can be represented as V = (JI + bl + et. I, J.k are unit
vectors in three onhogonal directions. This orthogonality is expressed by requiring the condition
that r.J=k· J= r.k = O. The orthogonalily condition ensures that lIle representation of any
- - -''eClor using i. j. le is unique.
2.1.1 Bases
We call i.].k as bases of space 9{1. By this we mean that any vector in 9{l can be represented
using I. J. k vectors. We also say i. J. k span the space 9{1. Let V= al + b] + et be a vector
in 9{1. Continuously changing scalar a. b. c: we will go on get new vectors in 9{1. We imagine
that. set of all such ,'eClors constitute the vector space 9{1. We express this truth by
Span (al + b] + ct) iI 9{1
.,'
2.1.2 Ortnonormalily
Norm of a vector V =al + b] + ef is conventionally defined as Ja2
+ b2
+c2
and denoted
as IV I. IV I is equal to Jv. ii . We interpret this quantity as the magnitude of the vector. It
must be noted that mher definition,; are also possible for norm. What is interest to us is that our
basis vectors of 911. Le.. T. J. k are having unity nonn. Hence the name unit vectors. They are
also onhogonal. Therefore. they are called orthonorma1 vectors or orthonorma1 bast'$. How
does it help us'! Given a vector V= al + bJ + cf. we can easily (ell to which direetion I,], k.
the vector V. is more inclined by noting the value of a. b. c.
2.1.3 Projection
Given a vector V how shall we find its normal component vectors or in other words how shall
- - - -we find the scalar coefficients a. b and c'! Wc project V on to bases i , j, le to get a, b and
e. This is the direct result of orthogonality of our basi,; vectors. Since V = (JI + bJ + ef,
Y' I =«JI + bJ + ef)· I =(J
ii· J = «JI+bJ+ek)·J=b >od
ii· f =(al+b} +cf)·k=c
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
33. 18 • Insight inlo Wa~cLcts-From llIeory to Pmctice
So projecting vcctor 0 on to a base of space 9J (spanned by a sct onoonormal vectors) gives
the corresponding coefficient (component) of the base. There is a parullcl for this in Fourier
scries analysis.
With these ideas firmly cngral'cd in your mind. you arc ready to understand and visualize
Fourier series.
2.2 FUNCTIONS AND FUNCTION SPACES
In vcctor space. we represent a vector in terms of onoogonal unit vectors called bases. Fourier
serics is an extension of this idea for functions (cleclrooics engineers call them as signals). A
function is expressed in terms of a set of orthogonal funclions. To com:13le with ideas in
geometry. we now need to introduce the concept of onhogonality and norm with respect to
functions. Note Ihat. funclion is quantity which varies with respect 10 one or more running
parameter. usually lime and space. Orthogonality of functions depends on multiplication and
integration of functions. Multiplication and integration of function must be toought of as
equivalent to projection of a vector on to another. Here comes the requirement for abstract
thinking. mentioned at the stan of tile chapter.
2.2.1 Orthogonal Functions
Two real functions fl(/) and f2(/) are said to be onhogonal if and only if
-JJ;COiJ(I)dl =0 (2.1)
-
Ponder over this equation. What we are doing? We are first doing a point by point
muliiplicll1ion of t...."Q function. TIlen we are summing up the area under the resulting funclioo
(obtained after multiplication). If this area is zero. we say. the two functions are onhogonaJ. We
also interpret it as similarity of functionsfl(t) andfil}. Ifft(f) and fit) vary synchronously, that
is, ifft(l) goes upil(t) also goes up and ifft(t) goes downil(t) also goes down then we say the
two functioos have high simi larity. Otherwise. tlley are dissimilar. Magnitude or absolute value
-of Jf t(l)f2(Odl will be high if they "ary synchrOllOusly and zero if they vary perfectly
asynChronously.
For example consider two functions ft(t) = sin t and 12(1) :: cos I for f in the interval
(0,211"). Figure 2.1 shows the two functions and their point wise product. Observe that the net
area under the cur'C sin I cos f is zero. That is. the llfCa above the I axis is same as tile llfCa
below the I axis. For these two functions, it ean be easily shown that. for I in the range of
multiples of 211". area under the product curve is always zero. TIlerefore. we say that sin I and
cos I are onhogonal in the range (0. 211). Specification of range is very important when yOU say
orthogonality of functions. Because the deciding factor is area under the curve which in turn
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
34. o.
o.
o.
,
8
6
•,
0
~ o.
•,< -0.
..-0.
-0.
,
•
6
8
,
,,,
,,,
I
o
,,,,,
/"
'J,
/, ,
V ,,,
/ '.
, •
/ (
,,,
••,,
I, I
i
V,,,,,,
6
Fourier Series and Geometry • 19
f
,,,,,,
//
I
" iI' I
I . I
j , I
, .
_____ sinl
__._ .in 1 • cos 1
---- ,~ ,
, ,, ,,, ,
, ,, ,
8
" 12 J4
n GURF. 2.1 Qnh<>gona] fUI>(:'i""•.
depends on the range of inlegralion. Now here observe Ihe varialion of sin t and oos /. Whcn
sin I is increasing, cos I is de<:reasing and vice 'crsa. For range of I in multiples of 2K, the two
functions considered are said to be ortllogonal.
The goometrkal analog of f It (t) 11 (/) d/ '" 0 is the dot product, ; .J'" O. For the given
,
interval. when!,(t) is projected on to!ft) (proje<:tion here mearu multiplying and integrating).
if the result is tero • we say Ihe IWO functions are orthogonal in the given interval. Clearly look
al Ihe analogy. Taking B do! product in ve<:tor space (multiplying corresponding terms and
adding) is equivalenl to multiplying and inlegrating in Ihe funclion space (point wi~
multiplication and integration). The dOl product is mro;;mum when t'NO vectors are oollinear and
zero when they arc at 90". This analogy will help )'ou to casily visualize and inlerprel various
results in signal processing.
2.2.2 Orthonormal FUllctions
Two real fUI>C,j<>"s/,O) a",I/1(1) a", said ,0 be onhonormal if and only if
-and f[;(t) [;(t) dl '" I. for i:::: 1. 2.
-
2,2.3 Function Spaces
Just like unit orthogonaJ set of vectors span vector spaces. orthogonal set of fUilCtions can span
function spaces. Consider the following functions defined over a period T:
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
35. 20 • Insight into Wavelets-From Tlleofy to Practice
Let us assume that w'" 2l1fT. Then (2.2)
(i) u(t)= I forO S t S T
(ii) cos " all. n '" I. 2. ...: 0 S 1 S T
(iii) sin 1I!l1l. n = 1. 2. ...: 0 S t S T
Here we have infinite number of (unctions of sines and cosines. These sets of functions
Iln: mUlUally onhogonal in the interval (0. 1). Let us verify the truth through some representative
examples.
Let us take functions u(t) = I and sin WI for 0 S t S T. These functions and their product
function are shown in Figure 2.2. Point wise multiplicar.ion gives the Il'sulting curve as sin (l)/.
I> j
o·i~ T
•
I.,
,
0.'
V ~0
/ T-0.'
"--,
-I>
" ,
03
1/
"'"0 T
-03
"- /'-,
- 1.5
FIGURF. 2.2 OnI>ogonalily of functiofu.
What is the area under the curve sin 0)1 in the interval (0. 7)'! Obviously i!.ero. Mathematically.
T T
Ju(t) sinllJl dl = fl ·sin(2mfT)dl = 0
,.,
We say U(I) = I is orthogonal to sin WI in (0. T ). Do not forget the Il'lar.ion between wand T.
Simi larly we can show that u(t) '" 1 is onhogonal to cos rut in (0.7).
In general.
T
Ju(t) s in nox dl '" 0 al50
,.,
T
fIl()) cos nox dt = 0 for all n '" I. 2•....
,.,
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
36. Fourier Series and Ge<>mctry • 21
Consider cos Wf and sin Wf in (0. n. Figure 2.3 shows the two functions and
COITespondillg fuoction obtained after point wise multiplication. Area abo'e and below of the
2
, / -
"-
~ .,
0 T
"- /'-,
-2
•
I'
,
0.'
/ "".nQW
0 T
-0.'
"" /-,
- u
0.6
(0.<
om "" 00$ fUr
0.2
0 T
...(1.2
V V-0.'
-0.6
)'IGURE 2.3 !'Iou of sin WI. 00$ WI and sin (Qf cos QW.
resulting curve is same implyillg that the net area is zero. This ill turn imply that cos rut and
sin rut in (0. n are orthogollal.
Generalizing.
Similarly.
,
fsinllM cos mWfdt = 0 . for all m. n = 1,2.3.....
,..
,
Jsill nM sill mM dl = 0
,..(for all illteger m and n such that m ~ /1).
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
37. 22 • I"sight inl(l Wavd"i5--From 'Theory tu Pra.c:licc
Alw
,
JcosnaN cosmaN dl =0
,..(for all integer m and n such that m ~ n).
Thus, we have a set of functions defined over (0, n such thal they arc mutually
orthogonal.
2,2.4 Orthogonal Basis Functions
In vector algebra, we know that the unit orthogonal vectors t, j, k span the space :Xl. This
imply Ihat any veclor in :x, can be represented by unit vectors 01" bases i. j, k. Similarly the
sel of functions,
(i) u(l)= I forOSI ST
(ii) cosnnn. n = I. 2. ...: 0 S 1ST
(iii) sin nW/. n = 1. 2....; 0 S 1ST
defined over 0 S t S T span a function space and we denote il as L1 (set of square integrable
funclions). Most of the practical signals or functions are assumed 10 belong in the space of Ll.
This means Ihal. practically any continuous signal in the interval (0. n can be represented using
Ihe above base•. ~tplhematicany.
f(t) E L1 = f(t) = 00 + 11'1 COSM +"1 COS2M + ...
+ ~ sinaN + ~ sin2aN + ... (2.3)
Lel us look at the concept from another angle. Consider Ih" sumj(t) for each / in the range
(0. T) given in Eq. (2.3). Choose a set of coefficients Do. al..... and bl. bz, .... and make the
function (or signal) f(l> in the range (0. T). Now go on choosing different set of coeffici"nts 10
gel a different signal. The set of all such possible signals generated is our signal space Ll in
the range (0. 1). Practically all smoothly varying fUllCIions that you can imagine in Ihe interval
(0.1) belongs 10 Ll. Or in other words. practically all smoothly varying functions thal you can
imagine in Ihe illlerYlll (0. T) can be represented by Ihe equation:
f(t) = 00 + 11'1 cos ea + al cos 2ea + ... + bt sin aN + bz sin 2M + ...
The number of coefficients required 10 represent a signal depends Qn bow smooth and how
fast your signal is varying in the range (0. T).
2.2.5 Orthonormality and the Method of Finding the
Coefficients
The problem that we are addressing is finding the coefficient set ao. 11'1. bl. 11'1, b-:, etc. In veclor
algebra. onhooonnalily of basis vectors allowed us 10 find the component of a vector along a
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
38. Fourier Series and Geometry • 23
panicular base by projecting the vector on to that base. The propeny that lil =Ijl =1.&:1 " I
where Ii 1=.Jt·7 and 7.]. k are onhogonal implies thal for a veclor If e 9). if
If· i '" a. If· ] = b, V· k =c Ihen V '" ai + b] + ek . [I is a unique representation using Ihe
bases i.]. k. To reilerale the principle, 10 find a panitular coefficienl, project the vector on 10
the torresponding basis. 1be same mental pittuTe is applicable to Founer series representation
of signals.
In the last section. we have foufld out a set of functions that are onhogonal among
themselves in a given range (0. n. These functions are onhogonal but we haven't thccked
whether they form an orthooormal set. In function spa<;e, onhonormalily of two functions 11(1)
and h(l) in (0. T) requires that
, , ,
f/i(/) 12(1) dl = o. fft(t) 11 (t) dl '" 1
, ,
and ff 2(t) f 2(t} dt '" I
,
Consider Ihe basis funttion sin nw.
f' . . d f'· ['~'J .['~'Jd TsmnM smlltIX 1 = SIn T sm Tt'" 2"
, ,
The result is a function of T and is IlOI equal to I. For any integer n > 0, the integral is TI2.
So. we say our sine bases are not normalized.
Similarly it can be easily SMwn that,
jtosnaw COS lItIX dl =jcose;nt)cos(2-;'1) dl = ~
, ,
for 11 = 1.2.....
So our cosine bases are also oot nonnalized bases. What aboul our finlt basis function (which
is also called OC base) u(t) = I. in (0. T)'I
A,.
,
f[2 dt = T. U(I) is also not normalized.
,
Therefore, none of our bases are normalized. So how shall we normaLize our bases? Multiply
all tile sine and cosine bases with J2fT and u(t) '" I with M.
N~.
f' = = . ' f' .['''''J.[''''J 'T...,2/TsinnM...,2/TsmnMdt= T sm T sm T dl "'T ' "2'" I
, ,
, ,
f= = 2f [2"'''J [2"'''J 2T...,2fT cosllM...,2fT cosnM dl = T cos -T- cos -r dl = T . 2" = 1
, ,
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
39. 24 • Insight into WavdelS-From 1brory to Practice
, ,
JJIIT../Ifrdt:o ~ fl dt:o ~ ' T=I
, ,
The set of nonnaliz.ed bases of Fourier series are:
{~, #cos M, #sin M, #cos 2ar, Hsin 2C1X • .•. , gcos IIflJ1. #sin IlflJl ••. .•}
All these functions are defined in the interval (0. 1) and ro = 21f1T.
Let us now take a functionf(t) ELl defined in the interval (0.1) and express il using the
nonnalized sine .and cosine bases defined over the same period (0, T). NOIe that the length of
bases are same as the length of the signal. Also note that the furKl.amcntal frequency of the
sinusoidal (also 'oosinusoidal") series is given by flJ:o 21f1T. (This is something which students
tend to ignore or forget and hence the repetition of the fact in many places.)
f
,I , f2 " f2 .(1) :0 Or! 7r + at V"Tcos ar + "I V"TStn fa + ..•
'H 'H'+ a - cos IlCIX + b - Sill liar + ...
W T · T
(2.4)
Here 00. al' ai, .... bj. bi.... are called normalized Fourier series coefficients.
Now. given th3!. such a n:presentation is possible. how shall we find out the nonnaliU!d
coefficients. The answer is, projeaj(l) on to the corresponding bases. Suppose we want to find
a;. Then project fit) on to J21T cos IlltW. that is,
a~ = jf(t)H cos mu dt
,
Note the analogy. In vector algebra, to find a component coefficient, we project the vector on
to the corresponding unil vector (base). Let us verify the troth through mathematical
microscope. Multiplying both sides of Eq. (2.4) by J21T cos Ilwt and integrating over (0. T).
we obtain
,
Jf(t )·hlT COSIIM' dt
,
,
:0 aO ~. JJUT cos mm dt
,T ,
,
+ a; JJ2fT cos wtJ2IT cos 1l{J}l df
,
,
+ hi fJ21T sin OX J21T cos lIax dt + ...
,
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
40. Fourier Series and Geometry • 2S
,
+ a~J../2/T COS nM ../2/T COS nM dr
•,
+ b~ J../21T sin nM ../21T sin nM dr + ...
•= /I'
•, ,
Thus. J10) ../2fT cos nM dr = a; since J../2fT cos nM ../2fT cos nM dr = I and all OIher
• •integrals vanish because of orthogonality property of tile bases.
,
Similarly b~ = Jl(r) ../21T sin nM dr. That is. project 1(1) on to the corresponding
•normalized base.
What about <lQ1 ProjcctJtt) on to the base w(r) = .JlfT defined over (0. 7). giving
,
~ = Jf(l)M dr
•
Well. we ullderstood how to get normalized coefficients in Fourier representation of signals. To
get a particular coefficient simply project 10) on to the com:sponding normalized base.
Projection in function space meaRS poiRt wise multiplication and integration of tile functions
concerned over the defined period.
After obtaining the IIOnnalized coefficients. we substitute in Eq. (2.4) 10 get Founer series
representation of the signal. Let us simplify Eq. (2.4) to avoid scalars ..J2IT and .JIlT.
Let <lQ IfJT =I/o so that
, ,
<lo = .JlfT<lQ =.JlfT J./liTI{t) dl =~ Jl(t) dl
• •
(Note that we substituted the Cltpression fo r <lQ to get /10-)
Similarly let o~ .J2IT = 0•. Then
, ,
a. =../2IT 0; =.J2IT J1(1).J2/T cos nM dl =: JI(t) cos nM dt
• •
, ,
b. = J21T b~ = .J2IT J1(1) ../2IT sin nM dt = : J/(I) sin ntu dl
• •
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
41. 26 • JruiShl inlo Wa~lcts From Theory to Practice
Thus. we can rewrite Eq. (2.4) as:
J{I) "" ao + al cos rut + bl sin rut + ... + a. cos new + b. sin nrut + ... (2.5)
when:
,
~= ~ fJ(r)dl
•,
a.=: f f(t) cos nar dl. n = 1,2, 3, ....
•,
b. = ; ff(t) sin 'laM dl. n = I. 2. 3.....
•
These are the formulae given in most of the textbooks.
2.2.6 Complex Fourier Series
We know that
N~,
a COSM+" sinM= (al -jb,) tiU + (al +jb,) t~ja
I '1 2 2
Let us denote C =(al - jb,) and C =(al + jb,)I 2 ~I 2
The Fourier series can now be rewrittcn as:
(2.6)
(2.7)
(2.8)
(2.9)
Note that the formulation involvcs complcx cxponcntials and the concept of negative
angular frequcncy -ru. 1ltere is nothing imaginary in any signal and there is no negative
frequency. Complex exponcntials facilitate easy manipulation of multiplication and integration
involvcd in thc calculation of Fouricr serics coefficicnts than multiplication and integration
using plain sine and cosine fUlI(:tions. Let U$ filS! try to visualize t}oH, when: fI) = 2trrr and T
is the period of our signal.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
42. Fouricr Series and Geometry • 27
By Euler's fannula,
~OA = cos (J)l + j sin (J)l
We can thiok of this fUlICtion as 11. 'bundle' of sine and cosine fUlICtion kept togethcr side by
side such that they do not 'mix' with each other. The fUlICtion of j cao be thought of as a
separator indicating that the two parts. I'iz. cos 001 and sin oot are orthogonal.
2,2,7 Orthogonality of Complex Exponential Bases
The set of bases. (I. ~. t-iu ... " ejoOA. e-jolll ... ,,) are orthogonal over the interval [0, n,
where 00 = 21rIT and T is the duration of the signal which we are representing using the bases.
However. there is a twist with respect to interpretation of orthogonality and projection of a
fUlICtion on to a complex function. To make you prepared fOf" that twist and to lICCept that twist,
let us consider If.: aT + bJ + ek. an element of ')1. n.e nonn of this vector (or magnitude) is
obtained by projecting V on to itself and then taking the square root of resulting quantity. That
;,
But in case of a complex number,
l=a +jb
to find the magnitude. we have 10 multiply l with its complex conjugate l and then take the
square root. In case of real function, say f(t), we find square of its nonn by
Jf(t) f(t) df
In case our function is compleJO, say Z(f), its square nonn is given by
Jl(t) l(f) df
This has implication in the formula for projection of a fUlClion on to a complex function.
We must multiply the fil'$t function with complex conjugate of the second functon a~ then
integrate. With this twist in mind, we will prove that the set of functions u(t). t"", e-JfM
.....
ej<lu. t - j<lu, ... ) are orthogonal. We denote the projection offt") on to h(f) as (fl(f), fit».
CIISe I : u(t) = 1 and el-', for 0 S; t S; T and 00 = 21f1T
, , ,
("(f), t i.) '" JI · ti-l< dr = Je-j.>tt dr = Je-j(lmff) dr '" 0
, _0 0 (I
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
43. 28 • Insight inlo Wavdet5 From Theory to Practicc
CtI$r 2: tj<t·" and t-il-' where 11 and m are positive integers.
, , ,
(tftttU , t -J"') == JrJou t J... dr == Jd<.''')'' dr =Jd{o+..~lJfIT), dl = 0
•• 0 0 0
CtI$e J: ep..., and eJ...,' where 11 and m are either positive or negative integers with m ~ 11.
, , ,
(~. ~) = J~M t!/ow dr = f~.-.) ..dt = Jeil.-",){2JffT)' dt ". 0
• • 0 0 0
Cases l. 2 and 3 prove that the functions are orthogona!. What about orthononnality of
comple" e)[ponential functions?
Since.
>od
,
(l!'*", r) = Jr"'Mt ftttU dt =
...
,
,
f]· dl=T
•
(U(I).U(I» = fI·dl =T. the functions are not orthooonnal
•
The functions though orthogonal do not salisfy the requirement of orthononnality. So we
multiply the functions with a scalar to make them onhOflOrmal WllOng themselves.
For unit function and comple" uponentials, scaling constant is .fliT. Therefore,
orthonormal bases are:
We denote the signal space generated by these bases as L2. Now. any signal which is an element
of Ll can be written as:
To get a particular coefficient, we project f(l} on to the corresponding base.
I.e.,
,
Col = ff(I).jlfT u(t) dr
•,
C; = Jf(I).JIiT r-;- dr
•
(2.10)
(2.11)
(2.12)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
44. I'ouricr Series and GeomcU'y • 29
,
C~. = f1(1) JilT t i- dl (2. 13)
"This resull follows from the OJIhonormality of bases used in (2.10).
Now, we simplify the expn:ssion given in Eq. (2.10) by introducing new constants.
Let Co = coJIiT
Since
Therefore,
,
Co = fl(lr/lfi 1.1(1) dt
", , ,
Co = J IlT ff(t).fliT 1.1(1) dr = ~ ff(t ) uet) dr = ~ ff(t ) dt
" " "Similarly
,
C. = c;JIiT =~ f1{I)e-i'fM dl
",,' ,
C. = C:.JUT = ~ fl(rY- dr
"
Thus, we oblain complex fourier series in slandard form:
with the coefficients dcfiled in Eqs. (2.14) to (2.16).
(2.4)
(2. 15)
(2.16)
(2.17)
This completes formal inU"Oduction to Fourier series. The n:adcr who is nO!: familiar with
advanced Founer methods can skip Chapter 3 and proceed reading Chapter 4.
SUMMARY
It is no undcrstatementthat the Founer senes especially discrete Founer transform is considered
one of the crowning achievements of the 20th Century. The theory of Founer series has strong
connection with geometry or at lcast we can understand the Founer transform thoory from a
geometrical viewpoint. We h3d the honour and a privilege to bring it to the reader in this user
friendly, low fat and no cholesterol form. In the forthcoming chopters on wovelets, this
geometrical viewpoint will help you to digest the theory in a beller way. It will surely prevent
intellectual diarrhoea.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
45. 30 .. Insight into Wavelets From Theory to Practice:
EXERCISES
2.1 Consider the interval [- I, IJ and the basis functions (1. ,. ,2, ,1, ...• ) for L2!_1, I].
,
The inner product on this vector space is defined as <to g) = J/(1) 8(1) dl . Wc can
-,
show that Ihese functions do nOl fonn an onoonormal basis. Given a finite or
countably infinite set of linearly independent vectors Xj, we can construct an
orthonormal set )'j with the same span as X j as follows:
S . h ".. tan wit )'1 = r:I
,'"
.. Then. recursively set )'2 =
.. Therefore.
This procedure is known as Gram·Schmldt orthogonali:zalion and it can be used
10 obtain an onhononnal basis from any other basis. Follwing is a MATLAB
function Ihat implement the Gram-Schmidt algorithm. Your input must be a set of
linearly independent fUll(:tions. 1be output will be a set of onhooormaJ functions that
5pan Ihe same spa<:e.
function ~_g~amschmidt( x,N .M);
fo ~ i_l:N;
s/i,:).x (i, : ),
end;
ell) .s/l. :) · conj (s(l,:).');
phi(l, :) - s(1, :)/sq~t (e(l);
fo~ i_2:N;
th(i, :).ze ~oa(1,M);
fo~ ~.i·l:·l:l;
th(i,:) _th( i ,:) .. (a (i,:) · conj (phi (~,:) . ')) · phi (r,:);
end;
th(i, : I- s (i,:) -th(i,:);
e(i)_th(i,:) · conj(th(i,:).'); 11 note.' means
transpose without conjugation
phi(i,:I·th(i,:l/sq~t(e ( i));
end;
z _phi Cl :N,:);
Apply the algorithm on Legendrc polynomials and plO! the first six orthonormal
basis functions.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
46. Fwrier Series Il!Id Geometry • 31
2.2 Project the function j(x) : x on to the space spanned by ji(x), ¥1x), ¥12x),
!l'(2x - I)e L2[O, I] where
fI(X) :
1
1.
o.
OSxS I/2
otherwi se
(
I OS x S l12
If(x): - 1 112 SxS I
o otherwise
Sketch the projection oflex) to this space.
2.3 Show that the set of functions /",(x) '" Hsin mx. m = l. 2, ..., is an or1hononnal
system in L2[O, If].
2.' Given that
112 112 1/.J2 0
In In -11../2 0
H. :
112 - In 11../20
,n - 112 0 - 1/./2
9
7
is an orthononnal basis for :It. and l : [l IE '" e :lt4
• where E is the standard
3
,
basis for :lt4
• that is.
I 0 0 0
0 I 0 0
E=
0 0 l 0
0 0 0
Find z in tenns of basis in H•.
2.5 (a) Find the Fourier series of/ (1) = 1 for - If S t < If.
(b) Using Plancl1erel's fonnula and (a), evaluate the series
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
47. 32 • Insigh! imo Waveleu~From Theory w Pn.c:!ice
[Hinl: Acoording 10 Plancherei's formula I If = L. 1c. 1
2
, where the C,.s are
""complex Fourier series coefficients of the function I(/). U/li2 in the given problem
"
is 2~ JIf(t)1
2
dt I
-"
2.6 Find the Fourier series of
f·HI(x)=ln
for-nSx<O
forOSx<n
Suggested Further Readings
Amold. Dave. I am the Resource jar all that iJ Math: A COnlemporory Perspective. C.R.. CA.
1998.
Burden, Richard L., Numerical Analysis. BrooklCole, Alban)', 1997.
LeQn, Sleven, Unear Algebra wirh Applications. Prenlice HaiL New York, 1998.
Lyons. Richard. Unde ,jumding Digital Signal Processing. Addison-Wesley. Menlo ParIc. CA.
1998.
Ramiret, Roben W., The FFT: Fundnmenlals and Concepu. Prentice Hall. New York. 1985.
Slnlng. Gilbert. Inlroduction to Unear Algebra. Cambridge Press. Massachusetts. 1998.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
48. Continuous Wavelet and Short
TIme Fourier Transform
INTRODUCTION
The aperiodic, noisy. intenninent, transient signals are the type of signals for which wavelet
transforms are panicularly useful. Wavelets lIave special ability to examine signals simuh-
aneousJy in both lime and frequency. This has resulted in the development of a variety of
wavelet based methods for signal manipulation and interrogation. Current applications of
wavelet include climate analysis, financial lime series analysis. hcan monitoring, condition
monitoring of rotating machinery, seismic signat denoising, deooising of astronomical images,
crack surface c haractcril.ation, characterization of turbulent intermittcncy. audio and video
oompression, compression of medical and thump impression records, fasl solution of partial
differential equations, computer graphics and so on. Some of these applications require
(:(lntinuQUs wavelet transform, which we will explon: in this chapter and find out how it differs
from classical mc:thods Ihat deals with aperiodic lJOisy signals.
3.1 WAVELET TRANSFORM-A FIRST LEVEL
INTRODUCTION
Wavelet means 'small wave' . So wavelet analysis is about analyzing signal with short duration
finite energy functions.1bey transform the signal under investigation into another representation
which presents the signal in a more useful form. This tnlnsformation of tbe signal is called
wavelet transfOl'm. Unlike Fourier tnmsfonn, we have a variety of wavelets that ~ used for
signal analysis. Choice of a particular wavelet depends on the type of application in hand.
Figures 3.1 to 3.4 show examples of some real and complex wavelets.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
49. 34 • I "gh . W le F
"" t ,"IQ m ~~ """" p,,,," '"
1
1
0.2
0.2
"0
0
..... -<1.2
-<1.2 - 1
- 20 o 20 - 20 - 10 o 10
FIGURE J.l Real and imaginary pans Qf Shan wavdet
1.2,--_-_--_-_-_-_--_-,
0.8
0'
01--_,
-<1.'
- 3 - 2 - 1 o 3 •
FlGURE).2 Quintic Splinc wavelet
FIGURE J.J Real and imaginary pans of Gabor ",,..."del.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
50. Continuous Wavelet: and Shoft Time Fouricr Tm1sfonn • 35
1.0
1.0
0.'
0.'
o0 t-~
o.ot------
-<I.'
-1.0 ':----:----;--:---0- 4 - 2 0 2 4
- 1.0':-----";----'-;----;:------;
- 4 - 2 0 4
, ,
FIGURE 3.4 Real and imaginary potU of Morlet wavelet.
We manipulate wavelet in two ways. The first one is translation. We change the central
position of the wavelet along the time axis. The second one is scaling. Figures 3.5 and 3.6 show
translated and scaled versions of wavelets.
" ",, ' ,, " ,, ,,
~, ;.
, '" "'.' '~, '.,
FIGURE J.5 Translation (change in position) of wavelets.
f~ucnq
(Scale)(Lcyd)
vv
TIme shin
,
FIGURE J.6 Change in Kale (aI~ called level) of wavelcu.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
51. 36 • lnsigllt into Wavelets-From Thcofy to Practice
Figure: 3.7 shows a schematic of the wavelet transform which basically quantifies the local
matching of the wavelet with the signal. If the wavelet matches the shape of the signal well at
a specific scale and location, as it happens to do in the top plO( of the Figure 3.7, then a large
transfonn value is obtained. If, however, the wavelet and signal do not correlate well, a low
value of transform is obtained. The transform value is then ploned in the two-dimensional
transfonn plane shown at the bottom of the Figure: 3.7. (indicated by a dot). The transfonn is
computed at various locations of the signal and for various scales of the wavelet, thus filling
up the transform plane. If the process is done in a smooth and continuous fashion (i.e., if scale
and position is varied very smoothly) then the transform is called continuous wavelet
transrOnR. If the scale and position an: changed in discrete steps, the transfonn is called
diM:rde wavelet transrorm.
W.~,,:,~,"~--=-"r-=--../ local =hing between signal
and wavelet Ic:odJ 10 large
transform coeffic;"ntWavelet transform
_ Cu=n' ",ale
"", I~---
-----------------------~
Wavelet transform plO!
(Two-dlmeru'onol)
: Higher lhe coefficient more
: darker tbe point
r CUrrent location .........
Position ---_.
FIGURE 3.7 n.e signal. wovelel and lnUI,form.
Planing the wavelet transform allows a picrure 10 be built up of correlation between
wavelet- at VMooS scales and locations-and the signal. Figure 3.g sllows an e~ample of a
signal and corresp:lllding spectrum. Commercial software packages use various colours and its
gradations to slIow the transform values on the two-dimensional plot. Note that in case of
Fourier transform. spectrum is one-dimensional array of values whereas in wavelet transfonn,
we get a two-dimensional array of values. Also note Ihat the spectrum depends 011 the type of
wavelet used for analysis.
Mathematically. we denote a wavelet as:
I ('-b)~•.•(,). JI"I ~ -;;-
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
52. Continuous Wa" elel and Shon Time Foorier TfWlSform • 37
Timel)
HGURE J.lI SiglW iIId iu ""iI,·(]tl tJillSfl1l1ll lipc.;lJUm.
where b is location parameter and 11 is scaling parameter. For the function to be a wavelet. it
should be time limited. For a given scaling parameter a. we translate the wavelet by varying the
parameter b.
We define wavelet transform as:
V(u.b)= ff(t)JtaII"C:b)dl (3.1)
According to Eq. (3.1). for every (Cl. b)• ....e have a wavelet transform coefficient.
rePf"CSenting how much the scaled wavelet is similar to the function at location I = (bla).
In the following section (Section 3.2) we will explore the classical time-frequency
representation of signals. and associmed uncenainties in frequency and time. 1llen we shall
compare the same with that of wavelet transfOl"ms.
3.2 MATHEMATICAL PRELlMINARIES-FOURIER
TRANSFORM
Continuous Fourier tmnsform
Assume that/is a complex-valued function defined on R. Its Fourier transform j is defined as:
If fELt. the integral makes sense fOl" every value of w and f is a continuous. bounded
function which goes to zero at infinity (this last fact is called Riemann-Lebesgue Lemma).
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
53. 38 • Insight into Wavelets- From Theory to Practice
The Fourier transform is also defined for [E:' L2. In this case, the integral may oot be
defined in the usual sense. One way to define it is:
L
' •
" I IlII f 'few) = c:= [(I)e-"OJ dl
,,2K R -J ...
-.In this case j is also in Ll.
If I E L1 or [E Ll it can be wrinen in tenns of its Fourier transform as:
-I f - .I(r) = r::-= I(wje"'" dw
,2.-where the integral may need to be imerpreted as a limit of finite integrals, as before. In general,
every locally integrable function [has a Fourier transform but j may not be a fu nction any
1IlQI"C, rather a generolized[unetlort or dlslrlm.lion. We will assume from oow onwards that all
functions have suitable imegrability properties so that all Fourier transforms (and other
operations that show up) are well defined. Some propenies of the Fourier U"lUlsform are:
• The Fouricr transform preserves L2
norms and inner products (this is called the
Parseval-Planchen:1 Theon:m). Thus. if f, g E Ll then
(I,g) = <i,g)
• 11Je Fourier transform turns convolutions imo products and vice vema. 1lte convolution
of f, g is defined as:
We find
(f. g)(I) = JI(y) g(t - y) dt = JI(t - y) g(y) dy
(f. g)" (w) = J2K j(M)g(M)
(f. g)" (M) = J2K(j. g}(/lJ)
• The Fourier transform turns translatioo into modulation and vice versa. The lrons/mion
of I by a E 9t is defined as:
T.[(t) = I(t - a)
The modulmion of[by a e 'X is defined as:
E.f(t) = e;"I [(t)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
54. Continuous Wavelet and Shon Time Founer Transform • 39
. .
(EJ) " (m) '" f(m - a) '" T.f«(I)
• The Fourier trlUlsform turns dilation inlO inverse dilation. The dilOliQn off by SE 9f
is given by
I 1
-112
D.f(t) '" s f(tls)
The factor in front is choscn so thal ID.f ll
'" Ifb.The Fountr transfonn relationship
IS:
I 1
1/2 " "
(D.f)'" (m) '" s f(sm) '" o",.f(m)
• The Fourier transform turns differentiation into multiplication by iro and vice ve~
(n" (w) :: iw j(W)
(If(t» " (m) '" ij'(m)
3.2.1 Continuous Time-frequency Representation of Signals
Assume that f{1) is a complex valued function on 91 which represcnts some signal (think of t
as lime).
The Fourier transform
-. 'f ·f(m ) '" ~ JO)t-"'" dl
,2. (3.2)
-is used to decomposc f into its frequcocy components. The inversion formula is o;pressed as:
(3.3)
Equation (3.3) can be interpreted as writing f as a superposition of time-harmonic waves t 4
-.
If i is large near some frequency then f has a large component that is periodic with that
frequency.
This approach works well for analyung signals that are produced by some periodic
process. However, in other applicalions, like speech analysis. we would like to localize the
frequency components in time as well and Fourier transform is nol suitable for thal.
We will consider two methods that attempt to provide information on both time and
frequency: the Windowed Founer Transfonn (wFI1. also called Short Time Fourier Transform
(STFl') and the Continuous Wavelet Transform (CWl). Both of lhem map a function of one
variable (lime) inlo a fuoction of two variables (time and frequency). A large value of the
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
55. 40 • Insight intQ Wa~elet. From Theory to Practice
transform near time I, and frequency ro i$ interpn:tcd as: [he signal! contains a large componenl
willl frequency W near lime I. lllere is a lot more theory [0 botll of these transforms IlIan we
will cover in [his chapter but our main interest lies elsewhere.
3.2,2 The Windowed Fourier Transform (Short Time Fourier
Transform)
Fix a function IV E L2, IlIe window function. W sllould be localized in time neat 1 '" O. witll a
spread IT (we will define "spread"' more precisely later).
Typical choices are:
(i) W = XI-I .11 willl IT = 1I.Jj (see Figure 3.9)
X. is the characteristic function of the set S, which has value I on the set. 0 otherwise.
w
----'---+_L---::+---'------+ ,
Sigma
.'IGURE 3.9 Char.lCtm>lic function of W .. XI-l.It
(ii) W", (J +cos 21)12 for I € (~.R"12. xl2j, IT == J/.J3 (see Figure 3. 10)
(iii)
Sigma
FIGURE ).10 1'101 of window function W .. (l + cos 21)/2.
w" _'_ t-r'12 (Gabor willdow). 0'" I (see Figure 3.11)
J2Jc
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
56. CQnlinUOlU Wa""kt and ShQrt Time F<>urier Transfmm • 41
Sigma
FIGURE 3.11 PlO! Qf window fUI>Ction W ~ ';"e-..n.,2,
The WfT with window W of f is defined as:
-Ifwf(a.b) = ,j~lf L!(r)to-;bl W(t a) dl
Thus. a is l!le time parameter, b is t!le frequency.
(3.4)
V'wf(a. b) can be interpreted as inner product of f with [he leSI function eiblW(1 - a).
A typical test funclion looks like the one given in Figure 3.12.
// y-
FIGURE 3.12 Shifted and modulated window.
Note how the time resolution u (rel3led to the window width) is C()nstanl. independent of
the frequency.
3.2.3 The Uncertainty Principle and Time-frequency Tiling
Assume a function f e l}. The II1C3I iJf of f is defined as:
(3.5)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
57. 42 • Insi):hl intQ Wavekts-Frorn TheQry IQ Practice
The uncertainty CIf of! is defined as:
(3.6)
Remark: If!E Lllhen !2/I!f is a probability distribution. i.e. a non-negali·e function with
integral L. IJf and CIf are simply the mean and stlllldard deviation of this distribution in the
statistical sense.
IiJ measures where! is localized in time and CIf measures how spread OUI (or uncertain)
this time measurement is. Pt localizes the frequency and Cl; measures the uncertainty in
frequency.
H,~
(3.7)
where
-" I J .! (w) = ~ !(t)e-"· dr
,2. -"d
(J.8)
The uncertainty principle states Ihat for any f
(3.9)
If a function is localized in time, it must be spread out in the frequency domain and vice
versa (see Figure 3.13). The optimal value of 112 is achieved if and only iff is a Gaussian
distribution. To visualize this, consi(jer a hypothetical function F(t, w) over the time-frequency
plane. F(I, w) represents the component of! with frequency wat time /.
The uneertainly principle says that it makes no sense to try to assign point-wise values to
F(I, w). All we can do is to assign meaning to averages of F over rectangles of area atleasl 2.
(The signal is localized in time to [,u - Cl, IJ + a:l and likewise for frequency, so the rectangle
has area (2C1p x (20).
The uncertainty principle applies 10 both WfT and CWT but in di fferent ways. Let 1J... 1lIld
Cl... represent 'localization' (mean) IlIld 'spread' of the wirKiow function W(I) in time domain
which are formally defined as:
-1-1... = 1 2 fIjW(t)11dl
IWO)I _
(3. 10)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
58. Conlinuous WUeltl and SIKH1 Time Foune, Tr.msfonn • 43
"I
D
Good time resolution
Bad f'<queDCY n:w1~!i0f
:;>
Good frequency resolution
Bad lime rUQlulion
nCURE J.B Po..ibt" lime-fn:qu<DCY ....indo......
(3.1 1)
Also Itt A.. and 0-.. represent F014,iu lrons/orms of 'locaJiution' (mean) and 'spread' of
the window funclion WCt). Then the inner product (f, W) contains information on / in
[.uoo - u.".u.. + u..J. Since <I,W) '" tj, IV) , it also contains (nfoonation on /(0) in
[.it.. - o-..,.it.. + it..]. Thus, (J, IV) represents the frequeno:ies between Pw - o-w and .it.. + 0-..
thal are present between time p.. - u.. and .u.. + u"'.
For WfT, the test function is the shifted and modulated window.
..iblW(I_ a) '" EhT. W
which is localized in time near .u.. + a with uno:ertainty a.".. Its Fourier transfonn is
which is localized near .it.. + b ....ith unCtnainly U...
Here E and T are respectively modulation and translation operators.
contain information on / in [.uw + a - u.... .uw + a+ u.. J and information on
[.it.. +b - 0-..,Pw +b + 0-00 ] .
(3.12)
"
f"
For a WFf with fixed window WCt), the time resolution is fixed at u..' the frequency
resolution is fixed at 0-,... We can shift the window around in both time and frequtncy but the
uncenainty box always has the same shape. n.e shape can only be changed by changing the
....indow W(I). Refer Figure 3.14.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
59. 44 • Insight into Wavelets From Theory to Practice
D
Time axi.
HGURE 3.14 Some possible t;me·(re<j""!IC)' window. of w;ndowc:d Fouric. transform.
Time and frequcocies are resolved equally well (or equally bad). A discrete WFT (with equally
spaced lime and frequency samples) gives a unifonn tiling as in Figure 3.15.
I
,
I' IGURE 3.15 lime.fre<j""'''''Y Lilinl of windowc:d Fouricr t.,,".form.
For CWT. lhe test function i5 of the form:
(3.13)
with Fourier transform E_bDlI8Y. Thus.
(j. V'Q.b) = (j. TbD.V') = 0.E_b DI18y) (3.14)
contains inronnation on f in lu,u,.+b-uu,..up",+b+uu",J and infonnalion On j in
[iJ"la - iJ",la. it"fa + iJ"fa]. The uncertainty boxes have differem shapes in different parts of
llIe lime·frequency plane iU in Figure 3. 16.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
60. Cominuous Wavelet and Short TIme Foorier Transfonn • 45
o
AI high frequency:
Good lime re""luliOl1
Bad freqllCOC)' r~lulion
AI low frequency:
[======1/Good freqllCncy re).OIi.>I;OII
i Bad I;me resolution
TIme aJIis
.'IGUIlE .3.1' f'lmiblc limc-frcquclI(Y tHings in the case of continuow wa...det transform.
3.3 PROPERTIES OF WAVELETS USED IN CONTINUOUS
WAVELET TRANSFORM
A wavelet 1f(1) is si mply a function of time t that obeyS a basic rule. known as the wavelet
admissibility condition:
- I~ )1c =f (U dw<- (3.15), w
o
where !i/(w) is the FourieT transform. This condition ensures that ]V{w) goes to zero quickly
as co -t O. In fact. to guarantee that Cl" < _, we must impose ]V(O) "" 0 whicl! is equivalent
" -fVlO) dl = 0
-A seCOl1dary condition imposed on wavelet function is unit energy. That is
-
-
3.4 CONTINUOUS VERSUS DISCRETE WAVELET
TRANSFORM
(3.16)
(3.17)
As mentioned previously. CWT is a function of two parameters and, therefore. eOnlai ns a I!igh
amount of extra(redundant) information when analYl.illg a functioll. illstead of continuously
varying Ihe parameters. we analyze the signal will! a small number of scales with varying
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;