SlideShare a Scribd company logo
1 of 177
Download to read offline
, ,, / - ,
. (~. Third Edition ..
11e'"1I
Insight into
~
stem
conomy
dition
FROM THEORYID PRACTICE
KP. SOMAN •KI. RAMACHANDRAN •N.G. RESMI
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
R... 375.00
INSIGHT INTO WAVEl£Ts--f'rom Theory to PracUce, 3rd MII. (wilt! CO-ROM)
K.P. Soman, K.I. A.amachandran, N.G. Rasmi
0 2010 by PHI Learning Private Umiled. New 0eIII. AI rights .--ved. No pari 0I1hIs book may be
reproduCad in any klnn. by mimeog...llln Of any 0Ihef ~. -..iIholA pemIisaion in writing from ".
publishef.
-----...,-..........- ...----.....-..~--_ ...... co--.n. .......... _ _ ....... _ ..... _ ... _ .. _ • 0l0I _ _ ..
_. , -. .. -.g ... COI. ... ~ ""..oo.=. ... _ .. lOo E_ _•
ISBN-978-81-203-40S3-4
The .~potI rights 01 IhII book Ire 11951ed IoIIIy w;., !he publisher.
S!Jc1t! Prlntl"1l (ThIn! Edition) M..-dI,2010
P~bli5h.d by A50h K. Gh-osh, PHI leamlng Privati llmitad. 1.4-97, Connaught Circus.
New [)aH.ll000t and Printed by Mudrak. 3O-A. Patparganj. Delhi-l10091 .
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Contents
Prrjoce
PrrjllCe /0 the Fir" Edition
A.ckno...led~mellU
1. The Age of Wavelets
inlmdMction--.l
1.1 The OrigiN of Waveleta-A~ They Fundllmcntally New? I
1.2 Wavelets and Other Rcality TrwlSfonns 3
1.3 Managing HeisenberJ', Uncenai!!!y: Gho6t j
1.4 Histor}' of Wavelel: from Morld 10 Daubechies via Mall&! 6
L4.LDiffcmrlLCommunitiu..oLWaveM:1L..9
1.4.L...Diffcreol EamiliCLoLWavcieuJ'jthin W'l'Ciet Communitics----.lO
.4.3 IlIIClt:5ting R«ent Devclopmcnb: 11
1.5 W.vt:lets in thr; Furun: 12
1.6 What llistOf}' of Wavelet T-=hu UI 15
S...........ry lj
SilIlUlM F,mJter RMding~ 15
1. Fourier Series and Geometry
lfllrOducliOfl /6
2.1 ¥«tot Space 17
2.U-----.S.uu.----.l7
2.1.2 Onhonormality 17
2. J.3 Proja;tion 17
22 Functions and Function S~ 18
2.2.1 Orthogon.l Fu.netions /8
=_OrtboooonalEullCtion5~9
2.23 Fwx:tion Spac:eI 19
2.2.4 0rtb0g0aa.L Basis Fl,lno;tions 22
2.2.5 OrthooormaIil)' and the Method of Finding the Coefficients 22
'"
Id'
16-32
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
viii • Contents
2.2.6 Comp~x Foorier Series 16
2.2.7 Orthogonality of Complex Exponential Bases 17
Sum/lUl_fl'...- 29
Ezerr.isu_ 30
SU8llt slt d Furthu Rtading~
3. Continuous Wavelet and Short TIme Fourier Transform
/fltroducliOll_ 33
3.LWavdeLJnnsform--A..Eim.L:.vdJnlrodLII:linn---.J3
3,2~athemalic;tl&eliminarie$-EQuri« TnlIl$form JJ
3.2.1 of ~gnal$ 39
3.2.2 The Founc! Transforml 40
3.2.3 The tli 41
3.3 Propenics of Wavelets In 45
3.4 Conlinuow; versus Discrete Wavelet Transform 45
SWllmQry 47
E:arr:isLL 48
Suggt~!td F...~r Rtadi~&s 49
33-4'
A. DisClocte Wankt...Transronn 50-74
/mroductio~ 50
4.1 Haar Scaling Functions and Function Spaces 50
4.1.1 Translation and &alinS of ;(1) 51
4.1.2 Orthosonal"l)' of Translates~O 51
4. 1.3 Function S~ V. 53
4.1.4 Finer Hair ScaliRJ Functions 55
4.2 Nested Spaces 56
43 Haar Wavdd Function 57
43.1 Scaled Haar Wavekt-Func6oos 59
4.4 OrtItogooality of ;(1) and 11(1) 64
45.......Nonnalizalion_oUiaaLBasa..ILDiffcrenLScak.s_ 65
4.6 Standardizing the Notations 67
4.7 Refinement Relation with Res~t to Normaliwl Base.< 67
4.8 Support of & Wavelet System 68
4.8.1 Triangle Scaling Function 69
4.9 Daubtthics Wavelets 70
4.10 Stocing tilt Hidden-PloI:ting tilt Daubtoch.iu Wa~It" 7J
Summary 73
Extrr.Ut~ 7J
Suggtutd FUr/htr R",di"ls 74
S. Designing OrthogonaJ Wavelet Systems-A Direct Approach 75-93
ImrodUClioll_ "
5.1 Refinement Relation for OrtItogo;>!!a1 Wavelet S)'stcms 75
5.2......Restrictionu.n-Filtcr Coefficients 76
5.2.1 Condition 1: Unit Area Undor Sc.a1inll Funetion 76
5.2.2 Coodition 2: Orthonorma.lity of Translates of Scaling Funetions 76
5.2.3 Condition 3: Orthooormality of ScalinS and Wavelet Functions 79
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Coments • ix
5.24 Condition 4: Approximation Condition5 (Smoolhne55 Conditions) 81
5.1.5 Condition 5;j!edundant) OrthonormaIity of Tl'IIl$lates of Wavelet
Fu.nctions 82
5.2.6 Condit~UIIdant) Orthooonnal'ty of ,(0and Translates of ttC!:) 82
5.3 Designing Daubt:chics Orthogonal Wavdet bstem Coefficients 83
5.3.1 ColIStnIints fow D.ubcchies· 6-1ap Scaling FLJ.Ilf;'tion 84
5.4 Design of Coiflet Wavelets 85
5.s StmlelS 86
5.6 An Intri,guin~y of Ortbogonal Scaling FWICtion 86
S.6..1~ SlImm·'ion.Form~88
5.6.2 Proof of Partition of Unity of Scaling Func:tion 90
5.6.3 Ne<:cni!), of Partition of Unity of Scaling Function 91
SIUftIIIQf)'_ "
E2~j!el-----.92
Sugg~sled Funher R~odill" 93
6.---»i.Krete WaYdet.Tnms!orm and Relation to f'Ute..-.lJank5 9+----.ll0
11llrOdJ.djOlt~
6.1 Signal Decom~ition (Analy~ 94
6.2 Relation with Filter Banks 97
6.3 F~ Response 102
6.4 Sig~1 Reconstruction: Synthesis from Coarse Scale to Fine Scale /03
6.4.1 Upsampling and filleTing 1()4
6.5 Perfect Match.i!!8 Filten 107
6.6 Computing Initial 'itt Coefficients 107
6.7 Vanishing Moments of Wavelet Function and Filter Properties 107
S"'""""ry---.JP9
Suglu/ed Funhu Rtadill8£ 110
7. Computing and PlottJng Scaling and Wavelet Functions 111-126
..
IItlmduClimt----.lH
1.1 Daubcchies-Lagatias Algorithm 112
1.1.1 Discrete Dilation EqlWion lJ4
1.1.2 Swcmcnt of Daubcchies-l...a8!!ias AI~thm /15
7.1.3 Generating Binary Equivalent of a Decimil Number JJ6
7.1.4 Implementation in Microsoft Excel (or Any Spreadsheet Pacuge) JJ6
7.1.5 CoIllR!!Un.LWavelet FuocUon 118
7.2 Sub:1ivis.ion..Scbem.L...l18
7.3 S""",,"';ve Appro.imation 121
8.1 Bionho&onali!J: in Vector S~ 127
8.2 Biorthogonal Wavelet Systems 129
S.3 Sigru.l~ UsinS_BiortlKlgonal Wavelet S)'Slem H2
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
" • Contents
8.4 Bioooogonal Analy~lJZ
8.S Coarse Scale to Fine Scale JJ.4
8.6 Wa""let~aum& 1.15
Summary-HJ
E:arru£L-L42
,o••1,"'••,., S)'~", or Cohen-Daubcchies-Fca.uveau
117
Suggesled FlIftMr Readings 141
9, Designing Wavekts-FrequelK)' Domain Approach 143-UI3
InJrrxluC/ioIL ...1j3
9.1 Basic Proptrtics of Filter Cocf("t<.:icntli 143
9.2 Fillcr Properties in Terms of H..l(li) 144
9.3 Filter Properties in TCITll5 of H,(z) /45
9.4 F~uelK)'_ Domain Characteril.lllion of Filter Coefficients 146
9.S Choice er Wavelcl Function Coeflkicnts Ig(.)) 150
9.6 Vanishing Moment CondiliO/lll in Fourier Domain 153
9.l.......DerivatiQn nLDaube:c.h.ic.LWavdcts.......15.4
9.7.1 Steps Involved in DerivaliOfl of Daubechies Wavelets 1S4
9.7.2 Daubechies Wavelets with I Vanishing Moment 1S7
9.7.3 Daubechies Wavelets with 2 Vanishing Moments 158
9.8 Parametric Design <.If O"~onal and Biortoollooat Wavelets 159
9.8.1 Polyphase Factorizatioo Approach 1<1 Onhogonal Wavelet Design 159
9.8.2 Bionoollonal Wavelet Desig.........A Factor Multiplication ~h 166
9.8.3 Exte",,""" to Orthog<llllll Wavelets 179
9.8.4 Applicatioos of Panunetcriud W.""lcts /81
10, Groebner Basis (or Wavelet Design 184-192
InJrrx/ucliOll--.-l84
lo..LGrocbner...Basu.--.l1tl
10.2 Deriving Grocbner Basis for Daubechiel Wavclcl Syaum 185
10.2. Daubedties' 6-tap W.ve!ct with 3 Vanishing Moments 187
10.3 Deriving Grocbnet Basis for Coiflet /88
10.3.1 ~rties of the Moments of Scaling Functions 189
10.3.2 Des'lln of 6-tap Coiflet 190
10.3.3 Gcnenolitcd Coif1c:t S)'stem 19/
Summ<l~ /92
11. WaveletPacket Ana1ysis
btl~IimL....JJ)J
1l.L ...HuLWaveIeLl'aculL ...194
~];:!:::':B~U:i'~SeIecIiOO f()!" SiS.nal CII" Image Compassion 100
S"""""ry 202
193-202
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
12. M-Band Wavelc:ts
/nlmdJJctian 203
l21---.Motivation----.203
12.2 Multi·resolution Formulation of M·B~nd Wavelet Symm 206
12.3 Derivation of tbc ProJ1C'rties of M-Band Filter Coefficients 209
12.4 SupportS of SQIIing FUrK:lion lJld Wavelets Zf4
Conknts • xi
203-228
12.S Dcaign of 4-Band Symmetric Or1hogonal Wavelet Filtu Banks Based on Il!oOO.i 215
12.S.1 Groebnc:r Based Design Example with Length = 12 and Regularity =2 216
12.6 Parametric: Da.ign of M-Band Wave1c1S 219
12.6.1 Down~g and IVlynomial Represcnt:>l.ion 219
Summary 228
13. Introduction to lfultiwaveiets
1.3.1 Refrnable Function Vcctor 229
13.I.I Fuoction Sp3Ce VI 231
13.1.2 Su~ of, 234
13.1.3 Symbol of, 234
13. 1.4 FourierTransfocm of f: 235
13.U OrthononnaI.ity of, 236
13..6 Orthonormality of , in ternlS of H. 237
IJ.2......Multiwavc:let.Eunctioo----.2J10
13.2.1 Symbol of tt(o) 240
1l.2.2_Fourier Transform of tl!!-240
13.2.3 Orthogonality..&J! and ~'l 241
13.L ..Momcnts_ 241
13.3.1 DiKrek Moment of; 24J
13.3.2 DUcrete Moment of ~,) 241
13.3.3 Relation Bd_n M. and H(CtI) 241
13.3.4 Relation Bdween N,{ij and G(I) (IV) 143
13.3.S Continuous Moment of;' 143
13.3.6 Conlinuous Moment of "I') 143
13.3.7 Relation Between Jl> and Fourier Transform of, 243
13.3.8 Relation Between vi') and Fourier Transform of ttI') 244
13.3.9 Relation Between J!t and M. 244
13.4 Approximation Order 246
I1..L...MuUiwavc..lo::l_Eilta..BaDk 149
13.S.1 Analysis and 149
13.S.2 250
13.'.3 Signal ProeC5&ing) 250
13.6
13.7 TImc-varyinLMultiwavelet Filter Bank 252
13.8 Balancing Condil;ons 256
1,3~.8~.~1>lB~"~~~;";8~O~f~""";~'~I.~';'~O~:25"13.8.2 J; _ I 258
13.9 259
162
229-277
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
xii • Contents
13.9.3 Constn>elion of InterJ1Qlating Mu.ltiwavdet 175
SU/MIOI)' 177
~8!sled Further Rwdl'!8s In
14. U fting Scheme 278-313
IIIJrD<luction--.l.18
]4.] Wavelct Transform Using Poly£1a$e M.m~ Factori~on 179
14.1.1 Invenc Lifting 183
14.].2 Example: Forward Wavelet Tran~form 184
14.2 Geometrical Found;Wons of Lifting Scheme 186
14.2.1
14.2.2
14.2.3 Using Lifting 291
[4.2.4 Higher Order Wavelet Transfonn 295
[4.3 Lifting Scheme in the Z-domain 296
14.3.1 Desi,n Example I 301
]4.3.2 Example 2: Lifting Haar Wavelet 305
14.4 Mathematical Pttliminaries for Polyphase Factorization 307
14.4.1 Laurent Polynomial 307
[4.4.2 The Eoclidcan AI,orithm 308
14.4.3 Factoring Wavelet Transform into Liftin~A Z-domain Al!Pf"OICh 314
]4.5 Dealinl_wilh Signal Bolmdir)'_ 1i9
1"-'..1 CircI,laLCooYOlutiOll_ 319
14.5.2 Padding Policies 310
14-'.3_ lteraLioo_Beh.aviour------111
SU/MIOr)' 31 J
Exercuu 311
15. Image Compression
lntrD<luctit11L-314
15.] Overvicwof
15.J. l
15.2 Wavelet Transform
]5.3 Quantization 331
15.3. 1 Uniform Quantjution 331
15.3.2 Subband Uniform Quanti~ion 332
15.3.3 Uniform Dead-zone Quantization 333
15,3.4 Non·uniform Quantization 333
]5.4 Entropy Encoding----.l34
15.4.1 Huffman Encodi"B 334
15.4.2 Run Length Encoding 3]5
15.5 ElW Coding (Embedded Zero-tree Wavelet Coding) 336
15.5.] EZW Pcrformar>ec 345
15.6 SPUIT (Set Partitioning in Hicran:hical Tree) 346
15.7 £Bear (Embedded Block Coding with Optimized Tl1.lICation) 346
SIUll/lW~ 346
Webk_ (Wtb Based Prob/e/llS) 347
Suggesftd FUrlM' ReMillgs 347
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Contents • xiii
16. Denolslng 349-365
IntrrJ<iJu:tion 349
16. A Sim~1c E;o;~lanaJjon and a 1-0 Eumpk....}49
16.2 DenoisinJl U,ing W~1d Shrinkagc-Stati51ical Modelling and Estimation 3JO
16.3....NoiJc.Es.tima!iIHL JJl
16.4 Shrinkage FUllClions JjJ
16. ~ Shrinkage Rules Jjj
16.~.1 Universal Jjj
16.~.2 Minimizing the False Discovery Rate JJJ
16.~.3 T~ J56
l6.M S.IU:C_ JJ6
16.~.S Translation Invariant Thresllolding J57
16.5.6 BayesShrink 357
16.6 Denoisina Images with MATI.AB 358
16.7 MAl1.A8 Programs for Denoising J60
16.8 Simulalion for Finding Effectiveness of Thresholding Method 361
SWMlClry 36J
EMn:ises 364
Su.ggulni Fu.rtMr R,.,Jillgs 365
17. Ber.ond Wavelets: The Ridgelets and Curvcleb
In1rrJ<iJu:lion 366
.8.
17.1 AWlOlIimation Rates 367
17.2 Why 1 and Curvclets.? J68
17.3 The Transform 369
17.3.3 Finite Radon Transform J7I
17.3.4 Applications of Rid~1et Transforms 371
17.4 The Digilal Curvclel Transform 377
17.4.1 Propenics of Curvclct Tmnsform 378
17.4.2 Analysis: The Curvc:lel Decomposition 379
17.4.3 Curvclel Dcc<>mposition Algorilhm 379
17.4.4 Sy_nlhcsis: Reoonslruct.ioo from !he CtmIelet Transform J80
17.4.5 Digit.aJ Implementation or the Curvclet Transform 38()
17.5 S«ond Gc~ralion ~lets. 382
17.~.1 Tight Frame of Curveleu 384
1.7.5L Cw:vekt Constructioo.....J8.S
17.~.J
l7.6
S~l
'"N_1i~ Fnme of Curvcletl 388
18.1.1 Splinc eurve. and,:"u.r{aces 395
18.1.2 Cubic Spli~ Interpolation Methods J99
18.1.3 Hennitc Spli~ Interpolation 399
366-391
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
xiv • Co~tents
18.1.4 CUbic SJllines #JI
S.1.5 Splines in Signal and Ima~ ProccssinS_ #J3
18. 1.6 Belier a.n'Ves and Surfaces 404
IS.1.7 Pr~rties of Buier CUrves 406
18.I.S Quadratic and Cubic Bezier Curves 407
18.1.9 Paramwic Cubic Surfaces 409
18..10 Buiu Su.rfaccs 4./1
IS.1.11 B·splincCu1'Ves 4/2
IS.1.12 Cubic Periodic B·splincs 414
IS. .1) Co,w"rsion Between .Icnnit". Be"i". and B·splinc Representations
IS.1.14 Non·uniform B·sjl:lines 416
IS.1.5 Relation Iktwo:en SJlline, BWe. and B·~linc ~s 418
IS.I.16 B.splinc Surfaces 419
IS. I.l1 Beta.splincs aoo Rational Spli""" 419
IS.2 Multiresolution Methods and Wavelet A~ysis 420
IS.1...Jhe_Filtu-.B~411
8.4 Ortbo.l!onaJ and Semi-ilrtboJOOal Wa~lets 413
IS.S Splinc Wa,,,,lcts 414
IS.6 ~rties of Splinc Wavelets 427
18.7 Advantages of B·splinc Based Wavelets in Signal Processing Applications
18.8 B·spliroe Filter Bank 430
IS./LMuIliresolutiOlLCun>a and Facc.s_ '"
18.10 Wavelet Based CUrve and Su.rfacc Ediling 431
18.11 Variational Modelling and the Finite Element Method 431
18.12 Ada~i..... Variational Modo!llina U£ina W.""lets 43S
SUnurt<l1)' 436
Sugge.ted Funhe, RetuJingJ 436
439-443
445-447
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Preface
Wavelet theory has matured and has entered into ils second pllase of development and evolution
in which practitioners an: finding newer applications in ever-widening scientific domains such
115 bio-informatics, computational drug discovery and nano-material simulation. Parallelly. the
theory of wavelel$ got more and more demystified and has become an e~~ryday tool for signal
and image processing. Postgraduate courses in mathematics and physics now ioclude a subject
on wavdet theory either as a separate eJoctive or as part of other related subjects. In many
technical universities, w3sclel has been introduced cven at the undergraduate level. In this third
edition of the book we have taken into account this increasing popularity and the needs of the
relatively 'young' readers from such wide range of backgrounds.
One of the main additions in this third edition is that we have shown how the ubiquitous
electronic spreadsbcct can be utilized for wavelet ba.o;cd signal and image processing. The theory
behind the algorithm for computing e)lact values of wavelet and scaling function is simplified
and implemented in Microsoft E)lcel as a workshcct function. Onc can now draw the fUlICtion
by writing and dragging an e)lcd formula in a cell. Many of the intriguing properties of wavelet
and scaling functions such as orthogonality of integer translates, partition of unity and
refinement relation of scaling functions can be easily visualized in spreadsheets. The
accompanying C D contains several worksheets that demonstrate the power of spreadshcct
packages as a computational and visualization 1001.
Recent years have secn heightened interest in 'parametric wavelet filter design' which
allows the tuning of wavelet fiheB for various applications. Theory of its design procedures arc
added in respt(;tive chapteB with several e:o;amples.
Another new feature is that parametric and !"IOn-parametric biorthogonal wavelet design
are e:o;plained in more detail.
M-band wavelets are finding increasing applications in Communication Engineering as a
tool for multirate signal procciSing and as signal modulators. So the chapter on M-band wavelet
is eJlpanded to include the more recent and simplified design procedures.
A scparate and elaborate chapter on Multiwavelet theory is added. Muhiwavelet represents
the highest level of generali:.tation in wavelet theory and provides short filtcrs with most Of all
of the desirable properties that a filtcr should possess. 1beory of baI:ma:d and interpolating
multiwavelets are discussed in detail.
"
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
xvi • Preface
We earnestly hope that this edition will meet the: needs of readen of different academic
backgrounds for their undcrgl1lduate, poslgraduate and research le'el sludies.
Finally, we acknowledge OUT heanfelt Ihanks here to our ex-students Ms, K. Hemalalha
and T. Ar.nhi who prepared all Ihe worksheets given in the accomp311ying CD.
K.P. SOMAN
K,I, RAMACHANDRAN
N.G. RESMI
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Preface to the First Edition
In the past few years, the study of wa'c1ets and the exploration of the principles governing their
behaviour have brought about sweeping changes in the disciplines of pure and applied
mathemaTics and sciences. One of the most significant development is the realization thai., in
addition 10 the canonical tool of representing a furn:tion by ils Fourier series, then:: is a different
representation more adapted 10 certain problems in data compression. noise removal, pauem
classification and fast scientific computation.
Many books are available on wavelets but most of them are wriuen at such a level Ihat
only research mathematicians can avail them. The purpose of this book is to make wavelets
accessible 10 anyone (for example. graduate and undergraduate students) wilh a modest
background in basic linear algebra and to serve as an introduction for the non-specialist. lbe
level of the applications and the format of this book are such as to make this suitable as a
textbook for an introductory course on wavelets.
Chapter I begins with a brief note on the origin of wavelets. mentioning the main early
contributors who laid the foundations of the theory and on the recent developments and
applications. Cbapter 2 introduces the basic concepts in FotJrier series and orients thc reader to
look at everything found in the Fourier kingdom from a geometrical point of view. In
Chapter 3, the focus is on the continuous wavelet transform and its relation with soort time
Fourier transform. Readers who have oot had much exposure to Foorier transforms carlicr may
skip this chapler. which is included only for the purpose of completeness.
Chapter 4 places the wavelet theory in a concrete selling usinS Ille Haar scaling and
wavelet function. lbe rcst of the book builds on Illis material. To urKlerstand the conceptS in Ihis
chapter fully, the reader need to have only an understanding of the basic concepts in linear
algebra: addition and multiplication of vectors by scallUl;, linear independence and dependence,
orthogonal bases, basis set. vector spaces and fUllClion spaces and projection of vectorffunction
on to the bases. TIle chapter introduces the concept of nested spaces. which is Ille comer stone
of mulliresolUlion analysis. Dauhechies' wavelets are also introduced. The chapter concludes
wilh a note on the fact that moSt of the wavelets art fractal in nature and that iterative methods
are required to display wavelets.
Designing wavelets is traditionally camed out in the Fourier domain. Readers who are IlOl
experts in Fourier analysis usually find the theoretical arguments and terminology used quite
ntl
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
xviii • Pref:>C(: to the l-irst Edition
baffling and tOlally out of Ihe world. This book. the~fore. adopts a lime domain approach 10
designing. lltc: orthogonality and smoothness/regularity constraints are di~ctly mapped on to
constraints on the scaling and wavelet filter coefficients, which can then be solved using solvers
available in Microsoft Excel-a spreadst.eet package or a scientific computillion package like
MAn.AB or MOlhematica.
Engineers onen view signal processing in terms of filtering by appropriate filters. Thus.
Chapter 6 is devoted to eSlablish the relationship between 'signal expansion in terms of wavelet
bas.c:s· and the ·filter bank· approach to signal analysis/synthesis.
Chapter 7 discuss.c:s tile theory behind parametric wa,·elets in an intuitive way I1Ither than
by using rigorous mathematical approach. The chapter also discusses various methods of
plotting scaling and wavelet functions. The focus of Chapter 8 is 011 biortoogonal wavelets
which is relatively a new concept. To drive oome this concept to the readers, biorthogooality
is explained using linear algebl1l. The chapter then goes on to discuss the design of elementary
S-spline biorthogonaJ wavelets.
Chapter 9 addresses orthogonal wavelet design using the Fourier domain approach.
Chapter 10 is devoted to the lifting scheme which provides a simple means to design wavelets
with the desil1lble propetties. lltc: chapter also shows how lhe lifting scheme allows faster
implementation of wavelet decomposition/reconstruction.
Chapters 11 to 13 describe applications of wavelets in Image Compression, Signal
Denoising and Computer Graphics. The notations used in Chapter 13 are that used by
~searchers in this patticular area and could be slightly different from those in the rest of the
chapters.
To make the book more useful 10 the readers. we propose 10 post lhe teaching malerial
(mainly PowerPoinl slides for each chapter, IlIld MATLABlExcel demonstralion programs) at
the companion webs-ile of the book: www.umritG.tdultrnlpublictdionslwu.vrkts.
We earnestly hope Ihat this book will initiate several persons 10 this exciting and
vigorously growing area. Though we have spared no pains 10 make this book free from
mislakes, some errors may slill have survived our scrutiny. We gladly welcome all com:ctions.
recommeOOatioos, suggestions aOO constructive criticism from our readers.
K.P. SOMAN
K.I. RAMACHANDRAN
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Acknowledgements
First and foremost. we would like 10 express our gratitude 10 Brahmachari Abhayamrita
Chaitanya. who persuaded us to lake this project. and never ceased to lend his encouragement
and support. We thank Dr. P. Venkal Rangan. Vice Chancellor of the university and
Dr. K.B.M. Nambudiripad. Dean (Research}-our guiding Slars--who c,mlinunlly showed us
what perfection means and demanded perfection in everything Ihal we did. We would like to
thank, especially, Dr. P. Murali Krishna, a scientist at NPOL, Cochin, for his endearing support
during the summer school on 'Wavelets Fractals and Chaos' thal we conducted in 1998. It was
then Ihal we learned wavelets seriously. We take this opportunity 10 thank our research students
C.R. Nitya. Shyam Divakar, Ajilh Peter, Sal1lhana Krishnan and V. Ajay for their help in
simplifying the concepts. G. Sreenivasan and S. Soornj. who helped liS in drnwing the various
figllres in the textbook, dcserve a special thanks. Finally. we express our sincere gl3titude to the
editors of PHI Learning.
K.P.SOMAN
K.I. RAMACIiANDRAN
N.G. RESMI
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
The Age of Wavelets
INTRODUCTION
Wavelct analysis is a new development in the area of applied mathematics. 'Illey were firsl
introduced in seislTIQlogy to provide a time dimension to seismic analysis that Fourier analysis
lacked. Fourier analysis is ideal for studying stationary data (data whose statistical propcnics
are invariant over time) bul is nol well suited for studying data with transient events that cann<ll
be statistically predicted from the data's past. Wavelets were: designed with such non-stationary
data in mind, and with their generality and strong results have quickly become useful 10 a
number of discipl incs.
I.I THE ORIGINS OF WAVELETS-ARE THEY
FUNDAMENTALLY NEW?
Research can be thought of as a continuous growing fractal (see Figure 1.1) which often folds
bad: onlo itself. This folding back definitely occurred several times in tne wavelet field. Even
though as an organized research topic wavelets is less than two decades old, it arises from a
"ollstellat;on of related eonc:t:pts developed o""r 11. period of ne;orly two ""nturies, repell.ledly
redisrovered by scientists wbo wanted to solve techni"al problems in their various disciplines.
Signal processors were seeking a way to transmit clear messages o~r telephone wires. Oil
prospectors wanted a better way to interpret seismic traces. Yet "wa~lets" did 001 become a
oousehold word among scientists until the theOf)' was liberated from the di~rse appliCalions in
which it arose and was synthesiud into a purely mathenwical thCQf)'. This synthesis, in turn,
opened scientists' eyes to new .ppliCalions. Today, for example, w.""lets are !K)( only the
workhorse in computer imaging and animation; they also are used by the FBI to encode its data
base of 30 million fingerprints (see Figure 1.2). in the future. scientists may put w.~kt analysis
for diagnosing breast cancer, looking for heart aboonnalities (look at Figure 1.3) or predicting
the weather.
•
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
2 • Insighl inlo Wavelets-From Theory 10 Praclice
FIGURE 1.1 Fnoctal. Wt fold Net on i~lf.
FIGURE 1.2 An FBI-digilized left thumb fingerprint (1lIc: image on the left i. the original; the one on the
right is reronsuuccr:d from a 16:1 """'pI"'ssion.)
,.,
LO
-<
0.'
'" 0.' - Hcallhy
- H~an failun:
0.'
0.0 0.' 0.' 0 3 0.'
,
FIGURE I.J MuhifractaJ 'pec(fum of hcar1 heat oscillations. h is a ""ph of singularity mcasun: versus flllCl&l
dimension. Thi'i spWrum clplum I di ff~n[ kind of iofoonalion Ihill CIIlOOl. be cljlluml by
a '1nl"''''')' spectrum. Wn-.:1c1S an: used for muhifraclal spectrum estimation.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
The Age of Wavelets • 3
1.2 WAVELETS AND OTHER REALITY TRANSFORMS
Wavelet analysis allows researchers to isolate and manipulate specific types of p<lttems hidden
in masses of data, in much the same way our eyes can pick out the trees in a forest, or our ears
can pick out the flute in a symphony. Otle approach to understanding how wavelets do this is
to stan with the difference between two kinds of sounds-a luning fork and lhe human voice
(see Figures 1.4 and 1.5). Strike a luning fork and you get a pure tone that lasts for a 'ery long
time. [n mathematical theory, such a tone is said to be "localized" in frequency, that is. il
consists of a single note with no higher-frequency overtol1oe.'l. A spoken word. by cOntl1lSt, laslll
for only a second. and thus is "localized'" in lime. It is not localized in frequency because the
word is not a single tone: but a combination of many different frequencies.
f f f
V V
I'TGURE lA Graphs of the sound WllveS prodoced by • tuning fQr< (top) and thr spol.C1 wool "grea.ly"
(bottom) iliu!MItc the diffm:l"ICc bet'.ieen • tone I<x:alized in freqUfflC)' and one I<x:ali«<i in
ti..... "The tuning fork produces a .imp'" "si"" wa>·c".
1/
FIGURE 1.5 A w."" and a _velcL
Graphs of the sound waves produced by the tuning fork and human voice highlight the
difference, as illUlitrated here. 1be vibrations of the tuning fork trace out what mathematicians
call a sine wa~. a smoothly undulating curve that, ill theory, could repeal forever. In contrast,
the graph of the word "greasy" contains a series of sharp spikes: there are nO oscillations.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
4 • Insight into Wavelets-From Theory to Practice
In the nineteenlh century, mathematicians perfected what might be called the tuning fork
version of reality, a theory known as Fourier u1Ullysis. lean Baptiste loseph Fourier, a French
mathematician, claimed in 1807 that any repeating waveform (or periodic function), like the
luning fork sound wave, can be cllpressed as an infinite sum of sine waves and cosine waves
of various frequencies. (A cosine wave is a sine wave shifted forward a quarter cycle.)
A familiar demonstration of Fourie(s thcory occurs in music. When a musician plays a
note, he or she creates an irregularly shaped sound wave. n.e same shape repeats itself for as
long as the musician holds the note. Therefore, according to Fourier, the note can be separa~d
into a sum of sine and cosine waves. The lowest frequency wave is called the fundamental
frequency of the note. and the higher fn::quency ones are called o,·ertoncs. For example, the
note A. played on a violin or a nute, has a fundamental frequency of 440 cycles per second and
ovenones with frequencies of S8O. 1320. and so on. Even if a violin and a flute arc playing the
same note, they will sound different because their ovenones have different strengths or
"amplitudes". As music synthesizers demonstrated in the 1960s, a very convincing imitation of
a violin or a nute can be obtained by recombining pure sine waves with the appropriate
amplitudes. That, of course. is ellactly what Fourier predicled back in IS07.
Mathematicians later elltendcd Fourier's idea to non-periodic [unctions (or waves) that
change over time, rather than repeating in the same shape forever. Mosl real-world waves arc
of this type: say, the sound of a m(){or that speeds up. slows down. and hiccups now and then.
In images, too. the distinction between repeating and non-repeating patterns is imponant.
A repeatins pattern may be seen as a texture or background while a non-repeating one is
picked out by the eye as an object. Periodic or repeating waves composed of a discrete series
of overtonc:s can be used to repres.c:nt repeating (background) pauerns in an image. Non-periodic
features can be resolved into a much more eomplu spectrum of frequencies, called the I'-ourier
transform, just as sunlight can be separated into a spectrum of colours. The Fourier transform
portrays the structure of a periodic wave in a much more revealing and conccntraled form than
a traditional graph of a wave would. For elllmple, a rallle in a motor will show up as a peak
at an unusual frequency in the Fourier tnulsform.
Fourier transforms have been a hit. During the nineteenth century they solved many
problems in physics and engineering. This promirtence led scientists and engineers 10 think of
them as the preferred way to analyze phenomena of all kinds. This ubiquity forced a close
examination of the method. As a n::SUII, througOOut the twentieth century, mathematicians,
physicists. and engineers came to realize a drawback of the Fourier transform: they have trouble
reproducing transient signals or signals with abrupt changes, such as the spoken word or the rap
of a snare drum. Music synthesizers. as good as they are, still do not match the sound of eoncert
violinists because the playing of a violinist contains transient features--such as the contact of
the bow on the string-that arc poorly imitated by representations based on sine waves.
The principle underlying this problem can be illustrated by what is known as the
Heisenberg Indeterminacy Principle. In 1927, the physicist Werner Heisenberg stated that the
position and the velocity of an object cannot be measured CIlactly at the SIlIl"Ie time even in
theory. In signal processing terms, this means it is impossible to know simultaneously the exact
frequency and the exact time of occurrence of this frequellCy in a signal. In order to know its
frequency, the signal must be spread in time or vicc versa. In musical terms. the trade-off means
that any signal with a short duration must have a complicated frequency spectrum made of a
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
The Age of Wavelets • 5
rich variety of sine waves whereas any signal made from a simple combination of a few sine
waves must have a complicated appear:J.nce in the time domain. Thus, we can't e;t;pect to
reproduce the sound of a drum with an orchestra of tuning forks.
1.3 MANAGING HEISENBERG'S UNCERTAINTY GHOST
Over the course of the twentieth century, scientists in different fields struggled to get around
these limitations, in order to allow representations of the data to adapt to the nature of the
infomullion. In essence, they wanted 10 capture both Ihe low-resolution forest- the repealing
background sigr"lal- aoo the high-resolution trees-the individual, localized variations in the
background. Although the scielllislS wcre trying to solve the problems panicular to their
respective fields. they began to arrh'e at the same conclusion- namely. Ihat Fourier InlIIsforms
themselves were 10 blame. They also arrived at essentially the same solution. Perhaps by
spliuing a signal into components that were not pure sine waves. it would be possible to
condense the information in bolh the time and frequerocy domains. This is the idea that would
ultimotely be known as wa"l~lcts.
Wavelet transforms allow timc·frcQuency localisation
The first entrant in the wavelet derby was 0 Hungarian mathematician nanw:d Alfred Haar,
who introduced in 1909 the furoctions that are now called Haar ",..ave1ets. These functions
consist si mply of a short positive pulse followed hy a short negative pulse. Although the short
pulses of Haar wavelets are e;t;cellent for teaching wavelet theory, they are less uscful for moSt
applkations because they yield jagged lincs instead of smooth curves. For example. an image
reconstrtlcted with Haar wavelcts looks like a cheap calculator display and a Haar wavelet
reconslrtiction of the sound of a i1me is too homh.
From time to time over the ne~t several decades, other precursors of wavelet theory arose.
In the 1930s, the English mathematicians 10hn Linlewood and R.E.A.C. Paley developed a
method of grouping frequencies by octaves thereby creating a signal that is well localized in
frequency (il$ spectrum lies within one octave) and also relatively well localized in time. In
1946, Dennis Gabor, a British-Hungarian physicist. introduced the Gabor transform. analogous
to the Fourier transform. which separatcs a w""e into "time-frequency packets" or "coherent
states" (see Figure 1.6) that have the gre.1test possible simultaneous localization in both time and
frequency.
And in the 1970s and 19805. the signal processing and image processing communities
introduced their ()wn versions of wavelet analysis. going by such names as "subband coding:'
"quadrature mirror filters" and the "pyramidal algorithm".
While not precisely identical, all of these techniques had similar features. They
decomposed or transformed signals into pieces that could be localized to any time interval and
could also be dilated or contracted to analyze the signal at different scales of rcsolulion. These
prC<:urso(S of wavelets had one other thing in common: no one knew about them beyond
individual specialized communities. But in 1984, wavelet theory finally came into its own.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
6 • Insighl into Wa~lelS-From Theory 10 Pnctice
••
•
•
•
•
•
•
•
•
••
•- -~I't------- --,,------
FIGURE 1.6 Oocomposing lignal inlo t;"",·f~l>e1lCy alom~. BOllom of the pictun: ~ twOtime fn:qumcy
atom•. The signal and the time·f""lllCnc:y map i. """"'n "'->ve that.
1.4 HISlURY OF WAVELET FROM MORLET TO DAUBECHIES
VIA MALLAT
lean Morle[ didn't plan to start a scientific revolution. He was merely trying [0 give geologists
a beller way 10 search for oil.
Petroleum geologists usually locale underground oil deposits by making lood noises.
Because sound waves lnIvel through different materials at differenl speeill;. geologisl.'l can infer
whal kind of malerial lies under the surface by sending seismic waves inlO the ground and
measuring how quickly they rehourKl. If the waves propagate especially quickly through onc:
layer. it may be it salt dome. which can trap a layer of oil underneath.
Figuring oot just how the geology translates into a sound wave (or vice versa) is a tricky
mathematical problem. and one Ihat engineers traditionally solve with Fourier analysis.
Unfortunately. seismic signals contain lots of transienl.'l-llbrupt changes in the wave as it passes
from one rock layer to another. Foorier analysis spreads thal spatial information oot all over the
place.
Morlet. an engineer for Elf-Aquitaine. developed his own way of analyzing the seismic
signals to creale componenl.'l that were localized in space. which he called Wllvdets of constant
shape. Later. they WQUld be known as Monet wllvelets. Whether the components are dilated.
compressed or shifted in time. they maintain the same shape. Other families of wavelets can be
huilt by Illking a different shape. called a motm-r WIIvelel. and dilating. compressing or shifting
it in time. Researchers woold find that the exact shape of the mother wavelet strongly affeclS
the accuracy and compression properties of the approximation. Many of the differences between
earlier versions of wavelets simply amounted to different choices for the mother wavelet.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
The Age of Wavelets • 7
Morlel's method wasn't in the books but it seemed to work. On his personal computer. he
could separate a wave into its wavelet components and then reassemble them into the original
wave. But he wasn't satisfied with this empirical proof and began asking other scientists if the
method was mathematically sound.
Morlct found the answer he wanted from Alex Grossmann. a physicist at the Centre de
Pltysique ThOOrique in MlIIOeilles. Grossmann worked with Morlet for a year to confirm that
waves could be re.constructed from their wavelet decompositions. In fact. wavelet transforms
turned out to work beller than Fourier transforms because they are much less sensitive to small
errorn in the computation. An error or an unwise truncation of the Fourier coefficients can turn
a smOOlh signal into a jumpy onc or vice versa: wavelets avoid such disastrous consequences.
Morlet and Grossmann's paper. the first to use the word "wavelet", was published in 1984.
Yvc:s Meyer. currently at the Ecole Normale Suptrieure de Cachan. widely acknowledged as
one of the founders of wavelet theory. heard about lheir work in the fall o f the same year. He
was the first to realize the connection between Morlet's wavelets and earlier mathematical
wavelets. such as those in the work of Littlewood and Palcy. (Indeed. Mt!yer has counted 16
separate rediscoveries of the wavelet concept before Morlet and Grossmann'S paper.)
Meyer went on 10 discover a new kind of wavelet. with a mathematical property called
orthogonality that made the wavelet lTansform as easy to work with and manipulate as a Fourier
transform. C·Qrthogonality" means that the information captured by one wavelet is completely
independent of the information captured by aoother.) Perhaps most importantly. he became the
IlCXUS of the emerging wavelet community.
In 1986. St6phane Mallat (see Figure 1.7). a former student of Meyer's who was working
on a doctorate in computer vision. linked the theory of wavelets 10 the existing literature on
nGURE 1.7 Sephane Malla! (CMAP. Ecoic PoIyIcchniq.... 911211 Palais.eau Cedeo.. Fr'anc:e).
sUbband coding and quadrature mirror filters which are the image processing community's
versions of wavelets. The idea of multiresolution ana1ysis--that is. looking at signals at different
scales of resolution-was already familiar to experts in image processing. MaUat, collaborating
with Meyer. showed that wavelets are implicit in the process of multiresolution analysis.
Thanks to Mallat's work. wavelets became much easier. One could now do a wavelet
analysis without knowing the formula for a mother wavelet. The process was reduced to simple
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
8 • Insight into Wawlets-From 'Theory to Practice
operations of avel'llging groups of pixels IOgether and taking their differences, over and over.
'The language of wavelets also became more comfortable to e l~trical engineers, who embraced
familiar teons such as "'filters"', "'high frequencies" and "Iow frequencies".
The final great salvo in the wavelet revolution was fired in 1987, when Ingrid Daubechies
(see Figure 1.8), while visiting the Courant I nstitUl~ at New York University and later during
tlGURE 1.8 Ingrid Daub«hits (Prof...,.,.-, Ikpartment 01 M3lhema.tiC$. Princo:ton University).
her appointment al AT&T Bell Laboratories, discovered a woole IICW class of wavelets [sec
Figure l.9(b)] which were not only otthogonal (like Meyer's) but which could be implemented
using simple digital filtering ideas, in fact, using shon digital filters. The new wavelets were
almost as simple to program and use as Haar wavelets but they were smooth, without the jumps
of Haar wavelets. Signal processors flOW had a dream tool: a way to breair;: up digital data into
contributions of various scales. Combining Daubechies and Mal1at's ideas. there was a simple.
orthogonal transform that could be rapidly computed on modem digital computers.
,, /~,) ,
I o.,
,n 0
"-, -0.,
1
,
o , 3
(a) Haar wavelet (b) Daulxdlic. · 4-...p Muna( WlI>"ek(
FIGURE 1.9 (Cont.).
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
11< Ag' 0 rw "''"
, • 9
0'
0.'
0.2
0
/-0.2
- 0.4
- 0.6
-5-4 - 3 - 2 - 10 , , , .(c) GaIIsi... ".""lel
n GUR£ 1.9 GrapJu of .....·era! different types of wa."lea
The Daubechics wa'eleLS have surprising features-such as intimate connections with the
theory of fractals. If their graph is viewed under magnification, characteristic jagged wiggles
can be seen, no matter bow strong Ihe magnification is. This exquisite !;omplexity of detail
means, there is no simple formula for these wavelets. They arc ungainly and asymmeuic;
nineletnth-eemury mathematicians would have recoiled from them in hOllOr. But like the
Model-T Ford. they are beautiful because they work. The Daubechies wavelets turn the theory
into a practical tool that can be easily programmed and used by any scientist with a minimum
of mathematical training.
1.4.1 Different Communities of Wavelets
There are several instances of functions (see Figures 1.9 and 1.10) that can be used for
muhiresolution analysis of data. All of them are referred to as wavelets. Some of these instances
=,
• Dyadic lrans/arts and di/mes of Olle funC/ion: These are classical wavelets.
• Wavelet fHlclcets: This is an extension of the dassi,al wavelets whi,h yields basis
fullO;lions with bener frequency localizalion at the ,ost of slightly more expensive
tnmsfonn.
• Local trigonometric basu: The main idea is to work with cosines and sines defined on
finite intervals combined with a simple but very powerful way 10 smoothly join Ihe
basis functions at the end points.
• Mulliwavtlets: Instead of using one fixed fun"ion 10 lranslate and dilate for making
basis fun,lions. we use a finite number of wavelet functions.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
10 • Insighl inlo Waveku From Theory 10 Prac:ti~
,.•
"
,1----"1 I1
-<l.' L_~__-'-'-'---_~__---'
-0.8 -0.4 0 0.4 0.8
(a) Model _velet
,.•
,.,
,
" I
-<l.,
-. , , •(b) Mexican Hal .....""lel
FIGURE 1.10 Wavelets uoed in conlinuoul wavelet tranJ(O,TU.
• Second gene,flIion wlll.'elefS; Here: one entirely abandons the idea of translation and
dilation, This gives elClra flelCibility which can be used 10 construct wavelets adapted to
im:gular samples.
1.4.2 Different Families of Wavelets within Wavelet
Communities
Like humans, wavelets also live in families. Each member of a family has cenain common
features that distinguish each member of a family. Some wavelets aJ"e fQr CQntinuQUs wavelet
transform and OIhe1; are: for discre:te wavelet transform. Some of the families that belong to
'classical' community of wavelets are (see Figures 1.10 and 1.11):
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
g. • •The A of W vcLets 11
2
,
0
I
-,
o 2 3
• ,
nGURE 1.11 Coinel wavelet.
• Wavelets for continuous wavelet transfoml (Gaussian. Morlel. Mexican Hat)
• Daubechies Maxllat wavelets
• Symleu
• Coillcts
• Biorthogonal splille wavelets
• Complex wavelets
1.4.3 Interesting Recent Developments
• Wavelet based denoising has opened up other fields and important tcchniques such as
dictionaries and non-linear approximation. smoothing and reduction to small
optimizalion problems are real achievements.
• Wavelets have had a big psychological impact. People from many different Ilreas
be!;ame interested in time-frequency and time-scale transfonns. There has beCII a
revolution in signal processing. There is less specialization and the subjcct is now
opened to new problems. More than just a simple tool, wavelet ideas prompt new points
of view. Some of the best ideas aren't writtcn down. The big diffcrence will come from
new gencnuion rescarchers flOw growing up amidlll wavelet ideas.
• Wavelets have advan~d our urKIelltanding of singularities. The singularity spectrum
completely characterizes the complcxity of the data. Now We must go to an
undelltanding of the underlying phenomenon to get an equation from the solution.
Wavelets don't give all the ansWCIl but lhey force us to ask right questions.
• Wavelets can be used to distinguish coherent VefSUS incoherent pans of turbulence in
fluid flow. They give some information but don't entirely solve the problem.
• The results on regularity, approximation power and wavelet design techniques have led
10 significant developments in signal and image processing.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
12 • Insigh.t into Wavelets-From Theory to Practice
1.5 WAVELETS IN THE FUTURE
Wavelet ar.alysis is unquestionably one of the beSt achieveme:nts of matllematics in the twentieth
century. Its initial applications were mainly in sparse signal representation and denoising of
signal and images. Probably, it was due to the fact that the theory was hard to understand at
that time. In recent years wavelets have spread in many fundame:ntal sciences other than
mathematics. such as me:dicine, biology, geophysics. physics, mechanics. economics, etc. This
is e,·ident from the title of the books that are appearing.
At present there are more Ihan hundred books on wavelets most of which are on
mathematical aspects of wavelet theory. Slowly but steadily many books are appearing in
specific application domains. Following are some: of them.
Wavelets in medicine and biology (by Akram Aldroubi and Michael Unser,
CRC Press, 1996)
This book explores application of wavelets in me:dical imaging and tomography biomedical
signal processing. wavelet based modeling of problems in biology.
Wavelets in physics (edited by J.C. van den Berg, 1999)
This book details the application of wavelets in many branches of pnysics such as acoustics,
spectroscopy. geophysics. astrophysics, fluid mechanics (turbulence), medical imagery, alomic
physics (laser-atom interaction), solid state physics (structure calculation).
Wavelets in chemistry (editor Beata Walczak, Elsevier, 2000)
This book is an introduction to wavelets intended for chemiSts. It covers all impor1ant aspects
of wavelet theol)' and presents wavelet applications in chemistry and in medical engineering.
It is addressed 0 analytical crn:mists. dealing wilh any type of spectral data, organic chemists,
in"Olved in combinatorial chemistry, chemists involved in chemome:trics and engineers involved
in process control.
ChcmomClrics: From basics to wavelet transfonn (by, Foo-Tim Chau, Vi-
Zeng Liang, Junbin Cao, Xue-Cuang Shao, WHey, 2004)
This book explores the use of wavelets in ·chemome:trics based signal processing'.
Wavelets for sensing technologies (by K. Chall, Cheng Peng. Arlech House
Publishers, 2003)
This reference book focuses on the processing of signals from Synthetic Aperture Radar (SAR).
Specific remOle sensing applications presented in the book include noise and clutter reduction
in SAR images. SAR image compression. texture and boundary enhancement in SAR images,
direction·al noise removal and general image processing.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
'TlIe Age of Wavelets • 13
Wavelet and wave analysis as applied to materials with micro or nano-
structure (by C. Cattani and J. Rushchitsky, World Scientific Publishing Co..
200?)
This book explores physical wavelets as solution of certain POEs anslng in solitary wave
propagation in elastic dispersive media. Three different types of physical wavelets. namely.
Kaiser physical (optical and acoustic) wavelets. Newland harmonic wavelets and elastic
wavelets are discussed. Optical wavelets own this name be<:ause they satisfy Ihe linear wave
equations of optics in the simplified form of Maxwcll electromagnetic equations. The acoustic
wavelets were proposed as those wavelets satisfying tlK: linear wave equations in acoustics.
Harmonic wavelets were suggested by Newland. The Newland hannonic wavelets can be
referred to physical family of wavelets for many reasons, btu mainly be<:ause they are especially
proposed for the analysis of physical problcms on oscillations.
Ultra-low biomedical signal processing: An analog wavelet filler approach
(or pacemakers (by Haddad and Serdijn. Springer, 2008)
In ultl1il low-power applications such as biomedical implantable devices. u IS not suilllble to
implement the wavelet transfonn by means of digital circuitry due to Ihe relatively high power
consumption associated with the required NO converter. Low-power analog realization of the
wal"elet transfoml enables its application in vivo, e.g.. in pacemakers. where the wavelet
transform provides a means to extremely reliable cardiac signal detection. The methodology
presented in this book focuses on the development of ultra low-power analog integrated circuits
that implement the required signal prucessing. taking into account the limitations imposed by
an implantable device.
Wavelets in wireless communication
Though there is no exclusive book on applicalion of wavelets in communication. there are
hundreds of research papers available on the topic. Wavelets give a new dimension to the
capacity of wireless communication. It provides "Waveform diversity'". to the physical
diversities cum:ntly exploited. namely. space (muhi-antenna wireless communication system),
frequency and time-diversity. Signa.! diversity which is similar to spread spectrum systems.
could be: exploited in a cellular communication system, where adjacent cells can be designated
different wavelets in onJer to minimize inter-cell interference. In addition wavelets provide the
following:
(a) Sc",i-u,birrury divisi<m of ,I.e sigll"l spuc" Ulld ",,,/,i"'lt: S1sr""'11: Wavetet
transform can create subcarriel'S of different bandwidth and symbol length. Since
each subcarrier has the same time-frequency plane area. an increase (or decrease) of
bandwidth is bound to a decrease (or increase) of subcamer symbol length. Such
characteristics of the wavelets can be: exploited to Create a muhirate system. From
11 communication perspecti·e. such a feature is favourable for systems that must
support multiple data streams with different transport delay requirements.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
14 • Insight into Wa~le!s-From Theory to Practice
(b) Flexibility ...irh ,ime-frequency ,i/ing: Another advantage of wavelets lies in their
ability to arrange the time-frequency tiling in a manller that minimizes the channel
disturbances. By nexibly aligning the time.frequency tiling. the effect of noise and
interference on the signal can be minimiZo!d. Wavelet based systems are capable of
ovt'rcoming kTlOwn channel disturbances at the transmitter. rather than waiting to
deal with them at the receiver. Thus. they can enhance the quality of service (QoS)
of wireless systems.
(c) Semi,i"iry ro chmmei ~ffecl5; The performance of a modulation scheme depends
On the set of waveforms that·the carriers use. 11le wavelet scheme. therefore. holds
the promise of reducing the sensitivity of the system 10 harmful channel effects like
inter-symbol interference (ISO and inter-carrier interference (ICI).
(d) Vltra widdxmd llpplicmiolls: Impulse radio. or ultra wideband (UWB) radio. is a
promising new technology for wireless communicruions. Rather than modulating the
information on a carrier. the data is transmiued using a coded series of very narrow
pulses. carrying information in the time and the frequency domain. Wavelet bases are
good candidates for these pulses.
Figure 1.12 shows communication engineering areas where wavelets are fiBding new
applications.
ChanllCI clumlct.erizluion
I. ChanllCl modeting
2. Electrorroagnctic compuuuions and "Jlltnna desis.n
3. SpeW estimation
COlnitive rmio
Inl~11i~nt co;>mmunkation
syst~ms
Uhra widcband communication
l. Impulse: radio
2. Multil>and OFDM
Muhiple access communical;on
t. COMA
2. SCDMA
3. MC-COMA
[nlt"l'f=e
1. Signal denoi.inl
2. O'lta amval estimation
3. Intcrf"= mitigation
4. ISI. ICI mitil'llion
Modulation aod rnuhipte.in,
l. Wa,~ 5ILapin,
2. Siolle carrier modulalion
3. MUhi-carrier modulation
4. FllIC1al modulation
~ . Multiplex;n,
Networking
1. Power con..........tion
2. Traffic proiction
). Net"'-ort< tnlffit modeling
4. Data rccon<IJUCtioo
5. Disuibull'(] data processinl
FIGURE 1.12 WaV(:iets in wireles~ communication.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
TIw: Age of Wavelet. • IS
In summary, we may safely bet that wavelets are here to stay and they have a bright future.
Of course wavelet do nOI solve every difficulty, and must be continually developed and
enriched. We can expecl proliferation of specialized wavelets each dedicated to a panicular type
of problem and an increasingly divcrse spectrum of applications.
1.6 WHAT HISTORY OF WAVELET TEACHES US
When asked 10 justify the value of mathematics, mathematicians often point out that ideas
developed 10 solve a pure mathematical problem can lead to unexpected applications years later.
BUI the story of wavelets paints a more complicated and somewhat more interesting picture. In
(his case, specific applicd research led to a new theoretical synthesis, which in lurn opened
sciemists' eyes to new applications. Perhaps the broader lesson of wavelets is that we should
not view basic and applied sciences as scparate endeavours. Good science requires us to see
both the theoretical forest and the practical trees.
SUMMARY
Though wavelet is an organized research IOpic. it is only two decades old, and has been in use
for a long time in various disciplines under different names. Morlet and Grossman were the first
10 use the word 'wavelet'. Stephan Mallat brought out the relation between wavelet
methodology used by Morlel and filter bank theOI)' used in image processing applications. TIle
greatest contribution came from Ingrid Daubechies who put tile whole theory on a strong
matherruuical foundation. Wavelets are now emerging as one of the fastest growing field with
application ranging from seismology to astrophysics.
Suggested Further Readings
Amara's wavelet page htlp:llwww.amara.com/currmll....ol·e/u/uml.
An Introduction 10 wavelets: hllp:llwww.amara.comll£££K"(JI.tII£££....al.eiel.hlm/f
Dilubel:lies, I.. Where do wavelet come from? A perwnal point of view, Proceedings of liltl
1£££ Specia/Issue on W'lI'e/eIS, S4 (4), pp. SIO-SJ3. April 1996.
Sweldons. W.. Wavelets what neltt? Procudill8s of 'he IEEE. Vol. 84. no. 4 April 1996.
Wa'elets; Seeing the Forests and the Trees, hltp:llw"'l'o'.M)"om/discol'ery.orgl.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fourier Series and Geometry
INTRODUCTION
Deemed one of the crowning achievements of the 20th Century, the Fourier series has
applications Ilia! are far n:aching in varioLls fields of science and mathematics. The Discrete
Fou';". "ansfurm i. one p"I1icular tool widely u&ed in today'~ age of compulers and solid state
electronics. From graphic equalizers in stereos 10 the moSI advanced scientific sampling
software, the userullless of this mathematical feal is ISlolJllding. Most of the readers of this book
might already know this fact but many of them may nor be knowing tha! Fourier series. as a
mathematical tooJ in analy:ting signals, h:J.S strong connection wilh geometry. In this chapter our
aim is to understand the theory of Fourier series from a geometrical viewpoillL
We assume that you are familial" with vector algebra and co-ordinate geometry. If you are
really comfonablc in using Ihc!iC topics, you can very well understand what Fourier series is,
computation and interprelalion of Fourier series coefficients, Fourier transform. discrete Fourier
transform. fast Fourier transform. etc. l1iere exists a strong analogy between what you do in
vector algebra and what you do in signal processing. This analogy will help you to visualize
and give interpretation to the processes and output of the processes that you do on signals.
Development of tllis geometrical mental picture about signal processing is the main aim of this
chapter. Einstein once said "Imagination is more imponant than knowledge". Signal processing,
which many students consider as dry and non·intuitive. demands imagination and some form
of abstract thinking from the pan of students. In fact. it requires only that. Once )'OIl develop
a conceptual link between geometry (vector algebra) and signal processing, you need nO(
remember any fonnula and every formula that you come across will become transparent to your
mind. Here we will refresh concepts from "Ulor spoce. The detailed exposition of vector space
is given in Appendix A. In this imroductory chapter, we won't try 10 be 100% mathematically
precise in our staternems regarding vector space. Instead our aim is to relate geometry and
Fourier series in an intuitive way.
..
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fourier Serie$ and Geometry • 17
2.1 VECTOR SPACE
Any vector in 3-dimensional ,;pace can be represented as V = (JI + bl + et. I, J.k are unit
vectors in three onhogonal directions. This orthogonality is expressed by requiring the condition
that r.J=k· J= r.k = O. The orthogonalily condition ensures that lIle representation of any
- - -''eClor using i. j. le is unique.
2.1.1 Bases
We call i.].k as bases of space 9{1. By this we mean that any vector in 9{l can be represented
using I. J. k vectors. We also say i. J. k span the space 9{1. Let V= al + b] + et be a vector
in 9{1. Continuously changing scalar a. b. c: we will go on get new vectors in 9{1. We imagine
that. set of all such ,'eClors constitute the vector space 9{1. We express this truth by
Span (al + b] + ct) iI 9{1
.,'
2.1.2 Ortnonormalily
Norm of a vector V =al + b] + ef is conventionally defined as Ja2
+ b2
+c2
and denoted
as IV I. IV I is equal to Jv. ii . We interpret this quantity as the magnitude of the vector. It
must be noted that mher definition,; are also possible for norm. What is interest to us is that our
basis vectors of 911. Le.. T. J. k are having unity nonn. Hence the name unit vectors. They are
also onhogonal. Therefore. they are called orthonorma1 vectors or orthonorma1 bast'$. How
does it help us'! Given a vector V= al + bJ + cf. we can easily (ell to which direetion I,], k.
the vector V. is more inclined by noting the value of a. b. c.
2.1.3 Projection
Given a vector V how shall we find its normal component vectors or in other words how shall
- - - -we find the scalar coefficients a. b and c'! Wc project V on to bases i , j, le to get a, b and
e. This is the direct result of orthogonality of our basi,; vectors. Since V = (JI + bJ + ef,
Y' I =«JI + bJ + ef)· I =(J
ii· J = «JI+bJ+ek)·J=b >od
ii· f =(al+b} +cf)·k=c
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
18 • Insight inlo Wa~cLcts-From llIeory to Pmctice
So projecting vcctor 0 on to a base of space 9J (spanned by a sct onoonormal vectors) gives
the corresponding coefficient (component) of the base. There is a parullcl for this in Fourier
scries analysis.
With these ideas firmly cngral'cd in your mind. you arc ready to understand and visualize
Fourier series.
2.2 FUNCTIONS AND FUNCTION SPACES
In vcctor space. we represent a vector in terms of onoogonal unit vectors called bases. Fourier
serics is an extension of this idea for functions (cleclrooics engineers call them as signals). A
function is expressed in terms of a set of orthogonal funclions. To com:13le with ideas in
geometry. we now need to introduce the concept of onhogonality and norm with respect to
functions. Note Ihat. funclion is quantity which varies with respect 10 one or more running
parameter. usually lime and space. Orthogonality of functions depends on multiplication and
integration of functions. Multiplication and integration of function must be toought of as
equivalent to projection of a vector on to another. Here comes the requirement for abstract
thinking. mentioned at the stan of tile chapter.
2.2.1 Orthogonal Functions
Two real functions fl(/) and f2(/) are said to be onhogonal if and only if
-JJ;COiJ(I)dl =0 (2.1)
-
Ponder over this equation. What we are doing? We are first doing a point by point
muliiplicll1ion of t...."Q function. TIlen we are summing up the area under the resulting funclioo
(obtained after multiplication). If this area is zero. we say. the two functions are onhogonaJ. We
also interpret it as similarity of functionsfl(t) andfil}. Ifft(f) and fit) vary synchronously, that
is, ifft(l) goes upil(t) also goes up and ifft(t) goes downil(t) also goes down then we say the
two functioos have high simi larity. Otherwise. tlley are dissimilar. Magnitude or absolute value
-of Jf t(l)f2(Odl will be high if they "ary synchrOllOusly and zero if they vary perfectly
asynChronously.
For example consider two functions ft(t) = sin t and 12(1) :: cos I for f in the interval
(0,211"). Figure 2.1 shows the two functions and their point wise product. Observe that the net
area under the cur'C sin I cos f is zero. That is. the llfCa above the I axis is same as tile llfCa
below the I axis. For these two functions, it ean be easily shown that. for I in the range of
multiples of 211". area under the product curve is always zero. TIlerefore. we say that sin I and
cos I are onhogonal in the range (0. 211). Specification of range is very important when yOU say
orthogonality of functions. Because the deciding factor is area under the curve which in turn
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
o.
o.
o.
,
8
6
•,
0
~ o.
•,< -0.
..-0.
-0.
,
•
6
8
,
,,,
,,,
I
o
,,,,,
/"
'J,
/, ,
V ,,,
 / '.
, •
/ (
,,,
••,,
I, I
 i
V,,,,,,
6
Fourier Series and Geometry • 19
f
,,,,,,
 //
I
" iI' I
I  . I
j ,  I
, .
_____ sinl
__._ .in 1 • cos 1
---- ,~ ,
, ,, ,,, ,
 , ,, ,
8
" 12 J4
n GURF. 2.1 Qnh<>gona] fUI>(:'i""•.
depends on the range of inlegralion. Now here observe Ihe varialion of sin t and oos /. Whcn
sin I is increasing, cos I is de<:reasing and vice 'crsa. For range of I in multiples of 2K, the two
functions considered are said to be ortllogonal.
The goometrkal analog of f It (t) 11 (/) d/ '" 0 is the dot product, ; .J'" O. For the given
,
interval. when!,(t) is projected on to!ft) (proje<:tion here mearu multiplying and integrating).
if the result is tero • we say Ihe IWO functions are orthogonal in the given interval. Clearly look
al Ihe analogy. Taking B do! product in ve<:tor space (multiplying corresponding terms and
adding) is equivalenl to multiplying and inlegrating in Ihe funclion space (point wi~
multiplication and integration). The dOl product is mro;;mum when t'NO vectors are oollinear and
zero when they arc at 90". This analogy will help )'ou to casily visualize and inlerprel various
results in signal processing.
2.2.2 Orthonormal FUllctions
Two real fUI>C,j<>"s/,O) a",I/1(1) a", said ,0 be onhonormal if and only if
-and f[;(t) [;(t) dl '" I. for i:::: 1. 2.
-
2,2.3 Function Spaces
Just like unit orthogonaJ set of vectors span vector spaces. orthogonal set of fUilCtions can span
function spaces. Consider the following functions defined over a period T:
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
20 • Insight into Wavelets-From Tlleofy to Practice
Let us assume that w'" 2l1fT. Then (2.2)
(i) u(t)= I forO S t S T
(ii) cos " all. n '" I. 2. ...: 0 S 1 S T
(iii) sin 1I!l1l. n = 1. 2. ...: 0 S t S T
Here we have infinite number of (unctions of sines and cosines. These sets of functions
Iln: mUlUally onhogonal in the interval (0. 1). Let us verify the truth through some representative
examples.
Let us take functions u(t) = I and sin WI for 0 S t S T. These functions and their product
function are shown in Figure 2.2. Point wise multiplicar.ion gives the Il'sulting curve as sin (l)/.
I> j
o·i~ T
•
I.,
,
0.'
V ~0
/ T-0.'
"--,
-I>
" ,
03
1/
"'"0 T
-03
"- /'-,
- 1.5
FIGURF. 2.2 OnI>ogonalily of functiofu.
What is the area under the curve sin 0)1 in the interval (0. 7)'! Obviously i!.ero. Mathematically.
T T
Ju(t) sinllJl dl = fl ·sin(2mfT)dl = 0
,.,
We say U(I) = I is orthogonal to sin WI in (0. T ). Do not forget the Il'lar.ion between wand T.
Simi larly we can show that u(t) '" 1 is onhogonal to cos rut in (0.7).
In general.
T
Ju(t) s in nox dl '" 0 al50
,.,
T
fIl()) cos nox dt = 0 for all n '" I. 2•....
,.,
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fourier Series and Ge<>mctry • 21
Consider cos Wf and sin Wf in (0. n. Figure 2.3 shows the two functions and
COITespondillg fuoction obtained after point wise multiplication. Area abo'e and below of the
2
, / -
"-
~ .,
0 T
"- /'-,
-2
•
I'
,
0.'
/ "".nQW
0 T
-0.'
"" /-,
- u
0.6
(0.<
om "" 00$ fUr
0.2
0 T
...(1.2
V V-0.'
-0.6
)'IGURE 2.3 !'Iou of sin WI. 00$ WI and sin (Qf cos QW.
resulting curve is same implyillg that the net area is zero. This ill turn imply that cos rut and
sin rut in (0. n are orthogollal.
Generalizing.
Similarly.
,
fsinllM cos mWfdt = 0 . for all m. n = 1,2.3.....
,..
,
Jsill nM sill mM dl = 0
,..(for all illteger m and n such that m ~ /1).
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
22 • I"sight inl(l Wavd"i5--From 'Theory tu Pra.c:licc
Alw
,
JcosnaN cosmaN dl =0
,..(for all integer m and n such that m ~ n).
Thus, we have a set of functions defined over (0, n such thal they arc mutually
orthogonal.
2,2.4 Orthogonal Basis Functions
In vector algebra, we know that the unit orthogonal vectors t, j, k span the space :Xl. This
imply Ihat any veclor in :x, can be represented by unit vectors 01" bases i. j, k. Similarly the
sel of functions,
(i) u(l)= I forOSI ST
(ii) cosnnn. n = I. 2. ...: 0 S 1ST
(iii) sin nW/. n = 1. 2....; 0 S 1ST
defined over 0 S t S T span a function space and we denote il as L1 (set of square integrable
funclions). Most of the practical signals or functions are assumed 10 belong in the space of Ll.
This means Ihal. practically any continuous signal in the interval (0. n can be represented using
Ihe above base•. ~tplhematicany.
f(t) E L1 = f(t) = 00 + 11'1 COSM +"1 COS2M + ...
+ ~ sinaN + ~ sin2aN + ... (2.3)
Lel us look at the concept from another angle. Consider Ih" sumj(t) for each / in the range
(0. T) given in Eq. (2.3). Choose a set of coefficients Do. al..... and bl. bz, .... and make the
function (or signal) f(l> in the range (0. T). Now go on choosing different set of coeffici"nts 10
gel a different signal. The set of all such possible signals generated is our signal space Ll in
the range (0. 1). Practically all smoothly varying fUllCIions that you can imagine in Ihe interval
(0.1) belongs 10 Ll. Or in other words. practically all smoothly varying functions thal you can
imagine in Ihe illlerYlll (0. T) can be represented by Ihe equation:
f(t) = 00 + 11'1 cos ea + al cos 2ea + ... + bt sin aN + bz sin 2M + ...
The number of coefficients required 10 represent a signal depends Qn bow smooth and how
fast your signal is varying in the range (0. T).
2.2.5 Orthonormality and the Method of Finding the
Coefficients
The problem that we are addressing is finding the coefficient set ao. 11'1. bl. 11'1, b-:, etc. In veclor
algebra. onhooonnalily of basis vectors allowed us 10 find the component of a vector along a
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fourier Series and Geometry • 23
panicular base by projecting the vector on to that base. The propeny that lil =Ijl =1.&:1 " I
where Ii 1=.Jt·7 and 7.]. k are onhogonal implies thal for a veclor If e 9). if
If· i '" a. If· ] = b, V· k =c Ihen V '" ai + b] + ek . [I is a unique representation using Ihe
bases i.]. k. To reilerale the principle, 10 find a panitular coefficienl, project the vector on 10
the torresponding basis. 1be same mental pittuTe is applicable to Founer series representation
of signals.
In the last section. we have foufld out a set of functions that are onhogonal among
themselves in a given range (0. n. These functions are onhogonal but we haven't thccked
whether they form an orthooormal set. In function spa<;e, onhonormalily of two functions 11(1)
and h(l) in (0. T) requires that
, , ,
f/i(/) 12(1) dl = o. fft(t) 11 (t) dl '" 1
, ,
and ff 2(t) f 2(t} dt '" I
,
Consider Ihe basis funttion sin nw.
f' . . d f'· ['~'J .['~'Jd TsmnM smlltIX 1 = SIn T sm Tt'" 2"
, ,
The result is a function of T and is IlOI equal to I. For any integer n > 0, the integral is TI2.
So. we say our sine bases are not normalized.
Similarly it can be easily SMwn that,
jtosnaw COS lItIX dl =jcose;nt)cos(2-;'1) dl = ~
, ,
for 11 = 1.2.....
So our cosine bases are also oot nonnalized bases. What aboul our finlt basis function (which
is also called OC base) u(t) = I. in (0. T)'I
A,.
,
f[2 dt = T. U(I) is also not normalized.
,
Therefore, none of our bases are normalized. So how shall we normaLize our bases? Multiply
all tile sine and cosine bases with J2fT and u(t) '" I with M.
N~.
f' = = . ' f' .['''''J.[''''J 'T...,2/TsinnM...,2/TsmnMdt= T sm T sm T dl "'T ' "2'" I
, ,
, ,
f= = 2f [2"'''J [2"'''J 2T...,2fT cosllM...,2fT cosnM dl = T cos -T- cos -r dl = T . 2" = 1
, ,
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
24 • Insight into WavdelS-From 1brory to Practice
, ,
JJIIT../Ifrdt:o ~ fl dt:o ~ ' T=I
, ,
The set of nonnaliz.ed bases of Fourier series are:
{~, #cos M, #sin M, #cos 2ar, Hsin 2C1X • .•. , gcos IIflJ1. #sin IlflJl ••. .•}
All these functions are defined in the interval (0. 1) and ro = 21f1T.
Let us now take a functionf(t) ELl defined in the interval (0.1) and express il using the
nonnalized sine .and cosine bases defined over the same period (0, T). NOIe that the length of
bases are same as the length of the signal. Also note that the furKl.amcntal frequency of the
sinusoidal (also 'oosinusoidal") series is given by flJ:o 21f1T. (This is something which students
tend to ignore or forget and hence the repetition of the fact in many places.)
f
,I , f2 " f2 .(1) :0 Or! 7r + at V"Tcos ar + "I V"TStn fa + ..•
'H 'H'+ a - cos IlCIX + b - Sill liar + ...
W T · T
(2.4)
Here 00. al' ai, .... bj. bi.... are called normalized Fourier series coefficients.
Now. given th3!. such a n:presentation is possible. how shall we find out the nonnaliU!d
coefficients. The answer is, projeaj(l) on to the corresponding bases. Suppose we want to find
a;. Then project fit) on to J21T cos IlltW. that is,
a~ = jf(t)H cos mu dt
,
Note the analogy. In vector algebra, to find a component coefficient, we project the vector on
to the corresponding unil vector (base). Let us verify the troth through mathematical
microscope. Multiplying both sides of Eq. (2.4) by J21T cos Ilwt and integrating over (0. T).
we obtain
,
Jf(t )·hlT COSIIM' dt
,
,
:0 aO ~. JJUT cos mm dt
,T ,
,
+ a; JJ2fT cos wtJ2IT cos 1l{J}l df
,
,
+ hi fJ21T sin OX J21T cos lIax dt + ...
,
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fourier Series and Geometry • 2S
,
+ a~J../2/T COS nM ../2/T COS nM dr
•,
+ b~ J../21T sin nM ../21T sin nM dr + ...
•= /I'
•, ,
Thus. J10) ../2fT cos nM dr = a; since J../2fT cos nM ../2fT cos nM dr = I and all OIher
• •integrals vanish because of orthogonality property of tile bases.
,
Similarly b~ = Jl(r) ../21T sin nM dr. That is. project 1(1) on to the corresponding
•normalized base.
What about <lQ1 ProjcctJtt) on to the base w(r) = .JlfT defined over (0. 7). giving
,
~ = Jf(l)M dr
•
Well. we ullderstood how to get normalized coefficients in Fourier representation of signals. To
get a particular coefficient simply project 10) on to the com:sponding normalized base.
Projection in function space meaRS poiRt wise multiplication and integration of tile functions
concerned over the defined period.
After obtaining the IIOnnalized coefficients. we substitute in Eq. (2.4) 10 get Founer series
representation of the signal. Let us simplify Eq. (2.4) to avoid scalars ..J2IT and .JIlT.
Let <lQ IfJT =I/o so that
, ,
<lo = .JlfT<lQ =.JlfT J./liTI{t) dl =~ Jl(t) dl
• •
(Note that we substituted the Cltpression fo r <lQ to get /10-)
Similarly let o~ .J2IT = 0•. Then
, ,
a. =../2IT 0; =.J2IT J1(1).J2/T cos nM dl =: JI(t) cos nM dt
• •
, ,
b. = J21T b~ = .J2IT J1(1) ../2IT sin nM dt = : J/(I) sin ntu dl
• •
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
26 • JruiShl inlo Wa~lcts From Theory to Practice
Thus. we can rewrite Eq. (2.4) as:
J{I) "" ao + al cos rut + bl sin rut + ... + a. cos new + b. sin nrut + ... (2.5)
when:
,
~= ~ fJ(r)dl
•,
a.=: f f(t) cos nar dl. n = 1,2, 3, ....
•,
b. = ; ff(t) sin 'laM dl. n = I. 2. 3.....
•
These are the formulae given in most of the textbooks.
2.2.6 Complex Fourier Series
We know that
N~,
a COSM+" sinM= (al -jb,) tiU + (al +jb,) t~ja
I '1 2 2
Let us denote C =(al - jb,) and C =(al + jb,)I 2 ~I 2
The Fourier series can now be rewrittcn as:
(2.6)
(2.7)
(2.8)
(2.9)
Note that the formulation involvcs complcx cxponcntials and the concept of negative
angular frequcncy -ru. 1ltere is nothing imaginary in any signal and there is no negative
frequency. Complex exponcntials facilitate easy manipulation of multiplication and integration
involvcd in thc calculation of Fouricr serics coefficicnts than multiplication and integration
using plain sine and cosine fUlI(:tions. Let U$ filS! try to visualize t}oH, when: fI) = 2trrr and T
is the period of our signal.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fouricr Series and Geometry • 27
By Euler's fannula,
~OA = cos (J)l + j sin (J)l
We can thiok of this fUlICtion as 11. 'bundle' of sine and cosine fUlICtion kept togethcr side by
side such that they do not 'mix' with each other. The fUlICtion of j cao be thought of as a
separator indicating that the two parts. I'iz. cos 001 and sin oot are orthogonal.
2,2,7 Orthogonality of Complex Exponential Bases
The set of bases. (I. ~. t-iu ... " ejoOA. e-jolll ... ,,) are orthogonal over the interval [0, n,
where 00 = 21rIT and T is the duration of the signal which we are representing using the bases.
However. there is a twist with respect to interpretation of orthogonality and projection of a
fUlICtion on to a complex function. To make you prepared fOf" that twist and to lICCept that twist,
let us consider If.: aT + bJ + ek. an element of ')1. n.e nonn of this vector (or magnitude) is
obtained by projecting V on to itself and then taking the square root of resulting quantity. That
;,
But in case of a complex number,
l=a +jb
to find the magnitude. we have 10 multiply l with its complex conjugate l and then take the
square root. In case of real function, say f(t), we find square of its nonn by
Jf(t) f(t) df
In case our function is compleJO, say Z(f), its square nonn is given by
Jl(t) l(f) df
This has implication in the formula for projection of a fUlClion on to a complex function.
We must multiply the fil'$t function with complex conjugate of the second functon a~ then
integrate. With this twist in mind, we will prove that the set of functions u(t). t"", e-JfM
.....
ej<lu. t - j<lu, ... ) are orthogonal. We denote the projection offt") on to h(f) as (fl(f), fit».
CIISe I : u(t) = 1 and el-', for 0 S; t S; T and 00 = 21f1T
, , ,
("(f), t i.) '" JI · ti-l< dr = Je-j.>tt dr = Je-j(lmff) dr '" 0
, _0 0 (I
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
28 • Insight inlo Wavdet5 From Theory to Practicc
CtI$r 2: tj<t·" and t-il-' where 11 and m are positive integers.
, , ,
(tftttU , t -J"') == JrJou t J... dr == Jd<.''')'' dr =Jd{o+..~lJfIT), dl = 0
•• 0 0 0
CtI$e J: ep..., and eJ...,' where 11 and m are either positive or negative integers with m ~ 11.
, , ,
(~. ~) = J~M t!/ow dr = f~.-.) ..dt = Jeil.-",){2JffT)' dt ". 0
• • 0 0 0
Cases l. 2 and 3 prove that the functions are orthogona!. What about orthononnality of
comple" e)[ponential functions?
Since.
>od
,
(l!'*", r) = Jr"'Mt ftttU dt =
...
,
,
f]· dl=T
•
(U(I).U(I» = fI·dl =T. the functions are not orthooonnal
•
The functions though orthogonal do not salisfy the requirement of orthononnality. So we
multiply the functions with a scalar to make them onhOflOrmal WllOng themselves.
For unit function and comple" uponentials, scaling constant is .fliT. Therefore,
orthonormal bases are:
We denote the signal space generated by these bases as L2. Now. any signal which is an element
of Ll can be written as:
To get a particular coefficient, we project f(l} on to the corresponding base.
I.e.,
,
Col = ff(I).jlfT u(t) dr
•,
C; = Jf(I).JIiT r-;- dr
•
(2.10)
(2.11)
(2.12)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
I'ouricr Series and GeomcU'y • 29
,
C~. = f1(1) JilT t i- dl (2. 13)
"This resull follows from the OJIhonormality of bases used in (2.10).
Now, we simplify the expn:ssion given in Eq. (2.10) by introducing new constants.
Let Co = coJIiT
Since
Therefore,
,
Co = fl(lr/lfi 1.1(1) dt
", , ,
Co = J IlT ff(t).fliT 1.1(1) dr = ~ ff(t ) uet) dr = ~ ff(t ) dt
" " "Similarly
,
C. = c;JIiT =~ f1{I)e-i'fM dl
",,' ,
C. = C:.JUT = ~ fl(rY- dr
"
Thus, we oblain complex fourier series in slandard form:
with the coefficients dcfiled in Eqs. (2.14) to (2.16).
(2.4)
(2. 15)
(2.16)
(2.17)
This completes formal inU"Oduction to Fourier series. The n:adcr who is nO!: familiar with
advanced Founer methods can skip Chapter 3 and proceed reading Chapter 4.
SUMMARY
It is no undcrstatementthat the Founer senes especially discrete Founer transform is considered
one of the crowning achievements of the 20th Century. The theory of Founer series has strong
connection with geometry or at lcast we can understand the Founer transform thoory from a
geometrical viewpoint. We h3d the honour and a privilege to bring it to the reader in this user
friendly, low fat and no cholesterol form. In the forthcoming chopters on wovelets, this
geometrical viewpoint will help you to digest the theory in a beller way. It will surely prevent
intellectual diarrhoea.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
30 .. Insight into Wavelets From Theory to Practice:
EXERCISES
2.1 Consider the interval [- I, IJ and the basis functions (1. ,. ,2, ,1, ...• ) for L2!_1, I].
,
The inner product on this vector space is defined as <to g) = J/(1) 8(1) dl . Wc can
-,
show that Ihese functions do nOl fonn an onoonormal basis. Given a finite or
countably infinite set of linearly independent vectors Xj, we can construct an
orthonormal set )'j with the same span as X j as follows:
S . h ".. tan wit )'1 = r:I
,'"
.. Then. recursively set )'2 =
.. Therefore.
This procedure is known as Gram·Schmldt orthogonali:zalion and it can be used
10 obtain an onhononnal basis from any other basis. Follwing is a MATLAB
function Ihat implement the Gram-Schmidt algorithm. Your input must be a set of
linearly independent fUll(:tions. 1be output will be a set of onhooormaJ functions that
5pan Ihe same spa<:e.
function ~_g~amschmidt( x,N .M);
fo ~ i_l:N;
s/i,:).x (i, : ),
end;
ell) .s/l. :) · conj (s(l,:).');
phi(l, :) - s(1, :)/sq~t (e(l);
fo~ i_2:N;
th(i, :).ze ~oa(1,M);
fo~ ~.i·l:·l:l;
th(i,:) _th( i ,:) .. (a (i,:) · conj (phi (~,:) . ')) · phi (r,:);
end;
th(i, : I- s (i,:) -th(i,:);
e(i)_th(i,:) · conj(th(i,:).'); 11 note.' means
transpose without conjugation
phi(i,:I·th(i,:l/sq~t(e ( i));
end;
z _phi Cl :N,:);
Apply the algorithm on Legendrc polynomials and plO! the first six orthonormal
basis functions.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Fwrier Series Il!Id Geometry • 31
2.2 Project the function j(x) : x on to the space spanned by ji(x), ¥1x), ¥12x),
!l'(2x - I)e L2[O, I] where
fI(X) :
1
1.
o.
OSxS I/2
otherwi se
(
I OS x S l12
If(x): - 1 112 SxS I
o otherwise
Sketch the projection oflex) to this space.
2.3 Show that the set of functions /",(x) '" Hsin mx. m = l. 2, ..., is an or1hononnal
system in L2[O, If].
2.' Given that
112 112 1/.J2 0
In In -11../2 0
H. :
112 - In 11../20
,n - 112 0 - 1/./2
9
7
is an orthononnal basis for :It. and l : [l IE '" e :lt4
• where E is the standard
3
,
basis for :lt4
• that is.
I 0 0 0
0 I 0 0
E=
0 0 l 0
0 0 0
Find z in tenns of basis in H•.
2.5 (a) Find the Fourier series of/ (1) = 1 for - If S t < If.
(b) Using Plancl1erel's fonnula and (a), evaluate the series
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
32 • Insigh! imo Waveleu~From Theory w Pn.c:!ice
[Hinl: Acoording 10 Plancherei's formula I If = L. 1c. 1
2
, where the C,.s are
""complex Fourier series coefficients of the function I(/). U/li2 in the given problem
"
is 2~ JIf(t)1
2
dt I
-"
2.6 Find the Fourier series of
f·HI(x)=ln
for-nSx<O
forOSx<n
Suggested Further Readings
Amold. Dave. I am the Resource jar all that iJ Math: A COnlemporory Perspective. C.R.. CA.
1998.
Burden, Richard L., Numerical Analysis. BrooklCole, Alban)', 1997.
LeQn, Sleven, Unear Algebra wirh Applications. Prenlice HaiL New York, 1998.
Lyons. Richard. Unde ,jumding Digital Signal Processing. Addison-Wesley. Menlo ParIc. CA.
1998.
Ramiret, Roben W., The FFT: Fundnmenlals and Concepu. Prentice Hall. New York. 1985.
Slnlng. Gilbert. Inlroduction to Unear Algebra. Cambridge Press. Massachusetts. 1998.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Continuous Wavelet and Short
TIme Fourier Transform
INTRODUCTION
The aperiodic, noisy. intenninent, transient signals are the type of signals for which wavelet
transforms are panicularly useful. Wavelets lIave special ability to examine signals simuh-
aneousJy in both lime and frequency. This has resulted in the development of a variety of
wavelet based methods for signal manipulation and interrogation. Current applications of
wavelet include climate analysis, financial lime series analysis. hcan monitoring, condition
monitoring of rotating machinery, seismic signat denoising, deooising of astronomical images,
crack surface c haractcril.ation, characterization of turbulent intermittcncy. audio and video
oompression, compression of medical and thump impression records, fasl solution of partial
differential equations, computer graphics and so on. Some of these applications require
(:(lntinuQUs wavelet transform, which we will explon: in this chapter and find out how it differs
from classical mc:thods Ihat deals with aperiodic lJOisy signals.
3.1 WAVELET TRANSFORM-A FIRST LEVEL
INTRODUCTION
Wavelet means 'small wave' . So wavelet analysis is about analyzing signal with short duration
finite energy functions.1bey transform the signal under investigation into another representation
which presents the signal in a more useful form. This tnlnsformation of tbe signal is called
wavelet transfOl'm. Unlike Fourier tnmsfonn, we have a variety of wavelets that ~ used for
signal analysis. Choice of a particular wavelet depends on the type of application in hand.
Figures 3.1 to 3.4 show examples of some real and complex wavelets.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
34 • I "gh . W le F
"" t ,"IQ m ~~ """" p,,,," '"
1
1
0.2
0.2
"0
0
..... -<1.2
-<1.2 - 1
- 20 o 20 - 20 - 10 o 10
FIGURE J.l Real and imaginary pans Qf Shan wavdet
1.2,--_-_--_-_-_-_--_-,
0.8
0'
01--_,
-<1.'
- 3 - 2 - 1 o 3 •
FlGURE).2 Quintic Splinc wavelet
FIGURE J.J Real and imaginary pans of Gabor ",,..."del.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Continuous Wavelet: and Shoft Time Fouricr Tm1sfonn • 35
1.0
1.0
0.'
0.'
o0 t-~
o.ot------
-<I.'
-1.0 ':----:----;--:---0- 4 - 2 0 2 4
- 1.0':-----";----'-;----;:------;
- 4 - 2 0 4
, ,
FIGURE 3.4 Real and imaginary potU of Morlet wavelet.
We manipulate wavelet in two ways. The first one is translation. We change the central
position of the wavelet along the time axis. The second one is scaling. Figures 3.5 and 3.6 show
translated and scaled versions of wavelets.
" ",, ' ,, " ,, ,,
~, ;.
, '" "'.' '~, '.,
FIGURE J.5 Translation (change in position) of wavelets.
f~ucnq
(Scale)(Lcyd)
vv
TIme shin
,
FIGURE J.6 Change in Kale (aI~ called level) of wavelcu.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
36 • lnsigllt into Wavelets-From Thcofy to Practice
Figure: 3.7 shows a schematic of the wavelet transform which basically quantifies the local
matching of the wavelet with the signal. If the wavelet matches the shape of the signal well at
a specific scale and location, as it happens to do in the top plO( of the Figure 3.7, then a large
transfonn value is obtained. If, however, the wavelet and signal do not correlate well, a low
value of transform is obtained. The transform value is then ploned in the two-dimensional
transfonn plane shown at the bottom of the Figure: 3.7. (indicated by a dot). The transfonn is
computed at various locations of the signal and for various scales of the wavelet, thus filling
up the transform plane. If the process is done in a smooth and continuous fashion (i.e., if scale
and position is varied very smoothly) then the transform is called continuous wavelet
transrOnR. If the scale and position an: changed in discrete steps, the transfonn is called
diM:rde wavelet transrorm.
W.~,,:,~,"~--=-"r-=--../ local =hing between signal
and wavelet Ic:odJ 10 large
transform coeffic;"ntWavelet transform
_ Cu=n' ",ale
"", I~---
-----------------------~
Wavelet transform plO!
(Two-dlmeru'onol)
: Higher lhe coefficient more
: darker tbe point
r CUrrent location .........
Position ---_.
FIGURE 3.7 n.e signal. wovelel and lnUI,form.
Planing the wavelet transform allows a picrure 10 be built up of correlation between
wavelet- at VMooS scales and locations-and the signal. Figure 3.g sllows an e~ample of a
signal and corresp:lllding spectrum. Commercial software packages use various colours and its
gradations to slIow the transform values on the two-dimensional plot. Note that in case of
Fourier transform. spectrum is one-dimensional array of values whereas in wavelet transfonn,
we get a two-dimensional array of values. Also note Ihat the spectrum depends 011 the type of
wavelet used for analysis.
Mathematically. we denote a wavelet as:
I ('-b)~•.•(,). JI"I ~ -;;-
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Continuous Wa" elel and Shon Time Foorier TfWlSform • 37
Timel)
HGURE J.lI SiglW iIId iu ""iI,·(]tl tJillSfl1l1ll lipc.;lJUm.
where b is location parameter and 11 is scaling parameter. For the function to be a wavelet. it
should be time limited. For a given scaling parameter a. we translate the wavelet by varying the
parameter b.
We define wavelet transform as:
V(u.b)= ff(t)JtaII"C:b)dl (3.1)
According to Eq. (3.1). for every (Cl. b)• ....e have a wavelet transform coefficient.
rePf"CSenting how much the scaled wavelet is similar to the function at location I = (bla).
In the following section (Section 3.2) we will explore the classical time-frequency
representation of signals. and associmed uncenainties in frequency and time. 1llen we shall
compare the same with that of wavelet transfOl"ms.
3.2 MATHEMATICAL PRELlMINARIES-FOURIER
TRANSFORM
Continuous Fourier tmnsform
Assume that/is a complex-valued function defined on R. Its Fourier transform j is defined as:
If fELt. the integral makes sense fOl" every value of w and f is a continuous. bounded
function which goes to zero at infinity (this last fact is called Riemann-Lebesgue Lemma).
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
38 • Insight into Wavelets- From Theory to Practice
The Fourier transform is also defined for [E:' L2. In this case, the integral may oot be
defined in the usual sense. One way to define it is:
L
' •
" I IlII f 'few) = c:= [(I)e-"OJ dl
,,2K R -J ...
-.In this case j is also in Ll.
If I E L1 or [E Ll it can be wrinen in tenns of its Fourier transform as:
-I f - .I(r) = r::-= I(wje"'" dw
,2.-where the integral may need to be imerpreted as a limit of finite integrals, as before. In general,
every locally integrable function [has a Fourier transform but j may not be a fu nction any
1IlQI"C, rather a generolized[unetlort or dlslrlm.lion. We will assume from oow onwards that all
functions have suitable imegrability properties so that all Fourier transforms (and other
operations that show up) are well defined. Some propenies of the Fourier U"lUlsform are:
• The Fouricr transform preserves L2
norms and inner products (this is called the
Parseval-Planchen:1 Theon:m). Thus. if f, g E Ll then
(I,g) = <i,g)
• 11Je Fourier transform turns convolutions imo products and vice vema. 1lte convolution
of f, g is defined as:
We find
(f. g)(I) = JI(y) g(t - y) dt = JI(t - y) g(y) dy
(f. g)" (w) = J2K j(M)g(M)
(f. g)" (M) = J2K(j. g}(/lJ)
• The Fourier transform turns translatioo into modulation and vice versa. The lrons/mion
of I by a E 9t is defined as:
T.[(t) = I(t - a)
The modulmion of[by a e 'X is defined as:
E.f(t) = e;"I [(t)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Continuous Wavelet and Shon Time Founer Transform • 39
. .
(EJ) " (m) '" f(m - a) '" T.f«(I)
• The Fourier trlUlsform turns dilation inlO inverse dilation. The dilOliQn off by SE 9f
is given by
I 1
-112
D.f(t) '" s f(tls)
The factor in front is choscn so thal ID.f ll
'" Ifb.The Fountr transfonn relationship
IS:
I 1
1/2 " "
(D.f)'" (m) '" s f(sm) '" o",.f(m)
• The Fourier transform turns differentiation into multiplication by iro and vice ve~
(n" (w) :: iw j(W)
(If(t» " (m) '" ij'(m)
3.2.1 Continuous Time-frequency Representation of Signals
Assume that f{1) is a complex valued function on 91 which represcnts some signal (think of t
as lime).
The Fourier transform
-. 'f ·f(m ) '" ~ JO)t-"'" dl
,2. (3.2)
-is used to decomposc f into its frequcocy components. The inversion formula is o;pressed as:
(3.3)
Equation (3.3) can be interpreted as writing f as a superposition of time-harmonic waves t 4
-.
If i is large near some frequency then f has a large component that is periodic with that
frequency.
This approach works well for analyung signals that are produced by some periodic
process. However, in other applicalions, like speech analysis. we would like to localize the
frequency components in time as well and Fourier transform is nol suitable for thal.
We will consider two methods that attempt to provide information on both time and
frequency: the Windowed Founer Transfonn (wFI1. also called Short Time Fourier Transform
(STFl') and the Continuous Wavelet Transform (CWl). Both of lhem map a function of one
variable (lime) inlo a fuoction of two variables (time and frequency). A large value of the
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
40 • Insight intQ Wa~elet. From Theory to Practice
transform near time I, and frequency ro i$ interpn:tcd as: [he signal! contains a large componenl
willl frequency W near lime I. lllere is a lot more theory [0 botll of these transforms IlIan we
will cover in [his chapter but our main interest lies elsewhere.
3.2,2 The Windowed Fourier Transform (Short Time Fourier
Transform)
Fix a function IV E L2, IlIe window function. W sllould be localized in time neat 1 '" O. witll a
spread IT (we will define "spread"' more precisely later).
Typical choices are:
(i) W = XI-I .11 willl IT = 1I.Jj (see Figure 3.9)
X. is the characteristic function of the set S, which has value I on the set. 0 otherwise.
w
----'---+_L---::+---'------+ ,
Sigma
.'IGURE 3.9 Char.lCtm>lic function of W .. XI-l.It
(ii) W", (J +cos 21)12 for I € (~.R"12. xl2j, IT == J/.J3 (see Figure 3. 10)
(iii)
Sigma
FIGURE ).10 1'101 of window function W .. (l + cos 21)/2.
w" _'_ t-r'12 (Gabor willdow). 0'" I (see Figure 3.11)
J2Jc
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
CQnlinUOlU Wa""kt and ShQrt Time F<>urier Transfmm • 41
Sigma
FIGURE 3.11 PlO! Qf window fUI>Ction W ~ ';"e-..n.,2,
The WfT with window W of f is defined as:
-Ifwf(a.b) = ,j~lf L!(r)to-;bl W(t a) dl
Thus. a is l!le time parameter, b is t!le frequency.
(3.4)
V'wf(a. b) can be interpreted as inner product of f with [he leSI function eiblW(1 - a).
A typical test funclion looks like the one given in Figure 3.12.
// y-
FIGURE 3.12 Shifted and modulated window.
Note how the time resolution u (rel3led to the window width) is C()nstanl. independent of
the frequency.
3.2.3 The Uncertainty Principle and Time-frequency Tiling
Assume a function f e l}. The II1C3I iJf of f is defined as:
(3.5)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
42 • Insi):hl intQ Wavekts-Frorn TheQry IQ Practice
The uncertainty CIf of! is defined as:
(3.6)
Remark: If!E Lllhen !2/I!f is a probability distribution. i.e. a non-negali·e function with
integral L. IJf and CIf are simply the mean and stlllldard deviation of this distribution in the
statistical sense.
IiJ measures where! is localized in time and CIf measures how spread OUI (or uncertain)
this time measurement is. Pt localizes the frequency and Cl; measures the uncertainty in
frequency.
H,~
(3.7)
where
-" I J .! (w) = ~ !(t)e-"· dr
,2. -"d
(J.8)
The uncertainty principle states Ihat for any f
(3.9)
If a function is localized in time, it must be spread out in the frequency domain and vice
versa (see Figure 3.13). The optimal value of 112 is achieved if and only iff is a Gaussian
distribution. To visualize this, consi(jer a hypothetical function F(t, w) over the time-frequency
plane. F(I, w) represents the component of! with frequency wat time /.
The uneertainly principle says that it makes no sense to try to assign point-wise values to
F(I, w). All we can do is to assign meaning to averages of F over rectangles of area atleasl 2.
(The signal is localized in time to [,u - Cl, IJ + a:l and likewise for frequency, so the rectangle
has area (2C1p x (20).
The uncertainty principle applies 10 both WfT and CWT but in di fferent ways. Let 1J... 1lIld
Cl... represent 'localization' (mean) IlIld 'spread' of the wirKiow function W(I) in time domain
which are formally defined as:
-1-1... = 1 2 fIjW(t)11dl
IWO)I _
(3. 10)
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Conlinuous WUeltl and SIKH1 Time Foune, Tr.msfonn • 43
"I
D
Good time resolution
Bad f'<queDCY n:w1~!i0f
:;>
Good frequency resolution
Bad lime rUQlulion
nCURE J.B Po..ibt" lime-fn:qu<DCY ....indo......
(3.1 1)
Also Itt A.. and 0-.. represent F014,iu lrons/orms of 'locaJiution' (mean) and 'spread' of
the window funclion WCt). Then the inner product (f, W) contains information on / in
[.uoo - u.".u.. + u..J. Since <I,W) '" tj, IV) , it also contains (nfoonation on /(0) in
[.it.. - o-..,.it.. + it..]. Thus, (J, IV) represents the frequeno:ies between Pw - o-w and .it.. + 0-..
thal are present between time p.. - u.. and .u.. + u"'.
For WfT, the test function is the shifted and modulated window.
..iblW(I_ a) '" EhT. W
which is localized in time near .u.. + a with uno:ertainty a.".. Its Fourier transfonn is
which is localized near .it.. + b ....ith unCtnainly U...
Here E and T are respectively modulation and translation operators.
contain information on / in [.uw + a - u.... .uw + a+ u.. J and information on
[.it.. +b - 0-..,Pw +b + 0-00 ] .
(3.12)
"
f"
For a WFf with fixed window WCt), the time resolution is fixed at u..' the frequency
resolution is fixed at 0-,... We can shift the window around in both time and frequtncy but the
uncenainty box always has the same shape. n.e shape can only be changed by changing the
....indow W(I). Refer Figure 3.14.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
44 • Insight into Wavelets From Theory to Practice
D
Time axi.
HGURE 3.14 Some possible t;me·(re<j""!IC)' window. of w;ndowc:d Fouric. transform.
Time and frequcocies are resolved equally well (or equally bad). A discrete WFT (with equally
spaced lime and frequency samples) gives a unifonn tiling as in Figure 3.15.
I
,
I' IGURE 3.15 lime.fre<j""'''''Y Lilinl of windowc:d Fouricr t.,,".form.
For CWT. lhe test function i5 of the form:
(3.13)
with Fourier transform E_bDlI8Y. Thus.
(j. V'Q.b) = (j. TbD.V') = 0.E_b DI18y) (3.14)
contains inronnation on f in lu,u,.+b-uu,..up",+b+uu",J and infonnalion On j in
[iJ"la - iJ",la. it"fa + iJ"fa]. The uncertainty boxes have differem shapes in different parts of
llIe lime·frequency plane iU in Figure 3. 16.
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
Cominuous Wavelet and Short TIme Foorier Transfonn • 45
o
AI high frequency:
Good lime re""luliOl1
Bad freqllCOC)' r~lulion
AI low frequency:
[======1/Good freqllCncy re).OIi.>I;OII
i Bad I;me resolution
TIme aJIis
.'IGUIlE .3.1' f'lmiblc limc-frcquclI(Y tHings in the case of continuow wa...det transform.
3.3 PROPERTIES OF WAVELETS USED IN CONTINUOUS
WAVELET TRANSFORM
A wavelet 1f(1) is si mply a function of time t that obeyS a basic rule. known as the wavelet
admissibility condition:
- I~ )1c =f (U dw<- (3.15), w
o
where !i/(w) is the FourieT transform. This condition ensures that ]V{w) goes to zero quickly
as co -t O. In fact. to guarantee that Cl" < _, we must impose ]V(O) "" 0 whicl! is equivalent
" -fVlO) dl = 0
-A seCOl1dary condition imposed on wavelet function is unit energy. That is
-
-
3.4 CONTINUOUS VERSUS DISCRETE WAVELET
TRANSFORM
(3.16)
(3.17)
As mentioned previously. CWT is a function of two parameters and, therefore. eOnlai ns a I!igh
amount of extra(redundant) information when analYl.illg a functioll. illstead of continuously
varying Ihe parameters. we analyze the signal will! a small number of scales with varying
Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9
In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9

More Related Content

What's hot

Linear block coding
Linear block codingLinear block coding
Linear block codingjknm
 
DSP_FOEHU - Lec 13 - Digital Signal Processing Applications I
DSP_FOEHU - Lec 13 - Digital Signal Processing Applications IDSP_FOEHU - Lec 13 - Digital Signal Processing Applications I
DSP_FOEHU - Lec 13 - Digital Signal Processing Applications IAmr E. Mohamed
 
Fir filter design (windowing technique)
Fir filter design (windowing technique)Fir filter design (windowing technique)
Fir filter design (windowing technique)Bin Biny Bino
 
Properties of fourier transform
Properties of fourier transformProperties of fourier transform
Properties of fourier transformNisarg Amin
 
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...Madhumita Tamhane
 
Fir filter design using windows
Fir filter design using windowsFir filter design using windows
Fir filter design using windowsSarang Joshi
 
DSP_FOEHU - Lec 10 - FIR Filter Design
DSP_FOEHU - Lec 10 - FIR Filter DesignDSP_FOEHU - Lec 10 - FIR Filter Design
DSP_FOEHU - Lec 10 - FIR Filter DesignAmr E. Mohamed
 
Windowing techniques of fir filter design
Windowing techniques of fir filter designWindowing techniques of fir filter design
Windowing techniques of fir filter designRohan Nagpal
 
DSP_2018_FOEHU - Lec 07 - IIR Filter Design
DSP_2018_FOEHU - Lec 07 - IIR Filter DesignDSP_2018_FOEHU - Lec 07 - IIR Filter Design
DSP_2018_FOEHU - Lec 07 - IIR Filter DesignAmr E. Mohamed
 
Dft and its applications
Dft and its applicationsDft and its applications
Dft and its applicationsAgam Goel
 
Motion Estimation - umit 5 (II).pdf
Motion Estimation  - umit 5 (II).pdfMotion Estimation  - umit 5 (II).pdf
Motion Estimation - umit 5 (II).pdfHeenaSyed6
 
NYQUIST CRITERION FOR ZERO ISI
NYQUIST CRITERION FOR ZERO ISINYQUIST CRITERION FOR ZERO ISI
NYQUIST CRITERION FOR ZERO ISIFAIZAN SHAFI
 

What's hot (20)

Array antennas
Array antennasArray antennas
Array antennas
 
Sampling Theorem
Sampling TheoremSampling Theorem
Sampling Theorem
 
Multirate DSP
Multirate DSPMultirate DSP
Multirate DSP
 
Linear block coding
Linear block codingLinear block coding
Linear block coding
 
DSP_FOEHU - Lec 13 - Digital Signal Processing Applications I
DSP_FOEHU - Lec 13 - Digital Signal Processing Applications IDSP_FOEHU - Lec 13 - Digital Signal Processing Applications I
DSP_FOEHU - Lec 13 - Digital Signal Processing Applications I
 
Fourier transform
Fourier transformFourier transform
Fourier transform
 
Fir filter design (windowing technique)
Fir filter design (windowing technique)Fir filter design (windowing technique)
Fir filter design (windowing technique)
 
Properties of fourier transform
Properties of fourier transformProperties of fourier transform
Properties of fourier transform
 
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
Convolution codes - Coding/Decoding Tree codes and Trellis codes for multiple...
 
Fir filter design using windows
Fir filter design using windowsFir filter design using windows
Fir filter design using windows
 
Fourier Transform
Fourier TransformFourier Transform
Fourier Transform
 
DSP_FOEHU - Lec 10 - FIR Filter Design
DSP_FOEHU - Lec 10 - FIR Filter DesignDSP_FOEHU - Lec 10 - FIR Filter Design
DSP_FOEHU - Lec 10 - FIR Filter Design
 
Windowing techniques of fir filter design
Windowing techniques of fir filter designWindowing techniques of fir filter design
Windowing techniques of fir filter design
 
Information theory
Information theoryInformation theory
Information theory
 
DSP_2018_FOEHU - Lec 07 - IIR Filter Design
DSP_2018_FOEHU - Lec 07 - IIR Filter DesignDSP_2018_FOEHU - Lec 07 - IIR Filter Design
DSP_2018_FOEHU - Lec 07 - IIR Filter Design
 
Sampling theorem
Sampling theoremSampling theorem
Sampling theorem
 
Dft and its applications
Dft and its applicationsDft and its applications
Dft and its applications
 
Motion Estimation - umit 5 (II).pdf
Motion Estimation  - umit 5 (II).pdfMotion Estimation  - umit 5 (II).pdf
Motion Estimation - umit 5 (II).pdf
 
NYQUIST CRITERION FOR ZERO ISI
NYQUIST CRITERION FOR ZERO ISINYQUIST CRITERION FOR ZERO ISI
NYQUIST CRITERION FOR ZERO ISI
 
Dif fft
Dif fftDif fft
Dif fft
 

Similar to In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9

Stephen Hawking - Properties of expanding universes
Stephen Hawking - Properties of expanding universesStephen Hawking - Properties of expanding universes
Stephen Hawking - Properties of expanding universesXps Gtx
 
Mathematical models of electric machines
Mathematical models of electric machinesMathematical models of electric machines
Mathematical models of electric machinesvijayabhaskar83
 
Electrical engineering-objective-questions-part-1 - copy
Electrical engineering-objective-questions-part-1 - copyElectrical engineering-objective-questions-part-1 - copy
Electrical engineering-objective-questions-part-1 - copysujit maurya
 
260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis
260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis
260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesisFlora Viale
 
Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...
Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...
Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...Richard L. Lachance
 
Ies electrical-engineering-paper-2-2003
Ies electrical-engineering-paper-2-2003Ies electrical-engineering-paper-2-2003
Ies electrical-engineering-paper-2-2003Venugopala Rao P
 
D021201031040
D021201031040D021201031040
D021201031040theijes
 
Marine Gyro Compasses For Ships Officers.pdf
Marine Gyro Compasses For Ships Officers.pdfMarine Gyro Compasses For Ships Officers.pdf
Marine Gyro Compasses For Ships Officers.pdfrulohaiti
 
Amma kalviyagam-free-formula-hand-book
Amma kalviyagam-free-formula-hand-bookAmma kalviyagam-free-formula-hand-book
Amma kalviyagam-free-formula-hand-bookSenthilKumar Selvaraj
 
Education Certificates.PDF
Education Certificates.PDFEducation Certificates.PDF
Education Certificates.PDFSandeep Tiwari
 
EVE 161 Winter 2018 Class 11
EVE 161 Winter 2018 Class 11EVE 161 Winter 2018 Class 11
EVE 161 Winter 2018 Class 11Jonathan Eisen
 
Ias mains-electrical-engineering-2009
Ias mains-electrical-engineering-2009Ias mains-electrical-engineering-2009
Ias mains-electrical-engineering-2009Mohitmathuria1
 
Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...
Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...
Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...Dmitry Novikov
 

Similar to In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9 (20)

Stephen Hawking - Properties of expanding universes
Stephen Hawking - Properties of expanding universesStephen Hawking - Properties of expanding universes
Stephen Hawking - Properties of expanding universes
 
Mathematical models of electric machines
Mathematical models of electric machinesMathematical models of electric machines
Mathematical models of electric machines
 
6th Semester (December; January-2014 and 2015) Mechanical Engineering Questio...
6th Semester (December; January-2014 and 2015) Mechanical Engineering Questio...6th Semester (December; January-2014 and 2015) Mechanical Engineering Questio...
6th Semester (December; January-2014 and 2015) Mechanical Engineering Questio...
 
s
ss
s
 
Electrical engineering-objective-questions-part-1 - copy
Electrical engineering-objective-questions-part-1 - copyElectrical engineering-objective-questions-part-1 - copy
Electrical engineering-objective-questions-part-1 - copy
 
5th Semester (December; January-2014 and 2015) Civil Engineering Question Papers
5th Semester (December; January-2014 and 2015) Civil Engineering Question Papers5th Semester (December; January-2014 and 2015) Civil Engineering Question Papers
5th Semester (December; January-2014 and 2015) Civil Engineering Question Papers
 
260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis
260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis
260613890_Viale_Flora_Atmospheric&OceanicSciences(AOS)_thesis
 
Testo
TestoTesto
Testo
 
Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...
Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...
Lachance 2000, Non-linearity correction of FTIR instruments, Fifth Workshop o...
 
6th Semester (December; January-2014 and 2015) Civil Engineering Question Papers
6th Semester (December; January-2014 and 2015) Civil Engineering Question Papers6th Semester (December; January-2014 and 2015) Civil Engineering Question Papers
6th Semester (December; January-2014 and 2015) Civil Engineering Question Papers
 
Ies electrical-engineering-paper-2-2003
Ies electrical-engineering-paper-2-2003Ies electrical-engineering-paper-2-2003
Ies electrical-engineering-paper-2-2003
 
D021201031040
D021201031040D021201031040
D021201031040
 
Marine Gyro Compasses For Ships Officers.pdf
Marine Gyro Compasses For Ships Officers.pdfMarine Gyro Compasses For Ships Officers.pdf
Marine Gyro Compasses For Ships Officers.pdf
 
4th Semester (December; January-2014 and 2015) Electronics and Communication ...
4th Semester (December; January-2014 and 2015) Electronics and Communication ...4th Semester (December; January-2014 and 2015) Electronics and Communication ...
4th Semester (December; January-2014 and 2015) Electronics and Communication ...
 
Amma kalviyagam-free-formula-hand-book
Amma kalviyagam-free-formula-hand-bookAmma kalviyagam-free-formula-hand-book
Amma kalviyagam-free-formula-hand-book
 
Education Certificates.PDF
Education Certificates.PDFEducation Certificates.PDF
Education Certificates.PDF
 
EVE 161 Winter 2018 Class 11
EVE 161 Winter 2018 Class 11EVE 161 Winter 2018 Class 11
EVE 161 Winter 2018 Class 11
 
Ias mains-electrical-engineering-2009
Ias mains-electrical-engineering-2009Ias mains-electrical-engineering-2009
Ias mains-electrical-engineering-2009
 
Iu3515201523
Iu3515201523Iu3515201523
Iu3515201523
 
Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...
Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...
Interparticle Interactions and Dynamics in Solutions of Copper (II), Cobalt (...
 

Recently uploaded

(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Serviceranjana rawat
 
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur High Profile
 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxAsutosh Ranjan
 
result management system report for college project
result management system report for college projectresult management system report for college project
result management system report for college projectTonystark477637
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxupamatechverse
 
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Christo Ananth
 
KubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlyKubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlysanyuktamishra911
 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCall Girls in Nagpur High Profile
 
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...ranjana rawat
 
AKTU Computer Networks notes --- Unit 3.pdf
AKTU Computer Networks notes ---  Unit 3.pdfAKTU Computer Networks notes ---  Unit 3.pdf
AKTU Computer Networks notes --- Unit 3.pdfankushspencer015
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service NashikCall Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service NashikCall Girls in Nagpur High Profile
 
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escortsranjana rawat
 
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )Tsuyoshi Horigome
 
UNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular ConduitsUNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular Conduitsrknatarajan
 
Java Programming :Event Handling(Types of Events)
Java Programming :Event Handling(Types of Events)Java Programming :Event Handling(Types of Events)
Java Programming :Event Handling(Types of Events)simmis5
 
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130Suhani Kapoor
 

Recently uploaded (20)

(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
(RIA) Call Girls Bhosari ( 7001035870 ) HI-Fi Pune Escorts Service
 
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptx
 
result management system report for college project
result management system report for college projectresult management system report for college project
result management system report for college project
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptx
 
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
 
KubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlyKubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghly
 
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
 
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
(SHREYA) Chakan Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Esc...
 
AKTU Computer Networks notes --- Unit 3.pdf
AKTU Computer Networks notes ---  Unit 3.pdfAKTU Computer Networks notes ---  Unit 3.pdf
AKTU Computer Networks notes --- Unit 3.pdf
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service NashikCall Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
 
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(PRIYA) Rajgurunagar Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
(MEERA) Dapodi Call Girls Just Call 7001035870 [ Cash on Delivery ] Pune Escorts
 
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )
 
UNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular ConduitsUNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular Conduits
 
Water Industry Process Automation & Control Monthly - April 2024
Water Industry Process Automation & Control Monthly - April 2024Water Industry Process Automation & Control Monthly - April 2024
Water Industry Process Automation & Control Monthly - April 2024
 
Roadmap to Membership of RICS - Pathways and Routes
Roadmap to Membership of RICS - Pathways and RoutesRoadmap to Membership of RICS - Pathways and Routes
Roadmap to Membership of RICS - Pathways and Routes
 
Java Programming :Event Handling(Types of Events)
Java Programming :Event Handling(Types of Events)Java Programming :Event Handling(Types of Events)
Java Programming :Event Handling(Types of Events)
 
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
 

In sight into wavelets from theory to practice , soman k.p. ,ramachandran k.i. , ch.1 ch.9

  • 1. , ,, / - , . (~. Third Edition .. 11e'"1I Insight into ~ stem conomy dition FROM THEORYID PRACTICE KP. SOMAN •KI. RAMACHANDRAN •N.G. RESMI Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 2. R... 375.00 INSIGHT INTO WAVEl£Ts--f'rom Theory to PracUce, 3rd MII. (wilt! CO-ROM) K.P. Soman, K.I. A.amachandran, N.G. Rasmi 0 2010 by PHI Learning Private Umiled. New 0eIII. AI rights .--ved. No pari 0I1hIs book may be reproduCad in any klnn. by mimeog...llln Of any 0Ihef ~. -..iIholA pemIisaion in writing from ". publishef. -----...,-..........- ...----.....-..~--_ ...... co--.n. .......... _ _ ....... _ ..... _ ... _ .. _ • 0l0I _ _ .. _. , -. .. -.g ... COI. ... ~ ""..oo.=. ... _ .. lOo E_ _• ISBN-978-81-203-40S3-4 The .~potI rights 01 IhII book Ire 11951ed IoIIIy w;., !he publisher. S!Jc1t! Prlntl"1l (ThIn! Edition) M..-dI,2010 P~bli5h.d by A50h K. Gh-osh, PHI leamlng Privati llmitad. 1.4-97, Connaught Circus. New [)aH.ll000t and Printed by Mudrak. 3O-A. Patparganj. Delhi-l10091 . Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 3. Contents Prrjoce PrrjllCe /0 the Fir" Edition A.ckno...led~mellU 1. The Age of Wavelets inlmdMction--.l 1.1 The OrigiN of Waveleta-A~ They Fundllmcntally New? I 1.2 Wavelets and Other Rcality TrwlSfonns 3 1.3 Managing HeisenberJ', Uncenai!!!y: Gho6t j 1.4 Histor}' of Wavelel: from Morld 10 Daubechies via Mall&! 6 L4.LDiffcmrlLCommunitiu..oLWaveM:1L..9 1.4.L...Diffcreol EamiliCLoLWavcieuJ'jthin W'l'Ciet Communitics----.lO .4.3 IlIIClt:5ting R«ent Devclopmcnb: 11 1.5 W.vt:lets in thr; Furun: 12 1.6 What llistOf}' of Wavelet T-=hu UI 15 S...........ry lj SilIlUlM F,mJter RMding~ 15 1. Fourier Series and Geometry lfllrOducliOfl /6 2.1 ¥«tot Space 17 2.U-----.S.uu.----.l7 2.1.2 Onhonormality 17 2. J.3 Proja;tion 17 22 Functions and Function S~ 18 2.2.1 Orthogon.l Fu.netions /8 =_OrtboooonalEullCtion5~9 2.23 Fwx:tion Spac:eI 19 2.2.4 0rtb0g0aa.L Basis Fl,lno;tions 22 2.2.5 OrthooormaIil)' and the Method of Finding the Coefficients 22 '" Id' 16-32 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 4. viii • Contents 2.2.6 Comp~x Foorier Series 16 2.2.7 Orthogonality of Complex Exponential Bases 17 Sum/lUl_fl'...- 29 Ezerr.isu_ 30 SU8llt slt d Furthu Rtading~ 3. Continuous Wavelet and Short TIme Fourier Transform /fltroducliOll_ 33 3.LWavdeLJnnsform--A..Eim.L:.vdJnlrodLII:linn---.J3 3,2~athemalic;tl&eliminarie$-EQuri« TnlIl$form JJ 3.2.1 of ~gnal$ 39 3.2.2 The Founc! Transforml 40 3.2.3 The tli 41 3.3 Propenics of Wavelets In 45 3.4 Conlinuow; versus Discrete Wavelet Transform 45 SWllmQry 47 E:arr:isLL 48 Suggt~!td F...~r Rtadi~&s 49 33-4' A. DisClocte Wankt...Transronn 50-74 /mroductio~ 50 4.1 Haar Scaling Functions and Function Spaces 50 4.1.1 Translation and &alinS of ;(1) 51 4.1.2 Orthosonal"l)' of Translates~O 51 4. 1.3 Function S~ V. 53 4.1.4 Finer Hair ScaliRJ Functions 55 4.2 Nested Spaces 56 43 Haar Wavdd Function 57 43.1 Scaled Haar Wavekt-Func6oos 59 4.4 OrtItogooality of ;(1) and 11(1) 64 45.......Nonnalizalion_oUiaaLBasa..ILDiffcrenLScak.s_ 65 4.6 Standardizing the Notations 67 4.7 Refinement Relation with Res~t to Normaliwl Base.< 67 4.8 Support of & Wavelet System 68 4.8.1 Triangle Scaling Function 69 4.9 Daubtthics Wavelets 70 4.10 Stocing tilt Hidden-PloI:ting tilt Daubtoch.iu Wa~It" 7J Summary 73 Extrr.Ut~ 7J Suggtutd FUr/htr R",di"ls 74 S. Designing OrthogonaJ Wavelet Systems-A Direct Approach 75-93 ImrodUClioll_ " 5.1 Refinement Relation for OrtItogo;>!!a1 Wavelet S)'stcms 75 5.2......Restrictionu.n-Filtcr Coefficients 76 5.2.1 Condition 1: Unit Area Undor Sc.a1inll Funetion 76 5.2.2 Coodition 2: Orthonorma.lity of Translates of Scaling Funetions 76 5.2.3 Condition 3: Orthooormality of ScalinS and Wavelet Functions 79 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 5. Coments • ix 5.24 Condition 4: Approximation Condition5 (Smoolhne55 Conditions) 81 5.1.5 Condition 5;j!edundant) OrthonormaIity of Tl'IIl$lates of Wavelet Fu.nctions 82 5.2.6 Condit~UIIdant) Orthooonnal'ty of ,(0and Translates of ttC!:) 82 5.3 Designing Daubt:chics Orthogonal Wavdet bstem Coefficients 83 5.3.1 ColIStnIints fow D.ubcchies· 6-1ap Scaling FLJ.Ilf;'tion 84 5.4 Design of Coiflet Wavelets 85 5.s StmlelS 86 5.6 An Intri,guin~y of Ortbogonal Scaling FWICtion 86 S.6..1~ SlImm·'ion.Form~88 5.6.2 Proof of Partition of Unity of Scaling Func:tion 90 5.6.3 Ne<:cni!), of Partition of Unity of Scaling Function 91 SIUftIIIQf)'_ " E2~j!el-----.92 Sugg~sled Funher R~odill" 93 6.---»i.Krete WaYdet.Tnms!orm and Relation to f'Ute..-.lJank5 9+----.ll0 11llrOdJ.djOlt~ 6.1 Signal Decom~ition (Analy~ 94 6.2 Relation with Filter Banks 97 6.3 F~ Response 102 6.4 Sig~1 Reconstruction: Synthesis from Coarse Scale to Fine Scale /03 6.4.1 Upsampling and filleTing 1()4 6.5 Perfect Match.i!!8 Filten 107 6.6 Computing Initial 'itt Coefficients 107 6.7 Vanishing Moments of Wavelet Function and Filter Properties 107 S"'""""ry---.JP9 Suglu/ed Funhu Rtadill8£ 110 7. Computing and PlottJng Scaling and Wavelet Functions 111-126 .. IItlmduClimt----.lH 1.1 Daubcchies-Lagatias Algorithm 112 1.1.1 Discrete Dilation EqlWion lJ4 1.1.2 Swcmcnt of Daubcchies-l...a8!!ias AI~thm /15 7.1.3 Generating Binary Equivalent of a Decimil Number JJ6 7.1.4 Implementation in Microsoft Excel (or Any Spreadsheet Pacuge) JJ6 7.1.5 CoIllR!!Un.LWavelet FuocUon 118 7.2 Sub:1ivis.ion..Scbem.L...l18 7.3 S""",,"';ve Appro.imation 121 8.1 Bionho&onali!J: in Vector S~ 127 8.2 Biorthogonal Wavelet Systems 129 S.3 Sigru.l~ UsinS_BiortlKlgonal Wavelet S)'Slem H2 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 6. " • Contents 8.4 Bioooogonal Analy~lJZ 8.S Coarse Scale to Fine Scale JJ.4 8.6 Wa""let~aum& 1.15 Summary-HJ E:arru£L-L42 ,o••1,"'••,., S)'~", or Cohen-Daubcchies-Fca.uveau 117 Suggesled FlIftMr Readings 141 9, Designing Wavekts-FrequelK)' Domain Approach 143-UI3 InJrrxluC/ioIL ...1j3 9.1 Basic Proptrtics of Filter Cocf("t<.:icntli 143 9.2 Fillcr Properties in Terms of H..l(li) 144 9.3 Filter Properties in TCITll5 of H,(z) /45 9.4 F~uelK)'_ Domain Characteril.lllion of Filter Coefficients 146 9.S Choice er Wavelcl Function Coeflkicnts Ig(.)) 150 9.6 Vanishing Moment CondiliO/lll in Fourier Domain 153 9.l.......DerivatiQn nLDaube:c.h.ic.LWavdcts.......15.4 9.7.1 Steps Involved in DerivaliOfl of Daubechies Wavelets 1S4 9.7.2 Daubechies Wavelets with I Vanishing Moment 1S7 9.7.3 Daubechies Wavelets with 2 Vanishing Moments 158 9.8 Parametric Design <.If O"~onal and Biortoollooat Wavelets 159 9.8.1 Polyphase Factorizatioo Approach 1<1 Onhogonal Wavelet Design 159 9.8.2 Bionoollonal Wavelet Desig.........A Factor Multiplication ~h 166 9.8.3 Exte",,""" to Orthog<llllll Wavelets 179 9.8.4 Applicatioos of Panunetcriud W.""lcts /81 10, Groebner Basis (or Wavelet Design 184-192 InJrrx/ucliOll--.-l84 lo..LGrocbner...Basu.--.l1tl 10.2 Deriving Grocbner Basis for Daubechiel Wavclcl Syaum 185 10.2. Daubedties' 6-tap W.ve!ct with 3 Vanishing Moments 187 10.3 Deriving Grocbnet Basis for Coiflet /88 10.3.1 ~rties of the Moments of Scaling Functions 189 10.3.2 Des'lln of 6-tap Coiflet 190 10.3.3 Gcnenolitcd Coif1c:t S)'stem 19/ Summ<l~ /92 11. WaveletPacket Ana1ysis btl~IimL....JJ)J 1l.L ...HuLWaveIeLl'aculL ...194 ~];:!:::':B~U:i'~SeIecIiOO f()!" SiS.nal CII" Image Compassion 100 S"""""ry 202 193-202 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 7. 12. M-Band Wavelc:ts /nlmdJJctian 203 l21---.Motivation----.203 12.2 Multi·resolution Formulation of M·B~nd Wavelet Symm 206 12.3 Derivation of tbc ProJ1C'rties of M-Band Filter Coefficients 209 12.4 SupportS of SQIIing FUrK:lion lJld Wavelets Zf4 Conknts • xi 203-228 12.S Dcaign of 4-Band Symmetric Or1hogonal Wavelet Filtu Banks Based on Il!oOO.i 215 12.S.1 Groebnc:r Based Design Example with Length = 12 and Regularity =2 216 12.6 Parametric: Da.ign of M-Band Wave1c1S 219 12.6.1 Down~g and IVlynomial Represcnt:>l.ion 219 Summary 228 13. Introduction to lfultiwaveiets 1.3.1 Refrnable Function Vcctor 229 13.I.I Fuoction Sp3Ce VI 231 13.1.2 Su~ of, 234 13.1.3 Symbol of, 234 13. 1.4 FourierTransfocm of f: 235 13.U OrthononnaI.ity of, 236 13..6 Orthonormality of , in ternlS of H. 237 IJ.2......Multiwavc:let.Eunctioo----.2J10 13.2.1 Symbol of tt(o) 240 1l.2.2_Fourier Transform of tl!!-240 13.2.3 Orthogonality..&J! and ~'l 241 13.L ..Momcnts_ 241 13.3.1 DiKrek Moment of; 24J 13.3.2 DUcrete Moment of ~,) 241 13.3.3 Relation Bd_n M. and H(CtI) 241 13.3.4 Relation Bdween N,{ij and G(I) (IV) 143 13.3.S Continuous Moment of;' 143 13.3.6 Conlinuous Moment of "I') 143 13.3.7 Relation Between Jl> and Fourier Transform of, 243 13.3.8 Relation Between vi') and Fourier Transform of ttI') 244 13.3.9 Relation Between J!t and M. 244 13.4 Approximation Order 246 I1..L...MuUiwavc..lo::l_Eilta..BaDk 149 13.S.1 Analysis and 149 13.S.2 250 13.'.3 Signal ProeC5&ing) 250 13.6 13.7 TImc-varyinLMultiwavelet Filter Bank 252 13.8 Balancing Condil;ons 256 1,3~.8~.~1>lB~"~~~;";8~O~f~""";~'~I.~';'~O~:25"13.8.2 J; _ I 258 13.9 259 162 229-277 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 8. xii • Contents 13.9.3 Constn>elion of InterJ1Qlating Mu.ltiwavdet 175 SU/MIOI)' 177 ~8!sled Further Rwdl'!8s In 14. U fting Scheme 278-313 IIIJrD<luction--.l.18 ]4.] Wavelct Transform Using Poly£1a$e M.m~ Factori~on 179 14.1.1 Invenc Lifting 183 14.].2 Example: Forward Wavelet Tran~form 184 14.2 Geometrical Found;Wons of Lifting Scheme 186 14.2.1 14.2.2 14.2.3 Using Lifting 291 [4.2.4 Higher Order Wavelet Transfonn 295 [4.3 Lifting Scheme in the Z-domain 296 14.3.1 Desi,n Example I 301 ]4.3.2 Example 2: Lifting Haar Wavelet 305 14.4 Mathematical Pttliminaries for Polyphase Factorization 307 14.4.1 Laurent Polynomial 307 [4.4.2 The Eoclidcan AI,orithm 308 14.4.3 Factoring Wavelet Transform into Liftin~A Z-domain Al!Pf"OICh 314 ]4.5 Dealinl_wilh Signal Bolmdir)'_ 1i9 1"-'..1 CircI,laLCooYOlutiOll_ 319 14.5.2 Padding Policies 310 14-'.3_ lteraLioo_Beh.aviour------111 SU/MIOr)' 31 J Exercuu 311 15. Image Compression lntrD<luctit11L-314 15.] Overvicwof 15.J. l 15.2 Wavelet Transform ]5.3 Quantization 331 15.3. 1 Uniform Quantjution 331 15.3.2 Subband Uniform Quanti~ion 332 15.3.3 Uniform Dead-zone Quantization 333 15,3.4 Non·uniform Quantization 333 ]5.4 Entropy Encoding----.l34 15.4.1 Huffman Encodi"B 334 15.4.2 Run Length Encoding 3]5 15.5 ElW Coding (Embedded Zero-tree Wavelet Coding) 336 15.5.] EZW Pcrformar>ec 345 15.6 SPUIT (Set Partitioning in Hicran:hical Tree) 346 15.7 £Bear (Embedded Block Coding with Optimized Tl1.lICation) 346 SIUll/lW~ 346 Webk_ (Wtb Based Prob/e/llS) 347 Suggesftd FUrlM' ReMillgs 347 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 9. Contents • xiii 16. Denolslng 349-365 IntrrJ<iJu:tion 349 16. A Sim~1c E;o;~lanaJjon and a 1-0 Eumpk....}49 16.2 DenoisinJl U,ing W~1d Shrinkagc-Stati51ical Modelling and Estimation 3JO 16.3....NoiJc.Es.tima!iIHL JJl 16.4 Shrinkage FUllClions JjJ 16. ~ Shrinkage Rules Jjj 16.~.1 Universal Jjj 16.~.2 Minimizing the False Discovery Rate JJJ 16.~.3 T~ J56 l6.M S.IU:C_ JJ6 16.~.S Translation Invariant Thresllolding J57 16.5.6 BayesShrink 357 16.6 Denoisina Images with MATI.AB 358 16.7 MAl1.A8 Programs for Denoising J60 16.8 Simulalion for Finding Effectiveness of Thresholding Method 361 SWMlClry 36J EMn:ises 364 Su.ggulni Fu.rtMr R,.,Jillgs 365 17. Ber.ond Wavelets: The Ridgelets and Curvcleb In1rrJ<iJu:lion 366 .8. 17.1 AWlOlIimation Rates 367 17.2 Why 1 and Curvclets.? J68 17.3 The Transform 369 17.3.3 Finite Radon Transform J7I 17.3.4 Applications of Rid~1et Transforms 371 17.4 The Digilal Curvclel Transform 377 17.4.1 Propenics of Curvclct Tmnsform 378 17.4.2 Analysis: The Curvc:lel Decomposition 379 17.4.3 Curvclel Dcc<>mposition Algorilhm 379 17.4.4 Sy_nlhcsis: Reoonslruct.ioo from !he CtmIelet Transform J80 17.4.5 Digit.aJ Implementation or the Curvclet Transform 38() 17.5 S«ond Gc~ralion ~lets. 382 17.~.1 Tight Frame of Curveleu 384 1.7.5L Cw:vekt Constructioo.....J8.S 17.~.J l7.6 S~l '"N_1i~ Fnme of Curvcletl 388 18.1.1 Splinc eurve. and,:"u.r{aces 395 18.1.2 Cubic Spli~ Interpolation Methods J99 18.1.3 Hennitc Spli~ Interpolation 399 366-391 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 10. xiv • Co~tents 18.1.4 CUbic SJllines #JI S.1.5 Splines in Signal and Ima~ ProccssinS_ #J3 18. 1.6 Belier a.n'Ves and Surfaces 404 IS.1.7 Pr~rties of Buier CUrves 406 18.I.S Quadratic and Cubic Bezier Curves 407 18.1.9 Paramwic Cubic Surfaces 409 18..10 Buiu Su.rfaccs 4./1 IS.1.11 B·splincCu1'Ves 4/2 IS.1.12 Cubic Periodic B·splincs 414 IS. .1) Co,w"rsion Between .Icnnit". Be"i". and B·splinc Representations IS.1.14 Non·uniform B·sjl:lines 416 IS.1.5 Relation Iktwo:en SJlline, BWe. and B·~linc ~s 418 IS.I.16 B.splinc Surfaces 419 IS. I.l1 Beta.splincs aoo Rational Spli""" 419 IS.2 Multiresolution Methods and Wavelet A~ysis 420 IS.1...Jhe_Filtu-.B~411 8.4 Ortbo.l!onaJ and Semi-ilrtboJOOal Wa~lets 413 IS.S Splinc Wa,,,,lcts 414 IS.6 ~rties of Splinc Wavelets 427 18.7 Advantages of B·splinc Based Wavelets in Signal Processing Applications 18.8 B·spliroe Filter Bank 430 IS./LMuIliresolutiOlLCun>a and Facc.s_ '" 18.10 Wavelet Based CUrve and Su.rfacc Ediling 431 18.11 Variational Modelling and the Finite Element Method 431 18.12 Ada~i..... Variational Modo!llina U£ina W.""lets 43S SUnurt<l1)' 436 Sugge.ted Funhe, RetuJingJ 436 439-443 445-447 Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 11. Preface Wavelet theory has matured and has entered into ils second pllase of development and evolution in which practitioners an: finding newer applications in ever-widening scientific domains such 115 bio-informatics, computational drug discovery and nano-material simulation. Parallelly. the theory of wavelel$ got more and more demystified and has become an e~~ryday tool for signal and image processing. Postgraduate courses in mathematics and physics now ioclude a subject on wavdet theory either as a separate eJoctive or as part of other related subjects. In many technical universities, w3sclel has been introduced cven at the undergraduate level. In this third edition of the book we have taken into account this increasing popularity and the needs of the relatively 'young' readers from such wide range of backgrounds. One of the main additions in this third edition is that we have shown how the ubiquitous electronic spreadsbcct can be utilized for wavelet ba.o;cd signal and image processing. The theory behind the algorithm for computing e)lact values of wavelet and scaling function is simplified and implemented in Microsoft E)lcel as a workshcct function. Onc can now draw the fUlICtion by writing and dragging an e)lcd formula in a cell. Many of the intriguing properties of wavelet and scaling functions such as orthogonality of integer translates, partition of unity and refinement relation of scaling functions can be easily visualized in spreadsheets. The accompanying C D contains several worksheets that demonstrate the power of spreadshcct packages as a computational and visualization 1001. Recent years have secn heightened interest in 'parametric wavelet filter design' which allows the tuning of wavelet fiheB for various applications. Theory of its design procedures arc added in respt(;tive chapteB with several e:o;amples. Another new feature is that parametric and !"IOn-parametric biorthogonal wavelet design are e:o;plained in more detail. M-band wavelets are finding increasing applications in Communication Engineering as a tool for multirate signal procciSing and as signal modulators. So the chapter on M-band wavelet is eJlpanded to include the more recent and simplified design procedures. A scparate and elaborate chapter on Multiwavelet theory is added. Muhiwavelet represents the highest level of generali:.tation in wavelet theory and provides short filtcrs with most Of all of the desirable properties that a filtcr should possess. 1beory of baI:ma:d and interpolating multiwavelets are discussed in detail. " Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 12. xvi • Preface We earnestly hope that this edition will meet the: needs of readen of different academic backgrounds for their undcrgl1lduate, poslgraduate and research le'el sludies. Finally, we acknowledge OUT heanfelt Ihanks here to our ex-students Ms, K. Hemalalha and T. Ar.nhi who prepared all Ihe worksheets given in the accomp311ying CD. K.P. SOMAN K,I, RAMACHANDRAN N.G. RESMI Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 13. Preface to the First Edition In the past few years, the study of wa'c1ets and the exploration of the principles governing their behaviour have brought about sweeping changes in the disciplines of pure and applied mathemaTics and sciences. One of the most significant development is the realization thai., in addition 10 the canonical tool of representing a furn:tion by ils Fourier series, then:: is a different representation more adapted 10 certain problems in data compression. noise removal, pauem classification and fast scientific computation. Many books are available on wavelets but most of them are wriuen at such a level Ihat only research mathematicians can avail them. The purpose of this book is to make wavelets accessible 10 anyone (for example. graduate and undergraduate students) wilh a modest background in basic linear algebra and to serve as an introduction for the non-specialist. lbe level of the applications and the format of this book are such as to make this suitable as a textbook for an introductory course on wavelets. Chapter I begins with a brief note on the origin of wavelets. mentioning the main early contributors who laid the foundations of the theory and on the recent developments and applications. Cbapter 2 introduces the basic concepts in FotJrier series and orients thc reader to look at everything found in the Fourier kingdom from a geometrical point of view. In Chapter 3, the focus is on the continuous wavelet transform and its relation with soort time Fourier transform. Readers who have oot had much exposure to Foorier transforms carlicr may skip this chapler. which is included only for the purpose of completeness. Chapter 4 places the wavelet theory in a concrete selling usinS Ille Haar scaling and wavelet function. lbe rcst of the book builds on Illis material. To urKlerstand the conceptS in Ihis chapter fully, the reader need to have only an understanding of the basic concepts in linear algebra: addition and multiplication of vectors by scallUl;, linear independence and dependence, orthogonal bases, basis set. vector spaces and fUllClion spaces and projection of vectorffunction on to the bases. TIle chapter introduces the concept of nested spaces. which is Ille comer stone of mulliresolUlion analysis. Dauhechies' wavelets are also introduced. The chapter concludes wilh a note on the fact that moSt of the wavelets art fractal in nature and that iterative methods are required to display wavelets. Designing wavelets is traditionally camed out in the Fourier domain. Readers who are IlOl experts in Fourier analysis usually find the theoretical arguments and terminology used quite ntl Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 14. xviii • Pref:>C(: to the l-irst Edition baffling and tOlally out of Ihe world. This book. the~fore. adopts a lime domain approach 10 designing. lltc: orthogonality and smoothness/regularity constraints are di~ctly mapped on to constraints on the scaling and wavelet filter coefficients, which can then be solved using solvers available in Microsoft Excel-a spreadst.eet package or a scientific computillion package like MAn.AB or MOlhematica. Engineers onen view signal processing in terms of filtering by appropriate filters. Thus. Chapter 6 is devoted to eSlablish the relationship between 'signal expansion in terms of wavelet bas.c:s· and the ·filter bank· approach to signal analysis/synthesis. Chapter 7 discuss.c:s tile theory behind parametric wa,·elets in an intuitive way I1Ither than by using rigorous mathematical approach. The chapter also discusses various methods of plotting scaling and wavelet functions. The focus of Chapter 8 is 011 biortoogonal wavelets which is relatively a new concept. To drive oome this concept to the readers, biorthogooality is explained using linear algebl1l. The chapter then goes on to discuss the design of elementary S-spline biorthogonaJ wavelets. Chapter 9 addresses orthogonal wavelet design using the Fourier domain approach. Chapter 10 is devoted to the lifting scheme which provides a simple means to design wavelets with the desil1lble propetties. lltc: chapter also shows how lhe lifting scheme allows faster implementation of wavelet decomposition/reconstruction. Chapters 11 to 13 describe applications of wavelets in Image Compression, Signal Denoising and Computer Graphics. The notations used in Chapter 13 are that used by ~searchers in this patticular area and could be slightly different from those in the rest of the chapters. To make the book more useful 10 the readers. we propose 10 post lhe teaching malerial (mainly PowerPoinl slides for each chapter, IlIld MATLABlExcel demonstralion programs) at the companion webs-ile of the book: www.umritG.tdultrnlpublictdionslwu.vrkts. We earnestly hope Ihat this book will initiate several persons 10 this exciting and vigorously growing area. Though we have spared no pains 10 make this book free from mislakes, some errors may slill have survived our scrutiny. We gladly welcome all com:ctions. recommeOOatioos, suggestions aOO constructive criticism from our readers. K.P. SOMAN K.I. RAMACHANDRAN Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 15. Acknowledgements First and foremost. we would like 10 express our gratitude 10 Brahmachari Abhayamrita Chaitanya. who persuaded us to lake this project. and never ceased to lend his encouragement and support. We thank Dr. P. Venkal Rangan. Vice Chancellor of the university and Dr. K.B.M. Nambudiripad. Dean (Research}-our guiding Slars--who c,mlinunlly showed us what perfection means and demanded perfection in everything Ihal we did. We would like to thank, especially, Dr. P. Murali Krishna, a scientist at NPOL, Cochin, for his endearing support during the summer school on 'Wavelets Fractals and Chaos' thal we conducted in 1998. It was then Ihal we learned wavelets seriously. We take this opportunity 10 thank our research students C.R. Nitya. Shyam Divakar, Ajilh Peter, Sal1lhana Krishnan and V. Ajay for their help in simplifying the concepts. G. Sreenivasan and S. Soornj. who helped liS in drnwing the various figllres in the textbook, dcserve a special thanks. Finally. we express our sincere gl3titude to the editors of PHI Learning. K.P.SOMAN K.I. RAMACIiANDRAN N.G. RESMI Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 16. The Age of Wavelets INTRODUCTION Wavelct analysis is a new development in the area of applied mathematics. 'Illey were firsl introduced in seislTIQlogy to provide a time dimension to seismic analysis that Fourier analysis lacked. Fourier analysis is ideal for studying stationary data (data whose statistical propcnics are invariant over time) bul is nol well suited for studying data with transient events that cann<ll be statistically predicted from the data's past. Wavelets were: designed with such non-stationary data in mind, and with their generality and strong results have quickly become useful 10 a number of discipl incs. I.I THE ORIGINS OF WAVELETS-ARE THEY FUNDAMENTALLY NEW? Research can be thought of as a continuous growing fractal (see Figure 1.1) which often folds bad: onlo itself. This folding back definitely occurred several times in tne wavelet field. Even though as an organized research topic wavelets is less than two decades old, it arises from a "ollstellat;on of related eonc:t:pts developed o""r 11. period of ne;orly two ""nturies, repell.ledly redisrovered by scientists wbo wanted to solve techni"al problems in their various disciplines. Signal processors were seeking a way to transmit clear messages o~r telephone wires. Oil prospectors wanted a better way to interpret seismic traces. Yet "wa~lets" did 001 become a oousehold word among scientists until the theOf)' was liberated from the di~rse appliCalions in which it arose and was synthesiud into a purely mathenwical thCQf)'. This synthesis, in turn, opened scientists' eyes to new .ppliCalions. Today, for example, w.""lets are !K)( only the workhorse in computer imaging and animation; they also are used by the FBI to encode its data base of 30 million fingerprints (see Figure 1.2). in the future. scientists may put w.~kt analysis for diagnosing breast cancer, looking for heart aboonnalities (look at Figure 1.3) or predicting the weather. • Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 17. 2 • Insighl inlo Wavelets-From Theory 10 Praclice FIGURE 1.1 Fnoctal. Wt fold Net on i~lf. FIGURE 1.2 An FBI-digilized left thumb fingerprint (1lIc: image on the left i. the original; the one on the right is reronsuuccr:d from a 16:1 """'pI"'ssion.) ,., LO -< 0.' '" 0.' - Hcallhy - H~an failun: 0.' 0.0 0.' 0.' 0 3 0.' , FIGURE I.J MuhifractaJ 'pec(fum of hcar1 heat oscillations. h is a ""ph of singularity mcasun: versus flllCl&l dimension. Thi'i spWrum clplum I di ff~n[ kind of iofoonalion Ihill CIIlOOl. be cljlluml by a '1nl"''''')' spectrum. Wn-.:1c1S an: used for muhifraclal spectrum estimation. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 18. The Age of Wavelets • 3 1.2 WAVELETS AND OTHER REALITY TRANSFORMS Wavelet analysis allows researchers to isolate and manipulate specific types of p<lttems hidden in masses of data, in much the same way our eyes can pick out the trees in a forest, or our ears can pick out the flute in a symphony. Otle approach to understanding how wavelets do this is to stan with the difference between two kinds of sounds-a luning fork and lhe human voice (see Figures 1.4 and 1.5). Strike a luning fork and you get a pure tone that lasts for a 'ery long time. [n mathematical theory, such a tone is said to be "localized" in frequency, that is. il consists of a single note with no higher-frequency overtol1oe.'l. A spoken word. by cOntl1lSt, laslll for only a second. and thus is "localized'" in lime. It is not localized in frequency because the word is not a single tone: but a combination of many different frequencies. f f f V V I'TGURE lA Graphs of the sound WllveS prodoced by • tuning fQr< (top) and thr spol.C1 wool "grea.ly" (bottom) iliu!MItc the diffm:l"ICc bet'.ieen • tone I<x:alized in freqUfflC)' and one I<x:ali«<i in ti..... "The tuning fork produces a .imp'" "si"" wa>·c". 1/ FIGURE 1.5 A w."" and a _velcL Graphs of the sound waves produced by the tuning fork and human voice highlight the difference, as illUlitrated here. 1be vibrations of the tuning fork trace out what mathematicians call a sine wa~. a smoothly undulating curve that, ill theory, could repeal forever. In contrast, the graph of the word "greasy" contains a series of sharp spikes: there are nO oscillations. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 19. 4 • Insight into Wavelets-From Theory to Practice In the nineteenlh century, mathematicians perfected what might be called the tuning fork version of reality, a theory known as Fourier u1Ullysis. lean Baptiste loseph Fourier, a French mathematician, claimed in 1807 that any repeating waveform (or periodic function), like the luning fork sound wave, can be cllpressed as an infinite sum of sine waves and cosine waves of various frequencies. (A cosine wave is a sine wave shifted forward a quarter cycle.) A familiar demonstration of Fourie(s thcory occurs in music. When a musician plays a note, he or she creates an irregularly shaped sound wave. n.e same shape repeats itself for as long as the musician holds the note. Therefore, according to Fourier, the note can be separa~d into a sum of sine and cosine waves. The lowest frequency wave is called the fundamental frequency of the note. and the higher fn::quency ones are called o,·ertoncs. For example, the note A. played on a violin or a nute, has a fundamental frequency of 440 cycles per second and ovenones with frequencies of S8O. 1320. and so on. Even if a violin and a flute arc playing the same note, they will sound different because their ovenones have different strengths or "amplitudes". As music synthesizers demonstrated in the 1960s, a very convincing imitation of a violin or a nute can be obtained by recombining pure sine waves with the appropriate amplitudes. That, of course. is ellactly what Fourier predicled back in IS07. Mathematicians later elltendcd Fourier's idea to non-periodic [unctions (or waves) that change over time, rather than repeating in the same shape forever. Mosl real-world waves arc of this type: say, the sound of a m(){or that speeds up. slows down. and hiccups now and then. In images, too. the distinction between repeating and non-repeating patterns is imponant. A repeatins pattern may be seen as a texture or background while a non-repeating one is picked out by the eye as an object. Periodic or repeating waves composed of a discrete series of overtonc:s can be used to repres.c:nt repeating (background) pauerns in an image. Non-periodic features can be resolved into a much more eomplu spectrum of frequencies, called the I'-ourier transform, just as sunlight can be separated into a spectrum of colours. The Fourier transform portrays the structure of a periodic wave in a much more revealing and conccntraled form than a traditional graph of a wave would. For elllmple, a rallle in a motor will show up as a peak at an unusual frequency in the Fourier tnulsform. Fourier transforms have been a hit. During the nineteenth century they solved many problems in physics and engineering. This promirtence led scientists and engineers 10 think of them as the preferred way to analyze phenomena of all kinds. This ubiquity forced a close examination of the method. As a n::SUII, througOOut the twentieth century, mathematicians, physicists. and engineers came to realize a drawback of the Fourier transform: they have trouble reproducing transient signals or signals with abrupt changes, such as the spoken word or the rap of a snare drum. Music synthesizers. as good as they are, still do not match the sound of eoncert violinists because the playing of a violinist contains transient features--such as the contact of the bow on the string-that arc poorly imitated by representations based on sine waves. The principle underlying this problem can be illustrated by what is known as the Heisenberg Indeterminacy Principle. In 1927, the physicist Werner Heisenberg stated that the position and the velocity of an object cannot be measured CIlactly at the SIlIl"Ie time even in theory. In signal processing terms, this means it is impossible to know simultaneously the exact frequency and the exact time of occurrence of this frequellCy in a signal. In order to know its frequency, the signal must be spread in time or vicc versa. In musical terms. the trade-off means that any signal with a short duration must have a complicated frequency spectrum made of a Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 20. The Age of Wavelets • 5 rich variety of sine waves whereas any signal made from a simple combination of a few sine waves must have a complicated appear:J.nce in the time domain. Thus, we can't e;t;pect to reproduce the sound of a drum with an orchestra of tuning forks. 1.3 MANAGING HEISENBERG'S UNCERTAINTY GHOST Over the course of the twentieth century, scientists in different fields struggled to get around these limitations, in order to allow representations of the data to adapt to the nature of the infomullion. In essence, they wanted 10 capture both Ihe low-resolution forest- the repealing background sigr"lal- aoo the high-resolution trees-the individual, localized variations in the background. Although the scielllislS wcre trying to solve the problems panicular to their respective fields. they began to arrh'e at the same conclusion- namely. Ihat Fourier InlIIsforms themselves were 10 blame. They also arrived at essentially the same solution. Perhaps by spliuing a signal into components that were not pure sine waves. it would be possible to condense the information in bolh the time and frequerocy domains. This is the idea that would ultimotely be known as wa"l~lcts. Wavelet transforms allow timc·frcQuency localisation The first entrant in the wavelet derby was 0 Hungarian mathematician nanw:d Alfred Haar, who introduced in 1909 the furoctions that are now called Haar ",..ave1ets. These functions consist si mply of a short positive pulse followed hy a short negative pulse. Although the short pulses of Haar wavelets are e;t;cellent for teaching wavelet theory, they are less uscful for moSt applkations because they yield jagged lincs instead of smooth curves. For example. an image reconstrtlcted with Haar wavelcts looks like a cheap calculator display and a Haar wavelet reconslrtiction of the sound of a i1me is too homh. From time to time over the ne~t several decades, other precursors of wavelet theory arose. In the 1930s, the English mathematicians 10hn Linlewood and R.E.A.C. Paley developed a method of grouping frequencies by octaves thereby creating a signal that is well localized in frequency (il$ spectrum lies within one octave) and also relatively well localized in time. In 1946, Dennis Gabor, a British-Hungarian physicist. introduced the Gabor transform. analogous to the Fourier transform. which separatcs a w""e into "time-frequency packets" or "coherent states" (see Figure 1.6) that have the gre.1test possible simultaneous localization in both time and frequency. And in the 1970s and 19805. the signal processing and image processing communities introduced their ()wn versions of wavelet analysis. going by such names as "subband coding:' "quadrature mirror filters" and the "pyramidal algorithm". While not precisely identical, all of these techniques had similar features. They decomposed or transformed signals into pieces that could be localized to any time interval and could also be dilated or contracted to analyze the signal at different scales of rcsolulion. These prC<:urso(S of wavelets had one other thing in common: no one knew about them beyond individual specialized communities. But in 1984, wavelet theory finally came into its own. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 21. 6 • Insighl into Wa~lelS-From Theory 10 Pnctice •• • • • • • • • • •• •- -~I't------- --,,------ FIGURE 1.6 Oocomposing lignal inlo t;"",·f~l>e1lCy alom~. BOllom of the pictun: ~ twOtime fn:qumcy atom•. The signal and the time·f""lllCnc:y map i. """"'n "'->ve that. 1.4 HISlURY OF WAVELET FROM MORLET TO DAUBECHIES VIA MALLAT lean Morle[ didn't plan to start a scientific revolution. He was merely trying [0 give geologists a beller way 10 search for oil. Petroleum geologists usually locale underground oil deposits by making lood noises. Because sound waves lnIvel through different materials at differenl speeill;. geologisl.'l can infer whal kind of malerial lies under the surface by sending seismic waves inlO the ground and measuring how quickly they rehourKl. If the waves propagate especially quickly through onc: layer. it may be it salt dome. which can trap a layer of oil underneath. Figuring oot just how the geology translates into a sound wave (or vice versa) is a tricky mathematical problem. and one Ihat engineers traditionally solve with Fourier analysis. Unfortunately. seismic signals contain lots of transienl.'l-llbrupt changes in the wave as it passes from one rock layer to another. Foorier analysis spreads thal spatial information oot all over the place. Morlet. an engineer for Elf-Aquitaine. developed his own way of analyzing the seismic signals to creale componenl.'l that were localized in space. which he called Wllvdets of constant shape. Later. they WQUld be known as Monet wllvelets. Whether the components are dilated. compressed or shifted in time. they maintain the same shape. Other families of wavelets can be huilt by Illking a different shape. called a motm-r WIIvelel. and dilating. compressing or shifting it in time. Researchers woold find that the exact shape of the mother wavelet strongly affeclS the accuracy and compression properties of the approximation. Many of the differences between earlier versions of wavelets simply amounted to different choices for the mother wavelet. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 22. The Age of Wavelets • 7 Morlel's method wasn't in the books but it seemed to work. On his personal computer. he could separate a wave into its wavelet components and then reassemble them into the original wave. But he wasn't satisfied with this empirical proof and began asking other scientists if the method was mathematically sound. Morlct found the answer he wanted from Alex Grossmann. a physicist at the Centre de Pltysique ThOOrique in MlIIOeilles. Grossmann worked with Morlet for a year to confirm that waves could be re.constructed from their wavelet decompositions. In fact. wavelet transforms turned out to work beller than Fourier transforms because they are much less sensitive to small errorn in the computation. An error or an unwise truncation of the Fourier coefficients can turn a smOOlh signal into a jumpy onc or vice versa: wavelets avoid such disastrous consequences. Morlet and Grossmann's paper. the first to use the word "wavelet", was published in 1984. Yvc:s Meyer. currently at the Ecole Normale Suptrieure de Cachan. widely acknowledged as one of the founders of wavelet theory. heard about lheir work in the fall o f the same year. He was the first to realize the connection between Morlet's wavelets and earlier mathematical wavelets. such as those in the work of Littlewood and Palcy. (Indeed. Mt!yer has counted 16 separate rediscoveries of the wavelet concept before Morlet and Grossmann'S paper.) Meyer went on 10 discover a new kind of wavelet. with a mathematical property called orthogonality that made the wavelet lTansform as easy to work with and manipulate as a Fourier transform. C·Qrthogonality" means that the information captured by one wavelet is completely independent of the information captured by aoother.) Perhaps most importantly. he became the IlCXUS of the emerging wavelet community. In 1986. St6phane Mallat (see Figure 1.7). a former student of Meyer's who was working on a doctorate in computer vision. linked the theory of wavelets 10 the existing literature on nGURE 1.7 Sephane Malla! (CMAP. Ecoic PoIyIcchniq.... 911211 Palais.eau Cedeo.. Fr'anc:e). sUbband coding and quadrature mirror filters which are the image processing community's versions of wavelets. The idea of multiresolution ana1ysis--that is. looking at signals at different scales of resolution-was already familiar to experts in image processing. MaUat, collaborating with Meyer. showed that wavelets are implicit in the process of multiresolution analysis. Thanks to Mallat's work. wavelets became much easier. One could now do a wavelet analysis without knowing the formula for a mother wavelet. The process was reduced to simple Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 23. 8 • Insight into Wawlets-From 'Theory to Practice operations of avel'llging groups of pixels IOgether and taking their differences, over and over. 'The language of wavelets also became more comfortable to e l~trical engineers, who embraced familiar teons such as "'filters"', "'high frequencies" and "Iow frequencies". The final great salvo in the wavelet revolution was fired in 1987, when Ingrid Daubechies (see Figure 1.8), while visiting the Courant I nstitUl~ at New York University and later during tlGURE 1.8 Ingrid Daub«hits (Prof...,.,.-, Ikpartment 01 M3lhema.tiC$. Princo:ton University). her appointment al AT&T Bell Laboratories, discovered a woole IICW class of wavelets [sec Figure l.9(b)] which were not only otthogonal (like Meyer's) but which could be implemented using simple digital filtering ideas, in fact, using shon digital filters. The new wavelets were almost as simple to program and use as Haar wavelets but they were smooth, without the jumps of Haar wavelets. Signal processors flOW had a dream tool: a way to breair;: up digital data into contributions of various scales. Combining Daubechies and Mal1at's ideas. there was a simple. orthogonal transform that could be rapidly computed on modem digital computers. ,, /~,) , I o., ,n 0 "-, -0., 1 , o , 3 (a) Haar wavelet (b) Daulxdlic. · 4-...p Muna( WlI>"ek( FIGURE 1.9 (Cont.). Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 24. 11< Ag' 0 rw "''" , • 9 0' 0.' 0.2 0 /-0.2 - 0.4 - 0.6 -5-4 - 3 - 2 - 10 , , , .(c) GaIIsi... ".""lel n GUR£ 1.9 GrapJu of .....·era! different types of wa."lea The Daubechics wa'eleLS have surprising features-such as intimate connections with the theory of fractals. If their graph is viewed under magnification, characteristic jagged wiggles can be seen, no matter bow strong Ihe magnification is. This exquisite !;omplexity of detail means, there is no simple formula for these wavelets. They arc ungainly and asymmeuic; nineletnth-eemury mathematicians would have recoiled from them in hOllOr. But like the Model-T Ford. they are beautiful because they work. The Daubechies wavelets turn the theory into a practical tool that can be easily programmed and used by any scientist with a minimum of mathematical training. 1.4.1 Different Communities of Wavelets There are several instances of functions (see Figures 1.9 and 1.10) that can be used for muhiresolution analysis of data. All of them are referred to as wavelets. Some of these instances =, • Dyadic lrans/arts and di/mes of Olle funC/ion: These are classical wavelets. • Wavelet fHlclcets: This is an extension of the dassi,al wavelets whi,h yields basis fullO;lions with bener frequency localizalion at the ,ost of slightly more expensive tnmsfonn. • Local trigonometric basu: The main idea is to work with cosines and sines defined on finite intervals combined with a simple but very powerful way 10 smoothly join Ihe basis functions at the end points. • Mulliwavtlets: Instead of using one fixed fun"ion 10 lranslate and dilate for making basis fun,lions. we use a finite number of wavelet functions. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 25. 10 • Insighl inlo Waveku From Theory 10 Prac:ti~ ,.• " ,1----"1 I1 -<l.' L_~__-'-'-'---_~__---' -0.8 -0.4 0 0.4 0.8 (a) Model _velet ,.• ,., , " I -<l., -. , , •(b) Mexican Hal .....""lel FIGURE 1.10 Wavelets uoed in conlinuoul wavelet tranJ(O,TU. • Second gene,flIion wlll.'elefS; Here: one entirely abandons the idea of translation and dilation, This gives elClra flelCibility which can be used 10 construct wavelets adapted to im:gular samples. 1.4.2 Different Families of Wavelets within Wavelet Communities Like humans, wavelets also live in families. Each member of a family has cenain common features that distinguish each member of a family. Some wavelets aJ"e fQr CQntinuQUs wavelet transform and OIhe1; are: for discre:te wavelet transform. Some of the families that belong to 'classical' community of wavelets are (see Figures 1.10 and 1.11): Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 26. g. • •The A of W vcLets 11 2 , 0 I -, o 2 3 • , nGURE 1.11 Coinel wavelet. • Wavelets for continuous wavelet transfoml (Gaussian. Morlel. Mexican Hat) • Daubechies Maxllat wavelets • Symleu • Coillcts • Biorthogonal splille wavelets • Complex wavelets 1.4.3 Interesting Recent Developments • Wavelet based denoising has opened up other fields and important tcchniques such as dictionaries and non-linear approximation. smoothing and reduction to small optimizalion problems are real achievements. • Wavelets have had a big psychological impact. People from many different Ilreas be!;ame interested in time-frequency and time-scale transfonns. There has beCII a revolution in signal processing. There is less specialization and the subjcct is now opened to new problems. More than just a simple tool, wavelet ideas prompt new points of view. Some of the best ideas aren't writtcn down. The big diffcrence will come from new gencnuion rescarchers flOw growing up amidlll wavelet ideas. • Wavelets have advan~d our urKIelltanding of singularities. The singularity spectrum completely characterizes the complcxity of the data. Now We must go to an undelltanding of the underlying phenomenon to get an equation from the solution. Wavelets don't give all the ansWCIl but lhey force us to ask right questions. • Wavelets can be used to distinguish coherent VefSUS incoherent pans of turbulence in fluid flow. They give some information but don't entirely solve the problem. • The results on regularity, approximation power and wavelet design techniques have led 10 significant developments in signal and image processing. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 27. 12 • Insigh.t into Wavelets-From Theory to Practice 1.5 WAVELETS IN THE FUTURE Wavelet ar.alysis is unquestionably one of the beSt achieveme:nts of matllematics in the twentieth century. Its initial applications were mainly in sparse signal representation and denoising of signal and images. Probably, it was due to the fact that the theory was hard to understand at that time. In recent years wavelets have spread in many fundame:ntal sciences other than mathematics. such as me:dicine, biology, geophysics. physics, mechanics. economics, etc. This is e,·ident from the title of the books that are appearing. At present there are more Ihan hundred books on wavelets most of which are on mathematical aspects of wavelet theory. Slowly but steadily many books are appearing in specific application domains. Following are some: of them. Wavelets in medicine and biology (by Akram Aldroubi and Michael Unser, CRC Press, 1996) This book explores application of wavelets in me:dical imaging and tomography biomedical signal processing. wavelet based modeling of problems in biology. Wavelets in physics (edited by J.C. van den Berg, 1999) This book details the application of wavelets in many branches of pnysics such as acoustics, spectroscopy. geophysics. astrophysics, fluid mechanics (turbulence), medical imagery, alomic physics (laser-atom interaction), solid state physics (structure calculation). Wavelets in chemistry (editor Beata Walczak, Elsevier, 2000) This book is an introduction to wavelets intended for chemiSts. It covers all impor1ant aspects of wavelet theol)' and presents wavelet applications in chemistry and in medical engineering. It is addressed 0 analytical crn:mists. dealing wilh any type of spectral data, organic chemists, in"Olved in combinatorial chemistry, chemists involved in chemome:trics and engineers involved in process control. ChcmomClrics: From basics to wavelet transfonn (by, Foo-Tim Chau, Vi- Zeng Liang, Junbin Cao, Xue-Cuang Shao, WHey, 2004) This book explores the use of wavelets in ·chemome:trics based signal processing'. Wavelets for sensing technologies (by K. Chall, Cheng Peng. Arlech House Publishers, 2003) This reference book focuses on the processing of signals from Synthetic Aperture Radar (SAR). Specific remOle sensing applications presented in the book include noise and clutter reduction in SAR images. SAR image compression. texture and boundary enhancement in SAR images, direction·al noise removal and general image processing. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 28. 'TlIe Age of Wavelets • 13 Wavelet and wave analysis as applied to materials with micro or nano- structure (by C. Cattani and J. Rushchitsky, World Scientific Publishing Co.. 200?) This book explores physical wavelets as solution of certain POEs anslng in solitary wave propagation in elastic dispersive media. Three different types of physical wavelets. namely. Kaiser physical (optical and acoustic) wavelets. Newland harmonic wavelets and elastic wavelets are discussed. Optical wavelets own this name be<:ause they satisfy Ihe linear wave equations of optics in the simplified form of Maxwcll electromagnetic equations. The acoustic wavelets were proposed as those wavelets satisfying tlK: linear wave equations in acoustics. Harmonic wavelets were suggested by Newland. The Newland hannonic wavelets can be referred to physical family of wavelets for many reasons, btu mainly be<:ause they are especially proposed for the analysis of physical problcms on oscillations. Ultra-low biomedical signal processing: An analog wavelet filler approach (or pacemakers (by Haddad and Serdijn. Springer, 2008) In ultl1il low-power applications such as biomedical implantable devices. u IS not suilllble to implement the wavelet transfonn by means of digital circuitry due to Ihe relatively high power consumption associated with the required NO converter. Low-power analog realization of the wal"elet transfoml enables its application in vivo, e.g.. in pacemakers. where the wavelet transform provides a means to extremely reliable cardiac signal detection. The methodology presented in this book focuses on the development of ultra low-power analog integrated circuits that implement the required signal prucessing. taking into account the limitations imposed by an implantable device. Wavelets in wireless communication Though there is no exclusive book on applicalion of wavelets in communication. there are hundreds of research papers available on the topic. Wavelets give a new dimension to the capacity of wireless communication. It provides "Waveform diversity'". to the physical diversities cum:ntly exploited. namely. space (muhi-antenna wireless communication system), frequency and time-diversity. Signa.! diversity which is similar to spread spectrum systems. could be: exploited in a cellular communication system, where adjacent cells can be designated different wavelets in onJer to minimize inter-cell interference. In addition wavelets provide the following: (a) Sc",i-u,birrury divisi<m of ,I.e sigll"l spuc" Ulld ",,,/,i"'lt: S1sr""'11: Wavetet transform can create subcarriel'S of different bandwidth and symbol length. Since each subcarrier has the same time-frequency plane area. an increase (or decrease) of bandwidth is bound to a decrease (or increase) of subcamer symbol length. Such characteristics of the wavelets can be: exploited to Create a muhirate system. From 11 communication perspecti·e. such a feature is favourable for systems that must support multiple data streams with different transport delay requirements. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 29. 14 • Insight into Wa~le!s-From Theory to Practice (b) Flexibility ...irh ,ime-frequency ,i/ing: Another advantage of wavelets lies in their ability to arrange the time-frequency tiling in a manller that minimizes the channel disturbances. By nexibly aligning the time.frequency tiling. the effect of noise and interference on the signal can be minimiZo!d. Wavelet based systems are capable of ovt'rcoming kTlOwn channel disturbances at the transmitter. rather than waiting to deal with them at the receiver. Thus. they can enhance the quality of service (QoS) of wireless systems. (c) Semi,i"iry ro chmmei ~ffecl5; The performance of a modulation scheme depends On the set of waveforms that·the carriers use. 11le wavelet scheme. therefore. holds the promise of reducing the sensitivity of the system 10 harmful channel effects like inter-symbol interference (ISO and inter-carrier interference (ICI). (d) Vltra widdxmd llpplicmiolls: Impulse radio. or ultra wideband (UWB) radio. is a promising new technology for wireless communicruions. Rather than modulating the information on a carrier. the data is transmiued using a coded series of very narrow pulses. carrying information in the time and the frequency domain. Wavelet bases are good candidates for these pulses. Figure 1.12 shows communication engineering areas where wavelets are fiBding new applications. ChanllCI clumlct.erizluion I. ChanllCl modeting 2. Electrorroagnctic compuuuions and "Jlltnna desis.n 3. SpeW estimation COlnitive rmio Inl~11i~nt co;>mmunkation syst~ms Uhra widcband communication l. Impulse: radio 2. Multil>and OFDM Muhiple access communical;on t. COMA 2. SCDMA 3. MC-COMA [nlt"l'f=e 1. Signal denoi.inl 2. O'lta amval estimation 3. Intcrf"= mitigation 4. ISI. ICI mitil'llion Modulation aod rnuhipte.in, l. Wa,~ 5ILapin, 2. Siolle carrier modulalion 3. MUhi-carrier modulation 4. FllIC1al modulation ~ . Multiplex;n, Networking 1. Power con..........tion 2. Traffic proiction ). Net"'-ort< tnlffit modeling 4. Data rccon<IJUCtioo 5. Disuibull'(] data processinl FIGURE 1.12 WaV(:iets in wireles~ communication. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 30. TIw: Age of Wavelet. • IS In summary, we may safely bet that wavelets are here to stay and they have a bright future. Of course wavelet do nOI solve every difficulty, and must be continually developed and enriched. We can expecl proliferation of specialized wavelets each dedicated to a panicular type of problem and an increasingly divcrse spectrum of applications. 1.6 WHAT HISTORY OF WAVELET TEACHES US When asked 10 justify the value of mathematics, mathematicians often point out that ideas developed 10 solve a pure mathematical problem can lead to unexpected applications years later. BUI the story of wavelets paints a more complicated and somewhat more interesting picture. In (his case, specific applicd research led to a new theoretical synthesis, which in lurn opened sciemists' eyes to new applications. Perhaps the broader lesson of wavelets is that we should not view basic and applied sciences as scparate endeavours. Good science requires us to see both the theoretical forest and the practical trees. SUMMARY Though wavelet is an organized research IOpic. it is only two decades old, and has been in use for a long time in various disciplines under different names. Morlet and Grossman were the first 10 use the word 'wavelet'. Stephan Mallat brought out the relation between wavelet methodology used by Morlel and filter bank theOI)' used in image processing applications. TIle greatest contribution came from Ingrid Daubechies who put tile whole theory on a strong matherruuical foundation. Wavelets are now emerging as one of the fastest growing field with application ranging from seismology to astrophysics. Suggested Further Readings Amara's wavelet page htlp:llwww.amara.com/currmll....ol·e/u/uml. An Introduction 10 wavelets: hllp:llwww.amara.comll£££K"(JI.tII£££....al.eiel.hlm/f Dilubel:lies, I.. Where do wavelet come from? A perwnal point of view, Proceedings of liltl 1£££ Specia/Issue on W'lI'e/eIS, S4 (4), pp. SIO-SJ3. April 1996. Sweldons. W.. Wavelets what neltt? Procudill8s of 'he IEEE. Vol. 84. no. 4 April 1996. Wa'elets; Seeing the Forests and the Trees, hltp:llw"'l'o'.M)"om/discol'ery.orgl. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 31. Fourier Series and Geometry INTRODUCTION Deemed one of the crowning achievements of the 20th Century, the Fourier series has applications Ilia! are far n:aching in varioLls fields of science and mathematics. The Discrete Fou';". "ansfurm i. one p"I1icular tool widely u&ed in today'~ age of compulers and solid state electronics. From graphic equalizers in stereos 10 the moSI advanced scientific sampling software, the userullless of this mathematical feal is ISlolJllding. Most of the readers of this book might already know this fact but many of them may nor be knowing tha! Fourier series. as a mathematical tooJ in analy:ting signals, h:J.S strong connection wilh geometry. In this chapter our aim is to understand the theory of Fourier series from a geometrical viewpoillL We assume that you are familial" with vector algebra and co-ordinate geometry. If you are really comfonablc in using Ihc!iC topics, you can very well understand what Fourier series is, computation and interprelalion of Fourier series coefficients, Fourier transform. discrete Fourier transform. fast Fourier transform. etc. l1iere exists a strong analogy between what you do in vector algebra and what you do in signal processing. This analogy will help you to visualize and give interpretation to the processes and output of the processes that you do on signals. Development of tllis geometrical mental picture about signal processing is the main aim of this chapter. Einstein once said "Imagination is more imponant than knowledge". Signal processing, which many students consider as dry and non·intuitive. demands imagination and some form of abstract thinking from the pan of students. In fact. it requires only that. Once )'OIl develop a conceptual link between geometry (vector algebra) and signal processing, you need nO( remember any fonnula and every formula that you come across will become transparent to your mind. Here we will refresh concepts from "Ulor spoce. The detailed exposition of vector space is given in Appendix A. In this imroductory chapter, we won't try 10 be 100% mathematically precise in our staternems regarding vector space. Instead our aim is to relate geometry and Fourier series in an intuitive way. .. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 32. Fourier Serie$ and Geometry • 17 2.1 VECTOR SPACE Any vector in 3-dimensional ,;pace can be represented as V = (JI + bl + et. I, J.k are unit vectors in three onhogonal directions. This orthogonality is expressed by requiring the condition that r.J=k· J= r.k = O. The orthogonalily condition ensures that lIle representation of any - - -''eClor using i. j. le is unique. 2.1.1 Bases We call i.].k as bases of space 9{1. By this we mean that any vector in 9{l can be represented using I. J. k vectors. We also say i. J. k span the space 9{1. Let V= al + b] + et be a vector in 9{1. Continuously changing scalar a. b. c: we will go on get new vectors in 9{1. We imagine that. set of all such ,'eClors constitute the vector space 9{1. We express this truth by Span (al + b] + ct) iI 9{1 .,' 2.1.2 Ortnonormalily Norm of a vector V =al + b] + ef is conventionally defined as Ja2 + b2 +c2 and denoted as IV I. IV I is equal to Jv. ii . We interpret this quantity as the magnitude of the vector. It must be noted that mher definition,; are also possible for norm. What is interest to us is that our basis vectors of 911. Le.. T. J. k are having unity nonn. Hence the name unit vectors. They are also onhogonal. Therefore. they are called orthonorma1 vectors or orthonorma1 bast'$. How does it help us'! Given a vector V= al + bJ + cf. we can easily (ell to which direetion I,], k. the vector V. is more inclined by noting the value of a. b. c. 2.1.3 Projection Given a vector V how shall we find its normal component vectors or in other words how shall - - - -we find the scalar coefficients a. b and c'! Wc project V on to bases i , j, le to get a, b and e. This is the direct result of orthogonality of our basi,; vectors. Since V = (JI + bJ + ef, Y' I =«JI + bJ + ef)· I =(J ii· J = «JI+bJ+ek)·J=b >od ii· f =(al+b} +cf)·k=c Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 33. 18 • Insight inlo Wa~cLcts-From llIeory to Pmctice So projecting vcctor 0 on to a base of space 9J (spanned by a sct onoonormal vectors) gives the corresponding coefficient (component) of the base. There is a parullcl for this in Fourier scries analysis. With these ideas firmly cngral'cd in your mind. you arc ready to understand and visualize Fourier series. 2.2 FUNCTIONS AND FUNCTION SPACES In vcctor space. we represent a vector in terms of onoogonal unit vectors called bases. Fourier serics is an extension of this idea for functions (cleclrooics engineers call them as signals). A function is expressed in terms of a set of orthogonal funclions. To com:13le with ideas in geometry. we now need to introduce the concept of onhogonality and norm with respect to functions. Note Ihat. funclion is quantity which varies with respect 10 one or more running parameter. usually lime and space. Orthogonality of functions depends on multiplication and integration of functions. Multiplication and integration of function must be toought of as equivalent to projection of a vector on to another. Here comes the requirement for abstract thinking. mentioned at the stan of tile chapter. 2.2.1 Orthogonal Functions Two real functions fl(/) and f2(/) are said to be onhogonal if and only if -JJ;COiJ(I)dl =0 (2.1) - Ponder over this equation. What we are doing? We are first doing a point by point muliiplicll1ion of t...."Q function. TIlen we are summing up the area under the resulting funclioo (obtained after multiplication). If this area is zero. we say. the two functions are onhogonaJ. We also interpret it as similarity of functionsfl(t) andfil}. Ifft(f) and fit) vary synchronously, that is, ifft(l) goes upil(t) also goes up and ifft(t) goes downil(t) also goes down then we say the two functioos have high simi larity. Otherwise. tlley are dissimilar. Magnitude or absolute value -of Jf t(l)f2(Odl will be high if they "ary synchrOllOusly and zero if they vary perfectly asynChronously. For example consider two functions ft(t) = sin t and 12(1) :: cos I for f in the interval (0,211"). Figure 2.1 shows the two functions and their point wise product. Observe that the net area under the cur'C sin I cos f is zero. That is. the llfCa above the I axis is same as tile llfCa below the I axis. For these two functions, it ean be easily shown that. for I in the range of multiples of 211". area under the product curve is always zero. TIlerefore. we say that sin I and cos I are onhogonal in the range (0. 211). Specification of range is very important when yOU say orthogonality of functions. Because the deciding factor is area under the curve which in turn Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 34. o. o. o. , 8 6 •, 0 ~ o. •,< -0. ..-0. -0. , • 6 8 , ,,, ,,, I o ,,,,, /" 'J, /, , V ,,, / '. , • / ( ,,, ••,, I, I i V,,,,,, 6 Fourier Series and Geometry • 19 f ,,,,,, // I " iI' I I . I j , I , . _____ sinl __._ .in 1 • cos 1 ---- ,~ , , ,, ,,, , , ,, , 8 " 12 J4 n GURF. 2.1 Qnh<>gona] fUI>(:'i""•. depends on the range of inlegralion. Now here observe Ihe varialion of sin t and oos /. Whcn sin I is increasing, cos I is de<:reasing and vice 'crsa. For range of I in multiples of 2K, the two functions considered are said to be ortllogonal. The goometrkal analog of f It (t) 11 (/) d/ '" 0 is the dot product, ; .J'" O. For the given , interval. when!,(t) is projected on to!ft) (proje<:tion here mearu multiplying and integrating). if the result is tero • we say Ihe IWO functions are orthogonal in the given interval. Clearly look al Ihe analogy. Taking B do! product in ve<:tor space (multiplying corresponding terms and adding) is equivalenl to multiplying and inlegrating in Ihe funclion space (point wi~ multiplication and integration). The dOl product is mro;;mum when t'NO vectors are oollinear and zero when they arc at 90". This analogy will help )'ou to casily visualize and inlerprel various results in signal processing. 2.2.2 Orthonormal FUllctions Two real fUI>C,j<>"s/,O) a",I/1(1) a", said ,0 be onhonormal if and only if -and f[;(t) [;(t) dl '" I. for i:::: 1. 2. - 2,2.3 Function Spaces Just like unit orthogonaJ set of vectors span vector spaces. orthogonal set of fUilCtions can span function spaces. Consider the following functions defined over a period T: Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 35. 20 • Insight into Wavelets-From Tlleofy to Practice Let us assume that w'" 2l1fT. Then (2.2) (i) u(t)= I forO S t S T (ii) cos " all. n '" I. 2. ...: 0 S 1 S T (iii) sin 1I!l1l. n = 1. 2. ...: 0 S t S T Here we have infinite number of (unctions of sines and cosines. These sets of functions Iln: mUlUally onhogonal in the interval (0. 1). Let us verify the truth through some representative examples. Let us take functions u(t) = I and sin WI for 0 S t S T. These functions and their product function are shown in Figure 2.2. Point wise multiplicar.ion gives the Il'sulting curve as sin (l)/. I> j o·i~ T • I., , 0.' V ~0 / T-0.' "--, -I> " , 03 1/ "'"0 T -03 "- /'-, - 1.5 FIGURF. 2.2 OnI>ogonalily of functiofu. What is the area under the curve sin 0)1 in the interval (0. 7)'! Obviously i!.ero. Mathematically. T T Ju(t) sinllJl dl = fl ·sin(2mfT)dl = 0 ,., We say U(I) = I is orthogonal to sin WI in (0. T ). Do not forget the Il'lar.ion between wand T. Simi larly we can show that u(t) '" 1 is onhogonal to cos rut in (0.7). In general. T Ju(t) s in nox dl '" 0 al50 ,., T fIl()) cos nox dt = 0 for all n '" I. 2•.... ,., Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 36. Fourier Series and Ge<>mctry • 21 Consider cos Wf and sin Wf in (0. n. Figure 2.3 shows the two functions and COITespondillg fuoction obtained after point wise multiplication. Area abo'e and below of the 2 , / - "- ~ ., 0 T "- /'-, -2 • I' , 0.' / "".nQW 0 T -0.' "" /-, - u 0.6 (0.< om "" 00$ fUr 0.2 0 T ...(1.2 V V-0.' -0.6 )'IGURE 2.3 !'Iou of sin WI. 00$ WI and sin (Qf cos QW. resulting curve is same implyillg that the net area is zero. This ill turn imply that cos rut and sin rut in (0. n are orthogollal. Generalizing. Similarly. , fsinllM cos mWfdt = 0 . for all m. n = 1,2.3..... ,.. , Jsill nM sill mM dl = 0 ,..(for all illteger m and n such that m ~ /1). Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 37. 22 • I"sight inl(l Wavd"i5--From 'Theory tu Pra.c:licc Alw , JcosnaN cosmaN dl =0 ,..(for all integer m and n such that m ~ n). Thus, we have a set of functions defined over (0, n such thal they arc mutually orthogonal. 2,2.4 Orthogonal Basis Functions In vector algebra, we know that the unit orthogonal vectors t, j, k span the space :Xl. This imply Ihat any veclor in :x, can be represented by unit vectors 01" bases i. j, k. Similarly the sel of functions, (i) u(l)= I forOSI ST (ii) cosnnn. n = I. 2. ...: 0 S 1ST (iii) sin nW/. n = 1. 2....; 0 S 1ST defined over 0 S t S T span a function space and we denote il as L1 (set of square integrable funclions). Most of the practical signals or functions are assumed 10 belong in the space of Ll. This means Ihal. practically any continuous signal in the interval (0. n can be represented using Ihe above base•. ~tplhematicany. f(t) E L1 = f(t) = 00 + 11'1 COSM +"1 COS2M + ... + ~ sinaN + ~ sin2aN + ... (2.3) Lel us look at the concept from another angle. Consider Ih" sumj(t) for each / in the range (0. T) given in Eq. (2.3). Choose a set of coefficients Do. al..... and bl. bz, .... and make the function (or signal) f(l> in the range (0. T). Now go on choosing different set of coeffici"nts 10 gel a different signal. The set of all such possible signals generated is our signal space Ll in the range (0. 1). Practically all smoothly varying fUllCIions that you can imagine in Ihe interval (0.1) belongs 10 Ll. Or in other words. practically all smoothly varying functions thal you can imagine in Ihe illlerYlll (0. T) can be represented by Ihe equation: f(t) = 00 + 11'1 cos ea + al cos 2ea + ... + bt sin aN + bz sin 2M + ... The number of coefficients required 10 represent a signal depends Qn bow smooth and how fast your signal is varying in the range (0. T). 2.2.5 Orthonormality and the Method of Finding the Coefficients The problem that we are addressing is finding the coefficient set ao. 11'1. bl. 11'1, b-:, etc. In veclor algebra. onhooonnalily of basis vectors allowed us 10 find the component of a vector along a Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 38. Fourier Series and Geometry • 23 panicular base by projecting the vector on to that base. The propeny that lil =Ijl =1.&:1 " I where Ii 1=.Jt·7 and 7.]. k are onhogonal implies thal for a veclor If e 9). if If· i '" a. If· ] = b, V· k =c Ihen V '" ai + b] + ek . [I is a unique representation using Ihe bases i.]. k. To reilerale the principle, 10 find a panitular coefficienl, project the vector on 10 the torresponding basis. 1be same mental pittuTe is applicable to Founer series representation of signals. In the last section. we have foufld out a set of functions that are onhogonal among themselves in a given range (0. n. These functions are onhogonal but we haven't thccked whether they form an orthooormal set. In function spa<;e, onhonormalily of two functions 11(1) and h(l) in (0. T) requires that , , , f/i(/) 12(1) dl = o. fft(t) 11 (t) dl '" 1 , , and ff 2(t) f 2(t} dt '" I , Consider Ihe basis funttion sin nw. f' . . d f'· ['~'J .['~'Jd TsmnM smlltIX 1 = SIn T sm Tt'" 2" , , The result is a function of T and is IlOI equal to I. For any integer n > 0, the integral is TI2. So. we say our sine bases are not normalized. Similarly it can be easily SMwn that, jtosnaw COS lItIX dl =jcose;nt)cos(2-;'1) dl = ~ , , for 11 = 1.2..... So our cosine bases are also oot nonnalized bases. What aboul our finlt basis function (which is also called OC base) u(t) = I. in (0. T)'I A,. , f[2 dt = T. U(I) is also not normalized. , Therefore, none of our bases are normalized. So how shall we normaLize our bases? Multiply all tile sine and cosine bases with J2fT and u(t) '" I with M. N~. f' = = . ' f' .['''''J.[''''J 'T...,2/TsinnM...,2/TsmnMdt= T sm T sm T dl "'T ' "2'" I , , , , f= = 2f [2"'''J [2"'''J 2T...,2fT cosllM...,2fT cosnM dl = T cos -T- cos -r dl = T . 2" = 1 , , Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 39. 24 • Insight into WavdelS-From 1brory to Practice , , JJIIT../Ifrdt:o ~ fl dt:o ~ ' T=I , , The set of nonnaliz.ed bases of Fourier series are: {~, #cos M, #sin M, #cos 2ar, Hsin 2C1X • .•. , gcos IIflJ1. #sin IlflJl ••. .•} All these functions are defined in the interval (0. 1) and ro = 21f1T. Let us now take a functionf(t) ELl defined in the interval (0.1) and express il using the nonnalized sine .and cosine bases defined over the same period (0, T). NOIe that the length of bases are same as the length of the signal. Also note that the furKl.amcntal frequency of the sinusoidal (also 'oosinusoidal") series is given by flJ:o 21f1T. (This is something which students tend to ignore or forget and hence the repetition of the fact in many places.) f ,I , f2 " f2 .(1) :0 Or! 7r + at V"Tcos ar + "I V"TStn fa + ..• 'H 'H'+ a - cos IlCIX + b - Sill liar + ... W T · T (2.4) Here 00. al' ai, .... bj. bi.... are called normalized Fourier series coefficients. Now. given th3!. such a n:presentation is possible. how shall we find out the nonnaliU!d coefficients. The answer is, projeaj(l) on to the corresponding bases. Suppose we want to find a;. Then project fit) on to J21T cos IlltW. that is, a~ = jf(t)H cos mu dt , Note the analogy. In vector algebra, to find a component coefficient, we project the vector on to the corresponding unil vector (base). Let us verify the troth through mathematical microscope. Multiplying both sides of Eq. (2.4) by J21T cos Ilwt and integrating over (0. T). we obtain , Jf(t )·hlT COSIIM' dt , , :0 aO ~. JJUT cos mm dt ,T , , + a; JJ2fT cos wtJ2IT cos 1l{J}l df , , + hi fJ21T sin OX J21T cos lIax dt + ... , Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 40. Fourier Series and Geometry • 2S , + a~J../2/T COS nM ../2/T COS nM dr •, + b~ J../21T sin nM ../21T sin nM dr + ... •= /I' •, , Thus. J10) ../2fT cos nM dr = a; since J../2fT cos nM ../2fT cos nM dr = I and all OIher • •integrals vanish because of orthogonality property of tile bases. , Similarly b~ = Jl(r) ../21T sin nM dr. That is. project 1(1) on to the corresponding •normalized base. What about <lQ1 ProjcctJtt) on to the base w(r) = .JlfT defined over (0. 7). giving , ~ = Jf(l)M dr • Well. we ullderstood how to get normalized coefficients in Fourier representation of signals. To get a particular coefficient simply project 10) on to the com:sponding normalized base. Projection in function space meaRS poiRt wise multiplication and integration of tile functions concerned over the defined period. After obtaining the IIOnnalized coefficients. we substitute in Eq. (2.4) 10 get Founer series representation of the signal. Let us simplify Eq. (2.4) to avoid scalars ..J2IT and .JIlT. Let <lQ IfJT =I/o so that , , <lo = .JlfT<lQ =.JlfT J./liTI{t) dl =~ Jl(t) dl • • (Note that we substituted the Cltpression fo r <lQ to get /10-) Similarly let o~ .J2IT = 0•. Then , , a. =../2IT 0; =.J2IT J1(1).J2/T cos nM dl =: JI(t) cos nM dt • • , , b. = J21T b~ = .J2IT J1(1) ../2IT sin nM dt = : J/(I) sin ntu dl • • Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 41. 26 • JruiShl inlo Wa~lcts From Theory to Practice Thus. we can rewrite Eq. (2.4) as: J{I) "" ao + al cos rut + bl sin rut + ... + a. cos new + b. sin nrut + ... (2.5) when: , ~= ~ fJ(r)dl •, a.=: f f(t) cos nar dl. n = 1,2, 3, .... •, b. = ; ff(t) sin 'laM dl. n = I. 2. 3..... • These are the formulae given in most of the textbooks. 2.2.6 Complex Fourier Series We know that N~, a COSM+" sinM= (al -jb,) tiU + (al +jb,) t~ja I '1 2 2 Let us denote C =(al - jb,) and C =(al + jb,)I 2 ~I 2 The Fourier series can now be rewrittcn as: (2.6) (2.7) (2.8) (2.9) Note that the formulation involvcs complcx cxponcntials and the concept of negative angular frequcncy -ru. 1ltere is nothing imaginary in any signal and there is no negative frequency. Complex exponcntials facilitate easy manipulation of multiplication and integration involvcd in thc calculation of Fouricr serics coefficicnts than multiplication and integration using plain sine and cosine fUlI(:tions. Let U$ filS! try to visualize t}oH, when: fI) = 2trrr and T is the period of our signal. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 42. Fouricr Series and Geometry • 27 By Euler's fannula, ~OA = cos (J)l + j sin (J)l We can thiok of this fUlICtion as 11. 'bundle' of sine and cosine fUlICtion kept togethcr side by side such that they do not 'mix' with each other. The fUlICtion of j cao be thought of as a separator indicating that the two parts. I'iz. cos 001 and sin oot are orthogonal. 2,2,7 Orthogonality of Complex Exponential Bases The set of bases. (I. ~. t-iu ... " ejoOA. e-jolll ... ,,) are orthogonal over the interval [0, n, where 00 = 21rIT and T is the duration of the signal which we are representing using the bases. However. there is a twist with respect to interpretation of orthogonality and projection of a fUlICtion on to a complex function. To make you prepared fOf" that twist and to lICCept that twist, let us consider If.: aT + bJ + ek. an element of ')1. n.e nonn of this vector (or magnitude) is obtained by projecting V on to itself and then taking the square root of resulting quantity. That ;, But in case of a complex number, l=a +jb to find the magnitude. we have 10 multiply l with its complex conjugate l and then take the square root. In case of real function, say f(t), we find square of its nonn by Jf(t) f(t) df In case our function is compleJO, say Z(f), its square nonn is given by Jl(t) l(f) df This has implication in the formula for projection of a fUlClion on to a complex function. We must multiply the fil'$t function with complex conjugate of the second functon a~ then integrate. With this twist in mind, we will prove that the set of functions u(t). t"", e-JfM ..... ej<lu. t - j<lu, ... ) are orthogonal. We denote the projection offt") on to h(f) as (fl(f), fit». CIISe I : u(t) = 1 and el-', for 0 S; t S; T and 00 = 21f1T , , , ("(f), t i.) '" JI · ti-l< dr = Je-j.>tt dr = Je-j(lmff) dr '" 0 , _0 0 (I Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 43. 28 • Insight inlo Wavdet5 From Theory to Practicc CtI$r 2: tj<t·" and t-il-' where 11 and m are positive integers. , , , (tftttU , t -J"') == JrJou t J... dr == Jd<.''')'' dr =Jd{o+..~lJfIT), dl = 0 •• 0 0 0 CtI$e J: ep..., and eJ...,' where 11 and m are either positive or negative integers with m ~ 11. , , , (~. ~) = J~M t!/ow dr = f~.-.) ..dt = Jeil.-",){2JffT)' dt ". 0 • • 0 0 0 Cases l. 2 and 3 prove that the functions are orthogona!. What about orthononnality of comple" e)[ponential functions? Since. >od , (l!'*", r) = Jr"'Mt ftttU dt = ... , , f]· dl=T • (U(I).U(I» = fI·dl =T. the functions are not orthooonnal • The functions though orthogonal do not salisfy the requirement of orthononnality. So we multiply the functions with a scalar to make them onhOflOrmal WllOng themselves. For unit function and comple" uponentials, scaling constant is .fliT. Therefore, orthonormal bases are: We denote the signal space generated by these bases as L2. Now. any signal which is an element of Ll can be written as: To get a particular coefficient, we project f(l} on to the corresponding base. I.e., , Col = ff(I).jlfT u(t) dr •, C; = Jf(I).JIiT r-;- dr • (2.10) (2.11) (2.12) Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 44. I'ouricr Series and GeomcU'y • 29 , C~. = f1(1) JilT t i- dl (2. 13) "This resull follows from the OJIhonormality of bases used in (2.10). Now, we simplify the expn:ssion given in Eq. (2.10) by introducing new constants. Let Co = coJIiT Since Therefore, , Co = fl(lr/lfi 1.1(1) dt ", , , Co = J IlT ff(t).fliT 1.1(1) dr = ~ ff(t ) uet) dr = ~ ff(t ) dt " " "Similarly , C. = c;JIiT =~ f1{I)e-i'fM dl ",,' , C. = C:.JUT = ~ fl(rY- dr " Thus, we oblain complex fourier series in slandard form: with the coefficients dcfiled in Eqs. (2.14) to (2.16). (2.4) (2. 15) (2.16) (2.17) This completes formal inU"Oduction to Fourier series. The n:adcr who is nO!: familiar with advanced Founer methods can skip Chapter 3 and proceed reading Chapter 4. SUMMARY It is no undcrstatementthat the Founer senes especially discrete Founer transform is considered one of the crowning achievements of the 20th Century. The theory of Founer series has strong connection with geometry or at lcast we can understand the Founer transform thoory from a geometrical viewpoint. We h3d the honour and a privilege to bring it to the reader in this user friendly, low fat and no cholesterol form. In the forthcoming chopters on wovelets, this geometrical viewpoint will help you to digest the theory in a beller way. It will surely prevent intellectual diarrhoea. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 45. 30 .. Insight into Wavelets From Theory to Practice: EXERCISES 2.1 Consider the interval [- I, IJ and the basis functions (1. ,. ,2, ,1, ...• ) for L2!_1, I]. , The inner product on this vector space is defined as <to g) = J/(1) 8(1) dl . Wc can -, show that Ihese functions do nOl fonn an onoonormal basis. Given a finite or countably infinite set of linearly independent vectors Xj, we can construct an orthonormal set )'j with the same span as X j as follows: S . h ".. tan wit )'1 = r:I ,'" .. Then. recursively set )'2 = .. Therefore. This procedure is known as Gram·Schmldt orthogonali:zalion and it can be used 10 obtain an onhononnal basis from any other basis. Follwing is a MATLAB function Ihat implement the Gram-Schmidt algorithm. Your input must be a set of linearly independent fUll(:tions. 1be output will be a set of onhooormaJ functions that 5pan Ihe same spa<:e. function ~_g~amschmidt( x,N .M); fo ~ i_l:N; s/i,:).x (i, : ), end; ell) .s/l. :) · conj (s(l,:).'); phi(l, :) - s(1, :)/sq~t (e(l); fo~ i_2:N; th(i, :).ze ~oa(1,M); fo~ ~.i·l:·l:l; th(i,:) _th( i ,:) .. (a (i,:) · conj (phi (~,:) . ')) · phi (r,:); end; th(i, : I- s (i,:) -th(i,:); e(i)_th(i,:) · conj(th(i,:).'); 11 note.' means transpose without conjugation phi(i,:I·th(i,:l/sq~t(e ( i)); end; z _phi Cl :N,:); Apply the algorithm on Legendrc polynomials and plO! the first six orthonormal basis functions. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 46. Fwrier Series Il!Id Geometry • 31 2.2 Project the function j(x) : x on to the space spanned by ji(x), ¥1x), ¥12x), !l'(2x - I)e L2[O, I] where fI(X) : 1 1. o. OSxS I/2 otherwi se ( I OS x S l12 If(x): - 1 112 SxS I o otherwise Sketch the projection oflex) to this space. 2.3 Show that the set of functions /",(x) '" Hsin mx. m = l. 2, ..., is an or1hononnal system in L2[O, If]. 2.' Given that 112 112 1/.J2 0 In In -11../2 0 H. : 112 - In 11../20 ,n - 112 0 - 1/./2 9 7 is an orthononnal basis for :It. and l : [l IE '" e :lt4 • where E is the standard 3 , basis for :lt4 • that is. I 0 0 0 0 I 0 0 E= 0 0 l 0 0 0 0 Find z in tenns of basis in H•. 2.5 (a) Find the Fourier series of/ (1) = 1 for - If S t < If. (b) Using Plancl1erel's fonnula and (a), evaluate the series Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 47. 32 • Insigh! imo Waveleu~From Theory w Pn.c:!ice [Hinl: Acoording 10 Plancherei's formula I If = L. 1c. 1 2 , where the C,.s are ""complex Fourier series coefficients of the function I(/). U/li2 in the given problem " is 2~ JIf(t)1 2 dt I -" 2.6 Find the Fourier series of f·HI(x)=ln for-nSx<O forOSx<n Suggested Further Readings Amold. Dave. I am the Resource jar all that iJ Math: A COnlemporory Perspective. C.R.. CA. 1998. Burden, Richard L., Numerical Analysis. BrooklCole, Alban)', 1997. LeQn, Sleven, Unear Algebra wirh Applications. Prenlice HaiL New York, 1998. Lyons. Richard. Unde ,jumding Digital Signal Processing. Addison-Wesley. Menlo ParIc. CA. 1998. Ramiret, Roben W., The FFT: Fundnmenlals and Concepu. Prentice Hall. New York. 1985. Slnlng. Gilbert. Inlroduction to Unear Algebra. Cambridge Press. Massachusetts. 1998. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 48. Continuous Wavelet and Short TIme Fourier Transform INTRODUCTION The aperiodic, noisy. intenninent, transient signals are the type of signals for which wavelet transforms are panicularly useful. Wavelets lIave special ability to examine signals simuh- aneousJy in both lime and frequency. This has resulted in the development of a variety of wavelet based methods for signal manipulation and interrogation. Current applications of wavelet include climate analysis, financial lime series analysis. hcan monitoring, condition monitoring of rotating machinery, seismic signat denoising, deooising of astronomical images, crack surface c haractcril.ation, characterization of turbulent intermittcncy. audio and video oompression, compression of medical and thump impression records, fasl solution of partial differential equations, computer graphics and so on. Some of these applications require (:(lntinuQUs wavelet transform, which we will explon: in this chapter and find out how it differs from classical mc:thods Ihat deals with aperiodic lJOisy signals. 3.1 WAVELET TRANSFORM-A FIRST LEVEL INTRODUCTION Wavelet means 'small wave' . So wavelet analysis is about analyzing signal with short duration finite energy functions.1bey transform the signal under investigation into another representation which presents the signal in a more useful form. This tnlnsformation of tbe signal is called wavelet transfOl'm. Unlike Fourier tnmsfonn, we have a variety of wavelets that ~ used for signal analysis. Choice of a particular wavelet depends on the type of application in hand. Figures 3.1 to 3.4 show examples of some real and complex wavelets. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 49. 34 • I "gh . W le F "" t ,"IQ m ~~ """" p,,,," '" 1 1 0.2 0.2 "0 0 ..... -<1.2 -<1.2 - 1 - 20 o 20 - 20 - 10 o 10 FIGURE J.l Real and imaginary pans Qf Shan wavdet 1.2,--_-_--_-_-_-_--_-, 0.8 0' 01--_, -<1.' - 3 - 2 - 1 o 3 • FlGURE).2 Quintic Splinc wavelet FIGURE J.J Real and imaginary pans of Gabor ",,..."del. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 50. Continuous Wavelet: and Shoft Time Fouricr Tm1sfonn • 35 1.0 1.0 0.' 0.' o0 t-~ o.ot------ -<I.' -1.0 ':----:----;--:---0- 4 - 2 0 2 4 - 1.0':-----";----'-;----;:------; - 4 - 2 0 4 , , FIGURE 3.4 Real and imaginary potU of Morlet wavelet. We manipulate wavelet in two ways. The first one is translation. We change the central position of the wavelet along the time axis. The second one is scaling. Figures 3.5 and 3.6 show translated and scaled versions of wavelets. " ",, ' ,, " ,, ,, ~, ;. , '" "'.' '~, '., FIGURE J.5 Translation (change in position) of wavelets. f~ucnq (Scale)(Lcyd) vv TIme shin , FIGURE J.6 Change in Kale (aI~ called level) of wavelcu. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 51. 36 • lnsigllt into Wavelets-From Thcofy to Practice Figure: 3.7 shows a schematic of the wavelet transform which basically quantifies the local matching of the wavelet with the signal. If the wavelet matches the shape of the signal well at a specific scale and location, as it happens to do in the top plO( of the Figure 3.7, then a large transfonn value is obtained. If, however, the wavelet and signal do not correlate well, a low value of transform is obtained. The transform value is then ploned in the two-dimensional transfonn plane shown at the bottom of the Figure: 3.7. (indicated by a dot). The transfonn is computed at various locations of the signal and for various scales of the wavelet, thus filling up the transform plane. If the process is done in a smooth and continuous fashion (i.e., if scale and position is varied very smoothly) then the transform is called continuous wavelet transrOnR. If the scale and position an: changed in discrete steps, the transfonn is called diM:rde wavelet transrorm. W.~,,:,~,"~--=-"r-=--../ local =hing between signal and wavelet Ic:odJ 10 large transform coeffic;"ntWavelet transform _ Cu=n' ",ale "", I~--- -----------------------~ Wavelet transform plO! (Two-dlmeru'onol) : Higher lhe coefficient more : darker tbe point r CUrrent location ......... Position ---_. FIGURE 3.7 n.e signal. wovelel and lnUI,form. Planing the wavelet transform allows a picrure 10 be built up of correlation between wavelet- at VMooS scales and locations-and the signal. Figure 3.g sllows an e~ample of a signal and corresp:lllding spectrum. Commercial software packages use various colours and its gradations to slIow the transform values on the two-dimensional plot. Note that in case of Fourier transform. spectrum is one-dimensional array of values whereas in wavelet transfonn, we get a two-dimensional array of values. Also note Ihat the spectrum depends 011 the type of wavelet used for analysis. Mathematically. we denote a wavelet as: I ('-b)~•.•(,). JI"I ~ -;;- Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 52. Continuous Wa" elel and Shon Time Foorier TfWlSform • 37 Timel) HGURE J.lI SiglW iIId iu ""iI,·(]tl tJillSfl1l1ll lipc.;lJUm. where b is location parameter and 11 is scaling parameter. For the function to be a wavelet. it should be time limited. For a given scaling parameter a. we translate the wavelet by varying the parameter b. We define wavelet transform as: V(u.b)= ff(t)JtaII"C:b)dl (3.1) According to Eq. (3.1). for every (Cl. b)• ....e have a wavelet transform coefficient. rePf"CSenting how much the scaled wavelet is similar to the function at location I = (bla). In the following section (Section 3.2) we will explore the classical time-frequency representation of signals. and associmed uncenainties in frequency and time. 1llen we shall compare the same with that of wavelet transfOl"ms. 3.2 MATHEMATICAL PRELlMINARIES-FOURIER TRANSFORM Continuous Fourier tmnsform Assume that/is a complex-valued function defined on R. Its Fourier transform j is defined as: If fELt. the integral makes sense fOl" every value of w and f is a continuous. bounded function which goes to zero at infinity (this last fact is called Riemann-Lebesgue Lemma). Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 53. 38 • Insight into Wavelets- From Theory to Practice The Fourier transform is also defined for [E:' L2. In this case, the integral may oot be defined in the usual sense. One way to define it is: L ' • " I IlII f 'few) = c:= [(I)e-"OJ dl ,,2K R -J ... -.In this case j is also in Ll. If I E L1 or [E Ll it can be wrinen in tenns of its Fourier transform as: -I f - .I(r) = r::-= I(wje"'" dw ,2.-where the integral may need to be imerpreted as a limit of finite integrals, as before. In general, every locally integrable function [has a Fourier transform but j may not be a fu nction any 1IlQI"C, rather a generolized[unetlort or dlslrlm.lion. We will assume from oow onwards that all functions have suitable imegrability properties so that all Fourier transforms (and other operations that show up) are well defined. Some propenies of the Fourier U"lUlsform are: • The Fouricr transform preserves L2 norms and inner products (this is called the Parseval-Planchen:1 Theon:m). Thus. if f, g E Ll then (I,g) = <i,g) • 11Je Fourier transform turns convolutions imo products and vice vema. 1lte convolution of f, g is defined as: We find (f. g)(I) = JI(y) g(t - y) dt = JI(t - y) g(y) dy (f. g)" (w) = J2K j(M)g(M) (f. g)" (M) = J2K(j. g}(/lJ) • The Fourier transform turns translatioo into modulation and vice versa. The lrons/mion of I by a E 9t is defined as: T.[(t) = I(t - a) The modulmion of[by a e 'X is defined as: E.f(t) = e;"I [(t) Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 54. Continuous Wavelet and Shon Time Founer Transform • 39 . . (EJ) " (m) '" f(m - a) '" T.f«(I) • The Fourier trlUlsform turns dilation inlO inverse dilation. The dilOliQn off by SE 9f is given by I 1 -112 D.f(t) '" s f(tls) The factor in front is choscn so thal ID.f ll '" Ifb.The Fountr transfonn relationship IS: I 1 1/2 " " (D.f)'" (m) '" s f(sm) '" o",.f(m) • The Fourier transform turns differentiation into multiplication by iro and vice ve~ (n" (w) :: iw j(W) (If(t» " (m) '" ij'(m) 3.2.1 Continuous Time-frequency Representation of Signals Assume that f{1) is a complex valued function on 91 which represcnts some signal (think of t as lime). The Fourier transform -. 'f ·f(m ) '" ~ JO)t-"'" dl ,2. (3.2) -is used to decomposc f into its frequcocy components. The inversion formula is o;pressed as: (3.3) Equation (3.3) can be interpreted as writing f as a superposition of time-harmonic waves t 4 -. If i is large near some frequency then f has a large component that is periodic with that frequency. This approach works well for analyung signals that are produced by some periodic process. However, in other applicalions, like speech analysis. we would like to localize the frequency components in time as well and Fourier transform is nol suitable for thal. We will consider two methods that attempt to provide information on both time and frequency: the Windowed Founer Transfonn (wFI1. also called Short Time Fourier Transform (STFl') and the Continuous Wavelet Transform (CWl). Both of lhem map a function of one variable (lime) inlo a fuoction of two variables (time and frequency). A large value of the Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 55. 40 • Insight intQ Wa~elet. From Theory to Practice transform near time I, and frequency ro i$ interpn:tcd as: [he signal! contains a large componenl willl frequency W near lime I. lllere is a lot more theory [0 botll of these transforms IlIan we will cover in [his chapter but our main interest lies elsewhere. 3.2,2 The Windowed Fourier Transform (Short Time Fourier Transform) Fix a function IV E L2, IlIe window function. W sllould be localized in time neat 1 '" O. witll a spread IT (we will define "spread"' more precisely later). Typical choices are: (i) W = XI-I .11 willl IT = 1I.Jj (see Figure 3.9) X. is the characteristic function of the set S, which has value I on the set. 0 otherwise. w ----'---+_L---::+---'------+ , Sigma .'IGURE 3.9 Char.lCtm>lic function of W .. XI-l.It (ii) W", (J +cos 21)12 for I € (~.R"12. xl2j, IT == J/.J3 (see Figure 3. 10) (iii) Sigma FIGURE ).10 1'101 of window function W .. (l + cos 21)/2. w" _'_ t-r'12 (Gabor willdow). 0'" I (see Figure 3.11) J2Jc Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 56. CQnlinUOlU Wa""kt and ShQrt Time F<>urier Transfmm • 41 Sigma FIGURE 3.11 PlO! Qf window fUI>Ction W ~ ';"e-..n.,2, The WfT with window W of f is defined as: -Ifwf(a.b) = ,j~lf L!(r)to-;bl W(t a) dl Thus. a is l!le time parameter, b is t!le frequency. (3.4) V'wf(a. b) can be interpreted as inner product of f with [he leSI function eiblW(1 - a). A typical test funclion looks like the one given in Figure 3.12. // y- FIGURE 3.12 Shifted and modulated window. Note how the time resolution u (rel3led to the window width) is C()nstanl. independent of the frequency. 3.2.3 The Uncertainty Principle and Time-frequency Tiling Assume a function f e l}. The II1C3I iJf of f is defined as: (3.5) Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 57. 42 • Insi):hl intQ Wavekts-Frorn TheQry IQ Practice The uncertainty CIf of! is defined as: (3.6) Remark: If!E Lllhen !2/I!f is a probability distribution. i.e. a non-negali·e function with integral L. IJf and CIf are simply the mean and stlllldard deviation of this distribution in the statistical sense. IiJ measures where! is localized in time and CIf measures how spread OUI (or uncertain) this time measurement is. Pt localizes the frequency and Cl; measures the uncertainty in frequency. H,~ (3.7) where -" I J .! (w) = ~ !(t)e-"· dr ,2. -"d (J.8) The uncertainty principle states Ihat for any f (3.9) If a function is localized in time, it must be spread out in the frequency domain and vice versa (see Figure 3.13). The optimal value of 112 is achieved if and only iff is a Gaussian distribution. To visualize this, consi(jer a hypothetical function F(t, w) over the time-frequency plane. F(I, w) represents the component of! with frequency wat time /. The uneertainly principle says that it makes no sense to try to assign point-wise values to F(I, w). All we can do is to assign meaning to averages of F over rectangles of area atleasl 2. (The signal is localized in time to [,u - Cl, IJ + a:l and likewise for frequency, so the rectangle has area (2C1p x (20). The uncertainty principle applies 10 both WfT and CWT but in di fferent ways. Let 1J... 1lIld Cl... represent 'localization' (mean) IlIld 'spread' of the wirKiow function W(I) in time domain which are formally defined as: -1-1... = 1 2 fIjW(t)11dl IWO)I _ (3. 10) Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 58. Conlinuous WUeltl and SIKH1 Time Foune, Tr.msfonn • 43 "I D Good time resolution Bad f'<queDCY n:w1~!i0f :;> Good frequency resolution Bad lime rUQlulion nCURE J.B Po..ibt" lime-fn:qu<DCY ....indo...... (3.1 1) Also Itt A.. and 0-.. represent F014,iu lrons/orms of 'locaJiution' (mean) and 'spread' of the window funclion WCt). Then the inner product (f, W) contains information on / in [.uoo - u.".u.. + u..J. Since <I,W) '" tj, IV) , it also contains (nfoonation on /(0) in [.it.. - o-..,.it.. + it..]. Thus, (J, IV) represents the frequeno:ies between Pw - o-w and .it.. + 0-.. thal are present between time p.. - u.. and .u.. + u"'. For WfT, the test function is the shifted and modulated window. ..iblW(I_ a) '" EhT. W which is localized in time near .u.. + a with uno:ertainty a.".. Its Fourier transfonn is which is localized near .it.. + b ....ith unCtnainly U... Here E and T are respectively modulation and translation operators. contain information on / in [.uw + a - u.... .uw + a+ u.. J and information on [.it.. +b - 0-..,Pw +b + 0-00 ] . (3.12) " f" For a WFf with fixed window WCt), the time resolution is fixed at u..' the frequency resolution is fixed at 0-,... We can shift the window around in both time and frequtncy but the uncenainty box always has the same shape. n.e shape can only be changed by changing the ....indow W(I). Refer Figure 3.14. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 59. 44 • Insight into Wavelets From Theory to Practice D Time axi. HGURE 3.14 Some possible t;me·(re<j""!IC)' window. of w;ndowc:d Fouric. transform. Time and frequcocies are resolved equally well (or equally bad). A discrete WFT (with equally spaced lime and frequency samples) gives a unifonn tiling as in Figure 3.15. I , I' IGURE 3.15 lime.fre<j""'''''Y Lilinl of windowc:d Fouricr t.,,".form. For CWT. lhe test function i5 of the form: (3.13) with Fourier transform E_bDlI8Y. Thus. (j. V'Q.b) = (j. TbD.V') = 0.E_b DI18y) (3.14) contains inronnation on f in lu,u,.+b-uu,..up",+b+uu",J and infonnalion On j in [iJ"la - iJ",la. it"fa + iJ"fa]. The uncertainty boxes have differem shapes in different parts of llIe lime·frequency plane iU in Figure 3. 16. Dr.M.H.Moradi;BiomedicalEngineeringFaculty;
  • 60. Cominuous Wavelet and Short TIme Foorier Transfonn • 45 o AI high frequency: Good lime re""luliOl1 Bad freqllCOC)' r~lulion AI low frequency: [======1/Good freqllCncy re).OIi.>I;OII i Bad I;me resolution TIme aJIis .'IGUIlE .3.1' f'lmiblc limc-frcquclI(Y tHings in the case of continuow wa...det transform. 3.3 PROPERTIES OF WAVELETS USED IN CONTINUOUS WAVELET TRANSFORM A wavelet 1f(1) is si mply a function of time t that obeyS a basic rule. known as the wavelet admissibility condition: - I~ )1c =f (U dw<- (3.15), w o where !i/(w) is the FourieT transform. This condition ensures that ]V{w) goes to zero quickly as co -t O. In fact. to guarantee that Cl" < _, we must impose ]V(O) "" 0 whicl! is equivalent " -fVlO) dl = 0 -A seCOl1dary condition imposed on wavelet function is unit energy. That is - - 3.4 CONTINUOUS VERSUS DISCRETE WAVELET TRANSFORM (3.16) (3.17) As mentioned previously. CWT is a function of two parameters and, therefore. eOnlai ns a I!igh amount of extra(redundant) information when analYl.illg a functioll. illstead of continuously varying Ihe parameters. we analyze the signal will! a small number of scales with varying Dr.M.H.Moradi;BiomedicalEngineeringFaculty;