Convolution and LTI Systems

8,613 views
8,479 views

Published on

Aah! Convolution finally made clear, simple and correct.

Published in: Technology, Education
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,613
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1,861
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Convolution and LTI Systems

  1. 1. CONVOLUTION & LTI SystemsSnigdha Suresh Savitha Sthawarmath Rajeev Rajkumar Mohit P. Tahiliani Department of Computer Science & Engineering N.M.A.M. Institute of Technology Nitte, Udupi District Karnataka- 574110 INDIA © May 2008 1
  2. 2. PrologueConvolution & LTI Systems are the heart and soul of Digital Signal Processing (DSP).Yet the CONVOLUTION operation has neither been defined nor explained clearly. Thispedagogical gem redeems this deficiency. The flow is smooth. The presentation is lucid,precise and concise.This monogram is also a tribute to the Management and Faculty of NMAMIT. True tothe alma mater, this work is par excellence. The authors (all students) are to becongratulated.Nitte Stephen VadakkanMay 2008 2
  3. 3. CONTENTS1 Introduction 52 Two Guitars Example 83 Asymmetric versus Symmetric 104 Distance Measurement using Convolution 135 Long Multiplication without Carry 176 Convolution Sum 197 Definition of Terms 228 A Simple Example 289 Padding with Zeros 3710 Super-position Curve 3811 Convolution Sums and Convolution Curve 4312 Convolution Interval 4513 Example from Communication Systems 4914 Convolution Interval and Convolution Index 5715 General Form of Convolution Sum 6016 Convolution Integral 6117 Linear Convolution 6418 Circular Convolution 6719 Linear Convolution of sin(ω0t) and cos(ω0t) 6920 Periodic Convolution of sin(ω0t) and cos(ω0t) 7021 Circular Convolution of Two Guitars Example 73 3
  4. 4. CONTENTS22 LTI Systems 7823 Shift Invariant 8024 Time Invariant 8125 Frequency Fidelity 8226 Complex Representation of an LTI System 8427 Deconvolution 8628 Linear 8729 Harmonics 9030 Linear Decomposition 9231 Unit Impulse Function 9432 Filtering 9633 Linear Constant Coefficient Difference Equations 9734 Infinite Impulse Response (IIR) 9835 Control Systems 99 4
  5. 5. INTRODUCTIONDigital Signal Processing (DSP) is a vast and rich subject. One sees the convergence ofmany beautiful ideas from Physics, Mathematics and Electronics. While it can be difficultin some areas, the basic concepts and calculations can be understood with a few goodexamples based on the knowledge of just Higher Secondary level Physics andMathematics.This modest module is intended as a preview to material in standard text-books. It isneither rigorous nor exhaustive. It is meant to be intuitive – to give the beginner a feel forthe subject. After going through this module it is strongly recommended that the studentreads one of the best books on this subject: The Scientist and Engineers Guide to Digital Signal Processing by Steven W. Smith.In high school we learn several types of operations on discrete operands: + and − , • and /,exponentiation as in xn and its two inverses: taking the nth root of x denoted byand taking the log of x to some base b denoted by logbx. We also learned two operations:differentiation and integration on well-behaved functions. We shall call these functionscontinuous operands. Let us briefly review the + and • binary operations as applied toboth types of operands. 5
  6. 6. OPERANDS DISCRETE CONTINUOUS fA(t) = t 0 ti t ANALOG DIGITAL a = //, b = /// a = 2, b = 3 +3 fB(t) = 3 0 ti tThere is also a third type of operand known as digitized. We may digitize discrete operandsand continuous operands. Digitizing continuous operands is a three step process: discretize,quantize and then encode.Continuous operands may also be digitalized. We may digitalize continuous operands usingnatural or what is called analog frequencies or we may digitalize continuous operands using(discrete) digital frequencies. Digitalized operands are also discrete.Very often when books say digital, as in Analog to Digital Converter (ADC), they actuallymean digitized. In digitized operands there is no frequency information. Where as indigitalized operands there is embedded/hidden somewhere the frequency information.Generally, after we digitize a continuous operand we cannot recover the original continuousoperand from the discrete digitized operand. But we can recover the original continuousoperand from a discrete digitalized operand. 6
  7. 7. ADDITION OPERATION: +ANALOG DIGITAL CONTINUOUS a+b=c fC(t) = fA(t) + fB(t) •(t , t + 3) i i// + /// = ///// 2+3=5 =t+3 +3 -3 0concatenation addition ti t Notice that the addition is at corresponding instants MULTIPLICATION OPERATION: •ANALOG DIGITAL CONTINUOUS a•b=cRepeated concatenation Repeated addition fC(t) = fA(t) • fB(t)where the number of where the addends =3•t • (ti , 3ti)strokes are the same are the same 0 ti t a • b = // // // = ////// a • b = 2+2+2 = 6 b • a = /// /// = ////// b•a=3+3=6 Notice that the multiplication is at corresponding instantsWe are now going to learn a new binary operation called CONVOLUTION denoted by *. Wemay apply this to both discrete operands and continuous operands. CONVOLUTION is theheart of the DSP. It is the operation in Linear Time Invariant (LTI) Systems. 7
  8. 8. TWO GUITARS EXAMPLEImagine you have two guitars; g and G. Guitar g has only one string {s3} and guitar G hasfive strings {S1, S2, S3, S4, S5}. Guitar g is with you and guitar G is a few meters awayfrom you. You strum guitar g. What happens?The sound waves or vibrations emanating from the sole string s3 of guitar g reach andimpact on all or EACH of the 5 strings of guitar G. What is the EFFECT of the impact of s3on EACH of the strings of G? s3•S1, s3•S2, s3•S3, s 3•S4, s3•S5.What is the sum total of all these EFFECTS: s3•S1 + s3•S2 + s3•S3 + s 3•S4 + s3 •S5. This wecall the TOTAL EFFECT SUM and can be written as (∑ s3•Sj) for j = 1, 2, 3, 4, 5.Only the corresponding string S3 of guitar G resonates or RESPONDS. So the TOTALEFFECT SUM is s3 • S3. This is known as the frequency response.Now reverse the scenario. Keep guitar G with you and place guitar g a few meters away.Strum all five strings of G. What happens?The sound waves or vibrations emanating from EACH of the five strings{S1, S2, S3, S4, S5}of G reach and impact the sole string s3 of guitar g. What is the EFFECT of EACH of thefive different sound waves from G on g? S1•s3, S2•s3, S3•s3, S4•s 3, S5 •s3What is the sum total of all these effects: S1•s3 + S2•s3 + S3•s3 + S4•s3 + S5•s3. This we callthe TOTAL EFFECT SUM and can be written as (∑ Sj •s3) for j = 1, 2, 3, 4, 5.The sole string s3 of guitar g RESPONDED only to the corresponding string S3 of guitar G.So the TOTAL EFFECT SUM is S3 • s3. The other waves had no EFFECT or produced zerofrequency response: Si. • sj = 0 when i ≠ j.Now let us upgrade guitar g to have five strings {s1, s2, s3, s4, s5}. Keep g with you. PlaceG a few meters away from you. Now strum all five strings of g. What happens?We know that the sound waves or vibrations of EACH of the five strings of g reach andimpact EACH of the five strings of G. An EFFECT is si •Sj. There are 5 x 5 = 25 such 8
  9. 9. EFFECTS. An EFFECT SUM i = (∑ si•Sj) for j = 1, 2, 3, 4, 5. We know this EFFECT SUM i isequal to si • Si . There are 5 such EFFECT SUM i for i = 1, 2, 3, 4, 5.We know from the previous two scenarios that EACH of the five strings of guitar Grespond to EACH of the corresponding strings of guitar g. Also, we know fromexperience, that we do not hear the individual responses separately. What we hear is aTOTAL EFFECT SUM or sum of the 5 EFFECT SUM i = {s1•S1 + s2•S2 + s3•S3 + s 4•S4 + s5 •S5}.How do we mathematically represent the computation process of the TOTAL EFFECT SUMof EACH string of g on EACH string of G?Let: g = {s1, s2, s3, s4, s5} sequence of strings G = {S1, S2, S3, S4, S5} sequence of stringsNote that in a sequence, as opposed to a set, the order of terms in the sequence isimportant.One way to represent the computation process is :{S1, S2,S3,S4,S5} {S1, S2, S3, S4, S5} {…} {…} {S1, S2, S3, S4, S5} • • • • • s1 s2 s3 s4 s5 s1• S1 + s2 • S2 + s3 • S 3 + s4 • S4 + s5 • S 5 EFFECT SUM EFFECT SUM EFFECT SUM EFFECT SUM EFFECT SUMThis is also referred to as a weighted sum where the Sj are “weighted” by si and thensummed up.The MULTIPLICATION or SUPERPOSITION computes the EFFECT of the string si on the stringSj. The EFFECT is: si.• Sj ≠ 0 for i = j and si • Sj = 0 for i ≠ jThis yields :{ s1•S1, s2•S2, s3•S3, s4•S4, s5•S5}, the EFFECT SUM sequence.The summation of the EFFECT SUM sequence is the TOTAL EFFECT SUM of EACH on EACH.This is: {s1•S1+s2•S2+s3•S3+s4•S4+s5•S5}, which is what we hear.We may formularize this computation process: 9
  10. 10. ASYMMETRIC versus SYMMETRICWhat happens if we REVERSE strum the strings of guitar g? That is to say , instead ofstrumming downwards {s1,s2,s3,s4,s5} we strum upwards {s5,s4,s3,s2,s1}.This yields :{ s5•S5, s4•S4, s3•S3, s2•S2, s1•S1}, the EFFECT SUM sequence.The EFFECT SUM sequences are not the same. They are ASYMMETRIC. In both cases thecomputation takes about (25+4) additions and about 25 multiplications.The TOTAL EFFECT SUM is the same in both cases. What we hear is the same. But there is adifference in the order in the EFFECT SUM sequence. And in some situations, knowing theindividual terms of a sequence and when they are computed or occur is of importance.Can we think of a computational process where the EFFECT SUM sequences or order of theindividual terms, are the same?Let us begin by REVERSING the strings of guitar g. So now g = {s5, s4, s3, s2, s1}.This process of REVERSING or REFLECTING is known as CONVOLVE. We may write downthe computation process as:Step 1: CONVOLVE g. G: S1 S2 S3 S4 S5 *g convolved: s5 s4 s3 s2 s1Step 2: SHIFT S1 S2 S3 S4 S5 * s5 s4 s3 s2 s1Step 3: MULTIPLY S1 S2 S3 S4 S5 * s5 s4 s3 s2 s1 s5 • S5Step 4: SHIFT AND MULTIPLY S1 S2 S3 S4 S5 * s5 s4 s3 s2 s1 s5 • S5 s5 • S 4 s4 • S5 10
  11. 11. We may continue with this step by step “SHIFT and MULTIPLY” process until EACH of thestrings of g has SHIFTED across and EFFECTED all or EACH of the strings of G to yield: S1 S2 S3 S4 S5 *s5 s4 s3 s2 s1 s1 • S1 s2 • S 1 s3 • S 1 s4 • S 1 s5 • S 1 s5 • S 2 s5 • S 3 s5 • S 4 s5 • S 5 s1 • S 2 s2 • S 2 s3 • S 2 s4 • S 2 s4 • S 3 s4 • S 4 s4 • S 5 s1 • S 3 s2 • S 3 s3 • S 3 s3 • S 4 s3 • S 5 s1 • S 4 s2 • S 4 s2 • S 5 s1 • S 5 s1 • S 1 0 s2 • S 2 0 s3 • S 3 0 s4 • S 4 0 s5 • S 5Finally, when there is no more SHIFTING and MULTIPLICATION (superposition) to be done,we can add up each individual column to get what we now call the CONVOLUTION SUM.The CONVOLUTION SUMS form a sequence: s1•S1, 0, s2•S2, 0, s3•S3, 0, s4•S4, 0, s5•S5.We may sum up these CONVOLUTION SUMS to get the TOTAL CONVOLUTION SUM.How would we formularize this computation process?We re-number the N = 5 strings with: i = 0 to 2N-1. Here 2N-1 = 9. Only strings 0 to 4exist. The extra strings s5 through s9 and S5 through S9 do NOT exist and hence are zero.This is known as PADDING WITH ZEROS. And now we may write out the SYMMETRICformula for the TOTAL CONVOLUTION SUM as: N NWe get nine CONVOLUTION SUMS which we number from 0 to 2N−2 = 8.This process or operation of finding the symmetric sequence of CONVOLUTION SUMS iscalled CONVOLUTION. We may denote this process of computation by g*G.The formula for CONVOLUTION SUM i is: 11
  12. 12. We note that the computation process is SYMMETRIC with respect to the sequence ofCONVOLUTION SUMS:The top two rows of G * g computation process is: S0 S1 S2 S3 S4 S5 S6 S7 S8 S9 * s9 s8 s7 s6 s5 s4 s3 s2 s1 s0The top two rows of g * G computation process is: s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 * S9 S8 S7 S6 S5 S4 S3 S2 S1 S0Despite this difference, the CONVOLUTION SUMS appear in the same order. G*g = g*GThe convolution operation * is commutative.Now it seems that we did some more work (computation) using the above SYMMETRICformulas to get the CONVOLUTION SUMS in the same order. It looks like we did some 45additions and 45 multiplications. As an exercise, you may write out the computationtables for (k = 0 to i) for each CONVOLUTION SUM i (i = 0 to 9).We notice that theCONVOLUTION SUMS with even index i (i = 0, 2, 4, 6 and 8) are NON-ZERO. The odd indexCONVOLUTION SUMS are zero. With such prior knowledge, we compute only the evenindex terms and we are down to (25 + 4) or so additions and 25 multiplications. It lookslike we got better information with the same computation effort. 12
  13. 13. DISTANCE MEASURMENT USING CONVOLUTIONLet us now see a simple example where the order of the CONVOLUTION SUMS is important.Transmitter& Receiver … a2 a1 a0 b0 b1 b2… T O W E 1 meter 1 meter RSPEED OF LIGHT=3 x 108 m/sec CLOCK PULSE: 3 x 108 cycles/sec [Hz]BELOW IS THE TIMING DIAGRAM... t5 t4 t3 t2 t1 t0 a5 a4 a3 a2 a1 a0 0 0 0 0 1 1 0 1 1 0 0 0 b5 b4 b3 b2 b1 b0Assuming we know the TRANSMITTED pulse sequence a[n] and the distance 2 meters, wemay determine the RECEIVED pulse sequences b[n] shown above.Now assume we do NOT know the distance. But we know the TRANSMITTED andRECEIVED pulse sequences a[n] and b[n].How can we find the distance? 13
  14. 14. Let us now CONVOLVE the transmitted signal a[n] and the received signal b[n] to get c[n] = a [n] * b[n]• • • a6 a5 a4 a3 a2 a1 a0 *•••0 0 0 0 0 1 1 0 0 0 1 1 0… b0 b1 b2 b3 b4 b5 b6 a0•b0 a1•b0 + 0 a0•b1 0+0 a2•b0+ a1•b1+ a0•b2 0+0+0 a3•b0+ a2•b1+ a1•b2 + a0•b3 0+0+0+1 a4•b0+ a3•b1+ a2•b2+ a1•b3+ a0•b4 0+0+0 +1+1 a5•b0+ a4•b1+ a3•b2+ a2•b3+ a1•b4+ a0•b5 0+0+0+ 0+0+1 1 2 1 0 0 0 ci CONVOLUTION SUM 5 4 3 2 1 0 i CONVOLUTION INDEXPeak CONVOLUTION SUM ci = 2 is at CONVOLUTION INDEX i = 4 , that is to say 4 clockpulses or 2 clock pulses each way. Distance = speed x time = (3 x 108 [m/sec]) x 2/(3 x 108 [sec]) = 2 meters 14
  15. 15. We may represent the pure sequences a[n], b[n] and c[n] with lollipop diagrams. 15
  16. 16. So: 1. We need to know the PEAK value. In this case it is 2. In general, we need to know each CONVOLUTION SUM ci . 2. We need to know where / when (in time) the PEAK occurs. In this case it is at index i = 4. In general, we need to know the CONVOLUTION INDEX.This is the principle behind a pulse radar. Does this look like our familiar LONGMULTIPLICATION?If we perform the LONG MULTIPLICATION of 11 and 11000, we get: 1 2 1 0 0 0index 5 4 3 2 1 0 16
  17. 17. LONG MULTIPLICATION WITHOUT CARRYThere is nothing new or mysterious about this method of computation. We encounteredthis method in fourth grade under the name LONG MULTIPLICATION. To compute 632 X521, we wrote down the process. 6 3 2 X 5 2 1 30 15 10 0 0 12 6 4 0 6 3 2 30 27 22 7 2 105 104 103 102 101 100This is what the result looks like without the CARRY. In fourth grade we also did theCARRY to the next higher place.With the CARRY the result is: 329272.What is the advantage of the CARRY?It is easy to compare two numbers in decimal digital form by inspection. For example, itis easier to see that, 101 100 101 100 101 100 101 100 3 2 < 3 3 rather than 3 2 ? 2 13 17
  18. 18. More precisely and formally we may write this procedure of 632 X 521 as: 6 3 2 * 1 2 5 1•2 1•3 2•2 SHIFT 1•6 MULTIPLY 2•3 ADD 5•2 2•6 5•3 5•6 30 27 17 7 2 : sequence of CONVOLUTION SUMS 105 104 103 102 101 100 5 4 3 2 1 0 :i CONVOLUTION INDEXWe may index the CONVOLUTION SUMS in the order in which they appear. Notice that it isnot necessary to know the most significant digits in advance. We may initiate theCONVOLUTION process given (• • • 632) * (• • • 521). This is useful in real timeapplications where we may not know all the input in advance, or in the design of InfiniteImpulse Response (IIR) filters.Notice that: 632 * 521 = 521 * 632. The sequence of the CONVOLUTION SUMS is the same.This method which we call LONG MULTIPLICATION (without carry) is actuallyCONVOLUTION. It was invented by Vedic Mathematicians after they invented zero and thedecimal digital system. This method is simple and fast when compared to the usualmethod of MULTIPLICATION: repeated addition where the addends are the same.Take a close look at the column of 102 for example. Will the products (1•6, 2•3, 5•2) inthat column appear in the same order in 521 * 632? Will it be inverted?If we were to plot these products or super-position in this column on a graph in the orderin which they are appear, will the graphs of this column in 632 * 521 and 521 * 632 bethe same?Later we shall actually plot the products in a column i and call it the super-position curvewith CONVOLUTION INDEX i. 18
  19. 19. CONVOLUTION SUMInstead of particular numbers like 632 and 521, let us represent them in an abstract wayand see if there is a pattern in the summation of each column which can be made in to aformula.particular: ••• 0 0 0 6 3 2 ••• 0 0 0 5 2 1abstract: a5 a4 a3 a2 a1 a0 b5 b4 b3 b2 b1 b0So now we have two pure sequences a[n] where ak = 0 for k > 2 and b[n] where bk = 0 for k > 2.This is known as PADDING WITH ZEROS. We are required to find the pure sequence c[n] = a[n] * b[n] a2 a1 a0 * b0 b1 b2 a2a2 a2b1 a2b0 a1b0 a0b0 a1b2 a1b1 a0b1 SHIFT MULTIPLY a0b2 ADD c4 c3 c2 c1 c0Pay careful attention to the subscripts of the products in each column. Do you see apattern? c0 = a0b0 c1 =a0b1 + a1b0 c2 =a0b2 + a1b1 + a2b1In each case the formula isNow c3 = a1b2 + a2b1 = a0b3 + a1b2 + a2b1 + a3b0 19
  20. 20. Again:because b3 = 0 and a3 = 0Now c4 = a2b2 = a0b4 + a1b3 + a2b2 + a3b1 + a4b0Again:because b4 = b3 = a3 = a4 = 0So due to the PADDING WITH ZEROS we can use the same formula for ci for the summationof the products in each column i.ci is known as the CONVOLUTION SUM.If we PAD WITH ZEROS all the way to -∞ to +∞, we may write the general form of theCONVOLUTION SUM as: 20
  21. 21. Exercise: To gain more insight, compute the following and write out the sequence ofCONVOLUTION TERMS with the appropriate CONVOLUTION INDEX below each term. a) 632 * 0.00521 b) 0.00632 * 521 c) 632000 * 521 d) 632 * 521000 e) 632000 * 0.00521 f) 0.00632 *521000Notice how the sequence of CONVOLUTION SUMS 30, 27, 22, 7, 2 is SHIFTED right or left.Despite this SHIFTING, the sequence of CONVOLUTION SUMS between the first non-zeroterm and last non-zero term is the same. We may call this property SHIFT INVARIANCE.It is not necessary that we stick to only decimal digital values in the operands. We coulduse any values, as we would expect if we were to discretize two real value continuousfunctions over the time axis.Exercise: CONVOLVE the two sequences below and write the appropriate CONVOLUTIONINDEX below each CONVOLUTION TERM. (1, √2, -3) and (√2, -5, 1/2) Index k: -1 0 1 2 3 4Exercise: You know that with decimal digital numbers: (632 * 521) * 489 = 632 * (521 * 489)This property is known as associativity. Verify that convolution is an associativeoperation: (a[n] * b[n]) * c[n] = a[n] * (b[n] * c[n]).We know that * is commutative: a[n] * b[n] = b[n] * a[n] .Commutativity allows us to change the order of the operands.Associativity allows us to change the order of the evaluation.CATALAN numbers tells us how many different ways of associating n applications of abinary operator. So, we have a lot of flexibility in the way we may perform thecalculations involving convolution. This directly translates into the flexibility of hardwaredesign of LTI Systems. 21
  22. 22. DEFINITION OF TERMSLets us state the terminology for the various formulae before we proceed further. Wemake a distinction between pure sequences and sequences fA[tk] and fB[tk] got from fA(t)and fB(t), continuous functions over time, made discrete using equal spacing Ts. We shallcall these discrete functions. DISCRETE pure sequences discrete functions ASYMMETRIC fA(t) fA[tk]1. EFFECT [TA─, TA+] [NA─, NA+]2. EFFECT SUM3. TOTAL EFFECT SUM fB(t) fB[tk] [TB─, TB+] [MB─, MB+] SYMMETRIC SYMMETRIC1. EFFECT 1. COMBINED EFFECT2. CONVOLUTION SUM 2. COMBINED EFFECT SUM3. TOTAL CONVOLUTION SUM 3. TOTAL COMBINED EFFECT SUMWe have 3 kinds of intervals to deal with in the CONVOLUTION process: 1. Equal sub-intervals Ts 2. Overlapping interval = contiguous overlapping sub-intervals. We plot the super-position curve over the overlapping interval. ∑ of the area under the super-position curve = COMBINED EFFECT SUM ∫ of the area under the super-position curve = CONVOLUTION INTEGRAL 3. Convolution interval = the entire interval over which the CONVOLUTION is taking place. continuous case: [TA─ + TB─, TA+ + TB+] is the convolution interval. discrete case: [NA─ + MB─, NA+ + MB+] is the convolution interval. We plot the convolution curve over the convolution interval. A point on the convolution curve is: discrete function case: COMBINED EFFECT SUM continuous function case: CONVOLUTION INTEGRALWe have two special INSTANTS: T T 22
  23. 23. Guitar strings form pure sequences. We want to be able to work with continuousfunctions and discrete functions. We also need to go back and forth between bothversions. So let us proceed to define more formally the CONVOLUTION SUM for discretefunctions and then derive the CONVOLUTION INTEGRAL for continuous functions.1. EFFECT AT AN INSTANT fA(t) = sin(t) fA(ti) is the effect at an instant ti 0 ti t Displacement /amplitude of an oscillating swing/pendulum fB(t) = e−at fB(ti) is the EFFECT at an INSTANT ti 0 ti t Friction function where a is the co-efficient of friction2. EFFECT AT CORRESPONDING INSTANTS fC(t) = fA(t) • fB(t) 0 ti t fA(ti)•fB(ti) is the EFFECT of fA(t) and fB(t) at CORESSPONDING instant ti. Another way to view this is: fC(ti) is the EFFECT at INSTANT ti. 23
  24. 24. 3. COMBINED EFFECT OVER A SUB-INTERVAL ( δt, t, Ts )In high school Calculus, assuming f(t) =CHANGE in F(t) = Area under f(t) = f(t)dtover interval [a,b] over interval[a,b]So: CHANGE in F(t) over δt = f(t) • δtNow: CHANGE EFFECTCOMBINED EFFECT of f(t) = EFFECT of f(t) over sub-interval δt = f(t) • δtHence: fA(t) = sin(t) 0 δti t COMBINED EFFECT of fA(ti) = EFFECT of fA(ti) over sub-interval δti = fA(ti) • δti fB(t) = e−at 0 δti t COMBINED EFFECT of fB(t) = EFFECT of fB(t) over sub-interval δti = fB(ti) • δti 24
  25. 25. 4. COMBINED EFFECT of fA(t) and fB(t) over CORRESPONDING SUB – INTERVALS of the equal size δti. fC(t) = fA(t)•fB(t) 0 δti t COMBINED EFFECT of fA(t) and fB(t) over CORESSPONDING SUB-INTERVAL δti is: COMBINED EFFECT of fC(ti) = fC(ti) • δti = fA(ti) • fB(ti) • δ ti We can do this because the sub-intervals for both functions fA(t) and fB(t) are of the same size. 5. COMBINED EFFECT of fA(t) and fB(t) over NON-CORRESPONDING SUB- INTERVALS of the equal size δt = δti = δtj: fA(t) = sin(t) fB(t) = e−at0 δti 0 δtj fA(ti) • fB(tj) • δt This step is crucial. We saw that in the CONVOLUTION process we do a SHIFT and MULTIPLY. To find the COMBINED EFFECT the SYMMETRIC way, the sub-intervals for both functions should be of the same size. Later, we shall refer to this important aspect in a different way and say: equally spaced points, implying sub-intervals of the same size. Hence we will say: without equal spacing there is no convolution. 25
  26. 26. 6. DISCRETISING a CONTINUOUS function with equal spacing Ts .Now we can slowly move away from the language and notation of Calculus that dealswith CONTINUOUS functions and get back to DISCRETE notation and calculations. Acontinuous function over an interval [0,T] can be made discrete by choosing a sub-interval of size Ts and dividing the interval [0,T] into N = T/Ts equal parts. The subscript sin Ts usually refers to the word sample.So, instead of an infinitesimally small sub-interval δt, we have a bigger (measurable) sub-interval t. We may denote a sub-interval by ti. They are of the same size Ts. Now wetake the discrete values f(ti), where the ti are the mid-points of the sub-intervals ti. CONTINUOUS DISCRETE f(t) = t2 f[ti] = ti2 • • • 0 0 T t t0 t1 t2 T = N • Ts tCOMBINED EFFECT of f[ti] = EFFECT of f[ti] over a sub-interval ti = f[ti] • ti or f[ti] • Ts.With this terminology and notation in mind we may write the COMBINED EFFECT of fA[ti]and fB[tj] over NON-CORRESPONDING sub-interval of EQUAL size is: fA[ti] • fB[tj] • Ts 26
  27. 27. 7. OVERLAPPING INTERVALDue to the SHIFT the two discrete functions fA[tk] and fB[ti-k] will overlap over more thanone sub-interval Ts. In fact, they will overlap over a contiguous set of sub-intervals. Thiscommon set of contiguous sub-intervals is called the overlapping interval.8. COMBINED EFFECTS AND SUPER-POSITION CURVEFor each SHIFT identified by the CONVOLUTION INDEX i we have the products (super-positions) in column i. These products (super-positions) or COMBINED EFFECTSfA[tk] • fB[ti-k] • Ts are over the range of k in the overlapping interval. We may plot theseCOMBINED EFFECTS to get the super-position curve over this overlapping interval.9. COMBINED EFFECT SUMFor each i we may sum up these COMBINED EFFECTS (in column i) to get the area under thesuper-position curve. The area under the super-position curve yields a single valuewhich in the discrete case is: A Band in the continuous case is the CONVOLUTION INTEGRAL.10. CONVOLUTION INTERVAL AND CONVOLUTION CURVEConvolution interval = the entire interval over which the CONVOLUTION is taking place.continuous case: [TA─ + TB─, TA+ + TB+] is the convolution interval.discrete case: [NA─ + MB─, NA+ + MB+] is the convolution interval.For each CONVOLUTION INDEX i in the CONVOLUTION INTERVAL, we plot the COMBINEDEFFECT SUM i for i • Ts on the time axis to get the CONVOLUTION CURVE.What is the area under the CONVOLUTION CURVE? TOTAL COMBINED EFFECT SUM: A BIn the continuous case, we have the CONVOLUTION INSTANT .For each CONVOLUTION INSTANT t in the CONVOLUTION INTERVAL, we plot theCONVOLUTION INTEGRAL for INSTANT t on the time axis to get the CONVOLUTION CURVE.What is the area under the CONVOLUTION CURVE? TOTAL COMBINED EFFECT INTEGRAL: A B ─ ─ ─ 27
  28. 28. A SIMPLE EXAMPLELet us now take two very simple continuous functions and make them discrete overinterval [0, 1] with Ts = ¼ sec implying N = 4. So the interval [0, 1] is divided into N=4EQUAL sub-intervals. Index k of ak and bk runs from 0 to N-1 = 3. fA[tn] fB[tn] a0 a1 a2 a3 b0 b1 b2 b3+1 • • • • +1 • • • • 0 1 2 3 4=N t 0 1 2 3 4=N t +1 sec +1 secIn general when making a function discrete, the index k is 0,1,2,3, • • • for the functionover [0, +∞) and k = -1,-2,-3, • • • for the function over [0, -∞).In the CONVOLUTION process, after we make fA(t) and fB(t) discrete choosing some particularsub-interval size Ts, we may treat the discrete values fA[tk] as a pure sequence and fB[tk] as a puresequence. We then find: fC [tn] = fA[tn] * fB[tn] more commonly written as: c[n] = a[n] * b[n] 28
  29. 29. Notation: some books use the notation b[−n] to denote b[n] convolved.There are several important aspects in the CONVOLUTION process. In the examples and exercisesthat follow we try to bring out these aspects • SUPERPOSITON • WEIGHTED • SHIFTING • OVERLAPPING INTERVALS • EQUAL SPACING • PADDING WITH ZEROS • CONVOLUTION INTERVAL • CONVOLUTION INDEXOnce we have found a CONVOLUTION SUM ci, we need to know: What is the (COMBINED EFFECT SUM)i? What is the ordinate on the time axis of the (COMBINED EFFECT SUM)i?We know Ts. So (COMBINED EFFECT SUM)i = ci • Ts. And its ordinate will be at i • Ts on the timeaxis. So getting the CONVOLUTION INDEX i right is extremely important. And vice versa, if we knowt on the time axis, we can determine the index i of ci : i = t/Ts . 29
  30. 30. We are ready to start the convolution process by fixing fA[tk] and SHIFTING CONVOLVED fB[tk] fromleft to right. We could have equally well fixed the CONVOLVED fB[tk] and SHIFTED the fA[tk] fromright to left. The sequence of CONVOLUTION SUMS c0, c1, c2, • • • will appear in the same order. Weknow that: fA[tk] * fB[tn] = fB[tn] * fA[tn]. fB[tn] convolved fA[tn] b3 b2 b1 b0 a0 a1 a2 a3 • • • • • • • • 0 1 2 3 4 5 6 7 8 t +1 sec +2 secNote that the index k of bk is still positive even though it is going in the negative direction(right to left). In the computation process we shift in the positive direction (left to right).So we make use of index of fA[tn] to determine the index of the SHIFT and hence theCONVOLUTION INDEX. fA[tn] : a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0CONVOLUTION SUM ci c0 c1 c2 c3 c4 c5 c6with index iVALUESWe show the computation of each CONVOLUTION SUM ci step by step.We have 7 = (4+4-1) = N+N-1 CONVOLUTION SUMS ci to compute. The ci are numberedfrom 0 to 2N-2, which is 0 to 6, using the CONVOLUTION INDEX i . We now SHIFTconvolved fB[tn] across fA[tn] sub-interval by sub-interval. 30
  31. 31. CONVOLUTION INDEX i=0 fB[tn] convolved fA[tn] b3 b2 b1 a0b0 a1 a2 a3 • • • • • • • 0 1 2 3 4 t +1 sec fC[t0] = fA[t0] • fB[t0]CONVOLUTION SUM c0 = a0 • b0 range of index k: 0COMBINED EFFECT SUM c0 • Ts = (a0 • b0) • Ts overlapping interval [0, 0.25] sec fA[tn]: a0 a1 a2 a3 *fB[tn]convolved: b3 b2 b1 b0 a0•b0CONVOLUTION SUM ci c0 c1 c2 c3 c4 c5 c6with index iVALUES 1 31
  32. 32. CONVOLUTION INDEX i=1 fB[tn] convolved fA[tn] b3 b2 a0b1 a1b0 a2 a3 • • • • • • 0 1 2 3 4 5 6 7 8 t +1 sec +2 sec fC[t1] = fA[t0] • fB[t1] + fA[t0] • fB[t1]CONVOLUTION SUM c1 = a0 • b1 + a1 • b0 range of index k: 0, 1COMBINED EFFECT SUM c1 • Ts = (a0 • b1 + a1 • b0) • Ts overlapping interval [0, 0.50] sec fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a0•b1 a1•b0CONVOLUTION SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 32
  33. 33. CONVOLUTION INDEX i=2 fB[tn] convolved and fA[tn] b3 a0b2 a1b1 a2b0 a3 • • • • • 0 1 2 3 4 5 6 7 8 t +1 sec +2 sec fC[t2] = fA[t0] • fB[t2] + fA[t1] • fB[t1] + fA[t2] • fB[t0]CONVOLUTION SUM c2 = a0 • b2 + a1 • b1 + a2 • b0 range of index k: 0, 1, 2COMBINED EFFECT SUM c2 • Ts = (a0 • b2 + a1 • b1 + a2 • b0) • Ts overlapping interval [0, 0.75] sec fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a0•b1 a1•b0 a0•b2 a1•b1 a2•b0CONVOLUTION SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 3 33
  34. 34. CONVOLUTION INDEX i=3 fB[tn] convolved and fA[tn] a0b3 a1b2 a2b1 a3b0 +1 • • • • 0 1 2 3 4 5 6 7 8 t +1sec +2 sec fC[t3] = fA[t0] • fB[t3] + fA[t1] • fB[t2] + fA[t2] • fB[t1] + fA[t3] • fB[t0]CONVOLUTION SUM c3 = a0 • b3 + a1 • b2 + a2 • b1 + a3 • b0 range of index k: 0, 1, 2, 3COMBINED EFFECT SUM c3 • Ts = (a0 • b3 + a1 • b2 + a2 • b1 + a3 • b0) • Ts overlapping interval [0, 1] sec fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a0•b1 a1•b0 a0•b2 a1•b1 a2•b0 a0•b3 a1•b2 a2•b1 a3•b0CONVOLUTION SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 3 4 34
  35. 35. CONVOLUTION INDEX i=4 fB[tn] convolved and fA[tn] a0 a1b3 a2b2 a3b1 b0 +1 • • • • 0 1 2 3 4 5 6 7 8 t +1 sec +2 sec fC[t4] = fA[t1] • fB[t3] + fA[t2] • fB[t2] + fA[t3] • fB[t1]CONVOLUTION SUM c4 = a1 • b3 + a2 • b2 + a3 • b1 range of index k: 1, 2, 3COMBINED EFFECT SUM c4 • Ts = (a1 • b3 + a2 • b2 + a3 • b1) • Ts overlapping interval [0.25, 1.0] sec fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a0•b1 a1•b0 a0•b2 a1•b3 a1•b1 a2•b2 a2•b0 a3•b1 a0•b3 a1•b2 a2•b1 a3•b0CONVOLUTION SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 3 4 3Here we need to explain how PADDING WITH ZEROS allows us to use the same formulainstead ofwhere the range of k = 1, 2, 3 is the overlapping interval. 35
  36. 36. Notice the overlapping interval due to the SHIFT. While the CONVOLUTION INDEX iidentifies the SHIFT position, the index k of the ak and bi-k lies within the overlappinginterval and is used to find the products ak • bi-k listed in the column i.Some very important points to note.1. For each CONVOLUTION INDEX i pay close attention to the overlapping interval of fB[tn] CONVOLVED and fA[tn] on the time axis of fA[tn]. For this CONVOLUTION INDEX i = 4 the overlapping interval is [0.25, 1.0] seconds.2. More importantly i • Ts = 1.0 seconds. So on the CONVOLUTION CURVE the COMBINED EFFECT SUM with index i = 4 will be plotted for t = i • Ts = 1.0 seconds on the time axis adjusted by Ts/2 to get the mid-point.3. The index k of the super-positions ak • bi−k that appear in column i = 4 lie within this overlapping interval [0.25, 1.0] seconds on the time axis of fA[tn]. For this CONVOLUTION INDEX i = 4 the range of k = 1, 2, 3. 36
  37. 37. PADDING WITH ZEROSNotice that outside the overlapping interval [0.25, 1.0] seconds on the time axis of fA[tn]for this CONVOLUTION INDEX i = 4, the super-positions ak• bi−k are zero. Why?Because outside the overlapping interval either ak does not exist and hence is zero, or bi−kdoes not exist and hence is zero. Hence these ak•bi-k products or super-positions are notshown in the respective columns of ci. Also we do not plot them on the super-positioncurve. So a4, a5, a6, b4, b5, b6 are zero. This we call PADDING WITH ZEROS. PADDING WITHZEROS allows us to use the same concise formula for the CONVOLUTION SUM andCOMBINED EFFECT SUM.The index k of the super-position has range 1, 2, 3 in the overlapping interval. But sinceb4 = 0, this implies a0 • b4 = 0. So we can extend the range of k left to 0. Likewise, since a4= 0, this implies a4 • b0 = 0. Again, we can extend the range of k right to 4 = i. So now therange of k is 0, 1, 2, 3, 4 = i. We may use the same formula:It is because of PADDING WITH ZEROS in both directions that we can have the sameformula for ci as the CONVOLUTION INDEX i varies over its entire range from 0 to 6. Alsothe formula matches the picture of the right edge of the convolved fB[tn] being atCONVOLUTION INDEX i as it SHIFTS (continuous case: slides) left to right over fA[tn].We may PAD WITH ZEROS fA[tn] and fB[tn] in both directions all the way to -∞ andto +∞. And now we may write the general form of theAlso notice that the overlapping interval is finite. Later, with Ts → 0, we get an infinitenumber of products ak•bi-k in this finite overlapping interval. Then the discrete ∑ of theseinfinite number of products ak•bi-k in column i, when multiplied by dτ (representinglimit (Ts→ 0)) becomes a continuous summation or an ∫ over the overlapping interval.This is the CONVOLUTION INTEGRAL, the continuous version of what we call theCOMBINED EFFECT SUM. 37
  38. 38. SUPER-POSITION CURVEWe now present an intermediate step which is usually not shown in text-books. We shallplot the super-position curve for this CONVOLUTION INDEX i = 4. super-position curve with CONVOLUTION INDEX i = 4 a1b3 a2b2 a3b1 +1 • • • ak• b4-k = fA[tk] • fB[ti-k] t = i •Ts = 1 sec 0 1 2 3 4 time axis of fA(τ) τ +0.25 sec +1 sec overlapping intervalGeometrically, we can find the area under the super-position curve over the overlappinginterval with Ts = ¼ sec.(COMBINED EFFECT SUM)i = ci • Ts = area under the super-position curve with index i.Note: this example is too simple to notice the difference. But what do you think the super-position curve over the same overlapping interval of fB[tn] * fA[tn] will look like? Will itlook reflected?Recall what we said in the example 632 * 521: the order of the products in each columnof 521 * 632 will be inverted. But the sum of the products in each column in both caseswill be the same.Exercise: Repeat this example with Ts = 1/8 sec. Plot the super-position curve over thesame overlapping interval and find the area. 38
  39. 39. What do you think will happen when Ts becomes smaller and smaller?What will the super-position curve look like over the same overlapping interval?We have the CONVOLUTION INDEX i and the index k on the overlapping interval on thetime axis of fA[tn]. We use this to plot the discrete super-position curve. discrete continuous fA[tk] • fB[ti-k] fA(t) • fB(t) fA[kTs] • fB[(i-k) •Ts] fA(τ) • fB(t-τ)As Ts → 0 the discrete super-position curve becomes a continuous super-position curve.This gives the expression fA(τ) • fB(t-τ) for the super-position curve in the continuouscase.Where did the τ come from? And why do we need it? We shall explain when we get tothe CONVOLUTION INTEGRALAfter we find the (COMBINED EFFECT SUM)i for each CONVOLUTION INDEX i, we may plotthem for each ordinate i • Ts to get the CONVOLUTION CURVE.Note that each point on the CONVOLUTION CURVE is ci • Ts = COMBINED EFFECT SUM withindex i for ordinate i • Ts on the time axis.What do you think will happen when Ts becomes smaller and smaller?What will the COMBINED EFFECT SUM ∑ ak • bi-k • Ts look like?Will it become an integral? ∑ ak•bi-k • Ts ∫ fA(τ) • fB(t-τ) • dτIn the continuous case, each point on the convolution curve is the CONVOLUTIONINTEGRAL at instant t where: i • Ts → t as Ts → 0 k • Ts → τ as Ts → 0 Ts → dτ as Ts → 0 39
  40. 40. Let us continue with our simple example CONVOLUTION INDEX i=5 fB[tn] convolved and fA[tn] a0 a1 a2b3 a3b2 a3b1 a3b0 • • • • • • 0 1 2 3 4 5 6 7 8 + 1sec +2 sec fC[t5] = fA[t2] • fB[t3] + fA[t3] • fB[t2]CONVOLUTION SUM c5 = a2 • b3 + a3 • b2 range of index k: 2, 3COMBINED EFFECT SUM c5 • Ts = (a2 • b3 + a3 • b2) • Ts overlapping interval [0.5, 1.0] sec fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a0•b1 a2•b3 a1•b0 a 3•b2 a0•b2 a1•b3 a1•b1 a2•b2 a2•b0 a3•b1 a0•b3 a1•b2 a2•b1 a3•b0CONV. SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 3 4 3 2 40
  41. 41. CONVOLUTION INDEX i=6 fB[tn] convolved and fA[tn] a0 a1 a2 a3b3 b2 b1 b0 • • • • • • • 0 1 2 4 5 6 7 8 +1 sec +2 sec fC[t6] = fA[t3] • fB[t3]CONVOLUTION SUM c6 = a3 • b3 range of index k: 3COMBINED EFFECT SUM c6 • Ts = (a3 • b3) • Ts overlapping interval [0.75, 1.0] sec fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a3•b3 a0•b1 a2•b3 a1•b0 a3•b2 a0•b2 a1•b3 a1•b1 a2•b2 a2•b0 a3•b1 a0•b3 a1•b2 a2•b1 a3•b0CONV. SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 3 4 3 2 1 41
  42. 42. CONVOLUTION INDEX i=7 fB[tn] convolved and fA[tn] a0 a1 a2 a3 b3 b2 b1 b0 • • • • • • • • 0 1 2 3 4 5 6 7 8 +1 sec +2 secThe convolution process is complete fA[tn]: a0 a1 a2 a3 *fB[tn] convolved: b3 b2 b1 b0 a0•b0 a3•b3 a0•b1 a2•b3 a1•b0 a 3•b2 a0•b2 a1•b3 a1•b1 a2•b2 a2•b0 a3•b1 a0•b3 a1•b2 a2•b1 a3•b0CONV. SUM ci c0 c1 c2 c3 c4 c5 c6VALUES 1 2 3 4 3 2 1 42
  43. 43. CONVOLUTION SUMS AND CONVOLUTION CURVE 43
  44. 44. We prefer to use lollipop diagrams (seen in text books) for pure sequences, and graphpaper (grid background diagrams) for discrete functions with the CONVOLUTION INDEX onthe horizontal axis and the time indicated below.Notice the difference in the diagrams. For pure sequences we do not have a time axis. Wehave a horizontal axis with the CONVOLUTION INDEX i: i = −∞, . . . -3, -2, -1, 0, 1, 2, 3, . . . , +∞For pure sequences like decimal digital numbers we write: c[n] = a[n] * b[n].Each ci or c[i] is called a CONVOLUTION SUM.For sequences that are discrete functions, like fA[tn] and fB[tn] got by discretisingcontinuous functions fA(t) and fB(t) over some specified interval or intervals, we shouldbe clear and use the notation: fC[tn] = fA[tn] * fB[tn] c[tn] = a[tn] * b[tn] y[tn] = x[tn] * h[tn] and so on.Here what we compute on the right hand side is the COMBINED EFFECT SUM. We may usethis expression to get to the CONVOLUTION INTEGRAL. We plot c[ti] over i • Ts on the timeaxis. 44
  45. 45. CONVOLUTION INTERVALSee where the COMBINED EFFECT SUMS occur on the time axis with T=1 sec & Ts = ¼ sec.To gain more insight, again with T = 1 sec choose Ts = 1/8 and plot the CONVOLUTIONCURVE of: fC[tn] = fA[tn] * fB[tn]How many COMBINED EFFECT SUMS will there be?Where will you plot them?What will be the CONVOLUTION INTERVAL?The CONVOLUTION INTERVAL is [0, 1.5] secs.PADDING WITH ZEROSIt is not necessary that the intervals over which fA(t) and fB(t) are defined be the same. Wecould have defined fA(t) over [0, T1] and fB(t) over [0, T2]. Let us say T1 < T2. Then wePAD WITH ZEROS the values of fA(t) over [T1, T2]. fC(t) will be over [0, T1 + T2].Now when we say fC(t) = fA(t) * fB(t) we mean: C (t) = A (t) * B (t) over over [0, T1] over [0, T2] [0, T1+T2] padded with padded with zeros over [T1, T1+T2] zeros over [T2, T1+T2]In the discrete versions we choose a certain Ts. Let N = T1/Ts and M = T2/Ts.Since T1 < T2 we will have N < M. So we pad the remaining (M-N) values of A [tk] withzeros and perform the CONVOLUTION. Now there will be (N+M-1) CONVOLUTION SUMS cior fC[ti] with the CONVOLUTION INDEX i running from 0 to (N+M-2). We shall see more onthis aspect when we talk about the CONVOLUTION INDEX. 45
  46. 46. Let us now formularize the CONVOLUTION SUM and COMBINED EFFECT SUM computationprocess:1. CONVOLUTION SUM for CONVOLUTION INDEX i = 0, 1, 2, 3, . . . , 2N−2. C A B A B2. COMBINED EFFECT SUM for CONVOLUTION INDEX i = 0, 1, 2, 3, . . . , 2N−2.[ C A B A BEQUAL SPACINGIf we look closely at the summations on the right hand side we note that Ts appears inboth fA[k • Ts] and fB[(i− k) • Ts]. Because the sub-intervals are of EQUAL size we canperform the CONVOLUTION. The CONVOLUTION formula works. Hence we say: withoutequal spacing there is no CONVOLUTION. This is the reason when sampling, we need totake equally spaced samples. When some of the samples are missing, the points orsamples will not be equally spaced. Then what do we do?NORMALIZED CONVOLUTIONFurther reading on the Internet: Normalized Convolution: A Tutorial by RobertaPiroddi and Maria Petrou (September 8, 2003).Example of Calculation of Normalized Convolution by Bob Fisher (09 March 2003). 46
  47. 47. A WORD OF CAUTIONWe were very careful to use the expression overlapping interval even though from thediagrams it looks like overlapping areas. In this simple example the difference is NOTnoticeable. But it is crucial. Some authors, when explaining CONVOLUTION graphicallyuse this simple example and say overlapping areas. But this is NOT correct. Moreover, itdoes NOT convey the sense of EFFECT computed by multiplication (superposition) overoverlapping sub-intervals of size Ts. Lets us see a slightly different example.Now when you CONVOLVE (reverse, reflect) one of them and SHIFT it over the other, theareas do not overlap. But contiguous sub-intervals do overlap. So we have an overlappinginterval. The EFFECT ak • bi−k is computed as before, and is: 2 • 3 = 6.Exercise: Work out this example in detail. Here Ts = 0.5 sec. Pad fA[tn] with zeros. Plot the super-position curve with CONVOLUTION INDEX i = 3. Plot the CONVOLUTION CURVE. 47
  48. 48. Exercise: Compute and plot C( )= A ( )* B( ) with T = 1 sec and Ts = ¼ seca) A (t) = 1, B (t) = tb) A( (t) = t , B (t) =tExercise: Repeat the above exercise with Ts = 1/8 sec.It is very important to note that • A (t) is defined over [0, T]. • fB(t) is defined over [0, T]. • But fC(t) = fA(t) * fB(t) is defined over the CONVOLUTION INTERVAL [0, 2T] no matter how many points we may take, or regardless of the size of Ts.To convey the notion that CONVOLUTION is being performed we may write: fC(t) = fA(t) * fB(t)But we have to specify the interval over which the CONVOLUTION of fA(t) and fB(t) istaking place. 48
  49. 49. EXAMPLE FROM COMMUNICATION SYSTEMSWe shall work out one last example adapted from Communication Systems by B. P. Lathi(page 84, fig 1-43), John Wiley, NY 1965, so that all the aspects of DISCRETECONVOLUTION can be highlighted. • Making two continuous functions discrete. • Using equal spacing Ts = 0.5 sec. • Padding with zeros • Then indexing the discrete functions correctly. • Convolving one of the functions and shifting • Finding the super-positions (products) ak • bi-k. • Finding the CONVOLUTION SUM ci. • Determining the picture for a chosen convolution index i or t = i • Ts.In the previous example we saw how PADDING WITH ZEROS allows us to use the sameCONVOLUTION SUM formula:throughout the CONVOLUTION process. The starting CONVOLUTION INDEX i (left end) is the(starting index of fA[tn]) + (starting index of fB[tn]) which in the earlier example was0 + 0 = 0.In this example fA(t) = t over [0, 2] sec and fB(t) = +2 over [─1, +1] sec. Let Ts = 0.5 sec.So fA[tk] = tk for k = 0, 1, 2, 3 and fB[tk] = +2 for k = ─2, ─1, 0, 1.We shall PAD WITH ZEROS in two steps. Initially we PAD WITH ZEROS fA[tk] for k = ─2, ─1.And we PAD WITH ZEROS fB[tk] for k = 2, 3.So now the range of k for both fA[tk] and fB[tk] is k = ─2, ─1, 0, 1, 2, 3.We do this for the sake of clarity and simplicity in the explanation that follows.The CONVOLUTION INDEX i will range from ─2 + (─2) = ─4 on the left to 3 + 3 = 6 on theright. 49
  50. 50. Even though we do not show it in the diagrams, we further PAD WITH ZEROS both fA[tk]and fB[tk] from ─5 to ─∞ on the left and from +4 to +∞ on the right.So now we can use the general form of the CONVOLUTION SUM throughout the example:Since we know that ci = 0 for i < ─4 and i > 6 we may use the form of the CONVOLUTIONSUM:So now the CONVOLUTION INDEX i varies from ─4 to +6. The CONVOLUTION INTERVAL is[─4, 6].Since we want to focus on the overlapping interval and the super-position curve over theoverlapping interval as convolved fB[tk] SHIFTS over fA[tk] we shall show only the rangeof k that lies in the overlapping interval.This way we can clearly depict the super-position curve over the overlapping interval asthe shifting progresses.Also we can clearly see how we get the CONVOLUTION SUMS for the lollipop diagram andthe COMBINED EFFECT SUMS for the CONVOLUTION CURVE. 50
  51. 51. 51
  52. 52. 52
  53. 53. 53
  54. 54. 54
  55. 55. 55
  56. 56. As an exercise find fC(t = 0.5 sec) with fA(t) convolved instead of fB(t).Sketch the overlapping of the two functions at t = 0.5 sec.What do you think the super-position curve fA(t-τ) • fB(t) with t = 0.5 sec will look like?Will this super-position curve look like the reflected version of the super-position curvewith CONVOLUTION INDEX i = 1?What is the area under the super-position curve? 56
  57. 57. CONVOLUTION INTERVAL AND CONVOLUTION INDEXWe are familiar with the CONVOLUTION (long multiplication without carry) of twodecimal digital numbers. For example: 6320 * 0.000521The interval of the least significant and most significant NON-ZERO digits is [1, 3] 103 102 101x[n] = . . . 0 0 6 3 2 0 . 0 0 0. . .xk = 2, 3, 6 for k = 1, 2, 3 respectivelyThe INDEX interval of the least significant and most significant NON-ZERO digits is[─6, ─4] 10-4 10-5 10-6y[n] = . . . 0 0 0 0 0 . 0 0 0 5 2 1 0 0...yk = 5, 2, 1 for k = ─4, ─5, ─6 respectivelyAfter the CONVOLUTION we get z[n] = x[n] * y[n].The least significant NON-ZERO digit will be under 10-5The most significant NON-ZERO digit will be under 10-1The CONVOLUTION INTERVAL is [1─6, 3─4] = [─5, ─1].The INDEX i of the CONVOLUTION SUMS we need to evaluate are: ─5, ─4, ─3, ─2, ─1.This is exactly the role of the CONVOLUTION INDEX even after PADDING WITH ZEROS.And 632 * 521 yields the values zi : 2, 7, 22, 27, 30 in the INDEXED order.In fact we may compute the value of any chosen term ziNotice that k + (i-k) = i.For example: z -3 = (x3.y-6) + (x2.y-5) + (x1.y-4) = 6 + 6 + 10 = 22 57
  58. 58. PADDING WITH ZEROS to −∞ and +∞Let us see a general example using two small finite sequences.Example: Given two sequencesx[n]= . . . .0, 0, 0, x-6, x-5, x-4, x-3, 0, 0, 0, . . .y[n]= . . . .0, 0, 0, y5, y6,, y7, y8, y9, 0, 0, 0, . . .What is CONVOLUTION INTERVAL of: z[n] = x [n]*y[n] ?Let k- be the INDEX of the left most NON-ZERO term.k+ be the INDEX of the right most NON-ZERO term. ─x [n] is in the interval [ kx , kx+ ] = [ ─6, ─3 ] ─y[n] is in the interval [ ky , ky+ ] = [ 5, 9 ] ─ ─ ─z[n] is in the interval [ iz , iz+ ] = [ kx + ky , kx+ + ky+ ] = [ ─6+5, -3+9 ] ─The CONVOLUTION INTERVAL = [ iz , iz+ ] = [ ─1, +6 ].The first and last NON-ZERO CONVOLUTION SUMS are z-1 and z6.The CONVOLUTION INDEX and CONVOLUTION SUMS are: . . . 0, 0, 0, z-1, z0, z1, z2, . . . , z6, 0, 0, 0, . . .From now on we PADDING WITH ZEROS x[n] and y[n] all the way to ─∞ and +∞. We knowexactly the INDEX of the first and last NON-ZERO CONVOLUTION SUMS of z[n] and theCONVOLUTION INTERVAL.How can we evaluate the CONVOLUTION SUMS? Let x[n] = . . . x─3, x─2, x─1, x0, x1, x2, x3, . . . y[n] = . . . y─3, y─2, y─1, y0, y1, y2, y3, . . .To find z[n] first CONVOLVE either one of the sequences and place it under the other. x[n] : . . . x─3, x─2, x─1, x0, x1, x2, x3, . . .y[-n] = y[n] CONVOLVED: . . . y3, y2, y1, y0, y─1, y─2, y─3, . . .Now z[0] = . . . + (x─3.y3) + (x─2.y2) + . . . + (x0.y0) + . . . + (x3.y─3) + . . . 58
  59. 59. Here i = 0. Notice how the index k and index i-k of the products (xk . yi-k) add up to i.To find z[1], SHIFT RIGHT y[-n] one step x[n] : . . . x─3, x─2, x─1, x0, x1, x2, x3, . . .y[-n] shifted right one step: . . . y3, y2, y1, y0, y─1, y─2, y─3, . . .Now z[1] = . . . + (x─2 • y3) + (x─1 • y2) + (x0 • y1) + (x1 • y0) + (x2 • y-1) + . . .Here i =1. Again notice how the index k and index i─k of the products (xk . yi─k) add upto i.To find z[2], SHIFT RIGHT , y[─n] two steps. Find all the products and add up.To find z[-3] SHIFT LEFT, y[-n] three steps. Find all the products and add up. ─The advantage of this method is that once we know the sequences x[n] and y[n] we canfind any chosen zi.When CONVOLVING discrete functions fA[tk] and fB[tk] using Ts , we may view thesesequences a[n] and b[n] as pure sequences. We may then find c[n] = a[n] * b[n]. In factwe may find any chosen ci.What is fC[ti] ? C A BWhere is ti on the time axis? ti = i .Ts TMay be adjusted by 2 to get the mid-point of the sub-interval.This is useful in real time processing. 59
  60. 60. GENERAL FORM OF CONVOLUTION SUMNotice that in the computation of each CONVOLUTION SUM zithere is an interval where the xk and yi-k overlap. Outside this interval the products xk .yi−kare zero. Why?Because outside this overlapping interval, either xk does not exist and hence is zero oryi−k does not exist and hence is zero.So after we PAD WITH ZEROS the given finite sequences x[n] and y[n] in both directions allthe way to ─∞ and +∞ and make the sequences infinite, we may write the more generalform of the CONVOLUTION SUM or 60
  61. 61. CONVOLUTION INTEGRALWe mentioned earlier that we are interested in each individual CONVOLUTION SUM. Weare also interested in each individual COMBINED EFFECT SUM. The COMBINED EFFECT SUMat instant i • Ts is C A B A Bwhere A (t) and B (t) are defined over [0, T].From the exercises with Ts= 1/4 and Ts = 1/8 we get the sense that the smaller the sub-interval Ts, the discrete versions become more accurate or closer to the continuousversions.Let t = i • Ts be the instant on the time axis between [0, 2T]. So now: C A B A BNow, instead of sub-dividing [0, T] into Ts= 1/4 or Ts = 1/8 sub-intervals, let Ts → 0.What happens? Limit (Ts → 0) = dτ, k • Ts becomes τ on the time axis and the ∑ becomesan ∫ over the overlapping interval. The COMBINED EFFECT SUM at instant i • Ts on the lefthand side becomes the CONVOLUTION INTEGRAL at instant t. We get: A B A B In going from discrete to continuous several things happen simultaneously: • First we let Ts→ 0 • Then we take limit (Ts→ 0). • In fB[(i─k) • Ts] the i • Ts→ t, the CONVOLUTION INSTANT. • We have used the letter t for the CONVOLUTION INDEX i. So we need to introduce another variable, say τ, for the index k. • And limit k • Ts→ τ the SUPER-POSITION INSTANT. • And limit (Ts→ 0) = dτ. 61
  62. 62. We make use of a τ (Greek letter t) as the dummy variable of integration because, wealready used t = i • Ts.On the right side we computed the continuous version of the discrete COMBINED EFFECTSUM. But the text books say (quite incorrectly) that it is the continuous version of theCONVOLUTION SUM.This we call the CONVOLUTION INTEGRAL. Standard text-books say that the CONVOLUTIONINTEGRAL is the continuous version of the discrete CONVOLUTION SUM. This is not quitecorrect. That is why we introduced some extra terminology: CONVOLUTION SUM for puresequences and COMBINED EFFECT SUM for discrete functions. And then quite naturally andprecisely: CONVOLUTION INTEGRAL for continuous functions. 62
  63. 63. It is important to see in the CONVOLUTION process: 1. When are we dealing with pure sequences where there is only a pure index on the horizontal axis? and, when are we dealing with discrete functions (sequences got from discretizing continuous functions over specified intervals on the time axis)? Here the CONVOLUTION INDEX i is to identify the sub-intervals with the specified intervals on the time axis. 2. In the case of the COMBINED EFFECT SUM we should know exactly where it occurs on the time axis within the specified CONVOLUTION INTERVAL. Likewise for the CONVOLUTION INTEGRAL.In the case of the CONVOLUTION SUM (used when pure sequences are convolved) weshould know the exact CONVOLUTION INDEX.Exercises: 1. Complete the following table fA(t) fB(t) fA(t) • fB(t) ∫ fA(t) • fB(t) fA(t) * fB(t) 1 1 1 t t tFrom the earlier related exercises can you see the COMBINED EFFECT SUM converging tothe CONVOLUTION INTEGRAL?2.a) What is sin(t) • cos(t)?b) What is sin(t) * cos(t) over one period [0, T].3.a)What is –sin(t) * 1? How is it related to d/dt sin(t)?b) What is –cos(t) * 1? How is it related to d/dt cos(t)?Can you achieve differentiation of sinusoidals using the CONVOLUTION operation? 63
  64. 64. LINEAR CONVOLUTIONSo far what we have been doing is LINEAR CONVOLUTION. This works with finite lengthsequences. To keep the explanation simple, assume only non-negative index. x[n] has length N with index k: 0 to N−1 y[n] has length M with index k: 0 to M−1 z[n] has length N+M-1 with index i: 0 to N+M−2EXAMPLE: index k: -3, - 2, -1, 0, 1, 2, 3, 4 x[n]: . . . . , 0, 0, 0, 2, 3, 0, 0, 0, . . . length N = 2 y[n]: . . . . . ,0, 0, 0, 4, 5, 0, 0, 0, . . . . . length M = 2 index k: -3, - 2, -1, 0, 1, 2, 3, 4 x[n]: . . . , 0, 0, 0, 2, 3, 0, 0, 0, . . . . y[-n]: . . ., 0, 0, 5, 4, 0, 0, 0, . . . .Convolution index k: -3, - 2, -1, 0, 1, 2, 3, 4. . . . . length N+M−1 = 3z[n] = x[n] * y[n] = . . ., 0, 0, 0, 8, 22, 15, 0, 0, . . . .Linear convolution: perform the convolution as usual and write out all the z[i] forconvolution index i from -∞ to +∞.Now let us see what happens when we have to deal with infinite sequences. Infinitesequences may be of 2 types: periodic and aperiodic. Let us consider the periodic case.Let us make both x[n] and y[n] periodic.So now: index k: -3, - 2, -1, 0, 1, 2, 3, 4 x[n]: . . . , 3, 2, 3, 2, 3, 2, 3, 2, . . . period N = 2 y[n]: . . . . ,5, 4, 5, 4, 5, 4, 5, 4, . . . . period M = 2 64
  65. 65. Now what is z[n]=x[n] * y[n] ?Obviously each CONVOLUTION SUM z[i] is unbounded. What can we do in thesesituations? What kind of useful information can we extract?Let us see another example where the periods of x[n] and y[n] are not the same. index k: -3, - 2, -1, 0, 1, 2, 3, 4 x[n]: . . . . , 6, 7, 8, 2, 3, 6, 7, 8, . . . period N = 5 y[n]: . . . . . ,5, 4, 9, 5, 4, 9, 5, 4, . . . . period M = 3 index k: -3, - 2, -1, 0, 1, 2, 3, 4We pad with zeroes y[n] to get: 5 4 9 0 0Now both x[n] and y[n] have the same period N.We take both x[n] and y[n] over one period, assume all the rest are PADDED WITH ZEROESall the way to −∞ and +∞. index k: -3, - 2, -1, 0, 1, 2, 3, 4 5 6 7 x[n]: . . . . , 0, 0, 0, 2, 3, 6, 7, 8, 0, 0, 0, . . . length N = 5 y[n]: . . . . . ,0, 0, 0, 5, 4, 9, 0, 0, 0, . . . . length M = 3Then we find: z[n] = x[n] * y[n] [0,N−1] [0,N−1]the usual linear convolution of two finite sequences. z[n] = z[0], z[1], z[2], z[3], z[4], z[5], z[6]z[n] has length N+M−1 = 7 with linear CONVOLUTION INDEX i: 0 to N+M−2 = 6. 65
  66. 66. MATRIX PICTURE FOR LINEAR CONVOLUTION 66
  67. 67. CIRCULAR CONVOLUTIONSuppose we want z[n] to be periodic with same period N, like x[n] and the PADDED WITHZEROES y[n].Instead of letting index i run from 0 to N+M−2, we take ic = (i) congruence modulo N.We getz[ic]: (z[0c] = z[0]+z[5]), (z[1c] = z[1]+z[6]), (z[2c] = z[2]), (z[3c] = z[3]), (z[4c] = z[4])where ic denotes the circular CONVOLUTION INDEX. Note that the period of z[ic] isN=5This we call circular convolution or periodic convolution. And it is written as: z[n] = x[n] ⊛ y[n].This same method applies even when M = Nx[n] is written inside circle in the positive direction (counter-clockwise).y[n] convolved is written outside the circle in the negative direction (clockwise).ic = 0: z[0c] = z[0] + z[5] = x0 . y0 + (x4 . y1 + x3 . y2)This is exactly what we get when we sum up the corresponding products in the circular picture with ic = 0. 67
  68. 68. Now: ROTATE y[n] convolved counter-clockwise one step.This is similar to a right SHIFT one step in linear convolution.ic =1 : z[1c] = z[1] + z[6] = (x1. y0 + x0 . y1) + (x4 . y2)This is exactly what we get when we sum up the corresponding products in the circularpicture with ic = +1.Notice the difference between linear convolution and circular convolution.z[2], z[3] and z[4] of linear convolution and circular convolution are the same forconvolution index from M to N-1.But z[0c] ≠ z[0], and z[1c] ≠ z[1].By now you must have realized how the z[ic] can be computed for positive ic.How do we compute z[ic] for negative ic?You guessed it! ROTATE y[n] convolved clockwise (negative direction).This corresponds to a left SHIFT in linear convolution. 68
  69. 69. LINEAR CONVOLUTION of sin(ω0t) and cos(ω0t)Let us work out the circular convolution of a(t) = sin(ω0t) and b(t) = cos(ω0t) over oneperiodT0 = 2π sec and Ts = π/4. So M = N = 8. Note that ω0 = 2π/T0. k 0 1 2 3 4 5 6 7 a[tk] 0.0 + 0.7 +1 + 0.7 0 − 0.7 −1 − 0.7 b[tk] +1 + 0.7 0 − 0.7 −1 − 0.7 0 + 0.7Let us first work out the LINEAR CONVOLUTION c[n] = a[n] * b[n]c0 = a0b0 0c1 = a0b1 + a1b0 0.7c2 = a0b2 + a1b1 + a2b0 1.49c3 = a0b3 + a1b2 + a2b1 + a3b0 1.4c4 = a0b4 + a1b3 + a2b2 + a3b1 + a4b0 0c5 = a0b5 + a1b4 + a2b3 + a3b2 + a4b1 + a5b0 − 2.1c6 = a0b6 + a1b5 + a2b4 + a3b3 + a4b2 + a5b1 + a6b0 − 3.47c7 = a0b7 + a1b6 + a2b5 + a3b4 + a4b3 + a5b2 + a6b1 + a7b0 − 2.8c8 = a1b7 + a2b6 + a3b5 + a4b4 + a5b3 + a6b2 + a7b1 0c9 = a2b7 + a3b6 + a4b5 + a5b4 + a6b3 + a7b2 + 2.1c10 = a3b7 + a4b6 + a5b5 + a6b4 + a7b3 + 2.47c11 = a4b7 + a5b6 + a6b5 + a7b4 + 1.4c12 = a5b7 + a6b6 + a7b5 0c13 = a6b7 + a7b6 − 0.7c14 = a7b7 − 0.49 69
  70. 70. PERIODIC CONVOLUTION of sin(ω0t) and cos(ω0t)Now let us computer the PERIODIC CONVOLUTION d[n] = a[n] ⊛ b[n] from the LINEARCONVOLUTION c[n]d0 = c0 + c8 = 0 + 0 =0d1 = c1 + c9 = 0.7 + 2.1 = 2.8d2 = c2 + c10 = 1.49 + 2.47 = 3.96d3 = c3 + c11 = 1.4 + 1.4 = 2.8d4 = c4 + c12 = 0 + 0 =0d5 = c5 + c13 = − 2.1 − 0.7 = − 2.8d6 = c6 + c14 = − 3.47 − 0.49 = − 3.96d7 = c7 = − 2.8This is the CIRCULAR CONVOLUTION of sin [ωo(kTs)] = discretised sin (ωot) and cos [ωo(kTs)] = discretised cos (ωot) with Ts = .Later, in LTI Systems we shall see the CIRCULAR CONVOLUTION of continuous sin (ωot)and cos (ωot) . 70
  71. 71. We could have computed the circular convolution a[n] ⊛ b[n] directly as shown below:#include<stdio.h>#include<conio.h>#include<math.h>void main(){float a[8]={0.0,0.7,1.0,0.7,0.0,-0.7,-1.0,-0.7};float b[8]={1.0,0.7,0.0,-0.7,-1.0,-0.7,0.0,0.7};int i,j,k;float c[8],b0;clrscr();for(i=0;i<=7;i++){ c[i]=a[0]*b[0]; for(j=1;j<=7;j++) { k=8-j; c[i]=c[i]+a[j]*b[k]; } printf("i=%d,c[i]=%6.4fn",i,c[i]); //rotate circular convolved B counter clockwise = left circular shift of linear B notconvolved. b0=b[0]; for(k=0;k<=6;k++) { b[k]=b[k+1]; } b[7]=b0;}getch();} 71
  72. 72. And the following is the output obtained. If we plot it we get a beautiful sine wave withsame period T0, only the amplitude and phase will be different.i = 0, c[i] = 0.0000i = 1, c[i] = 2.8000i = 2, c[i] = 3.9600i = 3, c[i] = 2.8000i = 4, c[i] = 0.0000i = 5, c[i] = −2.8000i = 6, c[i] = −3.9600i = 7, c[i] = −2.8000Which do you think is easier to compute, the linear convolution or the circularconvolution?Linear convolution is also referred to as acyclic convolution.Further reading:Algebraic Methods for Signal Processing and Communications Codingby Richard E. Blahut, Springer-Verlag, New York, 1991. Chapter 1 and Chapter 5. 72
  73. 73. CIRCULAR CONVOLUTION OF TWO GUITARS EXAMPLELet us take our two guitars example that we started with PADDED WITH ZEROS from 5 to 8. g = s0 s1 s2 s3 s4 s5 s6 s7 s8 G = S0 S1 S2 S3 S4 S5 S6 S7 S8With index k and padded with zeroes.Linear convolution g * G : s0S0 , 0 , s1S1 , 0, s2S2 , 0, s3S3, 0, s4S4Convolution index i : 0, 1, 2 , 3 ,4 , 5, 6 , 7, 8linear convolution index i: 0 1 2 3 4 values : s0S0 0 s1S1 0 s2S2 values : 0 s3S3 0 s4S4linear convolution index i : 5 6 7 8Cirular convolution g ⊛ G values : s0S0 s3S3 s1S1 s4S4 s2S2circular convolution index ic : 0c 1c 2c 3c 4cWe can see that it is possible to extract the linear convolution result from the circularconvolution. It is not always so easy or clear.Further reading: Orthogonality and the Graham-Smith method in Linear Algebra byGilbert Strang. 73

×