1.
Introduction to Signal Processing1 IntroductionIn this article, we discuss about signals, systems and transforms. Signals canbe considered as functions. Audio signal is a one dimensional function of timewhereas an image is a two dimensional function that varies in space. Systemsmanipulate signals, i.e they can be considered as a map from functions to func-tions. Transforms make signal processing easier by changing the domain inwhich the underlying signal is represented.2 SignalsSignals can be functions of time or space. For processing signals it is advanta-geous to represent them in in frequency domain. Several operations such as ﬁl-tering out unnecessary frequencies can be performed eﬃciently in the frequencydomain. Generally low frequency components represents the overall shape ofthe signal and high frequency components are caused due to noise or presenceof edges in an image. To remove noise, high frequency components are reduced.To sharpen an image, high frequency components are emphasized. Thus it iseasier to carry many operations in the frequency domain.2.1 From time domain to frequency domainFourier discovered that any periodic signal can be represented as combinationof sinusoids of diﬀerent amplitudes, frequencies and phases. Any periodic sig- 1nal s(t) with period T0 (thus having fundamental frequency f0 = T0 can berepresented as follows ∞ s(t) = an sin(2πnf0 t + φ) (1) n=0 1
2.
A phase shifted sine wave can be expressed in terms of sine and cosinefunctions due to the following identity. sin(A + φ) = sin(A)cos(φ) + sin(φ)cos(A) Thus we can express s(t) as sum of sines and cosines of diﬀerent amplitudesand frequencies. Equation 1 becomes ∞ s(t) = an cos(2πnf0 t) + bn sin(2πnf0 t) (2) n=0 which is also written as ∞ s(t) = a0 + an cos(2πnf t) + bn sin(2πnf t) n=1 If the function has non zero value for f (0), it indicates the presence of cosinecomponents. a0 represents the average value of the function over a time period.This is because the average value of sines and cosines over a time period is zero. it −it it −it Using the identities cos(t) = e +e 2 and sin(t) = e −e2i and letting An an −ibn ∗ an +ibndenote 2 and An denote 2 , Equation 2 can be represented by meansof a single complex exponential. ∞ s(t) = An ej2πnf0 t + A∗ ej2πnf0 t n n=0 which can be written as ∞ s(t) = An ej2πnf0 t n=−∞ The above formula is known as fourier decomposition of s(t) into complexexponentials. This decomposition is a way of transforming the signal from thetime domain to frequency domain. In the next section we will see how a signaland its decomposition into fourier series can be viewed in the vector space model.2.2 Vector Space ModelThe set of signals forms a vector space. A vector space is a collection of vectorsthat is closed under linear combination, i.e. if two vectors v, w are in the space,then the vector av + bw is also in the space for scalars a and b(The conditionsays the space is closed under vector addition and scalar multiplication). A 2
3.
familiar example of vector space is the Euclidean space R3 . Any set of objectswith operations addition and multiplication which are closed under linear com-bination forms a vector space. Under this deﬁnition polynomials, matrices andfunctions also form vector space. We can generalize familiar concepts in the Euclidean space to the moregeneral vector space. The length or norm of vector v is given by n v = 2 vi i=0 The distance between two vectors v1 and v2 is the norm of the diﬀerencevector v1 − v2 . The angle between two vectors is closely related to the innerproduct < v1, v2 > due to the following relation. < v1, v2 > cosθ = v1 ∗ v2 Two vectors are said to be orthogonal if their inner product is zero. The concepts of length, distance and angle can be deﬁned on any vectorspace. For instance, in the space of functions, the integral f (x).g(x) gen-eralizes the dot product in which summation is replaced by integration. The bfunctions f (x) and g(x) are said to be orthogonal on [a, b] if a f (x).g(x) = 0.For example the set of functions { ejn2πf t } n=0,1,2... are orthogonal. A basis is a minimal set of vectors that spans the entire space. If the elementsin basis are orthogonal and have unit length , then they form orthonormal basisas formed by (i, j, k) in the euclidean space. A point in the euclidean space givenby u = xi + yj + zk. The values x,y and z gives the magnitude of projection onthese basis. To compute the magnitude of projection along i, we compute theinner product of u and i, thus x = u.i. A function s(t) can be viewed as a point in the space of functions. Thefourier decomposition of the function given by ∞ s(t) = An ejn2πf0 t n=−∞can be viewed as projection of the function onto orthonormal basis given by thecomplex exponential functions ejn2πf0 t . Finding the magnitude of projection is 3
4.
equivalent to ﬁnding the fourier coeﬃcients. To compute the the magnitude ofprojection along ejn2πf0 t ,we compute the inner product of s(t) and ejn2πf0 t .Using the deﬁnition of inner product for functions, we get the fourier coeﬃcientsas T0 /2 1 An = s(t)e−jn2πf0 t dt T0 −T0 /23 SystemsSystems maps signals to signals. To characterize a system we need to ﬁnd itsresponse for each and every input function, which is infeasible. Fortunately sys-tems that are linear and time invariant (LTI) can be characterized if its responsefor a small set of inputs is known. In a linear system, the principle of superpo-sition holds - if x1 and x2 produces responses y1 and y2 respectively, then inputax1 + bx2 produces the response ay1 + by2 . A system is said to be time invari-ant if a delay in input by t seconds causes the output to be delayed by t seconds. In the following sections, we will see that to characterize an LTI system it issuﬃcient if we know its response for the impulse function and that the outputof an LTI system is just the convolution of the input with the impulse response.3.1 Impulse ResponseConsider a signal that lasts for w seconds and has amplitude 1/w. In the limit-ing case as w → 0, the signal is called impulse. LTI systems have a remarkableproperty that they can be completely characterized by its response to the im-pulse function. @ﬁgure The input signal is divided into smaller ones, each short enough to act as animpulse to the system. In other words, the input signal is decomposed into aninﬁnite number of scaled and shifted impulse functions. Each of these impulsesproduces a scaled and shifted version of the impulse response in the outputsignal. The ﬁnal output signal is then equal to the combined eﬀect, i.e., thesum of all of the individual responses. Thus if the response to the impulsefunction is known, the response of the input signal can be calculated. We showthat the output is the convolution of the input with the impulse response of thesystem. 4
5.
3.2 ConvolutionConvolution is a mechanism by which an input signal is shaped by another sig-nal to produce an output signal. We introduce convolution by considering theproblem of smoothing a signal. Later we show that the output of a system isthe convolution of the input with the impulse response. Consider a transformation that smoothes the signal by averaging adjacentsignals - bn = (an + an−1 )/2. These kinds of transformations are used for re-ducing eﬀects of noise. The transformation can also be described in terms ofpolynomial multiplication. If we represent the input sequence a0 , a1 , a2 .... bythe polynomial A(x) = a0 + a1 x + a2 x2 + a3 x3 .... , the coeﬃcient of bn in the 1product B(x) = A(x) 2 (1 + x) is given by bn = (an + an−1 )/2 which is the 1output sequence. The polynomial P (x) = 2 (1 + x) deﬁnes how the input signalis manipulated(convolved) to produce the output. Convolution and polynomialmultiplication are closely related. For the product B(x) = A(x) ∗ P (x), the coeﬃcient of the convolved sig- nnal bn is given by i=0 ai ∗ pn−i . In the continuous domain the summationis replaced by integration and the convolution of two function f (x) and g(x) ubecomes C(u) = 0 f (x) ∗ g(u − x) We now show that the output of a system can be viewed as a convolutionof the input signal with the impulse response of the system. Consider a systemwhose output is h(t) when given the input δ(t). Any input signal can be consid-ered as summation of scaled and shifted impulse functions i.e. For a continuous ∞signal, s(t) can be viewed as −∞ f (τ )δ(t − τ )dτ . Since the response of δ(t) is ∞h(t), the response for s(t) is −∞ f (τ )h(t − τ )dτ . Thus the convolution of theinput signal with the impulse response gives the output signal.4 Transforms and their applicationsTransforms make it easier for systems to manipulate signals. Any linear trans-formation can be described as a projection onto a new set of basis functions. Fourier Transform 5
6.
We have seen the fourier series for a periodic signal having frequency f0 is ∞given by s(t) = n=−∞ An ejn2πf0 t and that An is given by T0 1 2 An = s(t)e−jn2πf0 t dt T0 −T0 2 The following ﬁgure plots the frequency components and the magnitude ofAn . Note that here the amplitudes are function of frequencies deﬁned at discretepoints. @ﬁgure As the frequency f0 decreases, these amplitudes become more closely spaced.For a non periodic signal , the time period T0 → ∞ and thus f0 → 0. The sepa-ration between successive harmonics becomes small and the amplitudes becomea continuous function of frequency. Thus nf0 becomes the continuous variablef and the above formula for An becomes ∞ 1 Af = s(t)e−j2πf t dt (3) T0 −∞ The above transformation takes a function of time as input and producesa function of frequency i.e. s(t) → Af . This is called the Fourier Transform.The transform makes it easier to perform many operations that are diﬃcult inthe time domain. The fourier transform of the convolution of two functionss1 (t) and s2 (t) is equal to product of their fourier transforms. Intuitively, mul-tiplication emphasizes those frequencies that are present in both the signals.Thus for producing the convolution of two functions, it is easier to transformthem into frequency domain, multiply them and transform back to time domain. Discrete Fourier Transform While processing signals on a computer, variables are discrete and the con-tinuous fourier transform is approximated by Discrete Fourier Transform. Wesample a continuous function s(t) at intervals of ∆, generating a sequence ofsampled values sk = s(k∆) collecting a total of N samples (1/∆ is the samplingfrequency, which according to Nyquist theorem must be at least twice the fre-quency of the highest component present in the wave). In the continuous fourier transform, we have the values of the signal deﬁnedat each point of time and generate a function of frequency that is deﬁned forall values. In DFT, we have the values of the signal at discrete points tk = k∆ 6
7.
represented as s(k) and generate a function of frequency that are valid at discretefrequencies An . In the discrete case Equation 3 becomes N −1 An = s(k)e−2πikn/N (4) k=0 The equation xN = 1 is of degree N and thus has N roots given by ωN , n 2πi(n=0,1... N and ω = e N . Thus Equation 4 can also be written as N −1 kn An = s(k)ωN (5) k=0 The DFT can be viewed as a transformation of N point signal into N fouriercoeﬃcients or as a matrix that maps an N-dimensional vector to N-dimensionalvector. DFT can be used to speed up polynomial multiplication. We have alreadyseen that polynomial multiplication is related to convolution and it is easy toconvolve signals in frequency domain. We use these facts to speedup multipli-cation. Considering the coeﬃcients of a polynomial (of degree N) as our signal, weget N fourier coeﬃcients. (According to Equation 5, these correspond to thevalues of the polynomial evaluated at the complex roots of unity). These valuesgive the point-value representation of the polynomial. To multiply two polyno-mials f (x) and g(x), we evaluate the polynomials at roots of unity, multiply thecorresponding values to get the point-value representation of f (x) ∗ g(x) fromwhich we can construct the polynomial f (x) ∗ g(x). Fast Fourier Transform The fast fourier transform is a computationallyfast method to compute the DFT. Computing DFT directly for N points requiresO(N 2 ) complex multiplication. FFT exploits the structure of DFT to reducethe complexity. Rewriting Equation 5 we get N −1 kn An = s(k)ωN k=0 kn kn An = s(k)ωN + s(k)ωN k is even k is odd (N/2)−1 (N/2)−1 2jn (2j+1)n An = s(2j)ωN + s(2j + 1)ωN j=0 j=0 7
8.
2 Using the relation ωN = ωN/2 , we get (N/2)−1 (N/2)−1 jn n jn An = s(2j)ωN/2 + ωN s(2j + 1)ωN/2 j=0 j=0 Thus we have reduced the problem of computing N point DFT for s(k) toﬁnding two N/2 point DFTs for s(j) and s(2j + 1). This is a divide and conquerapproach where we have reduced the problem of size N to two subproblems ofsize N/2. It has complexity O(N logN ) Discrete Cosine Transform If the function is an even function, then thefourier transform consists of only cosine components and the transform becomesDiscrete Cosine Transform. Thus DCT can be deﬁned in terms of DFT by im-posing symmetry. DCT is used for compression in JPEG images. JPEG com-pression takes into account that the human eye, cannot detect high frequencycomponents. Using a process of quantization, we retain all low frequency com-ponents. (The high frequency components are retained only if they have largecoeﬃcients). On the resulting coeﬃcients run length encoding and huﬀmancompression is applied5 ConclusionWe have seen how a signal can be viewed as a function in the vector spacemodel of functions. Systems are used to manipulate signals and a special classof systems called LTI systems can be easily characterized by its impulse function.Transforms make it easier to manipulate signals. Transforms can be viewed aschange of basis in the vector space model. Transformations have a number ofapplications including ﬁltering, convolution, pattern recognition, solving partialdiﬀerential equations, multiplication of polynomials and large numbers, analysisof time series, solving system of linear equations 8
Be the first to comment