A localized nonlinear_method_for_the_contrast_enhancement_of_images


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

A localized nonlinear_method_for_the_contrast_enhancement_of_images

  1. 1. IMAGE LOCAL CONTRAST ENHANCEMENT USING ADAPTIVE NON-LINEAR FILTERS Tarik Arici, Yucel Altunbasak School of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, GA 30332 E-mail: tariq, yucel@ece.gatech.edu ABSTRACT around the edges. The second method uses an amplification factor that is inversely proportional to the local standard devi-We present a locallY adaptivE Non-lInear (YENI) filter to ob- ation (LSD) [3] so that the amplification around the edges istain the unsharp mask of an image. The unsharp mask obtained lowered. But this increases noise visibility in smooth regionsby the YENI filter preserves the edges in the image while fil- since LSD in smooth regions is relatively low.tering out the local details, which correspond to mid-range fre- We propose a locallY adaptivE Non-lInear (YENI) filter toquencies in the spectrum. The enhanced image using this un- find the unsharp mask of an image. YENI filter adapts to lo-sharp mask effectively prevents over/under (o/u) shooting arti- cal edge information and keeps the edges in the original imagefacts often observed with other unsharp masking techniques. while filtering out mid-range details. Therefore, the detail im-The enhanced frequency range also spans lower frequencies age obtained using the unsharp mask does not have large en-compared to the techniques that are based on Laplacian fil- ergy around the edges, and o/u shooting artifact is successfullyter variants. This improves the visual quality of the image, as avoided when the enhanced detail image is added back to themeasured subjectively and objectively in the real-video exper- original. To deal with the phase delay coming from the natureiments. Furthermore, since the YENI filter reduces to an IIR of the proposed filters, we employ two filters in opposite direc-filter at each pixel location, it has a low computational com- tions. The resultant phase shift coming from the two oppositeplexity. direction filters cancel out since we average the two filtered sig- Index Terms— Local contrast enhancement, IIR, recursive nals. Also, the proposed filter has low computational complex-filters ity and needs small memory resources, which makes it practical for use in television sets. 1. INTRODUCTION 2. IMAGE LOCAL CONTRAST ENHANCEMENTContrast enhancement techniques are widely used to increasethe visual image quality. Global contrast enhancement (GCE) In conventional ACE algorithms the enhanced image y(m, n)techniques remedy problems that manifest themselves in a global is obtained from the input image x(m, n) asfashion such as excessive/poor lightning conditions in the source y(m, n) = µ(m, n) + [1 + g(m, n)][x(m, n) − µ(m, n)], (1)environment. On the other hand, local contrast enhancementtries to enhance the visibility of local details in the image. Lo- where µ(m, n) is the local mean, g(m, n) is the enhancementcally enhanced images look more attractive than the originals gain, m is the row number, and n is the column number.because of the higher contrast [1]. With properly enhanced de- We employ our YENI filter to find the local mean by av-tails in the image, the HVS needs less amount of concentration eraging two opposite direction non-linear filters and we use ato discern the intensity differences. rational gain function that is designed to suppress noise visibil- Two well-known local contrast enhancement methods are ity in smooth regions.adaptive histogram equalization (AHE) [2] and adaptive con-trast enhancement (ACE) [3] [4]. AHE algorithms find local 2.1. Unsharp Masking Using YENI filtermappings using local histograms. Although the AHE improvescontrast, its computational burden is not acceptable for most The local mean µ(m, n) at row m and column n is the output ofapplications. A bilinear interpolation technique is presented YENI filter, which is the average of two different filter outputsin [2] for a block-based AHE. The major problem with AHE given bymethods is that it often over-enhances the image by creatingso called contrast objects that were not visible in the original µF (m, n) + µB (m, n) µ(m, n) = . (2)image. The enhanced image often does not look natural and is 2disturbing[5]. where µF (m, n) and µB (m, n) are the outputs of the two oppo- On the other hand, ACE algorithms utilize unsharp mask- site direction filters that run horizontally on a single row. Theing (UM) techniques. Unsharp mask (low-pass component) of first filter runs from left to right and is referred to as the for-the image is created by low-pass filtering and the detail mask ward filter. The forward filter outputs µF (m, n). The second(high-frequency component) is obtained by subtracting this un- filter runs from right to left and is similarly referred to as thesharp mask from the original image. The enhanced image is backward filter. The backward filter outputs µB (m, n). Theproduced by amplifying the detail mask and adding it back to two filters are single pole infinite impulse response (IIR) filtersthe unsharp mask. However, amplifying high-frequency com- at any given pixel location. The input-output relationship forponents creates o/u shooting artifacts around the edges. There the forward filtered µF (m, n) isare two conventional methods for selecting the amplificationfactor. The first one is to use a constant amplification fac- µF (m, n) = λ(m, n)µF (m, n − 1) + [1 − λ(m, n)]x(m, n),tor [4]. This method performs poorly in terms of o/u shooting (3)
  2. 2. where λ(m, n) is the edge adaptive delay coefficient. The rela- which has a zero phase. Thus, the 1-D local mean filter appliedtionship for the backward filtered µB (m, n) is similar1 . row-wise and given by (2) does not shift the 1-D input signal The adaptation of λ(m, n) to the edge information is cru- column-wise.cial for preventing the smoothing of edges. Considering that Next we discuss the ACE high-pass filter that is derivedλ(m, n) is the weight of the previous output, stronger λ(m, n) from the YENI filter.increases the low-pass characteristic of the filter. Hence, whenan edge is encountered, λ(m, n) must be decreased so that the 2.1.2. Frequency response of the ACE high-pass filteredge will be preserved in the output. The edge signal we use is|µF (m, n−1)−x(m, n)| for the forward filter, and |µB (m, n+ Let us rewrite the ACE relation in (1) as the original image plus1) − x(m, n)| for the backward filter. Both of the edge sig- an extra enhancement signal.nals are the differences between the original pixel value and y(m, n) = x(m, n) + g(m, n)[x(m, n) − µ(m, n)], (10)the previous filter output. Using these edge signals, λ(m, n) isobtained using The second term in (10) is the high-pass filtered original image. |µF (m, n − 1) − x(m, n)| α This can be seen considering that it is the difference between λ(m, n) = [1 − ] , (5) the original image and the low-pass filtered image. Then the 255 frequency response of this ACE high-pass filter generating thefor the forward filter, and similar for the backward filter2 . Here second term is found using (9)we use 255 for the maximum possible pixel value. As can be 1 − cos(w)observed from (5) and (6), strong edges reduce λ(m, n) more, 1 − H(w) = (λ2 + λ) , (11) 1 − 2λ cos(w) + λ2hence the low-pass characteristic of the signal at that locality islessened. Typical α values are in the range of [5-9]. Figure 1 shows the plot of (11). It is interesting to note 1 that the gain of the ACE high-pass filter is approximately con- 0.9 stant after some frequency threshold. Hence, not only high- 0.8 frequencies close to π are being enhanced, but also mid-range 0.7 frequencies in the spectrum are also enhanced with the same 0.6 magnitude. This increases the visual quality of the enhanced Increasing λ image because details that have their energy in the mid-range 1−H(ω) 0.5 0.4 frequency spectrum are also enhanced as well as high-frequency 0.3 ACE high−pass filters Laplacian filter details. Also, magnitude of the frequency response decreases 0.2 with decreasing λ. Since λ is a function of the edge signal, 0.1 this feature demonstrates that the high-pass filter adapts itself 0 to edges by reducing its magnitude. From (10), it is clear that 0 0.5 1 1.5 ω 2 2.5 3 enhancement around the locality of edges decreases because the magnitude of the enhancement signal decreases accordingly.Fig. 1. Adaptive high-pass filters used in proposed ACE method This adaptive behavior enables our proposed ACE method to successfully avoid o/u shooting artifacts.2.1.1. Frequency response of the YENI filter The representation of the ACE method given in (10) is ex-For ease of notation we denote the original pixel at the nth col- actly the same as the Laplacian unsharp masking (UM) repre-umn of row m as xm (n). Then, each row of the original image sentation in which the Laplacian filtered image is added to theis a 1-D signal. From (3) and (4) frequency response of the for- original image. A Laplacian filter has three taps {−1, 2, −1}.ward and backward filters at a locality with λ(m, n) = λ are Its frequency response is also given in Figure 1 and shows thatderived as below the Laplacian filter assigns an emphasis to the high frequency components. The gain at high frequencies is about 2 times 1−λ HF (w) = (7) larger than the mid-range frequencies. However, this creates 1 − λe−jw two sorts of problem: noise sensitivity and o/u shooting around 1−λ edges. To improve its noise sensitivity and reduce its o/u shoot- HR (w) = , (8) 1 − λe+jw ing artifacts, various modifications have been introduced [6]respectively. Here, we implicitly assume that λs for the two fil- [7] [8]. All of these modifications aim to suppress the Lapla-ters are equal since ideally edge information at the same locality cian filter’s gain at high frequencies. But this inevitably de-must be the same. From (7) and (8) we can see that the phase creases the enhancement gain at mid-range frequencies and re-of both filters is not zero causing a phase shift in the filtered duces the level of visual quality enhancement.output. In fact the forward filter lags, and the backward filter Figure 3 shows a 1-D enhancement example for 5 differentadvances the input signal xm (n). However, frequency response unsharp masking methods (linear UM [1], Cubic UM[6], OSof the local mean filter using (2), (7), and (8) can be obtained Laplacian [8], Rational UM [7]) including our proposed ACEas below as well as the original 1-D signal. The original signal has 3 edges with medium level detail existing between the edges (i.e. 1 − λ cos(w) H(w) = (1 − λ) , (9) sinusoidal oscillations) to model different textures in the image. 1 − 2λ cos(w) + λ2 The enhancement gain set for each tested algorithm is adjusted 1 so that increasing the gain does not bring any more enhance- µB (m, n) = λ(m, n)µB (m, n + 1) + [1 − λ(m, n)]x(m, n), (4) ment but only worsened the artifacts. Linear unsharp masking enhances both the edges and the medium level detail. But the 2 second edge does not look natural, in fact in an image it will |µB (m, n + 1) − x(m, n)| α λ(m, n) = [1 − ] , (6) look stair-like. Cubic UM does not produce stair-like edges but 255 it exhibits heavy o/ushooting artifacts. This is mainly because
  3. 3. 1 0.9 K Table 1. Results for Tempete 0.8 VR MOS NAR Enhancement Gain 0.7 Linear Lap. 1.72 5 4.50 0.6 Cubic UM 1.02 4.5 1.45 0.5 Rational UM 1.26 6 2.48 0.4 OS Lap. 1.09 5 1.74 0.3 Proposed 1.34 7.5 2.42 0.2 0.1 a b c 0 0 5 10 15 20 25 30 Output magnitude of ACE−high pass filter between 0 and 255. Hence, for λ and for the enhancement gain two look-up tables (LUTs) can be used. The total num- Fig. 2. Enhancement gain function ber of computations per pixel including the computation of theof its cubic dependence on the pixel value differences. Ratio- indices to these 2 LUTs is 2 multiplications, 6 additions andnal UM does a better job of avoiding o/u shooting artifacts but one bit shift. The memory needed for the LUTs is 256 bytes fordoes not enhance the medium level details and also creates a λ LUT, c bytes (typically 21) for enhancement gain LUT, onestair-like edge. OS Laplacian behaves similar to Cubic UM. In line-store for the forward filter’s output and an additional singleaddition it shows a common artifact of order statistics, that is register for backward filter’s output.producing patch like intensity regions that can be seen as smallconstant intermediate levels on the second edge. Our proposedACE method successfully avoids o/u shooting artifacts and alsoenhances the medium level details in the signal while still pro- 3. EXPERIMENTAL RESULTSducing natural looking edges. The reason for Laplacian modi-fied UM algorithms to create unnatural looking edge transitions We have tested our proposed method on a variety of video se-is that Laplacian filter puts an emphasis on the high-frequency quences and still images. We also compared it with above men-range of the spectrum, which includes aliased frequency com- tioned methods. To do a fair comparison, we modified all meth-ponents. As a result, enhancing the aliased components us- ods so that they only enhance row-wise.ing the modified Laplacian UM techniques produces unnatural In Figure 4 results using zoomed first frame of the tempetelooking and disturbing edge transitions. sequence is given. All of the modified Laplacian techniques show o/u shooting effects, especially on the up-right edge of2.2. Enhancement Gain Function the rock. Among the modified Laplacian techniques, linear UM and OS Laplacian has the worst performance in o/u shooting.There are two important design goals for the enhancement gain Rational UM is better in terms of o/u shooting but it producesfunction: avoiding noise visibility especially in smooth regions stair-like transitions and broken edges. On the other hand ourand preventing intensity saturation to minimum and maximum proposed method does not produce any o/u shooting and ex-possible intensity values (e.g. 0 and 255 for 1 byte per channel hibits natural looking transitions and edges.source format). To deal with these problems, the enhancementgain depends on the output magnitude (OM) of the ACE high- Performances of these methods on test video sequences arepass filter, that is the magnitude of the detail in that locality. measured using 3 different criteria: non-edge local variance ra- Since the low-pass component of the smooth regions is low, tio (VR), mean opinion score (MOS) and noise amplificationthe OM of the ACE high-pass filter will also be low. Hence, ratio (NAR). We have used non-edge local variance to computethe enhancement gain should be small when the filter’s OM is the contrast enhancement amount. Since modified Laplaciansmall to avoid noise visibility in smooth regions. As the details methods suffer from o/u shooting, we did not include edge pix-increase the gain should also be increasing. However, increas- els in variance computation not to inflate the local variancesing the gain function continuously may lead to saturation. This with artifacts. In any case, we believe that the effect of excludedcan be prevented by reducing the enhancement gain after some edge variances will be reflected in MOSs. Therefore non-edgepoint. We would like to note that since our YENI filter adapts local variance is found over a 1x16 window using only non-itself with edge information, saturation is not likely to be ob- edge pixels. For each enhanced image we report the ratio ofserved in practice. Considering these specifications, we have the non-edge local variance’s sum to that of the original image.designed a gain function that is an upward shifted cosine eval- We have used Canny [9] edge detector for identifying edge pix-uated in the 3rd quadrant when OM is in [a-b], and a cosine els. The noise sensitivity is tested by adding Gaussian noise toevaluated in the 1st quadrant when OM is in [b-c], where a,b,c the input image and computing the variance of the difference inare OM thresholds as shown in the example gain function given the enhanced image. The ratio of the difference image’s vari-in Figure 2. Here, a,b,c, and K are chosen as 1,7,21,1 respec- ance to the variance of the Gaussian noise added to the inputtively where K is the maximum achievable gain that determines image is the NAR. We have tried 3 different Gaussian signalsthe strength of the enhancement signal. with variances 10,20, and 40 and averaged the 3 corresponding NARs.2.3. Computational Complexity and Memory Requirement Results for CIF size ”Tempete” sequence is given in Ta- ble 1. Linear Laplacian has the largest VR, however, it per-Computational complexity of our proposed algorithm is extremely forms very poor in noise amplification. This is expected sincelow since at each given pixel we employ 2 one pole IIR filters linear UM boosts high-frequencies regardless. Our proposedgiven in (3) (4). The delay coefficient (λ) of the IIR filter does LCE method has the next largest VR and has approximatelynot have to be computed since λ is determined by (5) and the half NAR. Cubic UM and OS Laplacian have small NARs be-edge signal that is input to this function is always an integer cause their VRs are also correspondingly low.
  4. 4. 250 250 250 200 200 200 Pixel value Pixel value 150 150 150 Pixel value 100 100 100 50 50 50 0 0 0 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 Pixel index Pixel index Pixel index (a) Original (b) Linear UM (c) Cubic UM 250 250 250 200 200 200 Pixel value 150 150 150 Pixel value Pixel value 100 100 100 50 50 50 0 0 0 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 Pixel index Pixel index Pixel index (d) Rational UM (e) OS Laplacian (f) Proposed ACE Fig. 3. 1-D example (a) Original (b) Linear UM (c) Cubic UM (d) Rational UM (e) OS Laplacian (f) Proposed ACE Fig. 4. Zoomed first frame of the ”tempete” sequence enhanced with 5 different UM methods 4. REFERENCES [8] Y. H. Lee and S. Y. Park, “A study of convex/concave edges and edge-enhancing operators based on the laplacian,” IEEE Trans.[1] Mark J. T. Smith and Alen Docef, A Study Guide for Digital on Circuits and Systems, vol. 37, no. 7, pp. 940–946, 1990. Image Processing, Scientific Publishers, Georgia, 1999. [9] John Canny, “Computational approach to edge detection,” IEEE[2] S. M. Pizer, J. B. Zimmerman, and E. V. Staab, “Adaptive grey Transactions on Pattern Analysis and Machine Intelligence, vol. level assignment in ct scan display,” J. Comp. Assist. Tomogr., PAMI-8, no. 6, pp. 679–698, 1986. vol. 8, no. 2, pp. 300–305, 1984.[3] P. M. Narendra nd R. C. Fitch, “Rel-time adaptive contrast en- hancement,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI, no. 3, pp. 655–661, 1981.[4] J. S. Lee, “Digital image enhancement and noise filtering by using local statistics,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI, no. 2, pp. 165–168, 1980.[5] J. A. Stark, “Adaptive image contrast enhancement using general- izations of histogram equalization,” IEEE Transactions on Image Processing, vol. 9, no. 5, pp. 889–896, 2000.[6] G. Ramponi, “A cubic unsharp masking technique for contrast enhancement,” Signal Processing, vol. 67, no. 2, pp. 211–222, June 1998.[7] G. Ramponi and A. Polesel, “A rational unsharp masking tech- nique,” 1998.