Transcript of "A localized nonlinear_method_for_the_contrast_enhancement_of_images"
1.
IMAGE LOCAL CONTRAST ENHANCEMENT USING ADAPTIVE NON-LINEAR FILTERS Tarik Arici, Yucel Altunbasak School of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, GA 30332 E-mail: tariq, yucel@ece.gatech.edu ABSTRACT around the edges. The second method uses an ampliﬁcation factor that is inversely proportional to the local standard devi-We present a locallY adaptivE Non-lInear (YENI) ﬁlter to ob- ation (LSD) [3] so that the ampliﬁcation around the edges istain the unsharp mask of an image. The unsharp mask obtained lowered. But this increases noise visibility in smooth regionsby the YENI ﬁlter preserves the edges in the image while ﬁl- since LSD in smooth regions is relatively low.tering out the local details, which correspond to mid-range fre- We propose a locallY adaptivE Non-lInear (YENI) ﬁlter toquencies in the spectrum. The enhanced image using this un- ﬁnd the unsharp mask of an image. YENI ﬁlter adapts to lo-sharp mask effectively prevents over/under (o/u) shooting arti- cal edge information and keeps the edges in the original imagefacts often observed with other unsharp masking techniques. while ﬁltering out mid-range details. Therefore, the detail im-The enhanced frequency range also spans lower frequencies age obtained using the unsharp mask does not have large en-compared to the techniques that are based on Laplacian ﬁl- ergy around the edges, and o/u shooting artifact is successfullyter variants. This improves the visual quality of the image, as avoided when the enhanced detail image is added back to themeasured subjectively and objectively in the real-video exper- original. To deal with the phase delay coming from the natureiments. Furthermore, since the YENI ﬁlter reduces to an IIR of the proposed ﬁlters, we employ two ﬁlters in opposite direc-ﬁlter at each pixel location, it has a low computational com- tions. The resultant phase shift coming from the two oppositeplexity. direction ﬁlters cancel out since we average the two ﬁltered sig- Index Terms— Local contrast enhancement, IIR, recursive nals. Also, the proposed ﬁlter has low computational complex-ﬁlters ity and needs small memory resources, which makes it practical for use in television sets. 1. INTRODUCTION 2. IMAGE LOCAL CONTRAST ENHANCEMENTContrast enhancement techniques are widely used to increasethe visual image quality. Global contrast enhancement (GCE) In conventional ACE algorithms the enhanced image y(m, n)techniques remedy problems that manifest themselves in a global is obtained from the input image x(m, n) asfashion such as excessive/poor lightning conditions in the source y(m, n) = µ(m, n) + [1 + g(m, n)][x(m, n) − µ(m, n)], (1)environment. On the other hand, local contrast enhancementtries to enhance the visibility of local details in the image. Lo- where µ(m, n) is the local mean, g(m, n) is the enhancementcally enhanced images look more attractive than the originals gain, m is the row number, and n is the column number.because of the higher contrast [1]. With properly enhanced de- We employ our YENI ﬁlter to ﬁnd the local mean by av-tails in the image, the HVS needs less amount of concentration eraging two opposite direction non-linear ﬁlters and we use ato discern the intensity differences. rational gain function that is designed to suppress noise visibil- Two well-known local contrast enhancement methods are ity in smooth regions.adaptive histogram equalization (AHE) [2] and adaptive con-trast enhancement (ACE) [3] [4]. AHE algorithms ﬁnd local 2.1. Unsharp Masking Using YENI ﬁltermappings using local histograms. Although the AHE improvescontrast, its computational burden is not acceptable for most The local mean µ(m, n) at row m and column n is the output ofapplications. A bilinear interpolation technique is presented YENI ﬁlter, which is the average of two different ﬁlter outputsin [2] for a block-based AHE. The major problem with AHE given bymethods is that it often over-enhances the image by creatingso called contrast objects that were not visible in the original µF (m, n) + µB (m, n) µ(m, n) = . (2)image. The enhanced image often does not look natural and is 2disturbing[5]. where µF (m, n) and µB (m, n) are the outputs of the two oppo- On the other hand, ACE algorithms utilize unsharp mask- site direction ﬁlters that run horizontally on a single row. Theing (UM) techniques. Unsharp mask (low-pass component) of ﬁrst ﬁlter runs from left to right and is referred to as the for-the image is created by low-pass ﬁltering and the detail mask ward ﬁlter. The forward ﬁlter outputs µF (m, n). The second(high-frequency component) is obtained by subtracting this un- ﬁlter runs from right to left and is similarly referred to as thesharp mask from the original image. The enhanced image is backward ﬁlter. The backward ﬁlter outputs µB (m, n). Theproduced by amplifying the detail mask and adding it back to two ﬁlters are single pole inﬁnite impulse response (IIR) ﬁltersthe unsharp mask. However, amplifying high-frequency com- at any given pixel location. The input-output relationship forponents creates o/u shooting artifacts around the edges. There the forward ﬁltered µF (m, n) isare two conventional methods for selecting the ampliﬁcationfactor. The ﬁrst one is to use a constant ampliﬁcation fac- µF (m, n) = λ(m, n)µF (m, n − 1) + [1 − λ(m, n)]x(m, n),tor [4]. This method performs poorly in terms of o/u shooting (3)
2.
where λ(m, n) is the edge adaptive delay coefﬁcient. The rela- which has a zero phase. Thus, the 1-D local mean ﬁlter appliedtionship for the backward ﬁltered µB (m, n) is similar1 . row-wise and given by (2) does not shift the 1-D input signal The adaptation of λ(m, n) to the edge information is cru- column-wise.cial for preventing the smoothing of edges. Considering that Next we discuss the ACE high-pass ﬁlter that is derivedλ(m, n) is the weight of the previous output, stronger λ(m, n) from the YENI ﬁlter.increases the low-pass characteristic of the ﬁlter. Hence, whenan edge is encountered, λ(m, n) must be decreased so that the 2.1.2. Frequency response of the ACE high-pass ﬁlteredge will be preserved in the output. The edge signal we use is|µF (m, n−1)−x(m, n)| for the forward ﬁlter, and |µB (m, n+ Let us rewrite the ACE relation in (1) as the original image plus1) − x(m, n)| for the backward ﬁlter. Both of the edge sig- an extra enhancement signal.nals are the differences between the original pixel value and y(m, n) = x(m, n) + g(m, n)[x(m, n) − µ(m, n)], (10)the previous ﬁlter output. Using these edge signals, λ(m, n) isobtained using The second term in (10) is the high-pass ﬁltered original image. |µF (m, n − 1) − x(m, n)| α This can be seen considering that it is the difference between λ(m, n) = [1 − ] , (5) the original image and the low-pass ﬁltered image. Then the 255 frequency response of this ACE high-pass ﬁlter generating thefor the forward ﬁlter, and similar for the backward ﬁlter2 . Here second term is found using (9)we use 255 for the maximum possible pixel value. As can be 1 − cos(w)observed from (5) and (6), strong edges reduce λ(m, n) more, 1 − H(w) = (λ2 + λ) , (11) 1 − 2λ cos(w) + λ2hence the low-pass characteristic of the signal at that locality islessened. Typical α values are in the range of [5-9]. Figure 1 shows the plot of (11). It is interesting to note 1 that the gain of the ACE high-pass ﬁlter is approximately con- 0.9 stant after some frequency threshold. Hence, not only high- 0.8 frequencies close to π are being enhanced, but also mid-range 0.7 frequencies in the spectrum are also enhanced with the same 0.6 magnitude. This increases the visual quality of the enhanced Increasing λ image because details that have their energy in the mid-range 1−H(ω) 0.5 0.4 frequency spectrum are also enhanced as well as high-frequency 0.3 ACE high−pass filters Laplacian filter details. Also, magnitude of the frequency response decreases 0.2 with decreasing λ. Since λ is a function of the edge signal, 0.1 this feature demonstrates that the high-pass ﬁlter adapts itself 0 to edges by reducing its magnitude. From (10), it is clear that 0 0.5 1 1.5 ω 2 2.5 3 enhancement around the locality of edges decreases because the magnitude of the enhancement signal decreases accordingly.Fig. 1. Adaptive high-pass ﬁlters used in proposed ACE method This adaptive behavior enables our proposed ACE method to successfully avoid o/u shooting artifacts.2.1.1. Frequency response of the YENI ﬁlter The representation of the ACE method given in (10) is ex-For ease of notation we denote the original pixel at the nth col- actly the same as the Laplacian unsharp masking (UM) repre-umn of row m as xm (n). Then, each row of the original image sentation in which the Laplacian ﬁltered image is added to theis a 1-D signal. From (3) and (4) frequency response of the for- original image. A Laplacian ﬁlter has three taps {−1, 2, −1}.ward and backward ﬁlters at a locality with λ(m, n) = λ are Its frequency response is also given in Figure 1 and shows thatderived as below the Laplacian ﬁlter assigns an emphasis to the high frequency components. The gain at high frequencies is about 2 times 1−λ HF (w) = (7) larger than the mid-range frequencies. However, this creates 1 − λe−jw two sorts of problem: noise sensitivity and o/u shooting around 1−λ edges. To improve its noise sensitivity and reduce its o/u shoot- HR (w) = , (8) 1 − λe+jw ing artifacts, various modiﬁcations have been introduced [6]respectively. Here, we implicitly assume that λs for the two ﬁl- [7] [8]. All of these modiﬁcations aim to suppress the Lapla-ters are equal since ideally edge information at the same locality cian ﬁlter’s gain at high frequencies. But this inevitably de-must be the same. From (7) and (8) we can see that the phase creases the enhancement gain at mid-range frequencies and re-of both ﬁlters is not zero causing a phase shift in the ﬁltered duces the level of visual quality enhancement.output. In fact the forward ﬁlter lags, and the backward ﬁlter Figure 3 shows a 1-D enhancement example for 5 differentadvances the input signal xm (n). However, frequency response unsharp masking methods (linear UM [1], Cubic UM[6], OSof the local mean ﬁlter using (2), (7), and (8) can be obtained Laplacian [8], Rational UM [7]) including our proposed ACEas below as well as the original 1-D signal. The original signal has 3 edges with medium level detail existing between the edges (i.e. 1 − λ cos(w) H(w) = (1 − λ) , (9) sinusoidal oscillations) to model different textures in the image. 1 − 2λ cos(w) + λ2 The enhancement gain set for each tested algorithm is adjusted 1 so that increasing the gain does not bring any more enhance- µB (m, n) = λ(m, n)µB (m, n + 1) + [1 − λ(m, n)]x(m, n), (4) ment but only worsened the artifacts. Linear unsharp masking enhances both the edges and the medium level detail. But the 2 second edge does not look natural, in fact in an image it will |µB (m, n + 1) − x(m, n)| α λ(m, n) = [1 − ] , (6) look stair-like. Cubic UM does not produce stair-like edges but 255 it exhibits heavy o/ushooting artifacts. This is mainly because
3.
1 0.9 K Table 1. Results for Tempete 0.8 VR MOS NAR Enhancement Gain 0.7 Linear Lap. 1.72 5 4.50 0.6 Cubic UM 1.02 4.5 1.45 0.5 Rational UM 1.26 6 2.48 0.4 OS Lap. 1.09 5 1.74 0.3 Proposed 1.34 7.5 2.42 0.2 0.1 a b c 0 0 5 10 15 20 25 30 Output magnitude of ACE−high pass filter between 0 and 255. Hence, for λ and for the enhancement gain two look-up tables (LUTs) can be used. The total num- Fig. 2. Enhancement gain function ber of computations per pixel including the computation of theof its cubic dependence on the pixel value differences. Ratio- indices to these 2 LUTs is 2 multiplications, 6 additions andnal UM does a better job of avoiding o/u shooting artifacts but one bit shift. The memory needed for the LUTs is 256 bytes fordoes not enhance the medium level details and also creates a λ LUT, c bytes (typically 21) for enhancement gain LUT, onestair-like edge. OS Laplacian behaves similar to Cubic UM. In line-store for the forward ﬁlter’s output and an additional singleaddition it shows a common artifact of order statistics, that is register for backward ﬁlter’s output.producing patch like intensity regions that can be seen as smallconstant intermediate levels on the second edge. Our proposedACE method successfully avoids o/u shooting artifacts and alsoenhances the medium level details in the signal while still pro- 3. EXPERIMENTAL RESULTSducing natural looking edges. The reason for Laplacian modi-ﬁed UM algorithms to create unnatural looking edge transitions We have tested our proposed method on a variety of video se-is that Laplacian ﬁlter puts an emphasis on the high-frequency quences and still images. We also compared it with above men-range of the spectrum, which includes aliased frequency com- tioned methods. To do a fair comparison, we modiﬁed all meth-ponents. As a result, enhancing the aliased components us- ods so that they only enhance row-wise.ing the modiﬁed Laplacian UM techniques produces unnatural In Figure 4 results using zoomed ﬁrst frame of the tempetelooking and disturbing edge transitions. sequence is given. All of the modiﬁed Laplacian techniques show o/u shooting effects, especially on the up-right edge of2.2. Enhancement Gain Function the rock. Among the modiﬁed Laplacian techniques, linear UM and OS Laplacian has the worst performance in o/u shooting.There are two important design goals for the enhancement gain Rational UM is better in terms of o/u shooting but it producesfunction: avoiding noise visibility especially in smooth regions stair-like transitions and broken edges. On the other hand ourand preventing intensity saturation to minimum and maximum proposed method does not produce any o/u shooting and ex-possible intensity values (e.g. 0 and 255 for 1 byte per channel hibits natural looking transitions and edges.source format). To deal with these problems, the enhancementgain depends on the output magnitude (OM) of the ACE high- Performances of these methods on test video sequences arepass ﬁlter, that is the magnitude of the detail in that locality. measured using 3 different criteria: non-edge local variance ra- Since the low-pass component of the smooth regions is low, tio (VR), mean opinion score (MOS) and noise ampliﬁcationthe OM of the ACE high-pass ﬁlter will also be low. Hence, ratio (NAR). We have used non-edge local variance to computethe enhancement gain should be small when the ﬁlter’s OM is the contrast enhancement amount. Since modiﬁed Laplaciansmall to avoid noise visibility in smooth regions. As the details methods suffer from o/u shooting, we did not include edge pix-increase the gain should also be increasing. However, increas- els in variance computation not to inﬂate the local variancesing the gain function continuously may lead to saturation. This with artifacts. In any case, we believe that the effect of excludedcan be prevented by reducing the enhancement gain after some edge variances will be reﬂected in MOSs. Therefore non-edgepoint. We would like to note that since our YENI ﬁlter adapts local variance is found over a 1x16 window using only non-itself with edge information, saturation is not likely to be ob- edge pixels. For each enhanced image we report the ratio ofserved in practice. Considering these speciﬁcations, we have the non-edge local variance’s sum to that of the original image.designed a gain function that is an upward shifted cosine eval- We have used Canny [9] edge detector for identifying edge pix-uated in the 3rd quadrant when OM is in [a-b], and a cosine els. The noise sensitivity is tested by adding Gaussian noise toevaluated in the 1st quadrant when OM is in [b-c], where a,b,c the input image and computing the variance of the difference inare OM thresholds as shown in the example gain function given the enhanced image. The ratio of the difference image’s vari-in Figure 2. Here, a,b,c, and K are chosen as 1,7,21,1 respec- ance to the variance of the Gaussian noise added to the inputtively where K is the maximum achievable gain that determines image is the NAR. We have tried 3 different Gaussian signalsthe strength of the enhancement signal. with variances 10,20, and 40 and averaged the 3 corresponding NARs.2.3. Computational Complexity and Memory Requirement Results for CIF size ”Tempete” sequence is given in Ta- ble 1. Linear Laplacian has the largest VR, however, it per-Computational complexity of our proposed algorithm is extremely forms very poor in noise ampliﬁcation. This is expected sincelow since at each given pixel we employ 2 one pole IIR ﬁlters linear UM boosts high-frequencies regardless. Our proposedgiven in (3) (4). The delay coefﬁcient (λ) of the IIR ﬁlter does LCE method has the next largest VR and has approximatelynot have to be computed since λ is determined by (5) and the half NAR. Cubic UM and OS Laplacian have small NARs be-edge signal that is input to this function is always an integer cause their VRs are also correspondingly low.
4.
250 250 250 200 200 200 Pixel value Pixel value 150 150 150 Pixel value 100 100 100 50 50 50 0 0 0 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 Pixel index Pixel index Pixel index (a) Original (b) Linear UM (c) Cubic UM 250 250 250 200 200 200 Pixel value 150 150 150 Pixel value Pixel value 100 100 100 50 50 50 0 0 0 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 Pixel index Pixel index Pixel index (d) Rational UM (e) OS Laplacian (f) Proposed ACE Fig. 3. 1-D example (a) Original (b) Linear UM (c) Cubic UM (d) Rational UM (e) OS Laplacian (f) Proposed ACE Fig. 4. Zoomed ﬁrst frame of the ”tempete” sequence enhanced with 5 different UM methods 4. REFERENCES [8] Y. H. Lee and S. Y. Park, “A study of convex/concave edges and edge-enhancing operators based on the laplacian,” IEEE Trans.[1] Mark J. T. Smith and Alen Docef, A Study Guide for Digital on Circuits and Systems, vol. 37, no. 7, pp. 940–946, 1990. Image Processing, Scientiﬁc Publishers, Georgia, 1999. [9] John Canny, “Computational approach to edge detection,” IEEE[2] S. M. Pizer, J. B. Zimmerman, and E. V. Staab, “Adaptive grey Transactions on Pattern Analysis and Machine Intelligence, vol. level assignment in ct scan display,” J. Comp. Assist. Tomogr., PAMI-8, no. 6, pp. 679–698, 1986. vol. 8, no. 2, pp. 300–305, 1984.[3] P. M. Narendra nd R. C. Fitch, “Rel-time adaptive contrast en- hancement,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI, no. 3, pp. 655–661, 1981.[4] J. S. Lee, “Digital image enhancement and noise ﬁltering by using local statistics,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI, no. 2, pp. 165–168, 1980.[5] J. A. Stark, “Adaptive image contrast enhancement using general- izations of histogram equalization,” IEEE Transactions on Image Processing, vol. 9, no. 5, pp. 889–896, 2000.[6] G. Ramponi, “A cubic unsharp masking technique for contrast enhancement,” Signal Processing, vol. 67, no. 2, pp. 211–222, June 1998.[7] G. Ramponi and A. Polesel, “A rational unsharp masking tech- nique,” 1998.
Be the first to comment