This document proposes a locally adaptive nonlinear (YENI) filter for image local contrast enhancement. The YENI filter adapts to local edge information to find the unsharp mask of an image, preserving edges while filtering out mid-range details. This avoids overshooting/undershooting artifacts often seen with other techniques. The YENI filter is implemented as a single-pole infinite impulse response filter at each pixel location, keeping computational complexity low. Experimental results on real video show improved visual quality compared to other methods, both subjectively and objectively.

1. IMAGE LOCAL CONTRAST ENHANCEMENT USING ADAPTIVE NON-LINEAR FILTERS
Tarik Arici, Yucel Altunbasak
School of Electrical and Computer Engineering
Georgia Institute of Technology
Atlanta, GA 30332
E-mail: tariq, yucel@ece.gatech.edu
ABSTRACT around the edges. The second method uses an amplification
factor that is inversely proportional to the local standard devi-
We present a locallY adaptivE Non-lInear (YENI) filter to ob-
ation (LSD) [3] so that the amplification around the edges is
tain the unsharp mask of an image. The unsharp mask obtained
lowered. But this increases noise visibility in smooth regions
by the YENI filter preserves the edges in the image while fil-
since LSD in smooth regions is relatively low.
tering out the local details, which correspond to mid-range fre-
We propose a locallY adaptivE Non-lInear (YENI) filter to
quencies in the spectrum. The enhanced image using this un-
find the unsharp mask of an image. YENI filter adapts to lo-
sharp mask effectively prevents over/under (o/u) shooting arti-
cal edge information and keeps the edges in the original image
facts often observed with other unsharp masking techniques.
while filtering out mid-range details. Therefore, the detail im-
The enhanced frequency range also spans lower frequencies
age obtained using the unsharp mask does not have large en-
compared to the techniques that are based on Laplacian fil-
ergy around the edges, and o/u shooting artifact is successfully
ter variants. This improves the visual quality of the image, as
avoided when the enhanced detail image is added back to the
measured subjectively and objectively in the real-video exper-
original. To deal with the phase delay coming from the nature
iments. Furthermore, since the YENI filter reduces to an IIR
of the proposed filters, we employ two filters in opposite direc-
filter at each pixel location, it has a low computational com-
tions. The resultant phase shift coming from the two opposite
plexity.
direction filters cancel out since we average the two filtered sig-
Index Terms— Local contrast enhancement, IIR, recursive nals. Also, the proposed filter has low computational complex-
filters ity and needs small memory resources, which makes it practical
for use in television sets.
1. INTRODUCTION
2. IMAGE LOCAL CONTRAST ENHANCEMENT
Contrast enhancement techniques are widely used to increase
the visual image quality. Global contrast enhancement (GCE) In conventional ACE algorithms the enhanced image y(m, n)
techniques remedy problems that manifest themselves in a global is obtained from the input image x(m, n) as
fashion such as excessive/poor lightning conditions in the source y(m, n) = µ(m, n) + [1 + g(m, n)][x(m, n) − µ(m, n)], (1)
environment. On the other hand, local contrast enhancement
tries to enhance the visibility of local details in the image. Lo- where µ(m, n) is the local mean, g(m, n) is the enhancement
cally enhanced images look more attractive than the originals gain, m is the row number, and n is the column number.
because of the higher contrast [1]. With properly enhanced de- We employ our YENI filter to find the local mean by av-
tails in the image, the HVS needs less amount of concentration eraging two opposite direction non-linear filters and we use a
to discern the intensity differences. rational gain function that is designed to suppress noise visibil-
Two well-known local contrast enhancement methods are ity in smooth regions.
adaptive histogram equalization (AHE) [2] and adaptive con-
trast enhancement (ACE) [3] [4]. AHE algorithms find local 2.1. Unsharp Masking Using YENI filter
mappings using local histograms. Although the AHE improves
contrast, its computational burden is not acceptable for most The local mean µ(m, n) at row m and column n is the output of
applications. A bilinear interpolation technique is presented YENI filter, which is the average of two different filter outputs
in [2] for a block-based AHE. The major problem with AHE given by
methods is that it often over-enhances the image by creating
so called contrast objects that were not visible in the original µF (m, n) + µB (m, n)
µ(m, n) = . (2)
image. The enhanced image often does not look natural and is 2
disturbing[5]. where µF (m, n) and µB (m, n) are the outputs of the two oppo-
On the other hand, ACE algorithms utilize unsharp mask- site direction filters that run horizontally on a single row. The
ing (UM) techniques. Unsharp mask (low-pass component) of first filter runs from left to right and is referred to as the for-
the image is created by low-pass filtering and the detail mask ward filter. The forward filter outputs µF (m, n). The second
(high-frequency component) is obtained by subtracting this un- filter runs from right to left and is similarly referred to as the
sharp mask from the original image. The enhanced image is backward filter. The backward filter outputs µB (m, n). The
produced by amplifying the detail mask and adding it back to two filters are single pole infinite impulse response (IIR) filters
the unsharp mask. However, amplifying high-frequency com- at any given pixel location. The input-output relationship for
ponents creates o/u shooting artifacts around the edges. There the forward filtered µF (m, n) is
are two conventional methods for selecting the amplification
factor. The first one is to use a constant amplification fac- µF (m, n) = λ(m, n)µF (m, n − 1) + [1 − λ(m, n)]x(m, n),
tor [4]. This method performs poorly in terms of o/u shooting (3)

2. where λ(m, n) is the edge adaptive delay coefficient. The rela- which has a zero phase. Thus, the 1-D local mean filter applied
tionship for the backward filtered µB (m, n) is similar1 . row-wise and given by (2) does not shift the 1-D input signal
The adaptation of λ(m, n) to the edge information is cru- column-wise.
cial for preventing the smoothing of edges. Considering that Next we discuss the ACE high-pass filter that is derived
λ(m, n) is the weight of the previous output, stronger λ(m, n) from the YENI filter.
increases the low-pass characteristic of the filter. Hence, when
an edge is encountered, λ(m, n) must be decreased so that the 2.1.2. Frequency response of the ACE high-pass filter
edge will be preserved in the output. The edge signal we use is
|µF (m, n−1)−x(m, n)| for the forward filter, and |µB (m, n+ Let us rewrite the ACE relation in (1) as the original image plus
1) − x(m, n)| for the backward filter. Both of the edge sig- an extra enhancement signal.
nals are the differences between the original pixel value and
y(m, n) = x(m, n) + g(m, n)[x(m, n) − µ(m, n)], (10)
the previous filter output. Using these edge signals, λ(m, n) is
obtained using The second term in (10) is the high-pass filtered original image.
|µF (m, n − 1) − x(m, n)| α This can be seen considering that it is the difference between
λ(m, n) = [1 − ] , (5) the original image and the low-pass filtered image. Then the
255
frequency response of this ACE high-pass filter generating the
for the forward filter, and similar for the backward filter2 . Here second term is found using (9)
we use 255 for the maximum possible pixel value. As can be 1 − cos(w)
observed from (5) and (6), strong edges reduce λ(m, n) more, 1 − H(w) = (λ2 + λ) , (11)
1 − 2λ cos(w) + λ2
hence the low-pass characteristic of the signal at that locality is
lessened. Typical α values are in the range of [5-9]. Figure 1 shows the plot of (11). It is interesting to note
1
that the gain of the ACE high-pass filter is approximately con-
0.9
stant after some frequency threshold. Hence, not only high-
0.8
frequencies close to π are being enhanced, but also mid-range
0.7
frequencies in the spectrum are also enhanced with the same
0.6
magnitude. This increases the visual quality of the enhanced
Increasing λ image because details that have their energy in the mid-range
1−H(ω)
0.5
0.4
frequency spectrum are also enhanced as well as high-frequency
0.3
ACE high−pass filters
Laplacian filter details. Also, magnitude of the frequency response decreases
0.2
with decreasing λ. Since λ is a function of the edge signal,
0.1
this feature demonstrates that the high-pass filter adapts itself
0
to edges by reducing its magnitude. From (10), it is clear that
0 0.5 1 1.5
ω
2 2.5 3
enhancement around the locality of edges decreases because the
magnitude of the enhancement signal decreases accordingly.
Fig. 1. Adaptive high-pass filters used in proposed ACE method This adaptive behavior enables our proposed ACE method to
successfully avoid o/u shooting artifacts.
2.1.1. Frequency response of the YENI filter The representation of the ACE method given in (10) is ex-
For ease of notation we denote the original pixel at the nth col- actly the same as the Laplacian unsharp masking (UM) repre-
umn of row m as xm (n). Then, each row of the original image sentation in which the Laplacian filtered image is added to the
is a 1-D signal. From (3) and (4) frequency response of the for- original image. A Laplacian filter has three taps {−1, 2, −1}.
ward and backward filters at a locality with λ(m, n) = λ are Its frequency response is also given in Figure 1 and shows that
derived as below the Laplacian filter assigns an emphasis to the high frequency
components. The gain at high frequencies is about 2 times
1−λ
HF (w) = (7) larger than the mid-range frequencies. However, this creates
1 − λe−jw two sorts of problem: noise sensitivity and o/u shooting around
1−λ edges. To improve its noise sensitivity and reduce its o/u shoot-
HR (w) = , (8)
1 − λe+jw ing artifacts, various modifications have been introduced [6]
respectively. Here, we implicitly assume that λs for the two fil- [7] [8]. All of these modifications aim to suppress the Lapla-
ters are equal since ideally edge information at the same locality cian filter’s gain at high frequencies. But this inevitably de-
must be the same. From (7) and (8) we can see that the phase creases the enhancement gain at mid-range frequencies and re-
of both filters is not zero causing a phase shift in the filtered duces the level of visual quality enhancement.
output. In fact the forward filter lags, and the backward filter Figure 3 shows a 1-D enhancement example for 5 different
advances the input signal xm (n). However, frequency response unsharp masking methods (linear UM [1], Cubic UM[6], OS
of the local mean filter using (2), (7), and (8) can be obtained Laplacian [8], Rational UM [7]) including our proposed ACE
as below as well as the original 1-D signal. The original signal has 3
edges with medium level detail existing between the edges (i.e.
1 − λ cos(w)
H(w) = (1 − λ) , (9) sinusoidal oscillations) to model different textures in the image.
1 − 2λ cos(w) + λ2
The enhancement gain set for each tested algorithm is adjusted
1 so that increasing the gain does not bring any more enhance-
µB (m, n) = λ(m, n)µB (m, n + 1) + [1 − λ(m, n)]x(m, n), (4) ment but only worsened the artifacts. Linear unsharp masking
enhances both the edges and the medium level detail. But the
2 second edge does not look natural, in fact in an image it will
|µB (m, n + 1) − x(m, n)| α
λ(m, n) = [1 − ] , (6) look stair-like. Cubic UM does not produce stair-like edges but
255
it exhibits heavy o/ushooting artifacts. This is mainly because

3. 1
0.9
K
Table 1. Results for Tempete
0.8 VR MOS NAR
Enhancement Gain
0.7
Linear Lap. 1.72 5 4.50
0.6
Cubic UM 1.02 4.5 1.45
0.5
Rational UM 1.26 6 2.48
0.4
OS Lap. 1.09 5 1.74
0.3
Proposed 1.34 7.5 2.42
0.2
0.1 a b c
0
0 5 10 15 20 25 30
Output magnitude of ACE−high pass filter between 0 and 255. Hence, for λ and for the enhancement
gain two look-up tables (LUTs) can be used. The total num-
Fig. 2. Enhancement gain function ber of computations per pixel including the computation of the
of its cubic dependence on the pixel value differences. Ratio- indices to these 2 LUTs is 2 multiplications, 6 additions and
nal UM does a better job of avoiding o/u shooting artifacts but one bit shift. The memory needed for the LUTs is 256 bytes for
does not enhance the medium level details and also creates a λ LUT, c bytes (typically 21) for enhancement gain LUT, one
stair-like edge. OS Laplacian behaves similar to Cubic UM. In line-store for the forward filter’s output and an additional single
addition it shows a common artifact of order statistics, that is register for backward filter’s output.
producing patch like intensity regions that can be seen as small
constant intermediate levels on the second edge. Our proposed
ACE method successfully avoids o/u shooting artifacts and also
enhances the medium level details in the signal while still pro- 3. EXPERIMENTAL RESULTS
ducing natural looking edges. The reason for Laplacian modi-
fied UM algorithms to create unnatural looking edge transitions We have tested our proposed method on a variety of video se-
is that Laplacian filter puts an emphasis on the high-frequency quences and still images. We also compared it with above men-
range of the spectrum, which includes aliased frequency com- tioned methods. To do a fair comparison, we modified all meth-
ponents. As a result, enhancing the aliased components us- ods so that they only enhance row-wise.
ing the modified Laplacian UM techniques produces unnatural In Figure 4 results using zoomed first frame of the tempete
looking and disturbing edge transitions. sequence is given. All of the modified Laplacian techniques
show o/u shooting effects, especially on the up-right edge of
2.2. Enhancement Gain Function the rock. Among the modified Laplacian techniques, linear UM
and OS Laplacian has the worst performance in o/u shooting.
There are two important design goals for the enhancement gain Rational UM is better in terms of o/u shooting but it produces
function: avoiding noise visibility especially in smooth regions stair-like transitions and broken edges. On the other hand our
and preventing intensity saturation to minimum and maximum proposed method does not produce any o/u shooting and ex-
possible intensity values (e.g. 0 and 255 for 1 byte per channel hibits natural looking transitions and edges.
source format). To deal with these problems, the enhancement
gain depends on the output magnitude (OM) of the ACE high- Performances of these methods on test video sequences are
pass filter, that is the magnitude of the detail in that locality. measured using 3 different criteria: non-edge local variance ra-
Since the low-pass component of the smooth regions is low, tio (VR), mean opinion score (MOS) and noise amplification
the OM of the ACE high-pass filter will also be low. Hence, ratio (NAR). We have used non-edge local variance to compute
the enhancement gain should be small when the filter’s OM is the contrast enhancement amount. Since modified Laplacian
small to avoid noise visibility in smooth regions. As the details methods suffer from o/u shooting, we did not include edge pix-
increase the gain should also be increasing. However, increas- els in variance computation not to inflate the local variances
ing the gain function continuously may lead to saturation. This with artifacts. In any case, we believe that the effect of excluded
can be prevented by reducing the enhancement gain after some edge variances will be reflected in MOSs. Therefore non-edge
point. We would like to note that since our YENI filter adapts local variance is found over a 1x16 window using only non-
itself with edge information, saturation is not likely to be ob- edge pixels. For each enhanced image we report the ratio of
served in practice. Considering these specifications, we have the non-edge local variance’s sum to that of the original image.
designed a gain function that is an upward shifted cosine eval- We have used Canny [9] edge detector for identifying edge pix-
uated in the 3rd quadrant when OM is in [a-b], and a cosine els. The noise sensitivity is tested by adding Gaussian noise to
evaluated in the 1st quadrant when OM is in [b-c], where a,b,c the input image and computing the variance of the difference in
are OM thresholds as shown in the example gain function given the enhanced image. The ratio of the difference image’s vari-
in Figure 2. Here, a,b,c, and K are chosen as 1,7,21,1 respec- ance to the variance of the Gaussian noise added to the input
tively where K is the maximum achievable gain that determines image is the NAR. We have tried 3 different Gaussian signals
the strength of the enhancement signal. with variances 10,20, and 40 and averaged the 3 corresponding
NARs.
2.3. Computational Complexity and Memory Requirement Results for CIF size ”Tempete” sequence is given in Ta-
ble 1. Linear Laplacian has the largest VR, however, it per-
Computational complexity of our proposed algorithm is extremely forms very poor in noise amplification. This is expected since
low since at each given pixel we employ 2 one pole IIR filters linear UM boosts high-frequencies regardless. Our proposed
given in (3) (4). The delay coefficient (λ) of the IIR filter does LCE method has the next largest VR and has approximately
not have to be computed since λ is determined by (5) and the half NAR. Cubic UM and OS Laplacian have small NARs be-
edge signal that is input to this function is always an integer cause their VRs are also correspondingly low.

4. 250 250
250
200 200
200
Pixel value
Pixel value
150 150
150
Pixel value
100 100
100
50 50
50
0 0
0 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140
0 20 40 60 80 100 120 140
Pixel index Pixel index Pixel index
(a) Original (b) Linear UM (c) Cubic UM
250 250 250
200 200 200
Pixel value
150 150 150
Pixel value
Pixel value
100 100 100
50 50 50
0 0 0
0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140
Pixel index Pixel index Pixel index
(d) Rational UM (e) OS Laplacian (f) Proposed ACE
Fig. 3. 1-D example
(a) Original (b) Linear UM (c) Cubic UM
(d) Rational UM (e) OS Laplacian (f) Proposed ACE
Fig. 4. Zoomed first frame of the ”tempete” sequence enhanced with 5 different UM methods
4. REFERENCES [8] Y. H. Lee and S. Y. Park, “A study of convex/concave edges and
edge-enhancing operators based on the laplacian,” IEEE Trans.
[1] Mark J. T. Smith and Alen Docef, A Study Guide for Digital on Circuits and Systems, vol. 37, no. 7, pp. 940–946, 1990.
Image Processing, Scientific Publishers, Georgia, 1999. [9] John Canny, “Computational approach to edge detection,” IEEE
[2] S. M. Pizer, J. B. Zimmerman, and E. V. Staab, “Adaptive grey Transactions on Pattern Analysis and Machine Intelligence, vol.
level assignment in ct scan display,” J. Comp. Assist. Tomogr., PAMI-8, no. 6, pp. 679–698, 1986.
vol. 8, no. 2, pp. 300–305, 1984.
[3] P. M. Narendra nd R. C. Fitch, “Rel-time adaptive contrast en-
hancement,” IEEE Trans. Pattern Anal. Machine Intell., vol.
PAMI, no. 3, pp. 655–661, 1981.
[4] J. S. Lee, “Digital image enhancement and noise filtering by using
local statistics,” IEEE Trans. Pattern Anal. Machine Intell., vol.
PAMI, no. 2, pp. 165–168, 1980.
[5] J. A. Stark, “Adaptive image contrast enhancement using general-
izations of histogram equalization,” IEEE Transactions on Image
Processing, vol. 9, no. 5, pp. 889–896, 2000.
[6] G. Ramponi, “A cubic unsharp masking technique for contrast
enhancement,” Signal Processing, vol. 67, no. 2, pp. 211–222,
June 1998.
[7] G. Ramponi and A. Polesel, “A rational unsharp masking tech-
nique,” 1998.