Enhancement of Color Images by Scaling
the DCT C...
There are also other advantages of using compres...
In fact, from (7), it can be seen th...
There are also other advantages for using each o...
Fig. 4. Enhanced images by scaling b...
Fig. 7. Few more examples of original and enhanc...
Finally, we are also interested to o...
Fig. 8. Enhancement of image18.
Fig. 9. Zoomed p...
Fig. 12. Enhancement of image24.
the transform coefficients. The uniqu...
Sanjit K. Mitra (S’59–M’63–F’74–LF’00) received
Upcoming SlideShare
Loading in …5

Enhancement of Color Images by Scaling the DCT Coefficients


Published on

For more projects feel free to contact us @

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Enhancement of Color Images by Scaling the DCT Coefficients

  1. 1. IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 1783 Enhancement of Color Images by Scaling the DCT Coefficients Jayanta Mukherjee, Senior Member, IEEE, and Sanjit K. Mitra, Life Fellow, IEEE Abstract—This paper presents a new technique for color en- hancement in the compressed domain. The proposed technique is simple but more effective than some of the existing techniques reported earlier. The novelty lies in this case in its treatment of the chromatic components, while previous techniques treated only the luminance component. The results of all previous techniques along with that of the proposed one are compared with respect to those obtained by applying a spatial domain color enhancement technique that appears to provide very good enhancement. The proposed technique, computationally more efficient than the spa- tial domain based method, is found to provide better enhancement compared to other compressed domain based approaches. Index Terms—Blocking artifact, color enhancement, colorful- ness, discrete cosine transform (DCT), DC and AC coefficients, JPEG images, quality metrics, Y-Cb-Cr color space. I. INTRODUCTION I MAGE enhancement is required mostly for better visual- ization or rendering of images to aid our visual percep- tion. There are various reasons, why a raw image data requires processing before display. The dynamic range of the intensity values may be small due to the presence of strong background illumination, as well as due to the insufficient lighting. It may be the other way also. The dynamic range of the original image may be too large to be accommodated by limited number of bit-planes of a display device. The problem gets more compli- cated when the illumination of the scene widely varies in the space. In that case, in some places the scene appears to be too dark while in some other places it is too bright. An example of such an image is shown in Fig. 1(a). It can be seen that the brightness of the blue sky affects the display of the reflection on the glass window of the car, where many details are not vis- ible. In such images it is necessary to improve the local contrast. The result of such a processing [1], [2] with improved display is shown in Fig. 1(b). Image enhancement very often deals with such improvement of image contrast as it is related to the sharpness of the details. Manuscript received November 4, 2007; revised May 29, 2008. Current ver- sion published September 10, 2008. This work was supported in part by the USC-IIT Kharagpur partnership program and in part by a University of Cali- fornia MICRO program with matching supports from the Xerox Corporation. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Peyman Milanfar. J. Mukherjee is with the Department of Computer Science and Engi- neering, Indian Institute of Technology, Kharagpur, India 721302 (e-mail: S. K. Mitra is with the Ming Hsieh Department of Electrical Engineering, University of Southern California, Los Angeles, CA 90089 USA (e-mail: Digital Object Identifier 10.1109/TIP.2008.2002826 Fig. 1. (a) Original image and (b) enhanced image. These methods are designed on the principle of amplifying the local intensity or color variations within an image to increase the visibility of texture details and other features [3]–[5]. A majority of techniques advanced so far have focused on the enhancement of gray-level images in the spatial domain. These methods include adaptive histogram equalization, un- sharp masking, constant variance enhancement, homomorphic filtering, high-pass, and low-pass filtering, etc. (see [3] and [4] and references therein). These methods have also been adapted for color image enhancement [3], [6]. However, later approaches for enhancing color images have taken into account also the chromatic information as well. In many such algorithms [6]–[14] the - - color coordinates are transformed into a different space such as - - - - etc., where chro- matic components are more uncorrelated from the achromatic component. This allowed the representation of the color in terms of hue, saturation, and intensity in closer agreement with the physiological models which describe the color processing of the human visual system [4], [5], [15]. There are also a few work reported in the - - space. For example, Jobson et al. has used retinex theory leading to excellent quality of the enhanced images [as shown in Fig. 1(b)] [2]. However, their technique is computationally intensive as it requires filtering with multiscale Gaussian kernels and postprocessing stages for adjusting colors. There are also techniques reported using equalization of the 3-D histograms in the - - space [16]–[18]. All above mentioned enhancement techniques are spa- tial-domain based. However, increasingly images are being represented in the compressed format for efficient storage and transmission. Hence, it has become imperative to investigate compressed domain enhancement techniques to eliminate the computational overheads for carrying out the inverse transform to the spatial domain and back into the compressed domain by forward transform. In particular, processing in the DCT domain has attracted significant attention of researchers due to its adoption in the JPEG and MPEG compression standards. 1057-7149/$25.00 © 2008 IEEE
  2. 2. 1784 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 There are also other advantages of using compressed domain representation. For example, due to the spectral separation, it is possible to enhance features by treating different frequency components differently. To this end, different algorithms have been advanced for both color and grey level images in the block DCT domain, such as alpha rooting [19], multicontrast en- hancement [20], [21], by processing the AC coefficients and its modified form by processing both DC and AC coefficients [22], etc. However, there are also some disadvantages in processing images in the block DCT domain. As in most cases blocks are independently processed, blocking artifacts may become more visible in the processed data. Some times superfluous edges may appear in the image boundaries due to the sharp discontinuities of the distribution at its boundaries. The display of a color image depends upon three fundamental factors, namely i) its brightness, ii) contrast, and iii) colors. Interestingly, all the previous work have considered either the brightness (such as adjustment of dynamic ranges) or the con- trast (such as image sharpening operations), and even in some cases a combination of both attributes. But none of these al- gorithms have considered the preservation of colors in the en- hanced image. In this work we have considered all three at- tributes while designing a simple computationally efficient al- gorithm. The simplicity of the proposed algorithm lies in the fact that the computation requires only scaling of the DCT coefficients mostly by a factor which remains constant in a block. In con- trast, previous algorithms [19], [20], and [22], dealt with nonuni- form scaling of DCT coefficients in a block. For example, in [19], the scale factors are computed for every coefficient (both the DC and AC coefficients) by taking their roots. In [20], the relative contrast between a pair of successive bands of AC coef- ficients is scaled by a factor, which remains constant for every block. This eventually requires computation of different scale factors for different bands of AC coefficients. In [22], a similar strategy has been incorporated. However, in this technique the amount of scaling of the relative contrast between the successive bands of AC coefficients varies from one block to the other. It also varies between the low-frequency bands and the high fre- quency bands. On the other hand, the proposed algorithm not only uses the same scale factor for both the DC and AC coef- ficients, but also scales the chromatic components as well with the same factor (with the exception of their DC coefficients as discussed later). This simplicity, however, does not make the scheme inferior to others. In fact in many respects, it has been found to provide better performance metrics, as observed in our experimentations. The major objective for processing the data in the compressed domain is to reduce the computational complexity and storage requirements. Hence, these two requirements should be taken into account in designing the pertinent algorithms. For many spatial domain processing, an equivalent computation in the DCT domain could be developed by using the properties of the DCT such as its linear and orthogonal property, sub-band rela- tionship [23], spatial relationship [24], and convolution-multi- plication property [25]. II. MATHEMATICAL PRELIMINARIES The Type II DCT is more commonly used in image compres- sion algorithms. In the case of a 2-D image it is given by (1) where is given by (2) The coefficient is the DC coefficient and the remaining are the AC coefficients for the block. The normalized transform coefficients are defined as (3) Let and denote the mean and standard deviation of an image. Contrast of an image is usually modeled with the Weber law , where is the difference in luminance between a stimulus and its surround, whereas is the luminance of the surround [26]. As provides a measure for surrounding luminance and is strongly correlated with , we redefine the contrast of an image as follows: (4) It may be emphasized here that the above definition is merely intuitive by drawing analogy from the Weber Law as mentioned before. It also reflects the local contrast measure and under the present context, we apply it to the individual blocks. In the following theorem, we state how the contrast of an image is related to the scaling of its DCT coefficients. Theorem 1: Let be the scale factor for the normalized DC coefficient and be the scale factor for the normalized AC coefficients of an image of size . Let the DCT coefficients in the processed image be given by (5) The contrast of the processed image then becomes times of the contrast of the original image. Proof: From the definition of the DCT, it follows that the mean of the image is given by (6) Similarly, from the Perseval’s Theorem, the standard deviation of the image can be expressed as (7)
  3. 3. MUKHERJEE AND MITRA: ENHANCEMENT OF COLOR IMAGES BY SCALING THE DCT COEFFICIENTS 1785 In fact, from (7), it can be seen that the sum of the squares of the normalized AC coefficients provide the variance of the image. Hence, any change in the DC component does not have any bearing on its standard deviation . So under scaling of the DCT coefficients by according to (5), the mean and standard deviation of the processed image become and , respectively. As a result the contrast of the pro- cessed image becomes times of that of the original image. In the block DCT space, the image is partitioned into a set of blocks (in the JPEG standard, ). One of its advantages of partitioning is that it captures the local statistics, though in a predefined manner. As one of our objectives for the enhancement is to preserve the local contrast of the image, if we keep in a block, the local contrast remains the same as before. However, the scaling of the component only does not preserve the colors in the processed image. By preservation of colors we mean that in the - - color space the color vector of a pixel in the processed image has the same direction as that in the original. For example in [27] this is carried out in the - - space (in the spatial domain) by scaling the and components with the same proportion of enhanced values in pixels. Similar strategy is employed in the proposed algorithm. However, one should take care of the nonlinearity in the transformation of the - - color space to - - color space. In the block DCT space this requires separate treatment for the DC coefficient and the AC coefficients. The following theorem summarizes these operations. Theorem 2: Let and denote the DCT coefficients of the and components, respectively. If the luminance component of an image is uniformly scaled by a factor , the colors of the processed image with and are preserved by the following operations: (8) (9) Proof: The - - color space is related to the - - color space as follows: (10) assuming 8 bits for each color component. Though the - - space is not linearly related to the - - space, the color space - - , with and , is linearly related to the - - space. Hence, under uniform scaling by factor of all samples of and , the magnitude of the color vectors in - - space is also scaled by , while keeping its direction the same. Let the DCT of and components be denoted by and , respectively. They are related to the and the as follows: (11) (12) From (11) and (12), (8) and (9) can be easily derived. III. THE PROPOSED ALGORITHM The proposed algorithm performs the color image enhance- ment operation in three steps. First, it adjusts the background illumination. The next step preserves the local contrast of the image and the last one preserves the colors of the image. More- over, in a block DCT space, the algorithm attempts to exploit the advantage of having localized information from the DCT coeffi- cients. The algorithm is designed in such a way that each block (of size 8 8) for all the components could be handled indepen- dently. This makes it more suitable for parallel implementation. A. Adjustment of Local Background Illumination In adjusting the local background illumination, the DC co- efficient of a block is used. The DC value gives the mean of the brightness distribution of the block. This adjustment may be performed by mapping the brightness values to a value in the desired range. This function should be monotonic in the given range. Let us denote the maximum brightness value of the image as (which may be available from the header of the com- pressed stream). Let the DCT coefficients of a 8 8 block of the luminance component be denoted by . Then is the DC coefficient and the rest are the AC coefficients. As before we denote the normalized DC and AC co- efficients by . In adjusting the local brightness, this DC coefficient is mapped to by using a monotonically increasing function in the interval of [0,1] as follows: (13) The function can be chosen in various ways. In our ex- perimentations we have considered some of the previously used functions for enhancement, namely, the twicing function (de- noted here by ) [28], a function used in [22] (denoted here by ) and the S-function [29] (denoted by ). Their func- tional forms are given in (14)–(16), shown at the bottom of the next page. The reasons for choosing these functions are as follows: i) they have been employed earlier in developing image enhance- ment algorithms, and ii) there is no single function which has been found to provide the best performance for every image. Our objective here is to observe the performances of our proposed algorithm with different choices of these mapping functions.
  4. 4. 1786 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 There are also other advantages for using each of the above func- tions. The first one, , is simple to implement. It has no pa- rameter, yet it is found to be quite effective in brightening the dark regions across different types of images. The second one, , is also relatively simpler (than the third one). It has a single parameter which should be centered around 2. This function works well both in the dark and bright regions. The last one, , is the most complex one. It has four parameters, and . However, it can effectively control the dark regions (be- tween ) and bright regions (between ) separately (by choosing different values of for dark region and for bright region). It may be noted also that . Plots of these functions are shown in Fig. 2. The results obtained by applying these functions on the image of Fig. 1(a) are shown in Fig. 3. It may be noted that though the dark regions get brightened by these processes, the images lack the sharpness or details of the original one. So to preserve the local contrast we apply the following simple procedure. B. Preservation of Local Contrast Let us define the enhancement factor for a block during ad- justment of its luminance as (17) where is the mapped DC coefficient and is the original DC coefficient. Since the DCT is a linear transform, multiplying all the coef- ficients of by results in the multiplication of the pixel values in the block by the same factor. This also preserves the contrast of the block. However, there is a risk of overflow of some of the pixel values beyond the maximum allowable representation (say ). This can be controlled by taking into account of the standard deviation and mean of the brightness distribu- tion of the block. Let us assume that the brightness values of this distribution lie within , where is a constant. In that case should lie within an interval as stated by the fol- lowing theorem. Theorem 3: If we assume that the values in a block lies within , the scaled values will not exceed if . Proof: Due to the scaling of the DCT coefficients by , the mean and standard deviation of the block become as and , respectively. A pixel value of the scaled block should lie within . This implies and the scaled value Fig. 2. Plots of the mapping functions: (a) (x), (b) (x), and (c) (x)(m = n = 0:5). Fig. 3. Enhanced image by scaling DC coefficients only using (a) (x), (b) (x), and (c) (x). will not exceed if . Hence, . (14) (15) (16)
  5. 5. MUKHERJEE AND MITRA: ENHANCEMENT OF COLOR IMAGES BY SCALING THE DCT COEFFICIENTS 1787 Fig. 4. Enhanced images by scaling both DC and AC coefficients using (a)(x), (b) (x), and (c) (x). Fig. 5. Enhanced images by scaling all the components including Cb and Cr using (a) (x), (b) (x), and (c) (x). The resulting images after the application of these operations are depicted in Fig. 4. It can be seen that the colors look less saturated in the enhanced images and there is a scope for further improvement on the display of colors. This task is performed in the next and final stage. C. Preservation of Colors As described earlier, we have performed a restricted scaling operation according to (8) and (9) for preserving the colors. It may be noted that the previous enhancement algorithms [22] did not perform this operation [22]. These techniques only change the luminance component and keep the chrominance com- ponents ( and , respectively) unaltered. Though in the - - color space the chrominance components are decor- related better than that in the - - color space, the increasing values in the component usually tend to desaturate the colors. Typically one may observe from the conversion matrix for going from the - - space to the - - space, for and increasing while keeping and unchanged re- duces both the and factors. This is why we be- lieve that the chromatic components should be also processed for preserving the colors. The improvement of the quality of the enhanced image is also quite apparent from Fig. 5. In this case, there is also a chance that the enhanced values of the chromatic components may fall outside the range of rep- resentation (in the spatial domain) due to the scaling operation. One may adopt also a similar strategy of limiting the enhance- ment factor as described in the previous section. However, in our experimentation with a different set of images, we have not observed any such case and this operation has been ignored in the proposed algorithm. D. Suppressing Blocking Artifacts The major problem in developing algorithms in the block DCT domain is the blocking artifacts that may be visible as a result of the independent processing of the blocks. One possible solution is to make use of the information from neighboring blocks for reducing or removing these artifacts as adopted in [30] and [31] and elsewhere. However, this comes at the cost of more computation and more buffer requirements. A simpler Fig. 6. Enhanced images with blocking artifact removal by decomposition and composition of DCT blocks using (a) (x), (b) (x), and (c) (x). strategy is to suppress some of the coefficients in the first row and column for removing such artifacts [32]. However, we have found these approaches not very effective. Sometimes applica- tion of these techniques have resulted in the occurrence of spu- rious horizontal or vertical edges in the enhanced images. In our proposed technique, we have tried to address this problem from a different perspective. We have observed that blocking ar- tifacts are more visible in the regions where brightness values vary significantly, specially near the edges of sharp transitions of luminance values. For suppressing these artifacts, therefore, it would be necessary to keep the variations of the DC coeffi- cients smooth or continuous. To this end, the mapping function has been found to perform better than the others. In fact, the strategy for adjusting its parameter depending on the ratio of high and low spectral energies has been adopted in [22]. Later these values are further smoothed by taking the average of pa- rameter values in the neighboring blocks. However, to keep the computations limited within a block for achieving better par- allelism, we have taken a different approach. We have decom- posed the DCT blocks into smaller blocks, if necessary, and per- formed computations on them. Later these smaller blocks are merged into the original block size. In this case, we have identi- fied the blocks having significant variations by examining their standard deviations . If the is beyond a threshold (denoted here as ), we decompose a 8 8 block into four 4 4 sub-blocks. Then the same enhancement algorithm is applied to each sub-block. Finally, the four enhanced sub-blocks are com- bined again to a 8 8 block. There exist efficient algorithms for these composition and decomposition operations with a mar- ginal increase in the computational overhead [33], [34]. These additional operations also do not increase any buffer require- ment. The results obtained after application of this technique are shown in Fig. 6. E. The Algorithm The overall algorithm is summarized below. As each block is independently processed, in the description processing with a single block is narrated. In our description we have also consid- ered a block of general size . Algorithm Color Enhancement by Scaling (CES) Input: : DCTs of three components of a block. Input Parameters: (the mapping function), , (Block size). Output: . 1. Compute and using (6) and (7).
  6. 6. 1788 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 Fig. 7. Few more examples of original and enhanced images using the tech- nique reported in [1]. (a) Image18, (b) enhanced image18, (c) image22, (d) en- hanced image22, (e) image24, (f) enhanced image24. 2. If , 2a. Decompose into DCT sub-blocks, 2b. For each block apply similar computations as described in Steps 3 through 5, and 2c. Combine 4 of these blocks into a single DCT block and return. 3. Compute the enhancement factor as follows: 3a. , 3b. , and 3c. 4. Scale the coefficients: 4a. , and 4b. Apply (11) and (12) on and for preserving colors. End Color Enhancement by Scaling (CES) It may be noted that the step-2 of the above algorithm may be applied recursively. This would definitely improve the quality of the results by further removing the blocking artifacts. In the lim- iting case, the operation will be equivalent to the application of mapping function [refer to (13)] to individual pixels. It is clear that these would increase the computation and the improvement would be mostly marginal, as it is expected that smaller blocks are having lower variance values. Under the present context, as the block size is 8 8 only, we have applied one level of decom- position and composition (until the size of 4 4) if required. This also keeps the algorithm simple by avoiding recursion or iteration. IV. COMPARISON METRICS We have compared the performance of the proposed enhance- ment technique with several other compressed domain methods. To this end, we considered how the compressed domain tech- niques perform compared to that of a very good spatial domain enhancement technique. For the latter we chose the scheme pro- posed in [1] which is based on a multiscale retinex processing. It essentially involves multiscale Gaussian filtering and point-wise center-surround subtraction in the logarithmic scale. Finally, the pixel values are remapped to their original scale, followed by an empirical adjustment of color components. The processing is performed in the - - color space and is computation- ally quite expensive. However, the algorithm appears to pro- vide excellent enhancement results as can be seen in the web- site http://dragon.larc.nasa.govt/retinex/pao/news. Fig. 7 shows some of the images from this website. For perceptual quality evaluation purposes, we have consid- ered these spatial-domain processed images as the reference images and have used the metric proposed in [35]. For future reference we call this metric the Wang–Bovic-Quality-Metric (WBQM) and its definition is given below. Let and be two distributions. The WBQM between these two distributions is de- fined as (18) where is the covariance between and and are the standard deviations of and respectively, and as well as are their respective means. It may be noted that this mea- sure takes into account of the correlation between the two dis- tributions and also their proximity in terms of brightness and contrast. The WBQM values should lie in the interval . Processed images with WBQM values closer to 1 are more sim- ilar in quality according to our visual perception. We have mea- sured WBQM measures independently for each component in the - - space and called them as -WBQM, Cb-WBQM and Cr-WBQM, respectively. Wang et al. [36] have also suggested another no reference metric for judging the image quality reconstructed from the block DCT space to take into account visible blocking and blur- ring artifacts. In [22], this metric has been used. In our com- parison also, we have used the same one and call it the JPEG- Quality-Metric (JPQM). The computation of this metric is de- scribed in [36]. We have used the MATLAB code available at their web-site˜wang/research/ nr_jpeg_quality/index.html to compute these values. Inciden- tally, in [22], same parameter set has been used for getting the JPQM values. It may be noted that for an image with good vi- sual quality, the JPQM value should be close to 10.
  7. 7. MUKHERJEE AND MITRA: ENHANCEMENT OF COLOR IMAGES BY SCALING THE DCT COEFFICIENTS 1789 Finally, we are also interested to observe the quality in terms of color enhancement. In this regard, we have used a no-ref- erence metric called colorfulness metric (CM) as suggested by Susstrunk and Winkler [26]. The definition for this metric in the - - color space is as given below. Let the red, green and blue components of an Image be denoted by , and , respectively. Let and . Then the colorfulness of the image is defined as (19) where and are standard deviations of and , respec- tively. Similarly, and are their means. In our compar- ison, however, we have used the ratio of CMs between the en- hanced image and its original for observing the color enhance- ment factor CEF. V. RESULTS We have compared the performance of the proposed approach with that of three existing DCT domain color enhancement tech- niques, namely alpha-rooting [19], multicontrast enhancement [20] technique, and multicontrast enhancement coupled with dynamic range compression [22]. In alpha rooting a DCT co- efficient is modified as given by the following equation: (20) In our experimentation, the value of has been kept as 0.98. This value has been chosen empirically after observing the per- formance of this technique with different values of . For multicontrast enhancement, the spectral energy for each diagonal band such that is expressed as the average of the absolute value of the DCT coefficients in the band. The algorithm computes a contrast measure as the ratio of the cumulative energy sums till the th band for the enhanced and original image. Then the coefficients at each band are scaled by the factor . In our experimentation, has been taken as 1.95. The contrast enhancement operation of [22] is almost sim- ilar to that reported in [20]. In [20], the norm has been used, while in [22], the norm has been used for defining relative contrast among the spatial frequency bands. Besides in [22], the technique is further refined by varying the parameter of dy- namic range compression for each block depending on its spec- tral content. The parameter is also smoothed by taking average of its values used in the neighboring blocks. In our experimenta- tion, we have used the same set of parameter values ( and ), as used in [22]. Further, following the similar implementation in [22], during application of multicon- trast enhancement on the bands of AC coefficients, we have kept the scaling parameter [22] as 1 for low frequency bands and as the enhancement factor (equivalent to in the Theorem 1 of this paper) for high frequency bands. In [22], a preprocessing step of noise removal and a postprocessing operation of smoothing the DC coefficients are also carried out for further improvement of the result. In our implementation, we have not considered these TABLE I LIST OF TECHNIQUES CONSIDERED FOR COMPARATIVE STUDY steps, as both these steps could be applied to other algorithms for similar improvement. We have implemented our proposed algorithms CES with dif- ferent choices of the mapping functions. As we would like to ob- serve the effect of employing blocking artifact removals (Step 2 of the algorithm CES), the algorithms for both the variations (with and without the computation described in the Step 2) have been implemented. The names of different techniques and the parameter set used in experimentations are listed in Table I. A. Enhanced Images: A Few Examples In this section, we present the results of different approaches used for processing the set of images shown in Fig. 7. The processed images are shown in Figs. 8, 10, and 12, respectively. For observing the enhancement near edges, portions of the enhanced images near typical edges, are shown in enlarged forms in Figs. 9, 11, and 13, respectively. Further, the per- formance metrics of different techniques are also tabulated in Tables II–IV. In the tables, the maximum performance mea- sures are shown as bold numbers. It has been found that the alpha rooting technique leads to an image that is almost similar to its original, and, hence, its JPQM measures are higher than other schemes. But its other performance indices are quite poor. Though the MCEDRC performs quite well in terms of JPQM, its measures related to colors (CEF, Cb-WBQM, and Cr-WBQM) are significantly poorer than those obtained by the CES techniques. Among CES techniques, TW-CES has per- formed the best in our examples, though its performance related to suppression of the blocking artifacts is usually poorer. It should be noted also that adoption of blocking artifact removal (Step 2 of the algorithm CES) has significantly improved the JPQM measures. This is also reflected by the improvement of the visual quality of the enhanced images. 1) Iterative Processing: It should be noted that the proposed approach can also be applied iteratively for additional enhance- ment if necessary. However, it has been generally observed that additional iterations increase the appearances of blocking and blurring artifacts in the processed images. But for a few of the images more than one iterations has improved the visual quality. Typical examples of applying TW-CES-BLK on the image 22
  8. 8. 1790 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 Fig. 8. Enhancement of image18. Fig. 9. Zoomed portion of the enhanced image18. with two or more iterations are shown in Fig. 14. It can be ob- served that an increase in the number of iterations has led to a steady decrease of the JPQM values (refer Table V). However, with two iterations there is a significant improvement of the CEF measure. B. Average Performances at Different Compression Levels We have experimented with a set of color images (image1 to image24 except the image21 which is not a color image). These images have been compressed using JPEG at different compression quality factors, namely 100, 75, 50 and 30. At each compression level, we have listed the average performance metrics of different techniques in Tables VI–IX, respectively. From these tables it should be noted that the TW-CES-BLK provide the best performance indices though it is relatively poor in reducing the blocking artifacts compared to that obtained using other mapping functions. The other two variations of Fig. 10. Enhancement of image22. Fig. 11. Zoomed portion of enhanced image22. CES (DRC-CES-BLK and SF-CES-BLK) are quite good in suppressing the blocking artifacts. It is also interesting to note that the performance measures related to colorfulness and similarity with the reference images do not vary significantly at varying levels of compression. However, as expected, the blocking artifacts are more and more visible with increasing levels of compression. C. Computational Complexity We have also compared the computational complexity in terms of number of additions, multiplications (or divisions) and exponentiation operations per pixel for the different techniques. These are expressed in the form of , where denotes the number of additions, denotes the number of multiplications (or divisions) and denotes the number of exponentiation operations for every pixel in the original image. In estimating the number of operations, we have also
  9. 9. MUKHERJEE AND MITRA: ENHANCEMENT OF COLOR IMAGES BY SCALING THE DCT COEFFICIENTS 1791 Fig. 12. Enhancement of image24. Fig. 13. Zoomed portion of enhanced image24. ignored normalization operations on the coefficients such as division by the block length (8) or maximum intensity value TABLE II PERFORMANCE MEASURES FOR DIFFERENT TECHNIQUES ON ENHANCING IMAGE18 TABLE III PERFORMANCE MEASURES FOR DIFFERENT TECHNIQUES ON ENHANCING IMAGE22 TABLE IV PERFORMANCE MEASURES FOR DIFFERENT TECHNIQUES ON ENHANCING IMAGE24 Fig. 14. Enhanced images after number of iterations: (a) 2, (b) 3, and (c) 4. TABLE V PERFORMANCE MEASURES ON ITERATIVE APPLICATIONS OF TW-CES-BLK ON ENHANCING IMAGE22 (may be taken as 255 in most cases). For different algorithms, the accounting for different operations are described in the following subsections. 1) Alpha Rooting (Ar): The computation according to (20) requires 1 multiplication and 1 exponentiation operation.
  10. 10. 1792 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 TABLE VI AVERAGE PERFORMANCE MEASURES WITHOUT COMPRESSION TABLE VII AVERAGE PERFORMANCE MEASURES WITH JPEG COMPRESSION AT QUALITY 75 TABLE VIII AVERAGE PERFORMANCE MEASURES WITH JPEG COMPRESSION AT QUALITY 50 TABLE IX AVERAGE PERFORMANCE MEASURES WITH JPEG COMPRESSION AT QUALITY 30 Hence, the computational complexity can be expressed as per pixel. 2) Multicontrast Enhancement (MCE): Computation of the cumulative energies for both enhanced and original blocks, re- quires 126 additions (ignoring the cost of absolute operations). For computing , 14 divisions are required and finally the scaling of the AC coefficients requires two multipli- cations each. The total number of operations for each block is TABLE X COMPUTATIONAL COMPLEXITIES OF DIFFERENT TECHNIQUES thus 140 multiplications and 126 additions. Hence, the number of operation per pixel becomes . 3) Multicontrast Enhancement With Dynamic Range Com- pression (MCEDRC): As this technique uses norm, the com- putation of cumulative energies becomes more expensive than the previous technique. In this case, the number of operations is 128 Multiplications and 126 Additions. In addition, the dy- namic range compression requires the computation of the func- tion with 2 exponentiation and 2 addition operations. Con- sidering all other factors similar to the previous one, the per pixel operation can be expressed as . Here, we have ignored the computational overhead of the dynamic adjust- ment of the parameter (refer Table I) for reducing the blocking artifacts. 4) Contrast Enhancement by Scaling (CES): In this algo- rithm, the scaling of the coefficients by a constant for each com- ponent is the major computational task. This would require 192 multiplications and four additions. The additions are necessary for translating (and retranslating back) the DC coefficients of the Cb and Cr components. Computation of the scaling factor depends on the type of functions used. In addition, there is an overhead of computing the standard deviation and the mean of the block, which requires 63 multiplications, 62 additions and one exponential operation (square root). The per pixel operations of different techniques are summa- rized in the Table X. For removing the blocking artifacts, the composition and decomposition operations both require operations for each component [33]. This implies additional overhead for such blocks would be roughly operations. However, the number of blocks requiring such decomposition should be small. It may be noted that the multiscale-retinex technique in the spatial domain [1] requires much higher computation. Typically, Gaussian smoothing with scale requires a convo- lution with a mask of size . Exploiting the symmetry of the Gaussian mask, this computation can be performed with multiplications and number of additions. In [1], the enhancement is performed with convolutions with three Gaussian masks with scales 15, 80, and 250, respectively. Besides, there are other operations such as subtraction in the logarithmic scale and color restoration of pixels. These are carried out for all color components in the R-G-B space. Hence, the per pixel computation requirement is given by . VI. CONCLUDING REMARKS In this paper, we have presented a simple approach for enhancing color images in the block DCT domain by scaling
  11. 11. MUKHERJEE AND MITRA: ENHANCEMENT OF COLOR IMAGES BY SCALING THE DCT COEFFICIENTS 1793 the transform coefficients. The unique feature of this algorithm is that it also treats chromatic components in addition to the processing of the luminance component improving the visual quality of the images to a great extent. A comparative study with different other schemes has been carried out on the basis of different performance criteria. It has been found that proposed schemes outperform the existing schemes in most cases. REFERENCES [1] D. J. Jobson, Z. Rahman, and G. A. Woodell, “A multi-scale retinex for bridging the gap between color images and the human observation of scenes,” IEEE Trans. Image Process., vol. 6, no. 7, pp. 965–976, Jul. 1997. [2] D. J. Jobson, Z. Rahman, and G. A. Woodell, “Properties and perfor- mance of a center/surround retinex,” IEEE Trans. Image Process., vol. 6, no. 3, pp. 451–462, Mar. 1997. [3] J. S. Lim, Two-Dimensional Signal and Image Processing. Engle- wood Cliffs, NJ: Prentice-Hall, 1990. [4] R. C. Gonzales and E. Woods, Digital Image Processing. Reading, MA: Addison-Wesley, 1992. [5] The Colour Image Processing Handbook, S. J. Sangwine and R. E. N. Horne, Eds. London, U.K.: Chapman Hall, 1998. [6] S. Wolf, R. Ginosar, and Y. Zeevi, “Spatio-chromatic image enhance- ment based on a model of humal visual information system,” J. Vis. Commun. Image Represent., vol. 9, no. 1, pp. 25–37, Mar. 1998. [7] A. F. Lehar and R. J. Stevens, “High-speed manipulation of the color chromaticity of digital images,” IEEE Trans. Comput. Graph. Anima- tion, pp. 34–39, 1984. [8] S.-S. Kuo and M. V. Ranganath, “Real time image enhancement for both text and color photo images,” in Proc. Int. Conf. Image Processing, Washington, DC, Oct. 23–26, 1995, vol. I, pp. 159–162. [9] I. Bockstein, “Color equalization method and its application to color image processing,” J. Opt. Soc. Amer. A, vol. 3, no. 5, pp. 735–737, May 1986. [10] R. N. Strickland, C.-S. Kim, and W. F. McDonnell, “Digital color image enhancement based on the saturation component,” Opt. Eng., vol. 26, no. 7, pp. 609–616, July 1987. [11] B. A. Thomas, R. N. Strickland, and J. J. Rodrigues, “Color image enhancement using spatially adaptive saturation feedback,” in Proc. Int. Conf. Image Processing, Santa Barbara, CA, Oct. 26–29, 1997, vol. 3, pp. 30–33. [12] A. Toet, “Multiscale color image enhancement,” Pattern Recognit. Lett., vol. 13, no. 3, pp. 167–174, Mar. 1992. [13] I. Pitas and P. Kiniklis, “Multichannel techniques in color image en- hancement and modeling,” IEEE Trans. Image Process., vol. 5, no. 1, pp. 168–171, Jan. 1996. [14] L. Lucchese, S. K. Mitra, and J. Mukherjee, “A new algorithm based on saturation and desaturation in the xy chromaticity diagram for en- hancement and re-rendition of color images,” in Proc. Int. Conf. Image Processing, Thessaloniki, Greece, 2001, pp. 1077–1080. [15] O. Faugeras, “Digital color image processing within the framework of a human visual model,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-27, no. 4, pp. 380–393, Aug. 1979. [16] P. E. Trahanias and A. N. Venetsanopoulos, “Color image enhancement through 3-D histogram equalization,” in Proc. 11th IAPR Int. Conf. Pattern Recognition, The Hague, The Netherlands, Aug. 30–Sep. 3, 1992, vol. III, pp. 545–548. [17] P. A. Mlsna and J. J. Rodriguez, “A multivariate contrast enhance- ment technique for multispectral images,” IEEE Trans. Geosci. Remote Sens., vol. 33, no. 1, pp. 212–216, Jan. 1995. [18] Q. Zhang, P. A. Mlsna, and J. J. Rodrigues, “A recursive technique for 3-D histogram enhancement of color images,” in Proc. IEEE Southwest Symp. Image Analysis and Interpretation, San Antonio, TX, Apr. 8–9, 1996, pp. 218–223. [19] S. Aghagolzadeh and O. K. Ersoy, “Transform image enhancement,” Opt. Eng., vol. 31, pp. 614–626, Mar. 1992. [20] J. Tang, E. Peli, and S. Acton, “Image enhancement using a contrast measure in the compressed domain,” IEEE Signal Process. Lett., vol. 10, no. 10, pp. 289–292, Oct. 2003. [21] J. Tang, J. Kim, and E. Peli, “An image enhancement algorithm in JPEG domain for low-vision patient,” IEEE Trans. Biomed. Eng., vol. 51, no. 11, pp. 2013–2023, Nov. 2004. [22] S. Lee, “An efficient content-based image enhancement in the com- pressed domain using retinex theory,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 2, pp. 199–213, Feb. 2007. [23] S.-H. Jung, S. K. Mitra, and D. Mukherjee, “Subband DCT: Definition, analysis and applications,” IEEE Trans. Circuits Syst. Video Technol., vol. 6, no. 3, pp. 273–286, Jun. 1996. [24] J. Jiang and G. Feng, “The spatial relationships of DCT coefficients between a block and its sub-blocks,” IEEE Trans. Signal Process., vol. 50, no. 5, pp. 1160–1169, May 2002. [25] S. A. Martucci, “Symmetric convolution and the discrete sine and cosine transforms,” IEEE Trans. Signal Process., vol. 42, no. 5, pp. 1038–1051, May 1994. [26] S. Susstrunk and S. Winkler, “Color image quality on the internet,” Proc. IST/SPIE Electronic Imaging: Internet Imaging V, vol. 5304, pp. 118–131, 2004. [27] Y. T. Kim, “Image enhancement circuit and method using mean matching/quantized mean matching histogram equalization and color compensation,” U.S. Patent Office, US6018588, Jan. 2000. [28] S. K. Mitra and T. H. Yu, “Transform amplitude sharpening: A new method of image enhancement,” Comput. Vis. Graph. Image Process., vol. 40, pp. 205–218, 1987. [29] T. K. De, “A simple programmable S-function for digital image pro- cessing,” in Proc. 4th IEEE Region 10th Int. Conf., Bombay, India, Nov. 1989, pp. 573–576. [30] Y. Luo and R. K. Wars, “Removing the blocking artifacts of block- based DCT compressed images,” IEEE Trans. Image Process., vol. 12, no. 7, pp. 838–842, Jul. 2003. [31] B. Jeon and J. Jeong, “Blocking artifact reduction in image compres- sion with block boundary discontinuity criterion,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 3, pp. 345–357, Jun. 1998. [32] B. Zeng, “Reduction of blocking effect in DCT-coded images using zero-masking techniques,” Signal Process., vol. 79, pp. 205–211, 1999. [33] J. Mukherjee and S. K. Mitra, “Arbitrary resizing of images in the DCT space,” IEE Proc. Vis., Image, Signal Process., vol. 152, no. 2, pp. 155–164, 2005. [34] R. Dugad and N. Ahuja, “A fast scheme for image size change in the compressed domain,” IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 4, pp. 461–474, Apr., 2001. [35] Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE Signal Process. Lett., vol. 9, no. 3, pp. 81–84, Mar. 2002. [36] Z. Wang, H. R. Sheikh, and A. C. Bovik, “No-reference perceptual quality assessment of JPEG compressed images,” in Proc. Int. Conf. Image Processing, Rochester, NY, Sep. 2002, vol. I, pp. 477–480. Jayanta Mukherjee (M’90–SM’04) received the B.Tech., M.Tech., and Ph.D. degrees in electronics and electrical communication engineering from the Indian Institute of Technology (IIT), Kharagpur, in 1985, 1987, and 1990, respectively. He joined the faculty of the Department of Elec- tronics and Electrical Communication Engineering, IIT, Kharagpur, in 1990, and later transferred to the Department of Computer Science and Engineering where he is presently a Professor. He served as the head of the Computer and Informatics Center at IIT, Kharagpur, from September 2004 to July 2007. He was a Humboldt Research Fellow at the Technical University of Munich in Germany for one year in 2002. He also has held short-term visiting positions at the University of California, Santa Barbara, University of Southern California, Los Angeles, and the National University of Singapore. His research interests are in image processing, pattern recognition, computer graphics, multimedia systems, and medical informatics. He has published over 100 papers in journals and conference proceedings in these areas. Dr. Mukherjee received the Young Scientist Award from the Indian National Science Academy in 1992.
  12. 12. 1794 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 17, NO. 10, OCTOBER 2008 Sanjit K. Mitra (S’59–M’63–F’74–LF’00) received his M.S. and Ph.D. degrees in electrical engineering from the University of California, Berkeley, in 1960 and 1962, respectively. He is presently with the Ming Hsieh Department of Electrical Engineering, University of Southern California, Los Angeles, as the Stephen and Etta Varra Professor. He has published over 700 papers in the areas of analog and digital signal processing, and image processing. He has also authored and co-authored twelve books, and holds five patents. Dr. Mitra is the recipient of the 1973 F. E. Terman Award and the 1985 ATT Foundation Award of the American Society of Engineering Education, the 1989 Education Award, and the 2000 Mac Van Valkenburg Society Award of the IEEE Circuits AND Systems Society, the Distinguished Senior U.S. Scientist Award from the Alexander von Humboldt Foundation of Germany in 1989, the 1996 Technical Achievement Award and the 2001 Society Award of the IEEE Signal Processing Society, the IEEE Millennium Medal in 2000, the McGraw-Hill/ Jacob Millman Award of the IEEE Education Society in 2001, the 2002 Tech- nical Achievement Award of the European Association for Signal Processing (EURASIP) and the 2005 SPIE Technology Achievement Award of the Interna- tional Soceity for Optical Engineers, the University Medal of the Slovak Tech- nical University, Bratislava, Slovakia, in 2005, and the 2006 IEEE J. H. Mul- ligan, Jr. Education Medal. He is the co-recipient of the 2000 Blumlein-Browne- Willans Premium of the the Institution of Electrical Engineers (London) and the 2001 IEEE Transactions on Circuits AND Systems for Video Technology Best Paper Award. He has received Honorary Doctorate degrees from the Tampere University of Technology, Finland, the Technical University of Bucharest, Ro- mania, and the Technical University of Iasi, Romania. He is a member of the U.S. National Academy of Engineering, the Norwegian Academy of Technolog- ical Sciences, an Academician of the Academy of Finland, and a corresponding member of the Croatian Academy of Sciences and Arts, and the Academy of Engineering, Mexico. He is a Foreign Fellow of the Indian National Academy of Engineering and the National Academy of Sciences, India. He is a Fellow of the AAAS and SPIE.