A New Perceptually Uniform Color Space with Associated Color Similarity Measure for Content-Based Image and Video Retrieval M. Sarifuddin Rokia Missaoui Departement d’informatique et d’ingenierie, ´ ´ Departement d’informatique et d’ingenierie, ´ ´ Universite du Quebec en Outaouais ´ ´ Universite du Quebec en Outaouais ´ ´ C.P. 1250, Succ. B Gatineau C.P. 1250, Succ. B Gatineau Quebec - Canada, J8X 3X7 ´ Quebec - Canada, J8X 3X7 ´ email@example.com firstname.lastname@example.orgABSTRACT visual features like color, shape and texture. Given a largeColor analysis is frequently used in image/video retrieval. range of images such as landscape, satellite, and medical im-However, many existing color spaces and color distances fail ages, human visual system has the capacity to distinguish,to correctly capture color diﬀerences usually perceived by recognize and interpret diﬀerent types of objects in images.the human eye. The objective of this paper is to ﬁrst high- However, computer programs can hardly recognize imagelight the limitations of existing color spaces and similarity objects even in a simple scene. In image processing andmeasures in representing human perception of colors, and computer vision, color analysis (e.g., dominant color identiﬁ-then to propose (i) a new perceptual color space model called cation, color-based object detection) is a low-level operationHCL, and (ii) an associated color similarity measure denoted which plays an important role in image/video retrieval.DHCL . Experimental results show that using DHCL on thenew color space HCL leads to a solution very close to hu- A variety of color spaces have been developed for color rep-man perception of colors and hence to a potentially more resentation such as RGB, perceptual color spaces HSL (hue,eﬀective content-based image/video retrieval. Moreover, the saturation, luminance), HSV/HSB (hue, saturation, valueapplication of the similarity measure DHCL to other spaces or brightness) [13, 14] and HSI (hue, saturation, intensity)like HSV leads to a better retrieval eﬀectiveness. as well as perceptually uniform color spaces like L*u*v*, and L*a*b* (luminance L*, chrominance u*, v*, a*, and b*) andA comparison of HCL against L*C*H and CIECAM02 spaces CIECAM02 [7, 15]. We recall that perceptual uniformity inusing color histograms and a similarity distance based on a given color space means that the perceptual similarity ofDirichlet distribution illustrates the good performance of two colors is measured by the distance between the two colorHCL for a collection of 3500 images of diﬀerent kinds. points. The objective of this paper is to ﬁrst illustrate the limi-Keywords tations of existing color spaces and similarity measures inColor spaces, content-based image retrieval, similarity mea- representing human perception of colors, and then to pro-sures. pose (i) a new color space model which aims at capturing the real color diﬀerence as perceived by human eye, and1. INTRODUCTION (ii) a new color similarity measure. The proposed space isChallenges in content-based image retrieval (CBIR) consist inspired from HSV (or HSL) and L*a*b*.not only to bridge the semantic gap (i.e., the mismatch be-tween the capabilities of CBIR techniques and the semantic The paper is organized as follows. Section 2 is a brief de-needs of the users) but also to exploit diﬀerent models of hu- scription of color spaces, their strengths and limitations.man image perception, and manage large image collections Section 3 presents a new color space called HCL while Sec-and incomplete query/image speciﬁcations . The human tion 4 presents a set of existing color distances, proposesvisual system does not perceive a given image as a mere a new similarity measure and provides a performance anal-and aleatory collection of colors and pixels, but rather as a ysis of color distances applied to a set of color spaces. Alayout of homogeneous objects and regions with respect to conclusion is given in Section 5. 2. COLOR SPACES The most commonly used and popular color space is RGB. However, this space presents some limitations: (i) the pres- ence of a negative part in the spectra, which does not allow the representation of certain colors by a superposition of the three spectra, (ii) the diﬃculty to determine color features like the presence or the absence of a given color, and (iii) the inability of the Euclidean distance to correctly capture
color diﬀerences in the RGB space. Figure 4 illustrates thelatter fact.Color spaces like HSV and HSL are also commonly used inimage processing. As opposed to the RGB model, HSL andHSV are considered as natural representation color models(i.e., close to the physiological perception of human eye). Inthese models, color is decomposed according to physiologicalcriteria like hue, saturation and luminance. Hue refers tothe pure spectrum colors and corresponds to dominant coloras perceived by a human. Saturation corresponds to therelative purity or the quantity of white light that is mixedwith hue while luminance refers to the amount of light in a (a) (b)color . Figure 1: a) L*a*b* and L*C*H* color space models.A great advantage of HSL/HSV models over the RGB model b) Chroma and Luminance variations for six hue values.lies in their capacity to recognize the presence/absence ofcolors in a given image. However, the main drawback ofHSL and HSV models concerns their luminance variationwhich does not correspond to human perception. Visually, and purple. One can notice that hue angle for blue variesa color with a great amount of white has small variation of between 2570 and 2740 .luminosity than a fully saturated color. Such a situation isnot correctly captured in these models.In the HSV model, saturated colors have the same intensityas colors with 100% of white color. However, this is not thecase for the HSL model since there is a great luminosity gapbetween saturated colors and colors with a great amount ofwhite. Therefore, using metric distances such as Euclidean(see Equation 6) and cylindric distances (see Equation 10)with HSV and HSL models does not capture the color dif-ference as human eye does.The CIE (Commission Internationale de l’Eclairage) has de-ﬁned two perceptually uniform or approximately-uniformcolor spaces L∗ a∗ b∗ and L∗ u∗ v ∗ . Further, the L∗ C ∗ H ∗ (a) (b)(Lightness, Chroma, and Hue) and L∗ t∗ θ∗ (t = Chromaand θ∗ = Hue) color spaces have been deﬁned as derivatives Figure 2: a) CIECAM02 color space model. b) Chromaof L∗ u∗ v ∗ and L∗ a∗ b∗ . The L*a*b* and L*C*H* color and luminance variations for six hue values.models are represented in Figure 1. Figure 1-a shows colordistribution in these models while Figure 1-b illustrates thevariation of chroma C ∗ et luminance L∗ for six diﬀerent 3. A NEW COLOR SPACEhue values H ∗ (red, yellow, green, cyan, blue and purple). While in  we propose new similarity semi-metric distancesOne can see that the luminosity of a hue (respectively the based on color histograms, the present paper investigateschroma) grows (respectively decreases) slowly according to color pixel similarity analysis on a new perceptually uni-the increase in the percentage of white. This variation cor- form color space that we call HCL (Hue, Chroma and Lu-responds to human perception and hence represents a good minance). Such a new color space exploits the advantagesfeature in L*a*b* and L*C*H* color models. of each one of the color spaces: HSL/HSV and L∗ a∗ b∗ and discards their drawbacks.As pointed out by [7, 8], the spaces L*a*b* and L*C*H*have a signiﬁcant deﬁciency since they have weak hue con- We assume that the chroma and the hue of any color can bestancy for blues as illustrated by Figure 1-a) which shows deﬁned as a blend of the three chrominance elemental sensa-that the blue hue angle varies between 2900 to 3060 . Hue tions: R-G (from red to green), G-B (from green to blue) andconstancy means that a color object created by varying the B-R (from blue to red). Based on this assumption and theencoding values to obtain diﬀerent sensations in lightness or Munsell color system with the three color attributes closedchroma should still lead to the same hue over the entire ob- to human perceptions: hue (H), chroma (C) and luminanceject. Moreover, simple nonlinear channel editing should not (L), we deﬁne below a mapping from RGB space to HCLhave an impact on the hue of a color. In order to get such space.constancy, another color space called “CIE Color appear-ance model” (CIECAM02) has been proposed in . How- We recall that a color containing a lot of white is brighterever, CIECAM02 improves hue constancy for almost all col- than one with less white. A saturated color contains 0% ofors except the blue as illustrated in Figure 2-b which shows white and has a maximum value of chroma. An increasingthe variation of hue angles for red, yellow, green, cyan, blue value of white leads to a decreasing value of chroma and
a less saturated color. Concretely, a color is saturated if orM ax(R, G, B) is equal to R, G, or B, and M in(R, G, B) = 0. if ((R − G) ≥ 0 and (G − B) ≥ 0), then H = 2 H 3The saturation of a color is null (i.e., chroma =0) when if ((R − G) ≥ 0 and (G − B) < 0), then H = 4 H 3M in(R, G, B) = M ax(R, G, B). Therefore, we will use the if ((R − G) < 0 and (G − B) ≥ 0), then H = 180 + 4 H 3expressions M ax(R, G.B) and M in(R, G, B) to compute lu- if ((R − G) < 0 and (G − B) < 0), then H = 3 H − 180. 4minance L. (5)Human vision reacts in a non-linear (logarithmic) manner tocolor intensity. For example, a 20% reduction of luminosityis perceived as a 50% reduction. Based on the proportion-ality law of Van Kries, luminance L can be expressed byQ.Y where Y corresponds to the luminosity captured by aphoto-receptor. Color spaces YIQ, YUV, YCrCb, L*u*v*and L*a*b* express Y by Y = 0.299R + 0.587G + 0.114B,while spaces HSI, HSV, and HSL use Y = I = (R+G+B)/3,Y = L = M ax(R, G, B) and Y = L = (M ax(R, G, B) +M in(R, G, B))/2 respectively. (a)We deﬁne luminance L as a linear combination of M ax(R, G, B)and M in(R, G, B) as follows : Q.M ax(R, G, B) + (1 − Q).M in(R, G, B) L= (1) 2where Q = eαγ is a parameter that allows a tuning ofthe variation of luminosity between a saturated hue (color) (b)and a hue containing a great amount of white, with α = M in(R,G,B) 1 . M ax(R,G,B) Y0 and Y0 = 100. γ is a correction factorwhose value (= 3) coincides with the one used in L*a*b*space. It should be noted that when M in(R, G, B) = 0 andM ax(R, G, B) varies between 0 and 255, luminance L takesa value between 0 (black) and 128. When M ax(R, G, B) =255 and M in(R, G, B) varies between 0 and 255, luminancetakes a value between 128 and 135.In a similar way, we deﬁne chroma C = Q.Cn where Cn (c)represents a mixture of three diﬀerent combinations of R,G, and B components: red-green, green-blue and blue-red.The proposed formulae for C (Equation 2) ensures linearitywithin lines/planes of hue (see Figure 3-d). Q. R − G| + |G − B| + |B − R| C= (2) 3 (d)The hue value can be computed using the following equation: Figure 3: a) and c) HCL color space model with H com- puted using Equations 4 and 5 respectively. b) and d) G−B Variation of chroma C and luminance L for six diﬀerent H = arctan (3) R−G hue values.However, hue values (Equation 3) vary between −900 and+900 only. To allow hue values to vary in a larger interval Figure 3 shows the HCL color model where Figures 3-a andgoing from −1800 to 1800 we propose the following alternate 3-c are obtained using formula L, C as well as H computedformula (see ﬁgures 3-a and 3-c): using Equations 4 and 5 respectively. We can notice that the two variants of the HCL model (according to the two ways the hue H is computed) have a uniform hue angle. The chroma C decreases while the luminance L increases if ((R − G) < 0 and (G − B) ≥ 0), then H = 180 + H according to an increase of the white color. In Figure 3- if ((R − G) < 0 and (G − B) < 0), then H = H − 180 . b, the following colors: red, yellow, green, cyan, blue and (4) purple have a unique angle whose value is 00 , 900 , 1350 ,
1800 , 2700 and 3150 respectively. In Figure 3-d, the angle G=255, B=0). This reference color appears on the leftmostis 00 , 600 , 1200 , 1800 , 2400 et 3000 respectively. Such result top cell of each ﬁgure. The most similar colors returned byshows that HCL model oﬀers a better hue constancy than the selected distances (e.g., Euclidean, E94 , Dcyl ) are dis-L*C*H et CIECAM02 models. played in a decreasing order of similarity from left to right and top to bottom. Figures 4 to 6 give the sequences of4. COLOR SIMILARITY MEASURES colors returned by the Euclidean distance applied to RGB,The notion of uniform color perception is an important cri- L*a*b* and L*C*H* respectively. Figures 7 and 9 show theterion for classiﬁcation and discrimination between color list of colors returned by the application of E94 to thespaces. In order to capture perceptual uniformity in a color L*C*H* and CIECAM02 spaces. Figures 8 and 10 showrepresentation space, it is crucial to rely on the distance cri- the list of colors returned by the application of E00 to theterion which states that the distance D(c1 , c2 ) between two L*C*H* and CIECAM02 spaces while Figures 11 and 12 ex-colors c1 et c2 is correct if and only if the distance value is hibit the colors returned by the cylindric distance applied toclose to the diﬀerence perceived by the human eye . HSV and HCL respectively.Many distances have been proposed based on the existing From these ﬁgures, one can see that the application of thecolor models. The Euclidean distance (denoted by E) Euclidean distance to L*a*b* and L*C*H* spaces providesis frequently used in cubic representation spaces such as the worst answers, i.e., most of the returned colors are notRGB and L*a*b* and occasionally in cylindric spaces like close to the target color. Such a distance is appropriate toL*C*H* (see Equations 6 to 8). Another Euclidean-like dis- the RGB space, but is far from being uniform like humantance (Equation 9) was intensionally proposed for L*C*H perception. However, using the E94 and E00 distances. In Equation 10, a cylindric distance (denoted by Dcyl ) for color spaces like L*C*H* and CIECAM02 and the cylin- is used for cylindric and conic spaces like HSL, HSV and dric distance for color spaces like HSV and HCL oﬀers goodL*C*H*. Recently, another formulae for computing color results with a slight superiority of the HCL space (see Fig-diﬀerence (denoted by E00 in Equation 11) has been pro- ure 12) we deﬁned in this paper. However, all the providedposed in . results are not completely compatible with human percep- tion. ERGB = R2 + G2 + B2 (6) 4.1 A New Color Similarity Measure In the following we deﬁne a new color similarity measure called DHCL and based on the cylindric model with param- Eab = L∗ 2 + a∗2 + b∗ 2 (7) eters AL and ACH . This measure is particularly adapted to the new color space deﬁned in this paper. ECH = L∗ 2 + C∗2 + H ∗2 (8) DHCL = (AL L)2 + AH (C1 2 + C2 2 − 2C1 C2 cos( H)) (12) L∗ 2 C∗ 2 H∗ 2 E94 = + + (9) kL SL kC SC kH SH where AL is a constant of linearization for luminance from √ the conic color model to the cylindric model, and AH is awhere kL = kC = kH = 1, SL = 1, SC √ = 0.045 C1 C2 +1 and SH = 0.015 C1 C2 + 1 parameter which helps reduce the distance between colors having a same hue as the hue in the target (reference) color. Dcyl = L∗ 2 + C ∗ 1 2 + C ∗ 2 2 − 2C ∗ 1 C ∗ 2 cos( H) (10) In order to determine these two parameters, we consider a slice of the HCL model. For example, let us take a refer- ence pixel Pr of saturated purple (see Figure 3). We can see that a pixel Pa with the same hue ( H = 0) and the same L∗ 2 C∗ 2 H∗ 2 E00 = + + + R (11) luminance ( L = 0) with a diﬀerence in chroma equal to kL SL kC SC kH SH C = 50 is more similar to pixel Pr than pixel Pb havingWe have conducted an experimental study to ﬁrst analyze L = 0, C = 0 and H close to 80. Then, we can deter-the compatibility between these distances and the color spaces mine ACH as ACH = H + 8/50 = H + 0.16. Moreover,HSV, L*C*H* and CIECAM02, and then contrast these dis- the pixel Pb is more similar to pixel Pr than the pixel Pc hav-tances against human perception. To that end, we have se- ing H = 0 and C = 50, and being darker ( L = 37).lected ten diﬀerent colors as reference (target) colors. Each However, the pixel Pd with H = 0, C = 50 and a greaterone of them is compared to a collection of randomly gener- luminance ( L = 25) is more similar to pixel Pr than pixelated colors using each one of the proposed similarity mea- Pb does. Due to this luminance eﬀect, we proceed to a tri-sures. Colors are generated automatically by a variation of angulation computation which leads to a correction factorR, G and B values (0 ≤ R, G, B ≤ 255) using an increment equal to AL = 1.4456.equal to 15. This leads to a set of 4913 colors for each colorspace. Figure 13 illustrates the output provided by the new simi- larity measure DHCL when it is applied to the HCL colorTo illustrate the potential of the new color space HCL de- space. One can notice that the returned colors are closer toﬁned earlier, Figures 4 through 12 show an experimental the reference color (leftmost top cell) than those obtainedcase using a fully saturated and pure yellow color (R=255, using existing color distances and spaces (see Figures 4 to
Figure 10: Distance E00 applied to CIECAM02 space. Figure 4: Euclidean distance applied to RGB space. Figure 11: Cylindric distance Dcyl applied to HSV space.Figure 5: Euclidean distance applied to L*a*b* space. Figure 12: Cylindric distance Dcyl applied to HCLFigure 6: Euclidean distance applied to L*C*H* space. space. 11) or using Dcyl with the new HCL color space (see Figure 12). Experimental results on reference colors other than yel- low conﬁrm that the application of the new color distance DHCL to the new color space HCL leads to a better per- ceptual uniformity than HSV, HSL, L*a*b* et L*C*H* for which existing distances are used (see Equations 6 to 10). Figure 7: Distance E94 applied to L*C*H* space. Figure 13: New distance DHCL applied to HCL space. Figure 8: Distance E00 applied to L*C*H* space. 4.2 Empirical Analysis In order to compare the sequence of colors returned by the computer system (according to diﬀerent color spaces and distances) with the list returned by the human system, seven subjects were asked to evaluate the output. For each one of the ten cases (see Figures 4 to 13) corresponding to pairs of a given color space and a color distance, there are 48 cells: the reference color cell (leftmost top cell) and 47 (returned)Figure 9: Distance E94 applied to CIECAM02 space. color cells. Every subject has to choose and rank the top ten colors that are most similar to the reference color. If less than ten colors are selected by a subject for a given combina- tion of color distance and space (e.g., Euclidean distance and
RGB), then the rank of missing colors is given the value 48. HCL outperforms the other combinations of color distancesAt the end of the experimentation, all subjects concluded and spaces. The pair E00 and CIECAM02 provides goodthat using DHCL on HCL leads to better results than the results for yellow and green but the worst eﬀectiveness ratioother combinations of distance and space. Indeed, the com- for the three other colors. The pair E94 and L*C*H* givesbination of DHCL and HCL returns much more colors that the worst retrieval eﬀectiveness for all the selected colors.are similar to the reference color than any one of the othercombinations. Moreover, we conducted additional empirical studies to com- pare the proposed color space HCL against L*C*H* andFigure 14 exhibits ﬁve rows corresponding to diﬀerent colors. CIECAM02 on an image data set of 3500 images repre-The ﬁrst cell in each row identiﬁes the reference color (red, senting photographs et paintings of small, medium or highyellow, green, blue and purple) while the remaining cells resolution. This includes 500 images from the database ofhave a rank from 1 to 12 where rank 1 corresponds to the the Info-Muse network  containing museum collections incolor which is the most similar to the reference color. The Qu´bec (Canada) as well as images from diﬀerent web sites eranking is computed as the mean of the judgment of seven . The ﬁrst set contains art images related to paintings,subjects, three of them are experts in image processing. statues, medals and ancient clothing items. The whole col- lection is grouped under four overlapping semantic classes: painting, close-up, indoor and outdoor images. Each class (e.g., Outdoor) is further split into subgroups (e.g., city, landscape, etc.).Figure 14: Five reference colors with the average rank-ing of similar colors (from 1 to 12). Figure 15: Ranking according to eight pairs of distances and color spaces.Figure 15 provides the ranking for the purple color. The ﬁrstrow corresponds to the ranking (from the most similar tothe less similar) using the distance Dcyl and the HCL spacedeﬁned in the paper. The remaining rows give the rankingreturned by the pairs Dcyl and HSV, E and L*a*b*, Eand L*C*H*, E94 and L*C*H*, E00 and L*C*H*, E94and CIECAM02, and E00 with CIECAM02, respectively.To quantify the potential of each distance to return the col-ors that are close to human perception, we have applied thefollowing eﬀectiveness measure (see  for more details). Figure 16: Retrieval eﬀectiveness of six combinations of distances and color spaces. Rc 1 i=1 i Ef fsys = R Rc Rc . (13) Based on our previous work on similarity analysis , the 1 + log( Rc ) i=1 i+ i=1 |i − ri | comparison between two images makes use of color histogramswhere Rc is the total number of relevant colors (according to and a similarity distance involving the Dirichlet distribution.the user’s judgment) in the color set, R is the total number Figures 17 through 19 illustrate the retrieval output pro-of retrieved colors (R ≥ Rc ), i (= 1, 2, · · · , Rc ) is similarity vided by the system when CIECAM02, L*C*H* and HCLimage ranking by human judgment and ri corresponds to color spaces are used, respectively. When an image querysystem image ranking (in a decreasing relevance order). (leftmost top image) is submitted, the system returns im- ages in a decreasing order of similarity. A careful look atThe curves in Figure 16 illustrate the retrieval eﬀectiveness the three ﬁgures indicates that HCL outperforms the tworatio of color distance and space combinations pour ﬁve ref- other spaces. For example, one can see that the ﬁrst twoerence colors where the ordinate represents the average ef- rows in Figure 19 contain images with colors closer to thosefectiveness computed from the judgment of seven subjects. in the image query than images in the same rows of FiguresOne can see that the combination of DHCL and color space 17 (CIECAM02) and 18 (L*C*H*).
5. CONCLUSION In order to overcome the limitations of existing color spaces and color distances in correctly capturing color diﬀerences perceived by the human system, we have presented a new color space called HCL inspired from HSL/HSV and L*a*b* spaces as well as a new similarity measure labelled DHCL and tailored to the HCL space. Experimental results show that using DHCL on HCL leads to a solution very close to human perception of colors and hence to a potentially more eﬀective content-based image/video retrieval. We are currently studying the potential of our ﬁndings in three ﬁelds of image/video processing, namely : image seg- mentation, object edge extraction, and content-based image (or sub-image) retrieval. Acknowledgments The authors would like to thank the anonymous reviewers for their valuable comments and suggestions for improve-Figure 17: Image retrieval using CIECAM02 color ment. This work is part of CoRIMedia research projects thatspace. are ﬁnancially supported by Valorisation Recherche Qu´bec, e Canadian Heritage and Canada Foundation for Innovation.Figure 18: Image retrieval using L*C*H* color space. Figure 19: Image retrieval using HCL color space.
6. REFERENCES  D. Alman. Industrial color diﬀerence evaluation. Color Research and Application, no.3:137–139, 1993.  R. C. Gonzalez and R. E. Woods. Digital Image Processing. Prentice Hall, second edition, 2002.  B. Hill, T. Roger, and F. Vorhagen. Comparative analysis of the quantization of color spaces on the basis of the cielab color-diﬀerence formula. ACM Trans. on Graphics, 16:109–154, April 1997.  N. Info-Muse. Soci´t´ des Mus´es Qu´becois (SMQ); ee e e (http://www.smq.qc.ca/publicsspec/smq/services /infomuse/index.phtml). 2004.  M. R. Luo, G. Cui, and B. Rigg. The developpement of the cie 2000 colour diﬀerence formula: Ciede2000. COLOR Research and Application, 26:340–350, 2001.  R. Missaoui, M. Sarifuddin, and J. Vaillancourt. An eﬀective approach towards content-based image retrieval. In Proceedings of the International Conference on Image and Video Retrieval (CIVR 2004), Dublin, Ireland, pages 335–343, July 2004.  N. Moroney. The ciecam02 color appearance model. In Proceedings of the the Tenth Color Imaging Conference: Color Science, System and Application, pages 23–27, 2002.  N. Moroney. A hypothesis regarding the poor blue constancy of cielab. Color Research and application, 28, no.3:371–378, 2003.  G. Paschos. Perceptually uniform color spaces for color texture analysis: An exeprimental evaluation. IEEE Trans. on Image Processing, 10, no.6:932–937, 2001. K. Plataniotis and A. Venetsanopoulos. Color image processing and applications. Springer, Ch. 1, pp 268-269, 2000. W. sites. http://www.hemera.com, http://www.corbis.com; http://www.webshots.com; http://www.freefoto.com. 2004. A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349–1380, 2000. A. R. Smith. Color gamut transform pairs. Computer Graphics, 12, no.3:12–19, 1978. J. R. Smith. Integrated spatial and feature image system: retrieval, compression and analysis. In Ph.D. dissertation, Colombia Univ. New York, 1997. G. Wyszecki and W. S. Stiles. Color Science: Concepts and Methods, Quantitative Data and Formulae. John Wiley and Sons, second edition, 1982.