This document discusses improving content-based image retrieval through the selection of compact composite descriptors (CCDs). It proposes combining the CEDD and FCTH descriptors into a joint composite descriptor (JCD) and introduces an automatic descriptor selection (ADS) method to choose the most appropriate descriptor for each image pair. Experiments show that JCD and ADS improve retrieval performance over individual descriptors on various datasets, demonstrating the benefit of composite descriptors and adaptive selection for content-based image retrieval.

1. SELECTION OF THE PROPER COMPACT
COMPOSITE DESCRIPTOR FOR
IMPROVING CONTENT BASED IMAGE
RETRIEVAL Presenter: Savvas A. Chatzichristofis
Savvas Chatzichristofis, Mathias Lux and Yiannis Boutalis
Department of Electrical & Computer Engineering Democritus University of
Thrace – Greece
Institute of Information Technology ‐ Klagenfurt University
Klagenfurt, Austria
Signal Processing, Pattern Recognition and Applications SPPRA 2009

2. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
• Compact Composite Descriptors (CCD) are global image descriptors
capturing more than one feature at the same time, in a very
compact representation.
Natural Images Artificial Images Medical Images
SpCL
CEDD BTDH
FCTH

3. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Overview
• In this paper we propose a combination of two
recently introduced CCDs (CEDD and FCTH) into a
Joint Composite Descriptor (JCD).
• We further present a method for auto descriptor
selection.
• Similar techniques were applied to select the most
appropriate MPEG-7 descriptor, by extracting
information from all the images of a dataset.

4. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors
• The CEDD length is 54 bytes per image while FCTH
length is 72 bytes per image.
• The structure of these descriptors consists of n
texture areas. In particular, each texture area is
separated into 24 sub regions, with each sub region
describing a color.
• CEDD and FCTH use the same color information, as
it results from 2 fuzzy systems that map the colors of
the image in a 24-color custom palette.

5. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors

6. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Both Directions
-
7
High Energy
Vertical High
-
6
Energy
CEDD and FCTH Descriptors
135 Degree Horizontal
5
Diagonal High Energy
45 Degree Linear
4
Diagonal High Energy
Vertical Both Directions
3
Activation Low Energy
Horizontal Vertical Low
2
Activation Energy
Non Horizontal Low
1
Directional Energy
Linear Low
Linear
0
Energy
CEDD
FCTH

7. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
CEDD and FCTH Descriptors
WANG UCID NISTER
CCD
CEDD 0.25283 0.28234 0.11297
FCTH 0.27369 0.28737 0.09463
MPEG-7
DCD MPHSM 0.39460 - -
DCD QHDM 0.54680 - -
SCD 0.35520 0.46665 0.36365
CLD 0.40000 0.43216 0.2292
CSD 0.32460 - -
EHD 0.50890 0.46061 0.3332
HTD 0.70540 - -

8. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Joint Composite Descriptor (JCD)
• Based on the fact that the color information
given by the 2 descriptors comes from the same
fuzzy system, we can assume that joining the
descriptors will result in the combining of
texture areas carried by each descriptor.
• JCD is made up of 7 texture areas, with each
area made up of 24 sub regions that correspond
to color areas.

9. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Joint Composite Descriptor (JCD)
• The texture areas are as follows:
▫ JCD(0) Linear Area
▫ JCD(1) Horizontal Activation
▫ JCD(2) 45 Degrees Activation
▫ JCD(3) Vertical Activation
▫ JCD(4) 135 Degrees Activation
▫ JCD(5) Horizontal and Vertical Activation
▫ JCD(6) Non directional Activation

10. SELECTION BASED MEDICAL IMAGE INDEXING AND RETRIEVAL USING
CONTENT OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE DESCRIPTOR
A FUZZY COMPACT COMPOSITE RETRIEVAL
Descriptor Implementation
• We model the problem as follows:
• CEDD and FCTH be available for an image. The
indicator m symbolises the bin of the color of
each descriptor.
m ∈ [0, 23]
• The indicators n and n’ determine the texture
area for the CEDD and FCTH respectively
n ∈ [0,5] n ' ∈ [0, 7]

11. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Descriptor Implementation
• Each descriptor can be described in the
following way:
CEDD( j ) m , FCTH ( j ) m'
n n
CEDD( j )5 = bin(2 × 24 + 5) = bin(53)
4
The algorithm for the Joint Composite Descriptor
can be analysed as follows:

12. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Descriptor Implementation

13. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Auto Descriptor Selection (ADS)
• (i) The descriptor for search is chosen based
on the query image.
• (ii) The most appropriate descriptor is
chosen at similarity assessment time, so
within a single query the chosen descriptor
may be different for different image pairs.

14. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Auto Descriptor Selection (ADS)
• In retrieval scenarios a
combination of different
feature spaces within a
single query is often not
possible.
• Experiments on the Wang
data set have shown that
with normalized similarities
(mean of 0 and standard
derivation of 1) Distribution of (a) CEDD, (b) FCTH and
(c) JCD similarities / Wang 1000 image
distributions are similar database.
enough to be combined.

15. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Auto Descriptor Selection (ADS)
• Given that the color information in all two
descriptors is the same, the factor that will
determine the suitability and capability of each
descriptor is mainly found in the texture
information.
• The system that determines the most appropriate
descriptor is a Mamdani fuzzy system of three
inputs and one fuzzy output. The centroid method
was used to defuzzify the output of the Mamdani
model.

16. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Criterion 1: Maximum amount of
information.
• The first criterion shows
which CCD contains the
largest quantity of
information.

17. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Criterion 2: Percentage of information in
non-uniform texture areas.
• The most appropriate
descriptor is the one
that contains the
smallest percentage of
non uniform image
blocks.

18. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Criterion 3: The percentage of
information in texture areas.
• The third criterion
considers the most
appropriate descriptor to
be the one that has the
smallest percentage of
image blocks present in
linear areas.

19. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Experiments
• The proposed methods have been implemented and are available as
open source libraries under GNU - General public License (GPL) in
the image retrieval system img(Rummager) the on line application
img(Anaktisi) and image retrieval library LIRe.

20. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
•
• CEDD
• FCTH
• JCD
• Ranking
For use of multiple different descriptors within one query, the ADS unit also
needs to normalize the similarities based on their distribution. Based on
experiments we used the normalization values given in paper.

21. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Experiments
• To evaluate the performance of the proposed methods, the objective
measure called ANMRR is used.
WANG UCID NISTER
CEDD 0.25283 0.28234 0.11297
FCTH 0.27369 0.28737 0.09463
JCD 0.25606 0.26832 0.085486
ADS
0.24948 0.27952 0.09291
Based on Query descriptor
ADS
0.24876 0.27722 0.09291
Based on Pair wise descriptor

22. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Conclusions
• JCD and ADS methods are not suggested to improve the
retrieval procedure.
• The goal is to approach the best ANMRR that would
result from CEDD and FCTH.
• Nevertheless, the new JCD shows an increase in retrieval
performance.
• The methods for automatic selection of the most
appropriate descriptor (ASD) for retrieval increases
retrieval performance in all 3 experiments.

23. SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR
FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL
Thank You
Ευχαριστώ Πολύ
Download the img(Rummager) application from
http://www.img-rummager.com