The document discusses content-based image retrieval (CBIR) systems. It describes how CBIR systems use feature extraction to search large image databases based on visual content. The key components of CBIR systems are feature extraction, indexing, and system design. Feature extraction involves extracting information about images' colors, textures, shapes, and spatial locations. Effective features and indexing techniques are needed to make CBIR scalable for large image collections. Performance is evaluated based on how well systems return relevant images.

1. Evaluation of Content-Based Image Retrieval Systems
Based on Feature Extraction
Presented by
Fouzia Ashraf Khan

2. Motivation
 Exponential increase in computing power and electronic storage capacity
 Exponential increase in digital image/video database sizes
 Increase use of image and video:
Entertainment
Education
Commercial purposes
 Need abstractions for efficient and effective modeling
 Continuous data, multiple features and analysis of multi-layer
or multivariate data processing is difficult job.

3. Spatial Database
 Spatial Data
 Displayed, manipulated and analyzed by means of spatial attribute
which denotes a location on or near the surface of earth
Spatial attribute is presented in the form of co-ordinate pairs that
allow position and shape of a particular feature to be measured and
represented graphically.
• Spatial Database
A spatial database is a database that is optimized to store and query
data related to objects in space, including points, lines and polygons
Typical databases understand various numeric and character data
Need additional functionality spatial data types - called geometry or
feature.

4. CBIR
• Content-Based Image Retrieval
- a technique that utilizes the visual content of an image, to
search for similar images in large-scale image databases,
according to a user’s interest
- is a robust system based upon a combination of higher-
level and lower-level vision principles.
- Higher-level analysis uses perceptual organization,
inference and grouping principles to extract semantic
information
- Lower-level analysis employs a channel energy model to
describe image texture, shape and utilizes color histogram
techniques

5. CBIR System Components
 Three Major Components
 Feature Extraction
 Indexing
 System Design
Challenge:
Gap between low-level features and high level user
semantics

6. Feature Extraction
 Feature
A cluster of points or a boundary /region of pixels
 Primary Features
◦ Color
◦ Texture
◦ Shape
◦ Spatial location
• Feature extraction
Process of studying and locating areas and objects on the ground and deriving
useful information from images.
 Feature Extraction Methods
◦ Fuzzy approach
◦ Relevance feedback (supervised learning)
Example of a Classified Image

7. Color Based Feature Extraction
Approaches
 Conventional Color Histogram (CCH)
 Frequency of occurrence of every color in image.
 Refers probability mass function
 Captures the joint probabilities of intensities of color channels.
 hA,B,C(a,b,c) = N. Prob(A=a, B=b, C=c), where A, B and C are the three color channels and
N is the number of pixels in the image
 Fuzzy Color Histogram (FCH)
 Considers degree of color similarity between pixels
 A color space with K color bins, FCH of I is defined as
F(I)=[f1,f2,…fk] where fi = and
where N = # of pixels
μij = Membership value of jth pixel to ith color bin
dij = Euclidean distance between the color of pixel jth and the ith color bin,
ς = Average distance between colors in quantized color space

N
i
ij
N
)(
1


 dijxfij


1
1
)(

8. Color Based Feature Extraction
Approaches
 Color Correlogram (CC)
 Express how spatial correlation of pairs of colors changes with distance.
 A table indexed by color pairs
 For dth entry at location (i,j) is computed by
- Counting number of pixels of color j at a distance d from a pixel of color i,
divided by the total number of pixels in the image
 Color-Shape Based Method
 Includes area and shape information
 Area of each object is encoded as the number of pixels.
 Shape is characterized by “perimeter intercepted lengths”
 PIL - obtained by intercepting object perimeter with 8 line segments having 8
different orientations and passing through object center.

9. Comparative Study of Color Based
Feature Extraction

10. Texture Based Feature Extraction
Approaches
 Steerable Pyramid
◦ Basic filters are translation and rotation of a single function
◦ Filter is linear combination of basis functions
◦ Only for rotation-invariant texture retrieval
 Contourlet Transform
◦ Combination of a Laplacian pyramid and directional filter bank
◦ LP provides the multi‐scale decomposition,
◦ DFB provides the multi‐directional decomposition
 Gabor Wavelet
◦ Optimally achieves joint resolution in space and spatial frequency
◦ Transform dilates and rotates the two‐dimensional Gabor function.
◦ Image is convolved with each of the obtained Gabor functions
◦ Gabor function works in Fourier domain,
◦ Computationally intensive
 Complex Directional Filter Bank (CDFB)
◦ Retrieval results comparable with Gabor wavelet results
◦ Shift Invariant

11. Comparative Study of Texture
Based Feature Extraction

12. Shape Based Feature
Extraction Approaches
 Blob Detection
 Detecting points , regions are either brighter or darker than surrounding
 Two classes of blob detectors
- Differential method based on derivative expressions
- Methods based on local extrema of intensity
• Thresholding
 Classify pixels into two categories:
– Those to which some property measured from the image falls below a threshold,
and those at which the property equals or exceeds a threshold.
– Thresholding creates a binary image : binarization
- Choosing a threshold is a critical task.

13.  Hough Transform
- Find imperfect instances of objects within a certain class
of shapes by a voting procedure
- Voting procedure is carried out in a parameter space
- Object candidates are obtained as local maxima
 Template Matching
-For finding small parts of an image which match a template
-To detect edges in images.
-Simple to implement and understand, it is one of the slowest
methods.

14. High Dimensional Indexing
Techniques
 To make CBIR truly scalable to large size image collections
 Some Multi-dimensional Indexing Methods
 Self-Organizing Maps
- SOM are feed-forward, competitive artificial neural networks.
- Reduces complexity (both time and computation)
 Shape-Depth Representation
-Well-defined geometric objects, a novel strategy based on a 3D embedding
-Is between contour-based and region-based representations
-A very useful one for retrieving objects/images
 Fourier Transform of Shape-Depth and Indexing
- Invariant to transformations such as translation, scaling and rotation.
- Fourier transform is widely used for achieving the invariance

15. System Design
 Some factors of Design
- Random browsing
- Search by example
- Search by sketch
- Search by text (including keyword or speech) and navigation with
customized image categories.
Some Current Developed CBIR Systems
QBIC - Query By Image Content
Virage
RetrievalWare
VisualSEEk and WebSEEk
Netra

16. Performance Evaluation of CBIR System
Performance
 Observed Probability
 Distance Measure and Retrieval Score
 measure
- user is facing a target search task from a database D for an image I
belonging to class C D.
- Resembles the “target testing" method
- Obtained by implementing an “ideal screener“
- A computer program which simulates the human user by examining the
output of the retrieval system
- Marking the images returned by the system either as relevant (positive) or
non-relevant (negative).



17. Important Points
 CBIR has become a very active area research for two major research
communities, Database Management and Computer Vision
 Feature Extraction methods are easy, effective and less expensive
 FCH and Gabor wavelet transform are found to yield the highest color and
texture retrieval results, respectively, at the cost of higher computational
complexity.
 To make the content-based Image Retrieval truly scalable to large size
image collections, multi-dimensional indexing techniques are highly
essential.