This project was my undergrad final year project in which was taken from my internship at IIIT Ahmedabad, India. Little to know CBIR now being utilized everywhere in the image retrieval world. Google images do a great job of recognizing color palates.

1. CONTENT BASED
IMAGE
RETRIVAL TECHNIQUE
( CBIR )
Presented by:-
MAYANK RAJ
4TH yr. computer science
R.K.D.F.I.S.T, Bhopal

2. SEARCHING OF IMAGES
TEXT BASED CONTENT BASED
( IMAGE BASED )
SCENARY
SCENARY
S. K. Chang, and A. Hsu, "Image information systems: where do we go from here?" IEEE Trans. on
Knowledge and Data Engineering, Vol.5, No.5, pp. 431-442, Oct.1992

3. Query
Formation
Visual Content
Description
Feature
Vectors
Relevance
Feedback
Image
Database
Visual Content
Description
Feature
Database
Similarity
Comparison
Indexing &
Retrieval
Retrieval Results
HOW CBIR WORKS ?
user
Output
Global Local
FUNDAMENTALS OF
CONTENT-BASED
IMAGE RETRIEVAL
Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng
Page number 3 , figure 1.1

4. The chosen color space
is divided into `n` bins
For each image, a histogram is built
for each image by counting
the number of pixels
classified into each bin
The histogram becomes the feature
vector/descriptor of the image
During retrieval, the images are retrieved
and ranked according to the histogram
distances between the
query image and images in databases
Manhanttan
distance (L1)
Euclidean
distance (L2).

5. Image Content
Visual Semantic
Text Based
General Visual
Content
Domain Specific
Visual
Content
FUNDAMENTALS OF
CONTENT-BASED
IMAGE RETRIEVAL
Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng
Page number 3 , section 1.2

6. Visual Content Descriptor
Global Local
Visual Features
OF
Whole Image
Visual Features
OF
Regions & Objects
FUNDAMENTALS OF
CONTENT-BASED
IMAGE RETRIEVAL
Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng
Page number 3 , section 1.2

7. Presentation - Outline
 Introduction
 What is CBIR?
 Applications of CBIR
 Our Approach
Colour
Texture
Shape
Where We Are
Conclusion
Questions and Answers

8. Introduction - What is CBIR?
 The term [CBIR] describes the process of retrieving desired images
from a large collection on the basis of features (such as colour,
texture and shape) that can be automatically extracted from the
images themselves.
 Content-based image retrieval (CBIR), also known as query by
image content (QBIC) and content-based visual information
retrieval (CBVIR) is the application of computer vision to the image
retrieval problem, that is, the problem of searching for digital images
in large databases.
FUNDAMENTALS OF
CONTENT-BASED
IMAGE RETRIEVAL
Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng
Page number 1 , section 1.1 ( introduction )

9. Introduction - Reasons for its development
 In many current applications with large
image databases, traditional methods of
image indexing have proven to be
insufficient.
For example;
Finger print scanning
cannot be done using a
keyword search.
References : - www.google.com & www.wikipedia.com

10. Introduction - Applications
 Automatic face recognition systems
References : - www.google.com & www.wikipedia.com

11. Introduction - Applications
 Medical Image Databases
References : - www.google.com & www.wikipedia.com

12. Introduction - Applications
 Trademark Image Registration
References : - www.google.com & www.wikipedia.com

13. Our Approach - Image Features
 The image features that we will be
focusing on, for image retrieval are:
•Colour
•Texture
•Shape
•Spatial location
•Pixel intensity
•Sharpness
Other primitive features not considered are:
Future scopes

14. Our Approach - Colour
•J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes, Computer graphics: principles and
practice, 2nd ed., Reading, Mass, Addison-Wesley, 1990.
•J. Huang, S.R. Kumar, M. Metra, W. J., Zhu, and R. Zabith, "Spatial color indexing and
applications," Int’l J. Computer Vision, Vol.35, No.3, pp. 245-268, 1999.
•J. Huang, et al., "Image indexing using color correlogram," IEEE Int. Conf. on Computer Vision
and Pattern Recognition, pp. 762-768, Puerto Rico, June 1997.
References

15. Color moments have been successfully used in many retrieval
systems ( like QBIC ), especially when the images contains just the
object.
First Order :- MEAN
µ = 1/N ∑ Fij
Second Order :- VARIANCE
σ = ( 1/N ∑ ( f - µ ) )
N
j = 1
2 1/2
N
j = 1
i
i
i
i j
FUNDAMENTALS OF
CONTENT-BASED
IMAGE RETRIEVAL
Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng
Page number 5 , section 1.2

16. Third Order : - SKEWNESS
S = ( 1/N ∑ ( f - µ ) )
i i
j = 1
N
i j
3
1/3
Where:-
f i j = value of the ith color component of the image pixel j, and N is the
number of pixels in the image
FUNDAMENTALS OF
CONTENT-BASED
IMAGE RETRIEVAL
Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng
Page number 5 , section 1.2

17. Effective
representation of the
color content
Easy to compute
Effective in
Characterizing
( local & global )
In addition
It is robust to translation and rotation about the view axis and changes only with
the scale , occlusion and viewing angle.

19. Distribution of number of
pixels for each quantized
bin, can be defined for each
component.
Def.
E.g..
Courtesy:-
http://www.isixsigma.com/library/graphi
cs/histogram.gif
Courtesy:-
http://www.cee.hw.ac.uk/hipr/images/w
om2hst1.gif

20. Our Approach - Texture
o Texture is that innate property of all surfaces that describes visual
patters, and that contain important information about the structural
arrangement of the surface and its relationship to the surrounding
environment.
o An attribute representing the spatial arrangement of the gray levels
of the pixels in a region ( ref.: - www.cs.jhu.edu/.../ week9.2 )
What is Texture?

21. Our Approach - Texture
 Exam
ples:
Brick
Texture
Finger
print
Texture
Clouds
Texture
Rocks
Texture

22. TEXTURE REPRESENTATION
METHODS
Structural Statistical
Morphological
operator
Adjacency
graph
For regular
patterns
For irregular
patterns
Fourier power spectra
Co-occurrence matrices
Shift invariant principal
Tamura feature
Wold decomposition Etc…

23. References :
 P. Brodatz, "Textures: A photographic album for artists & designers," Dover, NY,
1966.
 T. Chang, and C.C.J. Kuo, "Texture analysis and classification with tree-structured
wavelet transform," IEEE Trans. on Image Processing, vol. 2, no. 4, pp. 429-441,
October 1993.
 J. M. Francos. "Orthogonal decompositions of 2D random fields and their
applications in 2D spectral estimation," N. K. Bose and C. R. Rao, editors, Signal
Processing and its Application, pp.20-227. North Holland, 1993.
 J. M. Francos, A. Narasimhan, and J. W. Woods, "Maximum likelihood parameter
estimation of textures using a Wold-decomposition based model," IEEE Trans. on
Image Processing, pp.1655-1666, Dec.1995.

24. Our Approach - Texture Properties
 Co-occurrence matrix:
o Based on the orientation and distance between
image pixels.
o From it we obtain statistics that represent:
•Coarseness
•Contrast
•Directionality
•Linelikeness
•Regularity
•Roughness
Texture
properties

25. Co occurrence matrix :
<mathematics> Given a position operator P(i,j), let A be a
nxn matrix whose element A[i][j] is the number of times that
points with grey level (intensity) g[i] occur, in the position
specified by P, relative to points with grey level g[j]. Let C be the
nxn matrix that is produced by dividing A with the total number
of point pairs that satisfy P. C[i][j] is a measure of the joint
probability that a pair of points satisfying P will have values g[i],
g[j]. C is called a cooccurrence matrix defined by P. Examples
for the operator P are: "i above j", "i one position to the right and
two below j", etc.

26. Our Approach - Shape
o Shape is the characteristic surface configuration that
outlines an object giving it a definite distinctive form.
o Fairly well-defined concept.
What is Shape?

27. Our Approach - Shape
 Examples:

28. Our Approach - Shape Features
 Aspect ratio
 Circularity
 Moment invariants
 Sets of consecutive
boundary segments

29. Our Approach - Shape Extraction
 Techniques under consideration:
o Fourier Descriptor
o Moment Invariants
o Directional Histograms

30. Where We Are ???
Courtesy:-
http://www.cee.hw.ac.uk/hipr/images/w
om2hst1.gif

31. Image Database

32. The advantage of content based image retrieval techniques is that they
can accept image queries and capture some features (such as some
irregular shapes and texture) difficult to describe using text.
The disadvantage is that they cannot capture high level semantic
concepts contained in the image.
Ref.:--http://personal.gscit.monash.edu.au/~dengs/resource/Project.html

33. Conclusion
o What is CBIR?
– The retrieval of images from a database based on
content features such as colour, texture and
shape.
o Reasons for its developments
– Insufficiency in certain applications

34. Conclusion
o Applications
– Finger print scanning systems
– Automatic face recognition systems
– Medical image databases
– Trademark image registration

35. Conclusion
o Our Approach
– Colour
– Texture
– Shape
o Where we are
– In the phase of understanding and implementing
shape.

36.  2D transformations can be converted into 3D
transformations
 3D based shape indexing technique
 Combining visual feature with semantics for a more
effective image retrieval
 Mapping Low-Level Features to High-Level Semantic
Concepts in Region-Based Image Retrieval
Cont…

37.  Other primitive features not considered are:
Spatial location
Pixel intensity
Sharpness
The proposed system attempts to narrow the gap
between content-based image retrieval and semantic-
based image retrieval

38. THANK YOU