search engine for images

Search Engine for Images
We are given a large database comprising several categories of images like: people, landscape, flowers, buses, trains, food items, animals etc.
There are numerous pictures in each category.

Our Goal
A user can pick any image (not necessarily inside the database) and our program (the search engine) should be able to pull out pictures from the database that are similar to the query image.

Examples of similar images

Our intention is to produce results as close to human perception as possible.
In other words, our search engine automatically categorizes the pictures.

Mathematical Tools Needed
The Wavelet Transform. A considerable amount of Linear algebra (particularly Inner product spaces) has to be covered before we are in a position to describe the wavelet transform.
The K-means clustering algorithm. This is a powerful statistical tool.

I.T. Tools Needed
We chose MATLAB 7.0 for the following reasons:
It handles matrices in a convenient manner
It supports a powerful programming language
It can handle images in almost every format

The Method
The core of this search engine is the Integrated Region Matching Scheme described by Wang, Li, and Wiederhold in the paper:
SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries, IEEE, 2001.
We now briefly outline the method.

A brief outline of the method
Partition a picture into 4x4 blocks and extract feature vectors for each block.
Group the feature vectors into a number of regions (using K-means).
Compute the signature of the image by combining the features of all its regions.
Save this signature to a database.
Repeat the process for every picture in the database.

A brief outline of the method
The feature vectors contain information regarding colour, texture and shape.
The query image given by the user also undergoes the same signature extracting process.
Finally distances between the query signature and each image signature are calculated and sorted to give the closest matches.

In the slides that follow we will make the above ideas precise.

What is a Feature vector?
It is simply a list of 6 numbers calculated for each 4x4 block.
The first three numbers are averages of the Red, Green and Blue components.
The next three numbers are the root mean square of wavelet coefficients arising by applying the wavelet transform once to rows and columns of the block.

Meaning of these components
The first three numbers hold information about the colour.
The next three numbers hold information about the texture.

Image Segmentation
This is what you get when K means is applied to the set of feature vectors for various values of K.
All these pictures (except the original!) were created in MATLAB.

7 regions
4 regions
Original
3 regions
2 regions

Remark
Figuring out an appropriate value of K for any image still remains an open problem in computer vision.
But Wang, Li and Weiderhold propose a scheme that calculates a K that works quite well for this experiment.

Integrated Region Matching
This method combines the properties of all the regions in a picture to measure the overall similarity between images.
The payoff is that the scheme provides robustness against poor segmentation.

Defining a similarity measure is nothing but defining a distance between sets of points (feature vectors) in a higher dimensional space.
The idea of distance must be carefully chosen so that it is consistent with a person’s idea of “closeness” of two images.

Suppose two images A and B are represented by region sets
A=r1,r2,…,rm
and
B=r′1,r′2,…,r′n

Denote by dij the distance between ri and r′j.
Wang et. al. give a simple prescription for this that they found experimentally.
The distance between the images is defined as the weighted sum of region-to-region matches:
dA,B=i,jsijdij

The weights sij are the elements of the significance matrix
s11⋯s1n⋮⋱⋮sm1⋯smn
Wang et.al. have devised an algorithm to calculate the significance matrix.

Meaning of the weights
The weights sij capture the importance of match.
For instance if a region consists of a body of an animal and is matched to various regions in the other picture then body-to-body match will be given more weight than a body-to-background or a body-to-tree match.

Distance between Regions
Two regions r and r′ comprise of several feature vectors.
We represent these two regions by single feature vectors f and f′ constructed as follows:
The ith component is the average of the ithcomponent of all FVs in that region.
Now the distance between r and r′ is given by
dr,r′=i=06fi−f′i2

Experimental Results
We coded up this strategy in MATLAB 7.0 and ran the program on a database of 1000 images.
Some outputs are shown in the next few slides.
The Top left image is the query image.

Final Remarks
One group (Eagle 2) has some original contributions to make:
They had come up with their own criteria of calculating the appropriate number of regions for an image.
They even implemented their own K-means routine instead of relying on MATLAB’s built-in function.

Final Remarks
One of our IAYM student
ShubhankarBiswas, NIT, Durgapur
has presented this project at the
Seminar on Applications of Computer and Embedded Technology
organized by the Variable Energy Cyclotron Centre (VECC), Kolkata, in Oct. 2009 (2 months after IAYM 2009).
His paper can be viewed at
http://www.vecc.gov.in/~sacet09/index_files/Page479.htm
He has acknowledged the MSF for all the guidance.

http://www.mathscifound.org	214
http://mathscifound.org	24
http://www.slideshare.net	2

search engine for images

by Anjani on Apr 13, 2010

Accessibility

Upload Details

Usage Rights

3 Embeds 240

Statistics

search engine for images — Presentation Transcript