Advanced relevance feedback strategy for precise image retrieval

International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-6367(Print),
ISSN 0976 - 6375(Online), Volume 6, Issue 1, January (2015), pp. 61-71© IAEME
61
ADVANCED RELEVANCE FEEDBACK STRATEGY FOR
PRECISE IMAGE RETRIEVAL
A. A. Khodaskara
, S. A. Ladhakeb
a
Computer Science and Engineering Department, SIPNA COET, Amravati, Maharashtra, India
b
Computer Science and Engineering Department, SIPNA COET, Amravati, Maharashtra, India
ABSTRACT
Relevance feedback is effective technique for bridging the semantic gap in image retrieval
which diminish semantic gap between low-level visual features and high-level semantic concepts for
image retrieval. Currently, crucial image retrieval system is content-based image retrieval. To
improve performance of proposed content based image retrieval system, automatic
relevance feedback technique is proposed which based on inductive learning involve inductive
concept learning by decision tree. We implement and tested proposed image retrieval system which
refine the retrieval result as per user requirement until get required accuracy. Experimental result
shows improved performance in term of precision, recall and accuracy.
Keywords: Content based image retrieval, Decision tree, Inductive learning, Inductive concept
learning, Relevance feedback, Semantic gap.
1. INTRODUCTION
As we know, an immense growth and development in internet technology give birth to
gigantic digital images which are exploited in variety of applications such as medical, academic,
virtual museums, military etc. it is really difficult for users to search large image databases for
particular required image. In order to address this requirement, initially text based and then content
based image retrieval system is proposed. In text based image retrieval system, images are retrieved
on the basic of text annotation. Text based retrieval systems are suffered from low precision, recall
and accuracy. To overcome these limitations, Content based image retrieval system is introduced. In
content based image retrieval system, initially visual features such as color, texture and shape are
extracted from images and store in feature vector which are used to represent image. When user fired
an image query, then a query model converts the image into a feature vector of query image and
retrieval module performs image retrieval by computing similarities between query image and
images in image database. Finally, the retrieval results are ranked according to computed similarity
INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING &
TECHNOLOGY (IJCET)
ISSN 0976 – 6367(Print)
ISSN 0976 – 6375(Online)
Volume 6, Issue 1, January (2015), pp. 61-71
© IAEME: www.iaeme.com/IJCET.asp
Journal Impact Factor (2015): 8.9958 (Calculated by GISI)
www.jifactor.com
IJCET
© I A E M E

62
values. Content based image retrieval systems are also critically suffer by semantic gap which
address by using relevance feedback mechanisms.
Relevance feedback mechanism provides way to interact human and computer to remodel
high-level queries to low-level features. Relevance feedback is a potent skill used to improve
accuracy and speed of retrieval in content based and text-based retrieval systems. Relevance
feedback is an iterative method, which filter and improve the retrieval result based on the previously
retrieved results user's feedback. Traditional there are two types of RF, short term RF and Long term
RF. Some retrieval system used merging of short-term and long-term learning with virtual feature
[8]. Relevance feedback automatically adjust an active query using the information provided by the
user about the earlier retrieved objects considering low-level features can capture high-level
concepts, the relevance feedback mechanism attempt to form the relation between high-level
concepts and low-level features from the user’s feedback.
1.1. Types of Relevance Feedback
Figure 1. Types of Relevance Feedback
1.2. Challenges Faced by Relevance Feedback
As mention earlier, relevance feedback is used to improve accuracy of image retrieval. There
are some challenges which need to address to improve relevance feedback strategy such as high level
feature extraction, imbalance feedback samples and real time processing.
1) High Level Semantics Extraction:
Image retrieval system is suffered from semantic gap. Extraction of high-level semantics from
images is difficult because RF used only low-level features.
2) Imbalance of feedback samples:
Machine learning algorithms are unable to accurate result, because smaller numbers of feedback
samples are labeled by the users during RF iteration which are far smaller feature space dimension.
Labelling of negative samples is generally greater than positive samples which result in imbalance of
training data. Thus the small positive sample definitely confines RF accuracy.
3) Real time processing:
Online relevance feedback takes more time which is addressed by implementing efficient image
representation and storage structures.

63
Figure 2. Challenges in content based image retrieval
2. ROLE OF RF IN CBIR
A number of relevance feedback techniques have been designed as a key tool to bridge the
semantic gap in content based image retrieval, and thus to improve the performance of CBIR
systems. By using relevance feedback scheme, a CBIR system learns from feedback given by the
user. Learning in CBIR systems is categorized into short-term learning, and long term learning [21].
• Short-term learning
CBIR system refines and tunes query using relevance feedback in single retrieval iteration. This type
of learning is usually known as intra-query learning or short term learning. A lot of standard machine
learning techniques are used in short-term learning, which include support vector machines,
Bayesian learning, boosting and feature weighting, discriminant analysis etc. short term learning is
online, which takes additional real time.
• Long term learning
The learning strategy analyzes the relationship between the current and past retrieval
iterations. This form of learning is inter-query or long term learning. Long-term learning methods
overcome the problem of short term learning. It confiscates inability of capturing semantics and
imbalance of feedback examples, and memory mechanism. It based on collaborative filtering.
3. ADVANCED DEVELOPMENT IN RF
This section highlights advance development in relevance feedback technique such as
Navigation-Pattern-based Relevance Feedback, Click-based relevance feedback, pseudo-relevance
feedback for multimedia retrieval, user historical feedback log, semi-supervised long-term
Relevance Feedback algorithm, a gaze-based Relevance Feedback, Gaussian based relevance
feedback, Advanced long-term relevance feedback system generalized biased discriminant analysis
relevance feedback algorithm etc.

64
View-based 3D model retrieval system implemented a relevance feedback method to balance
the contributions of multiple topic models with specified numbers of topics [1]. View-based 3D
model retrieval uses a set of views to represent each object. Click-based relevance feedback is
proposed which emphasizes the successful use of click-through data for identifying user search
intention [2]. Leverage click-through data can be considered as an implicit user feedback and help to
understand the query. Spoken term detection is a basic technology for spoken content retrieval,
which retrieve and browse multimedia content over the Internet. The concept of pseudo-relevance
feedback used in multimedia retrieval, which divide some spoken segments in the first-pass retrieved
results into relevant or pseudo-relevant and irrelevant or pseudo-irrelevant [3]. Adaptive wavelet-
based image characterizations used to characterize each query image which improves the retrieval
performance. Flexibility in wavelet adaptation also enhanced relevance feedback on image
characterization itself [4]. A novel subspace learning framework for learning an effective semantic
subspace is proposed on user historical feedback log data for a Collaborative Image Retrieval [5].
Basically, CPSL can efficiently assimilate discriminative information and geometrical information of
labeled log images, and the weakly similar information of unlabeled images together to learn a
reliable subspace.
Navigation-Pattern-based Relevance Feedback (NPRF) achieves the high efficiency and
effectiveness of CBIR in coping with the large-scale image data [6]. Proposed NPRF Search
discovered navigation patterns and three kinds of query refinement strategies, Query Point
Movement, Query Reweighting and Query Expansion, to match the user's goal successfully. A semi-
supervised long-term Relevance Feedback algorithm is designed which refine the multimedia data
representation and utilizes both the multimedia data distribution in multimedia feature space and the
history RF information provided by users [8]. Geometric optimum experimental design is a novel
active learning method selects multiple representative samples in the database as the most
informative ones for the user to label which improve relevance feedback [9]. A gaze-
based Relevance Feedback approach to region-based image retrieval is presented [12]. Proposed
object-based RF overcome the major drawback of region-based RF approaches, i.e., the frequently
inaccurate estimation of the regions of interest in the retrieved images. A biased maximum margin
analysis (BMMA) and a semisupervised BMMA (SemiBMMA) are proposed for integrating the
distinct properties of feedbacks and exploiting the information of unlabeled samples for SVM-based
RF schemes [14]. The BMMA distinguishes positive feedbacks from negative ones based on local
analysis, whereas the SemiBMMA can efficiently incorporate information of unlabeled samples by
introducing a Laplacian regularizer to the BMMA. Advanced long-term relevance feedback system
was proposed which integrates the user feedback from all iteration and instills memory into
the feedback system of content based retrieval without earlier retrieval log [16]. A new
relevance feedback approach for content-based image retrieval was introduced, which uses Gaussian
mixture (GM) models as image representations [17]. Proposed RF technique categorizes the relevant
and irrelevant images according to the preferences of the user. The generalized biased discriminant
analysis algorithm is the most promising relevance feedback way employed for content-
based image retrieval [18]. It evades the singular problem by adopting the differential scatter
discriminant criterion and manages the Gaussian distribution assumption.
New RF method with active sample-selecting and manifold learning was proposed for CBIR of SAR
images with limited feedback samples [19]. Particle swarm optimizer which merges
relevance feedback with an evolutionary stochastic algorithm was proposed which grasp user's
semantics through optimized iterative learning [20]. For improvement of performance, a real-time
textual query-based personal photo retrieval system for millions of Web images implemented
two relevance feedback methods using cross-domain learning, which efficiently employ both the
Web images and personal images [22].

65
4. PROPOSED SYSTEM
Proposed system has basic four components, image DB, features extraction, similarity
measure, and relevance feedback.
• Image database:
We proposed and tested image retrieval system based Inductive Learning strategy using
image database of 2400 images. Used image database consist of both image natural as well as
synthetic images.
• Feature extraction:
Feature extraction consists of extracting the significant information from the images. Once
the low level features are extracted, they are stored in the form of feature vector in Database. Feature
extraction is significantly reduces the information storage space as it represent an image for
understanding using image content. In the CBIR system, low level features of image such as color,
texture and shape are extracted using Discrete Wavelet Transform. Low level contents of the images
in the database are extracted and described by multi-dimensional feature vectors in image feature
database.
Figure 3. Workflow of proposed system
• Similarity measure:
In similarity measure, feature vector of user query image and database image feature vector
are compared using the Euclidean distance metric. The images are ranked based on the distance
value. For similarity measure, proposed system use Euclidean distance techniques for matching user

66
query image with images in database. Similarity measure is based on threshold vales of image
extracted features. If a Euclidean distance value is less than threshold value than image is irrelevant
otherwise image is relevant to query image.
• Advanced Relevance feedback strategy:
Basically, relevance feedback is a technique used in image retrieval to help the users for
retrieving the correct images from large database as per user requirement. The users can continue to
refine the search from the results of the previous search until they get the desired images or closest to
what they desire. In this framework, the Relevance Feedback functionality is described. The
Relevance Feedback is a composition of Inductive Learning Framework, Inductive Concept
Learning by Decision Tree.
Figure 4. CBIR with Relevance Feedback
a) Inductive Learning Framework:
Raw input data is processed to obtain a feature vector, x, that adequately all of the relevant
features for classifying examples. Each x is a list of (attribute, value) pairs. For example,
x = (Person = Sue, Eye-Color = Brown, Age = Young, Sex = Female)
The number of attributes (also called features) is fixed (positive, finite). Each attribute has a fixed,
finite number of possible values. Each example can be interpreted as a point in an n-dimensional
feature space, where n is the number of attributes.
b) Inductive Concept Learning by Learning Decision Trees
Build a decision tree for classifying examples as positive or negative instances of a concept.
Supervised learning, batch processing of training examples are employed using a preference bias. A
decision tree is a tree in which each non-leaf node has associated with it an attribute (feature), each
leaf node has associated with it a classification (+ or -), and each arc has associated with it one of the
possible values of the attribute at the node where the arc is directed from. For example,

67
Color
/ |
/ |
green/ red blue
/ |
Size + Shape
/ /
/ /
big/ small round square
/ /
- + Size -
/
/
big/ small
/
- +
Figure 5. Learning Decision Tree
Algorithm:
Function decision-tree-learning (examples, attributes, default)
;; examples are a list of training examples
;; attributes is a list of candidate attributes for the
;; current node
;; default is the default value for a leaf node if there
;; are no examples left
If empty (examples) then return (default)
If same-classification (examples) then return (class (examples))
If empty (attributes) then return (majority-classification (examples))
best = choose-attribute(attributes, examples)
tree = new node with attribute best
for each value v of attribute best do
v-examples = subset of examples with attribute best = v
subtree = decision-tree-learning (v-examples, attributes - best, majority-classification(examples))
add a branch from tree to subtree with arc labeled v
return (tree)
5. RESULT EVALUATION
Proposed automatic relevance feedback strategy is presented and tested to reduce the gap
between low-level visual features and high-level semantic during the process of relevance feedback.
We implemented and tested the proposed content based image retrieval system with advanced
relevance feedback based on Inductive Learning Framework using a 2400 image subset from the
COREL image database. The images used are assorted set comprising both natural and man-made
scenes and objects. As proposed system is based on color, texture and shape feature abstraction,
equity in terms of these low level features was desired. Therefore, the test set used required striking a
balance of low level image content. The main objective of experimentation was to compare and
refine retrieval result to achieve high degree of retrieval accuracy.

68
In the first phase as shown in figure.7, proposed system retrieved a set of relevant image for user
query which has low accuracy as per user expectation. To improve accuracy of image retrieval,
proposed relevance feedback technique used to refine result as per user requirement as shown in
figure 8. During each iteration accuracy of retrieval is improves as shown in table1 which ultimately
reduce semantic gap. Performance of proposed image retrieval system evaluated using parameters
such as precision recall and accuracy with dataset of 2400 images. Figure 6 shows improvement of
precision with iteration.
Figure 6. Precision per iteration for user query
Table 1.Precision with relevance feedback
Query Image Relevance Feedback
Iteration 1 Iteration 2 Iteration 3 Iteration 4
White rose 80.5 86.67 88.65 89.50
White Horse 77.45
89.91
90.46 91.56
Mountain 76.28 88.79 89.47 90.68
Beach 79.46 85.33 91.90 92.55
Figure 7. Result for user query

69
Figure 8. Result for user query with relevance feedback
6. CONCLUSION
We proposed automatic relevance feedback which reduces the semantic gap between low-
level features and high level concepts. We proposed content based Image retrieval system with
automatic relevance feedback which is an effective tool to improve the performance of CBIR in term
of accuracy, recall and precision. Proposed automatic relevance feedback is based on a composition
of Inductive Learning Framework, Inductive Concept Learning by Decision Tree. We used image
dataset of 2400 image to evaluate the result. Experimental result clearly shows improvement in
precision and accuracy of image retrieval.
REFERENCES
1. Leng, B. , Zeng, J. , Yao, M. , Xiong, Z., 3D Object Retrieval With Multitopic Model
CombiningRelevance Feedback and LDA Model, IEEE Trans. Image Processing, Vol.24
, No. 1, pp. 94 – 105, 2015.
2. Yongdong Zhang , Xiaopeng Yang , Tao Mei, Image Search Reranking With Query-
Dependent Click-Based Relevance Feedback, IEEE Trans. Image Processing, Vol.23 , No.
10, pp. 4448 – 4459, 2014.
3. Hung-Yi Lee , Lin-Shan Lee, Enhanced Spoken Term Detection Using Support Vector
Machines and Weighted Pseudo Examples, IEEE Trans. Audio, Speech, and Language
Processing, Vol.21 , No. 6, pp.1272 – 1284, 2013.

70
4. Quellec, G. , Lamard, M. , Cazuguel, G. , Cochener, B. , Roux, C., Fast Wavelet-Based
Image Characterization for Highly Adaptive Image Retrieval, IEEE Trans. Image Processing,
Vol. 21 , No. 4, pp. 1613 - 1623 , 2012.
5. Lining Zhang, Lipo Wang, Weisi Lin, Conjunctive Patches Subspace Learning With Side
Information for Collaborative Image Retrieval, IEEE Trans. Image Processing, Vol.21, No. 8,
pp. 3707 - 3720, 2012.
6. Ja-Hwung Su , Wei-Jyun Huang , Yu, P.S. , Tseng, V.S., Efficient Relevance Feedback for
Content-Based Image Retrieval by Mining User Navigation Patterns, IEEE Trans.
Knowledge and Data Engineering, Vol.23 , No. 3, 360 – 372, 2011.
7. Junwei Han, McKenna, S.J., Query-dependent metric learning for adaptive, content
based image browsing and retrieval, IEEE Trans. Image Processing, Vol.8, No. 10, pp. 610 –
618, 2014.
8. Peng-Yeng Yin, Bhanu, B. ; Kuang-Cheng Chang ; Anlei Dong, Long-Term Cross-Session
Relevance Feedback Using Virtual Features, IEEE Trans. Knowledge and Data Engineering,
Vol. 20 , no. 3, pp. 352 – 368, 2008
9. Lining Zhang, Lipo Wang, Weisi Lin, Shuicheng Yan , Geometric Optimum Experimental
Design for Collaborative Image Retrieval, IEEE Trans. Circuits and Systems for Video
Technology, Vol.24 , No. 2, pp.346 – 359, 2014.
10. Hao Ma , Jianke Zhu , Lyu, M.R.-T. , King, I., Bridging the Semantic Gap
Between Image Contents and Tags, IEEE Trans. Multimedia, Vol.12, No. 5, pp. 462 – 473,
2012.
11. Zilong Chen, Yang Lu, Using Text Classification Method in Relevance Feedback,
Springer LNCS Intelligent Information and Database Systems, Vol. 5991(2010) pp. 441-449
12. Papadopoulos, G.T. , Apostolakis, K.C. , Daras, P., Gaze-Based Relevance Feedback for
Realizing Region Based Image Retrieval, IEEE Trans. Multimedia, Vol.16 , No. 2, pp. 440 –
454, 2014.
13. Rahman, M.M., Antani, S.K., Thoma, G.R., A Learning-Based Similarity Fusion and
Filtering approach for Biomedical Image Retrieval Using SVM Classification
and Relevance Feedback, IEEE Trans. Information Technology in Biomedicine, Vol.15, No.
4, pp. 640 – 646, 2011.
14. Lining Zhang , Lipo Wang , Weisi Lin, Semisupervised Biased Maximum Margin Analysis
for Interactive Image Retrieval, IEEE Trans. Image Processing, Vol.21 , No. 4, pp. 2294 –
2308, 2012.
15. Imen Taktak, Mohamed Tmar, A. B. Hamadou, Query Reformulation Based on Relevance
Feedback, Springer LNCS Flexible Query Answering Systems, Vol. 5822, pp. 134-144,
2009.
16. Lakshmi, A., Nema, M. , Rakshit, S., Long Term Relevance Feedback: A Probabilistic Axis
Re-Weighting Update Scheme, IEEE Trans. SignalProcessingLetters,Vol.22, No.7, pp. 852 –
856, 2015.
17. Marakakis, A. , Siolas, G. , Galatsanos, N. , Likas, A. ,Stafylopatis,
A., Relevance feedback approach for image retrieval combining support vector machines and
adapted gaussian mixture models, IEEE Trans. Image Processing, Vol.5 , No. 6, pp. 531 –
540, 2011.
18. Lining Zhang, Lipo Wang, Weisi Lin, Generalized Biased Discriminant Analysis for
Content-Based Image Retrieval, IEEE Trans. Systems, Man, and Cybernetics, Part B:
Cybernetics, Vol.42 , No. 1, pp. 282 – 290, 2012.
19. Chen, R. , Cao, Y.F. , Sun, H., Active sample-selecting and manifold learning-based
relevance feedback method for synthetic aperture radar image retrieval, IEEE Trans. Radar,
Sonar & Navigation, Vol.5 , No. 2, pp. 118 – 127, 2011.

71
20. Broilo, M. , De Natale, F.G.B., A Stochastic Approach to Image Retrieval Using Relevance
Feedback and Particle Swarm Optimization, IEEE Trans. Multimedia, Vol.12, No. 4, pp. 267
– 277, 2010.
21. Gu, Hong, Zhao, Guangzhou, Qiu, Jun, Online metric learning for relevance feedback in e-
commerce image retrieval applications, IEEE Trans. Tsinghua Science and
Technology, Vol.16 , No. 4, pp. 377 – 385, 2011.
22. Yiming Liu , Dong Xu , Tsang, I.W. , Jiebo Luo, Textual Query of Personal Photos
Facilitated by Large-Scale Web Data, IEEE Trans. Pattern Analysis and Machine
Intelligence, Vol.33 , No. 5, pp. 1022 – 1036, 2011.
23. Tarun Dhar Diwan and Upasana Sinha, “Performance Analysis Is Basis on Color Based
Image Retrieval Technique” International journal of Computer Engineering & Technology
(IJCET), Volume 4, Issue 1, 2013, pp. 131 - 140, ISSN Print: 0976 – 6367, ISSN Online:
0976 – 6375.
24. Dr. Prashant Chatur, Pushpanjali Chouragade, “Visual Rerank: A Soft Computing Approach
For Image Retrieval From Large Scale Image Database” International journal of Computer
Engineering & Technology (IJCET), Volume 3, Issue 3, 2012, pp. 446 - 458, ISSN Print:
0976 – 6367, ISSN Online: 0976 – 6375.
25. Shivamurthy R.C, Manjunatha M B and Pradeep Kumar B.P, “A Comparative Study on
Procedures on Content-Based Image Retrieval In Medical Imaging” International journal of
Computer Engineering & Technology (IJCET), Volume 5, Issue 10, 2014, pp. 64 - 73, ISSN
Print: 0976 – 6367, ISSN Online: 0976 – 6375.

Advanced relevance feedback strategy for precise image retrieval

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to Advanced relevance feedback strategy for precise image retrieval

Similar to Advanced relevance feedback strategy for precise image retrieval (20)

More from IAEME Publication

More from IAEME Publication (20)

Recently uploaded

Recently uploaded (20)

Advanced relevance feedback strategy for precise image retrieval