Feature extraction techniques on cbir a review
Upcoming SlideShare
Loading in...5

Feature extraction techniques on cbir a review






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Feature extraction techniques on cbir a review Feature extraction techniques on cbir a review Document Transcript

  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 467 FEATURE EXTRACTION TECHNIQUES ON CBIR-A REVIEW Ajeesh S. S.1 , Indu M.S.2 1 Research Scholar, M. S. University, Tirunelveli, Tamilnadu 2 Research Scholar, M. G. University, Kottayam, Kerala ABSTRACT Content Based Image retrieval (CBIR) is the process of retrieving and displaying relevant images of users wish from a database on the basis of its visual content. Since traditional text based image retrieval (TBIR) doesn’t meet the users demand and due to the gigantic increase in image database sizes the need for CBIR development arose. This paper reviews the feature extraction methods, which has became one of the key factor in CBIR. Keywords- Content Based Image Retrieval, Logit boost, Relevance Feedback, Support Vector Machine, Self organizing Map. I. INTRODUCTION As the size of image databases grow exponentially, the running of large image databases became difficult which leads to the motivation of research communities to hunt new algorithms for feature extraction. From the historical insight the earlier image retrieval systems are text based where images are annotated and indexed using textual information. However, with the ample increase in the size of images as well as the size of image databases the task of TBIR became more difficult. To tackle these problems near the beginning of 1990s, the research community projected Content Based Image retrieval (CBIR) [1-3]. In the earlier systems, images will be indexed according to their low level features or a combination of all these. A wide range of applications for CBIR systems has been identified. A little of these are image search on internet [1-20], art galleries, museums, archeology, architecture / Engineering design, geographic information systems, weather forecast, medical imaging [9][10], trademark databases [11], home entertainment, criminal investigations, fashion and publishing etc. Again, the need for efficient tool to retrieve images from the large database systems became crucial. Therefore in order to solve these problems, relevance feedback and novel classification methods such as SVM, PCA has been gained more attention during recent years. INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING & TECHNOLOGY (IJCET) ISSN 0976 – 6367(Print) ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), pp. 467-474 © IAEME: www.iaeme.com/ijcet.asp Journal Impact Factor (2013): 6.1302 (Calculated by GISI) www.jifactor.com IJCET © I A E M E
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 468 This paper is organized as follows: In section II, a generic view of CBIR system is discussed. In section III, An overview of existing feature extraction techniques used so far are discussed. In section IV, commonly used image similarity measures are explained. Section V presents system evaluation methods. Finally a conclusion and future work is presented in section VI. II. GENERIC VIEW OF CBIR SYSTEM Figure 1: shows the various processing components of a content based image retrieval system. Figure 1: Generic View of CBIR System The processing steps used by the components of content based image retrieval system are: a) Feature extraction and Indexing of Image Database: Extracts effective features to represent images and index the feature vector in a database. b) Feature extraction of query image. c) Feature Comparison: Comparing the query image feature with feature vector (FV) of images in FV database. d) Similarity Matching: This computes the distance between query image and the images in the database by using the feature vectors. So that the images with zero distance i.e., the exact image or the images having minimum distance i.e. the closest images can be determined. e) User Interface and feedback: Helps to see query results and by giving relevance feedback which enables to display more refined query results. III. EXISTING FEATURE EXTRACTION TECHNIQUES – AN OVERVIEW Image retrieval techniques [12] are distinguished into three distinct levels. Level 1: Retrieval by primitive features such as color, texture, shape and spatial location. Images are compared based on low-level visual features, semantics are not considered.
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 469 Level 2: Retrieval of objects of given type that is Query By Example (QBE) which uses middle level semantics. For example: find the images containing horses. Here the queries and the search targets are image objects. Level 3: Retrieval of abstract attributes of images using high level reasoning. Example: “find the picture of a baby smiling”. In this, high level semantics of image objects are considered. Feature extraction and classification is the background process behind all CBIR systems. Research on image retrieval based on color features [13] proves that it is a partially reliable feature that enhances image search and does the improvement in accuracy of sorting. To provide fast search over huge image database, color histogram based segmentation approach [14] was proposed. Color plays a vital role in most of the CBIR systems, for e.g. VisualSEEK [2], Photobook [3], Virage [4], Blobworld [5], PicToSeek [6] or SIMPLIcity [7], QBIC [8] etc. Texture is the most important native property of all surfaces which describes the visual patterns that can do discrimination of image content. In view of the closeness to human perception and description of texture, investigation based on structural methods [15] of texture analysis was carried out. Three image features, namely color co-occurrence matrix (CCM), difference between pixels of scan pattern (DBPSP), and color histogram for K-mean (CHKM), are presented in a CBIR system [16] making use of color and texture features. Also it proposed an algorithm which effectively reduces the feature vector number of an image that reduced indexing time. A novel approach for effective color image retrieval scheme by combining the three features namely color, texture and shape information, was introduced [17], which demanded higher retrieval efficiency. In this, a fast quantization algorithm has been applied initially and then texture features are extracted, and finally the pseudo-Zernike moments of an image were considered for providing a better feature representation scheme. By combining Gabor filters (GF) and Zernike Moments (ZM) and considering texture and shape features, a new method [18] was proposed. GF and ZM are found effective for face database. Also GF is found effective for finger print database. Even though ZM are found effective for face database and MPEG-7 shape database, it is not effective for finger print database. A novel framework by combining all the primitive image features such as color, texture and shape was also proposed [19] to achieve high retrieval efficiency. Researches show that the performance of a CBIR system can be improved when spatial Relationship of colors is considered. A spatial chromatic histogram based approach [20] was proposed that measures the global spatial relationship of colors. A fuzzy membership function [21] was introduced with the distribution of the features, distances, and assigning a degree of worthiness to each feature based on its average performance. It aggregated memberships and feature weights which gave confidence that helps to rank the retrieved images. A fuzzy logic framework [22] was proposed to alleviate problems in traditional CBIR systems, by considering the semantic gap and the perception subjectivity. The proposed framework consists of two major parts, including model construction and query comparison. In the model construction part, fuzzy linguistic terms with associated fuzzy membership functions are automatically generated through an unsupervised fuzzy clustering algorithm. The linguistic terms provided a natural way of expressing user’s concepts, and the membership functions characterized the mapping between image features and human visual concepts. It also defined the syntax and semantics rules of a query description language to unify the query expression of textual descriptions, visual examples, and relevance feedbacks. In the query comparison part, a similarity function is inferred based on user’s feedbacks to measure the similarity
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 470 between the query and each image in the database. The user’s preference is also captured and retained in his/her own profile to achieve personalization. Experimental result showed that this framework reduced the semantic gap and the perception subjectivity problems. Many fuzzy methods have been applied to the content-based image retrieval (CBIR) to retrieve the similar images according to the similarity of fuzzy sets. In [23], the principle of fuzzy similarity measure for CBIR is deeply inspected, then the properties and the classes about fuzzy similarity measures are introduced and remarked, and developed a faster algorithm on similarity measure using center of gravity of fuzzy sets in CBIR. An experimental CBIR system [24] was developed which makes use of texture co-occurrence matrix. Fuzzy index of major colors are also used as color feature to improve performance. A new measure is suggested to find out the relevance of the retrieved images and to evaluate the CBIR system. Instead of using global features and local statistical features, a kind of distinctive local invariant feature i.e. Lowe's SIFT feature [25] for the purpose of CBIR was proposed. In this CBIR system, the visual contents of the query image and the database images are extracted and described by the 128-dimensional SIFT feature vectors. The KD-tree with the Best Bin First (BBF), an Approximate Nearest Neighbors (ANN) search algorithm, is used to index and match those SIFT features. As their contribution, a modified voting scheme called Nearest Neighbor Distance Ratio Scoring (NNDRS) was put forward to calculate the aggregate scores of the corresponding candidate images in the database respectively. By sorting the database images according to their aggregate scores in descending order, the top few similar images are shown to users as the retrieval results. Additionally, RANSAC was used as a geometry verification method to re-check the results and remove the false matches. Experiments proved that their approach has obtained high recall and high precision in the context of CBIR on the famous image databases ZuBud. When the gap between low level features and high level semantics exceeds, the user won’t get the desired images according to his/her wish. For similar image grouping a hierarchical clustering technique [26] was used. K-Means algorithm is then applied to these image groups and so obtained favored image results. The focus is now shifted from designing low- level image features to reducing the semantic gap between the visual features and richness of human semantics. Relevance Feedback (RF) is a widely used technique in incorporating user’s knowledge with the learning process for Content- Based Image Retrieval (CBIR). Strategies for relevance feedback [27] in image retrieval to reduce the semantic gap were proposed. Content-based image retrieval (CBIR) systems with user relevance feedback are considered in [28]. The influence of the type and the number of feature vector (FV) components on the retrieval efficiency was investigated. They compared a CBIR system with a very small number of FV components (only 25 components describing color and texture) with a system with a high- dimensional FV inspired by MPEG-7 (556 coordinates describing color, texture and line directions), as well as with a system using feature vector reduction (FVR) of about 90% (with about 50 FV components from the full-length 556-component FVs). The systems were tested over the annotated Corel 1K and Corel 60K datasets. Simulation results showed that a decreased number of FV components do not have significant influence on the quality of image retrieval, while the processing time is reduced compared to CBIR with full-length FV and/or FVR. As a supervised learning technique, RF has shown significant increase in the retrieval accuracy. However, as a CBIR system continues to receive user queries and user feedbacks, the information of user preferences across query sessions are often lost at the end of search, thus requiring the feedback process to be restarted for each new query. A few works targeting long-term learning have been done in general CBIR domain to alleviate this problem. However, none of them
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 471 address the needs and long-term similarity learning techniques for region-based image retrieval. A Latent Semantic Indexing (LSI) based method [29] to utilize users’ relevance feedback information was proposed. The proposed region-based image retrieval system is constructed on a Multiple Instance Learning (MIL) framework with One-class Support Vector Machine (SVM) as its core. Experiments showed that the proposed method can better utilize users’ feedbacks of previous sessions, thus improving the performance of the learning algorithm (One-class SVM). Conventionally, CBIR system used labeled images for learning, which was very time consuming. To tackle this problem a new technique relied on the concept of pseudo labeling method [30] was proposed. In this, using fuzzy rule, the images are labeled. To exploit the advantages of pseudo labeling method and fuzzy support vector machine (FSVM), an extended version of SVM a unified frame work called PLFSVM (Pseudo-label FSVM) was proposed. For feature vector dimensionality reduction researchers proposed Self Organizing Map (SOM) based clustering method [31]. A kind of problem in supervised learning method is MIL (Multiple Instance Learning). To solve this problem one-class SVM [32] was proposed. Relevance feedback was also combined to guide the learning process. For the last 20 years researches have been carried out on reducing the Semantic Gap in CBIR systems, which is not ended so far. IV. SIMILARITY COMPUTATION Searching large databases of images is a challenging task especially for retrieval by content. Most search engines calculate the similarity between the query image and all the images in the database and rank the images by sorting their similarities. Similarity measurement is a key to CBIR algorithms. These algorithms search image database to find images similar to a given query, so, they should be able to evaluate the amount of similarities between images. In similarity measure, the query image feature vector and database image feature vector are compared using the distance metric. The images are ranked based on the distance value. Novel image retrieval with empirical evaluation [3] did the detailed comparison of different metrics such as Manhattan, Bray-Curtis, weighted mean-variance, Euclidean, Chebychev, Canberra distance, Mahanobis etc. were done. They found that Canberra and Bray-Curtis distance metrics performed exceptionally well than all other distance metrics. But the most important metrics used by other researchers are: Euclidean distance, Quadratic distance, Chebyshev distance, Manhattan distance etc. V. SYSTEM EVALUATION Human perceptions can easily recognize the similarity between images. To test the effectiveness of a CBIR systems two evaluation measures namely precision and recall are commonly used. Recall measures how far a system is capable to present all relevant images. Precision measures how far a system can present only relevant images. To calculate these, the equations were given below. ܲ‫݊݋݅ݏ݅ܿ݁ݎ‬ ൌ ܰ‫.݋‬ ‫݂݋‬ ‫ݐ݊ܽݒ݈݁݁ݎ‬ ݅‫ݏ݉݁ݐ‬ ‫݀݁ݒ݁݅ݎݐ݁ݎ‬ ܶ‫݈ܽݐ݋‬ ܰ‫.݋‬ ‫݂݋‬ ‫ݐ݊ܽݒ݈݁݁ݎ‬ ݅‫ݏ݉݁ݐ‬ ‫݀݁ݒ݁݅ݎݐ݁ݎ‬ െ െ െ െሺ1ሻ ܴ݈݈݁ܿܽ ൌ ܰ‫.݋‬ ‫݂݋‬ ‫ݐ݊ܽݒ݈݁݁ݎ‬ ݅‫ݏ݉݁ݐ‬ ‫݀݁ݒ݁݅ݎݐ݁ݎ‬ ܰ‫.݋‬ ‫݂݋‬ ‫ݐ݊ܽݒ݈݁݁ݎ‬ ݅‫ݏ݉݁ݐ‬ ݅݊ ܿ‫݊݋݅ݐ݈݈ܿ݁݋‬ െ െ െ െሺ2ሻ
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 472 VI. CONCLUSION AND FUTURE WORK In this paper a brief review of feature extraction techniques for CBIR is presented. Most recent CBIR techniques are geared towards retrieval by some aspect of image appearance, depending on the automatic extraction and comparison of image features judged most likely to convey that appearance. The features most often used include color, texture, shape, spatial layout, and multi resolution pixel intensity transformations such as wavelets. This classification of feature set can be enhanced to heterogeneous (shape, texture) so that we can get more accurate result. It can also enhance to merging of heterogeneous features and by using neural network. Besides investigating suitable frameworks for image retrieval, early researchers have attempted to use existing techniques in different fashion to retrieve image information. Relevance feedback and Support Vector Machines (SVMs) have in the recent years. A machine learning approach called SVM is a supervised learning method for classifying images. Association rule mining is a typical approach used in data mining domain for uncovering interesting trends, patterns and rules in large data sets. Hence, SVMs and association rules are likely to be used more to accelerate image retrieval. Also an approach to classify the images based on regression is needed. Alternate methods for image classification methods like Logitboost algorithms [33] are to be considered in future research which may outperform the SVMs. VII. ACKNOWLEDGEMENT We would like to thank our guides, Dr. G. Raju, Head & Associate Professor, School of Information Science & Technology, Kannur University, Kerala, India and Dr. Elizabeth Sherly, Head and Principal Investigator(ILCI,ILMT), IIITM-K, Trivandrum for their valuable guidance and constant inspiration throughout the course of this work. REFERENCES [1] C. Faloutsos, R. Barber, M. Flickner, J. Hafner, W. Niblack, D. Petkovic, W. Equitz, Efficient and effective querying by image content, pp. 231–262, J. Intell. Inf. Syst., 1994. [2] J.R. Smith, S. F. Chang, VisualSEEK: A fully automated content-based image query system, pp. 87–98. Proc. ACM Multimedia,1996. [3] A. Pentland, R.W. Picard, S. Sclaroff, Photobook: content-based manipulation for image databases, pp. 233–254, Int. J. Comput. Vision 18,1996. [4] A. Gupta, R. Jain, Visual information retrieval, 40 (5), pp. 70–79, Commun. ACM, 1997. [5] C. Carson, M. Thomas, S. Belongie, J.M. Hellerstein, J. Malik, Blobworld: a system for region-based image indexing and retrieval, pp. 509–516, Proc. Visual Inf. Syst. 1999. [6] T. Gevers, A.W.M. Smeulders, PicToSeek: combining color and shape invariant features for image retrieval, 9 (1), pp. 102–119, IEEE Trans. Image Process, 2000. [7] J.Z. Wang, J. Li, G. Wiederhold, SIMPLIcity: semantics-sensitive integrated matching for picture libraries, pp. 947–963, IEEE Trans. Pattern Anal. Mach. Intell. 2001. [8] W.R. Niblack et al., The QBIC project: querying images by color, texture and shape, IBM Research Report RJ-9203, 1993. [9] Sameer Antani and George r Thoma, “Bridging the gap: Enabling CBIR in medical applications”, pp. 4-6, 21st IEEE Intl. Symbosium on Computer Based Medical Systems 2008.
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 473 [10] Ashnil Kumar, Jinman Kim, Weidong Cai, Stefan Eberl, Dagan Feng, “A Graph-Based Approach to the Retrieval of Dual-Modality Biomedical Images Using Spatial Relationships”, IEEE 2008. [11] Y. S. Kim and W. Y. Kim, Content-based trademark retrieval system using a visually salient features, Vol. 16, pp. 931-939, Image and Vision Computing, 1998. [12] J.P. Eakins, Techniques for image retrieval, Library and Information Brie"ngs 85, British Library and South Bank University, London, 1998. [13] Bai Xue, Liu Wanjun, Research of Image Retrieval Based on Color, , pp. 283-286, Intl. Forum on Computer Science-Technology and Applications”, IEEE 2009. [14] Zhenhua Zhang, Wenhui Li and Bo Li, An Improving Technique of Color Histogram in Segmentation-based Image Retrieval, , pp. 381-384, Fifth International Conference on Information Assurance and Security, IEEE 2009. [15] K. Jalaja, Chakravathy Bhagvati,.L. Deekshatulu, and Arun K.Pujari Texture element feature characterizations for CBIR, , pp. 733-736, IEEE 2005. [16] Chuen-Horng Lin, Rong-Tai Chen, Yung-Kuan Chan, A smart content-based image retrieval system based on color and texture feature, pp. 658–665, Image and Vision Computing, 2009. [17] Xiang-Yang Wang, Yong-Jian Yu, Hong-Ying Yang, An effective image retrieval scheme using color, texture and shape features, Elsevier B.V. 2010. [18] X. Fu, Y. Li,R. Harrison, S. Belkasim, Content-based Image Retrieval Using Gabor-Zernike Features, IEEE 2006. [19] P. S. Hiremath and Jagadesh Pujari, Content-based image retrieval using color, texture and shape features, pp. 780-784, Proc. of 15th Intl. Conf. on Advanced Computing and Communications 2007. [20] Suryani Lim and Guojun Lu, Spatial statistics for Content based image retrieval, Proceedings of the international conference on information technology: computers and communications, IEEE 2003. [21] Hichem, Frigui, Joshua Caudill and Ahammed Chamseddine Ben Abdallah, Fusion of multimodal features for Efficient Content based image retrieval, , pp. 1992-1998, Proceedings of the Intl. conf. on Fuzzy Systems, IEEE 2008. [22] Chih-Yi Chiu, Hsin-Chih Lin, Shi-Nine Yam, A FUZZY Logic CBIR System, IEEE 2003. [23] Yun Li, Jia-Ming Liu, Ji Li, Wei Deng Chun-Xuo Ye, Zhongfu, The Fuzzy Similarity Measures For Content Based Image Retrieval, IEEE 2003. [24] Sanjoy Kumar Saha, Amit Kumar Das, Bhabatosh Chanda, “CBIR using Perception based Texture and Colour Measure,pp 985-988, ICPR 2004. [25] Xu Wangming, Wu Jin, Liu Xinhai, Zhu Lei, Shi Gang, Application of Image SIFT Features to The Context of CBIR, IEEE 2008. [26] Murthy V.S., E.Vamsidhar, P.Sankara Rao, G.Samuel, Varaprasad Raju, Application of Hierarchical and K-Means Techniques in Content Based Image Retrieval, pp. 749- 755, vol. 2(5), IJEST 2010. [27] H. Muller, Muller, S. Machand-Maillet, and T. Pun, Strategies for positive and negative relevance feedback in image retrieval, presented at Poc. Intl. Conf. on Pattern recognition, ICPR 2000. [28] Stevan Rudinac, Goran Zajic, Marija Uscumlic, Maja Rudinac, Branimir Reljin, Comparison of CBIR Systems with Different Number of Feature Vector Components, IEEE 2007. [29] Xin Chen, Chengcui Zhang, Shu-Ching Chen, Min Chen, A Latent Semantic Indexing Based Method for Solving Multiple Instance Learning Problem in Region-Based Image Retrieval, IEEE 2005.
  • International Journal of Computer Engineering and Technology (IJCET), ISSN 0976- 6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 4, July-August (2013), © IAEME 474 [30] Kui Wu and Kim-Hui Yap, Fuzzy SVM for Content-BasedImage Retrieval A Pseudo-Label Support Vector Machine Framework, IEEE 2006. [31] Chen Guo, Campbell Wilson, Use of Self-Organizing Maps for Texture Feature Selection in Content-Based Image Retrieval, IEEE 2008. [32] Chengcui Zhang, Xin Chen, Min Chen, Shu-Ching Chen, Mei-Ling Shyu, A Multiple Instance Learning Approach For Content Based Image Retrieval Using One-Class Support Vector Machine, IEEE 2005. [33] Yu-Dong Cat, Kai-Yan Feng, Wen-Cong Lu, Kuo-Chen Chou, Using Logitboost classifier to predict protein structural classes, Elsiever. [34] Kranthi Kumar.K, Dr.T.Venu Gopal, K. Dasaradh Ramaiah and P.Prasanna Rani, “Relevance Feedback: A Novel Method to Associate User Subjectivity to Image Databases in CBIR”, International Journal of Computer Engineering & Technology (IJCET), Volume 4, Issue 2, 2013, pp. 325 - 339, ISSN Print: 0976 – 6367, ISSN Online: 0976 – 6375. [35] Shivamurthy.R.C, Dr. B.P. Mallikarjunaswamy and Pradeep Kumar B.P., “Dynamic Hand Gesture Recognition using CBIR”, International Journal of Computer Engineering & Technology (IJCET), Volume 4, Issue 3, 2013, pp. 340 - 352, ISSN Print: 0976 – 6367, ISSN Online: 0976 – 6375. [36] K.Ganapathi Babu, A.Komali, V.Satish Kumar and A.S.K.Ratnam, “An Overview of Content Based Image Retrieval Software Systems”, International Journal of Computer Engineering & Technology (IJCET), Volume 3, Issue 2, 2012, pp. 424 - 432, ISSN Print: 0976 – 6367, ISSN Online: 0976 – 6375.