PhD Program in Electronic and Computer Engineering
               PhD School in Information Engineering




              ...
Outline

• Relevance Feedback

• Image representation

• Weighted similarity measures

• State of the art: Estimation of F...
Aim of this work

• Exploiting neighborhood relations to weight
    feature sets

• Weight designed to improve Relevance
 ...
Distance weighted kth-Nearest
                Neighbor

                              pNN (I )
                           ...
Relevance Feedback

• Query by examples




            User


                                       System
             ...
Relevance Feedback


                                                             k best ranked images




            Use...
Relevance Feedback


                                                             image labelling




            User


 ...
Image representation
                                               I(F)                                           level

...
Image representation




color histogram layout

  f1,1 = [g1,1,1, g1,1,2, g1,1,3, g1,1,4 ]
 g1,1,1 = [f1,1,1, …, f1,1,8]
...
Image similarity

                                                          1
            #                               ...
Weighted similarity measures

In order to have good performance into images
retrieval systems
• Relevant images should be ...
Weighted similarity measures

                                                                   1
            #          ...
State of the art

• Inverse of standard deviation
Rui, Huang, Mehrotra. Int. Conf. on Image Processing , 1997


          ...
Neighborhood-Based
             Feature Weighting
• “Relevance” of different feature space is
    estimated in terms of th...
Neighborhood-Based
                  Feature Weighting

                                                                  ...
Neighborhood-Based
             Feature Weighting
• Evaluation of capability to exploit neighborhood
    relations in term...
Why better?


• Inverse of standard deviation
     – Doesn’t use information about neighborhood of
            relevant im...
Dataset

• Corel 19511 images
• 43 classes            (min: 96 - max: 1544 images)




6-05-2009      Neighborhood-Based F...
Feature sets

• 4 feature sets
     – Co-Occurrence Texture                           (4x4 subsets)
            • 4 direct...
Experiment setup


• 500 queries
• 9 iterations
• 20 images retrieved each iteration




6-05-2009        Neighborhood-Bas...
Legend

                Dw 2-NN no weight



        *       SVM no weight

                Dw 2-NN Probabilistic learning...
Experimental Results
                        Color Histogram




6-05-2009     Neighborhood-Based Feature Weighting - L. P...
Experimental Results
                                  Color Histogram




                    1
   F=
                1  ...
Experimental Results
                  Color Histogram (PFRL)




6-05-2009     Neighborhood-Based Feature Weighting - L. ...
Conclusions

• Reported results show that a weighted measure
    improve the performance of the NN technique

• Weighted d...
Upcoming SlideShare
Loading in...5
×

Wiamis2009 Pres

290

Published on

High retrieval precision in content-based image retrieval can be attained by adopting relevance feedback mechanisms. In this paper we propose a weighted similarity measure based on the nearest-neighbor relevance feedback technique that the authors proposed elsewhere. Each image is ranked according to a relevance score depending on nearest-neighbor distances from relevant and non-relevant images. Distances are computed by a weighted measure, the weights being related to the capability of feature spaces of representing relevant images as nearest-neighbors. This approach is proposed to weights individual features, feature subsets, and also to weight relevance scores computed from different feature spaces. Reported results show that the proposed weighting scheme improves the performances with respect to unweighed distances, and to other weighting schemes.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
290
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Wiamis2009 Pres

  1. 1. PhD Program in Electronic and Computer Engineering PhD School in Information Engineering Neighborhood-Based Feature Weighting for Relevance Feedback in Content-Based Retrieval Luca Piras luca.piras@diee.unica.it Giorgio Giacinto giacinto@diee.unica.it P R A Pattern Recognition and Applications Group Department of Electrical and Electronic Engineering Pattern Recognition and Applications Group University of Cagliari, Italy 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 1
  2. 2. Outline • Relevance Feedback • Image representation • Weighted similarity measures • State of the art: Estimation of Feature Relevance • Neighborhood-Based Feature Weighting 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 2
  3. 3. Aim of this work • Exploiting neighborhood relations to weight feature sets • Weight designed to improve Relevance Feedback based on Distance weighted kth- Nearest Neighbor • Dw k-NN estimate the relevance of an image according to the (non-)relevant one in its nearest neighborhood 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 3
  4. 4. Distance weighted kth-Nearest Neighbor pNN (I ) r I ! NN nr (I ) relevanceNN (I ) = = p r NN (I ) + p (I ) nr NN I ! NN r (I ) + I ! NN nr (I ) 1 where pNN (I ) = r N ( V I ! NN r (I ) ) ( ) and V I ! NN (I ) " I ! NN (I ) 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 4
  5. 5. Relevance Feedback • Query by examples User System Image Retrieval Images database 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 5
  6. 6. Relevance Feedback k best ranked images User System Image Retrieval Images database 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 6
  7. 7. Relevance Feedback image labelling User System Image Retrieval Images database 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 7
  8. 8. Image representation I(F) level image F=[ f1 … fi … fF ] feature color shape texture f1,1 … f1,i fi,1… fi,j fFi … representation col. hist. layout color moments co-occurrence texture f1,1,1 f1,i,1 fF,i,1 . . . . . . . . . components f1,1,j f1,i,j fF,i,j . . . . . . . . . f1,1,32 f1,i,9 fF,i,16 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 8
  9. 9. Image representation color histogram layout f1,1 = [g1,1,1, g1,1,2, g1,1,3, g1,1,4 ] g1,1,1 = [f1,1,1, …, f1,1,8] g1,1,2 = [f1,1,9, …, f1,1,16] g1,1,3 = [f1,1,17, …, f1,1,24] g1,1,4 = [f1,1,25, …, f1,1,32] 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 9
  10. 10. Image similarity 1 # & ( ) ( ) ( ) N p = % " I A fi, j, k ! I B fi, j, k p S fi, j ( components (lower) $ k =1 ' S ( fi ) = ! S fi, j j ( ) representation S = ! S ( fi ) feature (higher) i 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 10
  11. 11. Weighted similarity measures In order to have good performance into images retrieval systems • Relevant images should be considered as neighbors each others. • Non-relevant images should not be in the neighborhood of relevant ones. • Weighted similarity measures. • Weights related to the capability of feature spaces of representing relevant images as nearest-neighbors 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 11
  12. 12. Weighted similarity measures 1 # & ( ) ( ) ( ) N p = % " wi, j, k I A fi, j, k ! I B fi, j, k p S fi, j ( components (lower) $ k =1 ' ( ) G S fi, j = ! wg i d p ( I A , I B ) g component subset g =1 j ( ) S ( fi ) = ! wi, j i S fi, j representation S = ! wi i S ( fi ) i feature (higher) 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 12
  13. 13. State of the art • Inverse of standard deviation Rui, Huang, Mehrotra. Int. Conf. on Image Processing , 1997 1 w fj = fj is the j-th feature, !j is its standard deviation !j • Probabilistic learning (PFRL) Peng, Bhanu, Qing. Computer Vision and Image Understanding, 1999 e (T ir ( z)) fj w fj = F rfj(z) is the measure of relevance of the j-th ! e( T irl ( z )) feature for the query z l =1 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 13
  14. 14. Neighborhood-Based Feature Weighting • “Relevance” of different feature space is estimated in terms of their capability of representing relevant images as Nearest Neighbors • Relevance of an image is estimated according to the relevant and non-relevant images in its nearest nieghborhood 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 14
  15. 15. Neighborhood-Based Feature Weighting " d (I ,N ) fx wf = p r NN (f ) x = i !R min i x p r NN (f ) + p (f ) "d x nr NN x fx min (I ,R) + " d (I ,N ) i fx min i i !R i !R 1 where p r (f ) = V NN x r NN (f ) x 1 and VNN ( fx ) ! # d min (I i ,R ) r fx card(R) i "R 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 15
  16. 16. Neighborhood-Based Feature Weighting • Evaluation of capability to exploit neighborhood relations in terms of weighted similarity measures and in terms of weighted relevance score : – Components level – Component subset level ( ) S fi, j relevanceNN fi, j ( ) 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 16
  17. 17. Why better? • Inverse of standard deviation – Doesn’t use information about neighborhood of relevant images • Probabilistic learning (PFRL) – It considers only relevant images 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 17
  18. 18. Dataset • Corel 19511 images • 43 classes (min: 96 - max: 1544 images) 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 18
  19. 19. Feature sets • 4 feature sets – Co-Occurrence Texture (4x4 subsets) • 4 directions x 4 values – Color Moments (3x3 subsets) • first 3 moments x (H, S, V) – Color Histogram (4x8 subsets) • 8 ranges of H x 4 ranges of S – Color Histogram Layout (4x8 subsets) • 4 sub-images x 8 color 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 19
  20. 20. Experiment setup • 500 queries • 9 iterations • 20 images retrieved each iteration 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 20
  21. 21. Legend Dw 2-NN no weight * SVM no weight Dw 2-NN Probabilistic learning Dw 2-NN Inverse of standard deviation Dw 2-NN Neighborhood-Based Dw 2-NN N-Based component subset Dw 2-NN N-Based Score component subset 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 21
  22. 22. Experimental Results Color Histogram 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 22
  23. 23. Experimental Results Color Histogram 1 F= 1 1 + 2 ! prec 2 ! recall 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 23
  24. 24. Experimental Results Color Histogram (PFRL) 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 24
  25. 25. Conclusions • Reported results show that a weighted measure improve the performance of the NN technique • Weighted distance metric based on feature subset provided the best results • Neighborhood-Based weights provide similar or better results with respect to PFRL but without annoying tuning operations 6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 25
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×