Large Scale Tag Recommendation Using Different Image Representations Rabeeh Abbasi Marcin Grzegorzek Steffen Staab
Tag Recommendation http://www.flickr.com/photos/64255998@N00/283890351/ Clocktower Uhrturm Graz Austria Europe Herbst Autu...
Tag Recommendation http://www.flickr.com/photos/64255998@N00/283890351/ Clocktower Uhrturm Graz Austria Europe Herbst Autu...
Tag Recommendation Tags Geographical Coordinates Low-level Features Clocktower Uhrturm Graz Austria Europe Herbst Autumn N...
Tag Recommendation Tags Geographical Coordinates Low-level Features Which image features to use? Clocktower Uhrturm Graz A...
Features Available in Large Datasets <ul><ul><li>CoPhIR Dataset contains </li><ul><li>106  million images from Flickr with...
Five standard MPEG-7 Features </li></ul></ul></ul></ul>
Tag Recommender – System Overview Features Features Training Data Location 1 Location N Location 1 Clusters Location N Clu...
Tag Recommender – System Overview Features Features Image Features Tags Clocktower Graz Austria Europe Autumn Night Uhrtur...
Tag Recommender – Clustering <ul><li>Used Standard K-Means clustering </li></ul>
Tag Recommender – Representative Tags <ul><ul><li>Given a cluster containing homogeneous images </li><ul><li>Find most rep...
Tag Recommender – Classification <ul><ul><li>Map a new image to one of the existing clusters
Find the nearest cluster centroid </li><ul><li>Assign representative tags of the mapped cluster to the image </li></ul></u...
Evaluation <ul><ul><li>Evaluated on tags of ~400,000 Flickr images </li><ul><li>75% training data
25% test data / gold standard </li></ul><li>Data is randomly split based on users (not resources)
Ignored  </li><ul><li>10 most frequent tags for each location (city) and
Tags used by less than three users </li></ul><li>Evaluation based on the correctly identified tags of test data </li><ul><...
Recall
F-Measure </li></ul></ul></ul>
Micro Precision
Micro Recall
Upcoming SlideShare
Loading in …5
×

Large Scale Tag Recommendation Using Different Image Representations

1,219
-1

Published on

Nowadays, geographical coordinates (geo-tags), social annotations (tags), and low-level features are available in large image datasets. In our paper, we exploit these three kinds of image descriptions to suggest possible annotations for new images uploaded to a social tagging system. In order to compare the benefits each of these description types brings to a tag recommender system on its own, we investigate them independently of each other. First, the existing data collection is clustered separately for the geographical coordinates, tags, and low-level features. Additionally, random clustering is performed in order to provide a baseline for experimental results. Once a new image has been uploaded to the system, it is assigned to one of the clusters using either its geographical or low-level representation. Finally, the most representative tags for the resulting cluster are suggested to the user for annotation of the new image. Large-scale experiments performed for more than 400,000 images compare the different image representation techniques in terms of precision and recall in tag recommendation.

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,219
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
42
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Large Scale Tag Recommendation Using Different Image Representations

  1. 1. Large Scale Tag Recommendation Using Different Image Representations Rabeeh Abbasi Marcin Grzegorzek Steffen Staab
  2. 2. Tag Recommendation http://www.flickr.com/photos/64255998@N00/283890351/ Clocktower Uhrturm Graz Austria Europe Herbst Autumn Night Nacht 47° 4′ 32″ N 15° 26′ 12″ E Image Tags
  3. 3. Tag Recommendation http://www.flickr.com/photos/64255998@N00/283890351/ Clocktower Uhrturm Graz Austria Europe Herbst Autumn Night Nacht Input Output 47° 4′ 32″ N 15° 26′ 12″ E Tag Recommender
  4. 4. Tag Recommendation Tags Geographical Coordinates Low-level Features Clocktower Uhrturm Graz Austria Europe Herbst Autumn Night Nacht Input Output 47° 4′ 32″ N 15° 26′ 12″ E Tag Recommender
  5. 5. Tag Recommendation Tags Geographical Coordinates Low-level Features Which image features to use? Clocktower Uhrturm Graz Austria Europe Herbst Autumn Night Nacht Input Output 47° 4′ 32″ N 15° 26′ 12″ E Tag Recommender
  6. 6. Features Available in Large Datasets <ul><ul><li>CoPhIR Dataset contains </li><ul><li>106 million images from Flickr with </li><ul><li>Title, location, tag, comment, etc
  7. 7. Five standard MPEG-7 Features </li></ul></ul></ul></ul>
  8. 8. Tag Recommender – System Overview Features Features Training Data Location 1 Location N Location 1 Clusters Location N Clusters Model
  9. 9. Tag Recommender – System Overview Features Features Image Features Tags Clocktower Graz Austria Europe Autumn Night Uhrturm Nacht Herbst HDR Photomatix Input Output 47° 4′ 32″ N 15° 26′ 12″ E Tag Recommender Training Data Location 1 Location N Location 1 Clusters Location N Clusters Model
  10. 10. Tag Recommender – Clustering <ul><li>Used Standard K-Means clustering </li></ul>
  11. 11. Tag Recommender – Representative Tags <ul><ul><li>Given a cluster containing homogeneous images </li><ul><li>Find most representative tags </li><ul><li>Tag Rank ~ Frequency of Users </li></ul></ul></ul></ul>Clocktower (4) Graz (4) Austria (3) Europe (3) Autumn (2) Night (2) Uhrturm (1) Nacht (1) Herbst (1) HDR (1) Photomatix (1) Cluster Tags User 2 User 4 User 1 User 3
  12. 12. Tag Recommender – Classification <ul><ul><li>Map a new image to one of the existing clusters
  13. 13. Find the nearest cluster centroid </li><ul><li>Assign representative tags of the mapped cluster to the image </li></ul></ul></ul>Representative Tags Nearest Centroid
  14. 14. Evaluation <ul><ul><li>Evaluated on tags of ~400,000 Flickr images </li><ul><li>75% training data
  15. 15. 25% test data / gold standard </li></ul><li>Data is randomly split based on users (not resources)
  16. 16. Ignored </li><ul><li>10 most frequent tags for each location (city) and
  17. 17. Tags used by less than three users </li></ul><li>Evaluation based on the correctly identified tags of test data </li><ul><li>Precision
  18. 18. Recall
  19. 19. F-Measure </li></ul></ul></ul>
  20. 20. Micro Precision
  21. 21. Micro Recall
  22. 22. Micro F-Measure
  23. 23. Coverage <ul><li>Number of different tags correctly suggested </li></ul>
  24. 24. Edge Histogram vs. Color Layout
  25. 25. Cosine vs. Euclidean
  26. 26. Euclidean vs. Manhattan
  27. 27. Conclusions <ul><ul><li>New images can be automatically tagged
  28. 28. Geographical coordinates give best results </li><ul><li>Require less computational power (just 2 dimensions)
  29. 29. No low-level image feature analysis required
  30. 30. Limited to location based tag recommendations only! </li></ul><li>Future Work </li><ul><li>Optimize individual modules of the system </li><ul><li>Clustering, representative tags, classification etc. </li></ul><li>Combination of multiple image features </li></ul></ul></ul>
  31. 31. Thank You ☺

×