IMPROVING IMAGE TAG RECOMMENDATION                                                                     USING FAVORITE IMAG...
Upcoming SlideShare
Loading in …5

Improving Image Tag Recommendation Using Favorite Image Context


Published on

Improving Image Tag Recommendation Using Favorite Image Context.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Improving Image Tag Recommendation Using Favorite Image Context

  1. 1. IMPROVING IMAGE TAG RECOMMENDATION USING FAVORITE IMAGE CONTEXT Wonyong Eom, Sihyoung Lee, Wesley De Neve, and Yong Man Ro Image and Video Systems Lab Korea Advanced Institute of Science and Technology (KAIST) Daejeon, South Korea e-mail: website: INTRODUCTION III. EXPERIMENTS- Observation 1. Experimental setup - the number of images shared on online social network services - We collected images from Flickr users meeting the following keeps growing at a fast rate requirements: 1) uploaded at least 100 images, 2) assigned at least- Problem 500 tags, and 3) bookmarked at least 500 favorite images - manual tagging of images is labor intensive and time consuming, - Consequently, using the Flickr API, we retrieved a total of 387,397 making it difficult to facilitate effective image retrieval images from 27 users (on September 30, 2010)- Novel solution - the images retrieved are either favorite images or images owned by - personalized tag recommendation using favorite image context the 27 users, and are annotated with 4,657,288 tags by 46,686 users • source of collective knowledge that consists of images and - To calculate visual distance, we used global and local image features associated tags that have been bookmarked by a particular user - assumptions made 2. Effectiveness of using favorite image context • favorite images and their associated tags are indicative of the - Recommending tags using tag statistics: Rtag(t, q) Context P@5 S@5 P@1 visual and topical interests of a user Personal 0.158 0.609 0.318 • people actively bookmark favorite images Collective 0.208 0.612 0.373 Favorite 0.247 0.729 0.457II. PROPOSED TAG RECOMMENDATION METHOD - Recommending tags using visual similarity: SCD-based Rimg(t, q)1. Number of favorite images on Flickr for users of MIRFLICKR-25000 Context P@5 S@5 P@1 Personal 0.187 0.629 0.384 1000000 Collective 0.208 0.611 0.324 Favorite 0.294 0.813 0.446 Number of favorite images 100000 - Recommending tags using visual similarity: BoVW-based Rimg(t, q) 10000 Context P@5 S@5 P@1 1000 Personal 0.206 0.697 0.367 Collective 0.309 0.767 0.523 100 Favorite 0.317 0.813 0.513 10 3. Influence of linear fusion and bookmarking activity 1 1 9861 0.4 187 2538 6872 8855 9861 User 0.3 Fig. 1. Number of favorite images per MIRFLICKR-25000 user2. Personal, collective, and favorite image context P@5 0.2 tag statistics visual similarity (SCD) 0.1 visual similarity (BoVW) tag statistics + visual similarity (SCD) tag statistics + visual similarity (BoVW) personal context 0 Level 1 Level 2 Level 3 Level 4 Type of user group favorite image context Fig. 3. P@5 for users with different levels of bookmarking activity ... 4. Example query images collective context ... tag statistics + query image tag statistics visual similarity visual similarity Fig. 2. Relation between personal, collective, and favorite image context, visualized nature, africa, photo, nature, wildlife, from the point-of-view of a user who uploaded a new image that is to be annotated nature, wildlife, image, macro, birds, africa, macro, birds, africa, moth, bird, wildlife, flower, animal, 3. Mathematical modeling animal, bird, flower, birds, flowers, safari,-The relevance of a set of tags Tq to the content of a query image q safari, flowers   macro, Australia butterflyTq  t t  T and R(t , q)   tag-R(t, q) is modeled by linearly fusing the output of two relevance functions sardegna, mare, italy, bw, red, italy, bw, red, green, donna, fitness, street, milano, green, street, street, milano,R(t , q)    Rtag (t , q)  (1   )  Rimg (t , q) red, luce, bw, green, silhouette, people, silhouette, light,-Rtag(t, q) is modeled by making use of tag statistics light paris, canon sardegna, shadow P(t | v), if P(t | v)  0R tag (t , q)  P(t )  vV   , otherwise Fig. 4. Example images with tags recommended using favorite image context-Rimg(t, q) is modeled using a MAP-based method IV. CONCLUSIONS P(q | t , Q ) P(t | Q ) - Tag recommendation using favorite image context is, for the usersR img (t , q)  P(t | q, Q )  , selected, more effective than the use of personal and collective context P(q | Q ) - Linearly fusing tag statistics and visual similarity allows for a higher effectiveness in terms of P@5, compared to their separate usage IEEE International Conference on Image Processing (ICIP), September 2011, Brussels (Belgium)