Cluster based landmark and event detection for tagged photo collections

1,658 views

Published on

A simple presentation of the article: "Cluster-based landmark and event detection for tagged photo collections" on the IEEE MultiMedia magazine.
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5611558

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,658
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Cluster based landmark and event detection for tagged photo collections

  1. 1. Cluster-based Landmark and Event Detectionon Tagged Photo CollectionsSymeon Papadopoulos, Christos Zigkolis,Yiannis Kompatsiaris, Athena Vakali
  2. 2. user generated content creates newopportunities
  3. 3. real-world depicted in users’ online collections
  4. 4. potential for many insights into what peoplesee, do and like need new tools for content organization
  5. 5. image clustering
  6. 6. clusters  landmarks + events landmark event
  7. 7. the framework
  8. 8. + +photos tags geo
  9. 9. overview1 2 landmark landmark event4 3
  10. 10. step 1: create photo similarity graph 1 2 landmark landmark event 4 3
  11. 11. SURF SIFTvisual similarity casa mila, la pedrera tag similarity co-occurrence latent semantic indexing
  12. 12. step 2: use graph to cluster the photos 1 2 landmark landmark event 4 3
  13. 13. the concept of node structure neighborhood of node v + node itself = structure of node v v v v N(v) v Γ(v)
  14. 14. the concept of structural similarity (1) v u Γ(v) ∩ Γ(u) structural similarity between nodes v and u Γ(v)  Γ(u)
  15. 15. the concept of structural similarity (2) high structural similarity photo cluster 1 C A B photo cluster 2 low structural similarity
  16. 16. # edgescomplexity O (km  m) graph-based clustering average node degree # dimensions # clusters k-means clustering O (I  C  n  D) # iterations # nodes O (n2  log n) hierarchical agglomerative clustering
  17. 17. step 3: detect landmarks & events 1 2 landmark landmark event 4 3
  18. 18. #users / #photos baseline features [2 years, 50 users / 120 photos] [1 day, 2 users / 10 photos] Quack et al., CIVR 2008 duration
  19. 19. Landmark Tags additional features Event Tags
  20. 20. step 4: post-process landmark clusters 1 2 landmark landmark event 4 3
  21. 21. cluster merging based on proximity
  22. 22. cluster tag filtering CLUSTER TAGS helado tropical barcelona cielos spain field park güell jaume oller park sclupture el beso low frequency tags generic tags
  23. 23. results
  24. 24. 207,750 photos7,768 users33,959 unique tagscompare graph-based vs. k-means clustering user study geospatial coherence high geospatial coherence low geospatial coherence
  25. 25. user study VISUAL precision recall κ-statistic graph-based 1.000 0.110 1.000 k-means 0.806 0.324 0.226 TAG precision recall κ-statistic graph-based 0.950 0.182 0.820 k-means 0.848 0.307 0.564
  26. 26. geospatial coherence VISUAL radius std. deviation graph-based 357 m 1.18 km k-means 2.4 km 1.73 km TAG graph-based 456 m 1.15 km k-means 767 m 1.76 km
  27. 27. classification performance 16% - 23% improvement thanks to tag features
  28. 28. landmark localization accuracy sagrada familia, cathedral, catholic 15.2m la pedrera, casa mila 31.8m parc guell 9.6m boqueria, market, mercado, ramblas 82.1m camp nou, fc barcelona, nou camp 18.7m
  29. 29. event category composition music, concert, gigs, dj 43.1% conference, presentation 6.5% local traditional, parades 4.6% racing, motorbikes, f1 3.3%
  30. 30. clusttour www.clusttour.grtwitter.com/clusttour facebook.com/clusttour

×