Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Spatial Segmentation improves the Multimodal Geo-Tagging

918 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

How Spatial Segmentation improves the Multimodal Geo-Tagging

  1. 1. Pascal Kelm Kelm@nue.tu-berlin.de Communication Systems Groupwww.nue.tu-berlin.de Technische Universität Berlin Thursday, 04 October 2012
  2. 2. What is meant by Spatial Segmentation? 2 World map is iteratively divided into segments of different sizes Kelm: How Spatial Segmentation improves the Multimodal Geo-
  3. 3. Run4: only audio/visual information 3Descriptions are pooled for each spatial segment (k-d tree) inthe different hierarchy levelVisual nearest neighbour search in lowest hierarchy Kelm: How Spatial Segmentation improves the Multimodal Geo-
  4. 4. Visual Region Model 4 Returns the visually most similar areas, which arerepresented by a mean feature vector of all training imagesand videos of the respective area Kelm: How Spatial Segmentation improves the Multimodal Geo-
  5. 5. Run4: Results 5 UG-CU 100Th [km] TUB [%] [%] 901 0,1 0,110 0,1 0,7 8020 0,1 0,9 7050 0,1 1,1 Accuracy [%] 60100 0,2 2,6 50200 0,8 6,9 TUB UG-CU 40500 4,1 14,71000 14,8 21,2 302000 44,5 28,5 205000 81,0 29,6 1010000 98,7 91,4 015000 100,0 95,7 1 10 20 50 100 200 500 1000 2000 5000 100001500020000 Margin of Error [km]20000 100,0 100,0 Kelm: How Spatial Segmentation improves the Multimodal Geo-
  6. 6. Run1: No additional data or gazetteers 6combines textual and visual features: translation of tags andextracted words (NLP) from the title and the description.Porter stemmer and stop-word elimination for each segmentand granularity in the spatial segmentation.Visual Search for the k-nearest segments in the lowesthierarchy Kelm: How Spatial Segmentation improves the Multimodal Geo-
  7. 7. 7Term-location-distribution:Term frequency-inverse document frequency: Kelm: How Spatial Segmentation improves the Multimodal Geo-
  8. 8. Example 8 Condence scores of the visual approach (right) restricted to be in the most likely spatial segment determined by the textual approach (left) Kelm: How Spatial Segmentation improves the Multimodal Geo-
  9. 9. Run1: Results 9Th [km] TUB [%] 1001 13,7 9010 32,7 8020 36,5 7050 39,4100 41,8 60 Accuracy [%]200 44,8 50500 51,7 40 TUB1000 62,4 302000 76,5 205000 92,310000 99,4 1015000 100,0 0 1 10 20 50 100 200 500 1000 2000 5000 10000 15000 2000020000 100,0 Margin of Error [km] Kelm: How Spatial Segmentation improves the Multimodal Geo-
  10. 10. Run2: No additional data 10For the highest hierarchy level the boundaries extraction usinggazetteers (GeoNames, Wikipedia and Google Maps) for thespell checked words is added. Kelm: How Spatial Segmentation improves the Multimodal Geo-
  11. 11. Collaborative Systems: Example 11這是我上次去巴黎。在那裡,我得到了我的城堡在迪斯尼樂園看。… 這是我上次去巴黎。在那裡,我得到了我的城堡在迪斯尼樂園看。 Kelm: How Spatial Segmentation improves the Multimodal Geo-
  12. 12. Collaborative Systems: Example 12 這是我上次去巴黎。在那裡,我得到了我的城堡在迪斯尼樂園看。…Which language is it? Chinese This was my last trip to Paris. I visited the castle in Disneyland…Which words gives us information? Tags? Trip, Paris, Castle, DisneylandWhich of these nouns have got geographical information? Paris, Disneyland Kelm: How Spatial Segmentation improves the Multimodal Geo-
  13. 13. Geographical Ambiguity 13 Paris Disneyland R(ci) = Rank sum France China ci = Countries N = Number of toponym Canada USA Puerto France Rico … … Kelm: How Spatial Segmentation improves the Multimodal Geo-
  14. 14. Extracted geo. items 14 kauii hawaii usa00001: hawaii, kauai, usa Kelm: How Spatial Segmentation improves the Multimodal Geo-
  15. 15. Results 15 100 90 80 70 60Accuracy [%] 50 Run1 Run2 40 Run4 30 20 10 0 1 10 20 50 100 200 500 1000 2000 5000 10000 15000 20000 Margin of Error [km] Kelm: How Spatial Segmentation improves the Multimodal Geo-
  16. 16. Question 16Thanks for your attention! Kelm: How Spatial Segmentation improves the Multimodal Geo-
  17. 17. Training Set: Weighting 17 Kelm: How Spatial Segmentation improves the Multimodal Geo-
  18. 18. Training Set: Features 18 Kelm: How Spatial Segmentation improves the Multimodal Geo-

×