How Spatial Segmentation improves the Multimodal Geo-Tagging
Upcoming SlideShare
Loading in...5
×
 

How Spatial Segmentation improves the Multimodal Geo-Tagging

on

  • 685 views

 

Statistics

Views

Total Views
685
Views on SlideShare
685
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

How Spatial Segmentation improves the Multimodal Geo-Tagging How Spatial Segmentation improves the Multimodal Geo-Tagging Presentation Transcript

  • Pascal Kelm Kelm@nue.tu-berlin.de Communication Systems Groupwww.nue.tu-berlin.de Technische Universität Berlin Thursday, 04 October 2012
  • What is meant by Spatial Segmentation? 2 World map is iteratively divided into segments of different sizes Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Run4: only audio/visual information 3Descriptions are pooled for each spatial segment (k-d tree) inthe different hierarchy levelVisual nearest neighbour search in lowest hierarchy Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Visual Region Model 4 Returns the visually most similar areas, which arerepresented by a mean feature vector of all training imagesand videos of the respective area Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Run4: Results 5 UG-CU 100Th [km] TUB [%] [%] 901 0,1 0,110 0,1 0,7 8020 0,1 0,9 7050 0,1 1,1 Accuracy [%] 60100 0,2 2,6 50200 0,8 6,9 TUB UG-CU 40500 4,1 14,71000 14,8 21,2 302000 44,5 28,5 205000 81,0 29,6 1010000 98,7 91,4 015000 100,0 95,7 1 10 20 50 100 200 500 1000 2000 5000 100001500020000 Margin of Error [km]20000 100,0 100,0 Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Run1: No additional data or gazetteers 6combines textual and visual features: translation of tags andextracted words (NLP) from the title and the description.Porter stemmer and stop-word elimination for each segmentand granularity in the spatial segmentation.Visual Search for the k-nearest segments in the lowesthierarchy Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • 7Term-location-distribution:Term frequency-inverse document frequency: Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Example 8 Condence scores of the visual approach (right) restricted to be in the most likely spatial segment determined by the textual approach (left) Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Run1: Results 9Th [km] TUB [%] 1001 13,7 9010 32,7 8020 36,5 7050 39,4100 41,8 60 Accuracy [%]200 44,8 50500 51,7 40 TUB1000 62,4 302000 76,5 205000 92,310000 99,4 1015000 100,0 0 1 10 20 50 100 200 500 1000 2000 5000 10000 15000 2000020000 100,0 Margin of Error [km] Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Run2: No additional data 10For the highest hierarchy level the boundaries extraction usinggazetteers (GeoNames, Wikipedia and Google Maps) for thespell checked words is added. Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Collaborative Systems: Example 11這是我上次去巴黎。在那裡,我得到了我的城堡在迪斯尼樂園看。… 這是我上次去巴黎。在那裡,我得到了我的城堡在迪斯尼樂園看。 Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Collaborative Systems: Example 12 這是我上次去巴黎。在那裡,我得到了我的城堡在迪斯尼樂園看。…Which language is it? Chinese This was my last trip to Paris. I visited the castle in Disneyland…Which words gives us information? Tags? Trip, Paris, Castle, DisneylandWhich of these nouns have got geographical information? Paris, Disneyland Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Geographical Ambiguity 13 Paris Disneyland R(ci) = Rank sum France China ci = Countries N = Number of toponym Canada USA Puerto France Rico … … Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Extracted geo. items 14 kauii hawaii usa00001: hawaii, kauai, usa Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Results 15 100 90 80 70 60Accuracy [%] 50 Run1 Run2 40 Run4 30 20 10 0 1 10 20 50 100 200 500 1000 2000 5000 10000 15000 20000 Margin of Error [km] Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Question 16Thanks for your attention! Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Training Set: Weighting 17 Kelm: How Spatial Segmentation improves the Multimodal Geo-
  • Training Set: Features 18 Kelm: How Spatial Segmentation improves the Multimodal Geo-