CERTH @ MediaEval 2011 Social Event Detection Task


Published on

The presentation of the CERTH presentation in the Social Event Detection (SED) task @ MediaEval (Pisa, 2 September 2011).

1 Comment
  • The framework described in this presentation has been made publicly available in the form of a Java library: http://mklab.iti.gr/project/sed2011_certh
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CERTH @ MediaEval 2011 Social Event Detection Task

  1. 1. CERTH @ MediaEval 2011 SocialEvent Detection TaskSymeon Papadopoulos, Christos Zigkolis, YiannisKompatsiaris, Athena VakaliPisa, 1-2 September 2011
  2. 2. The problem• Identify social events in tagged photos collections: – Challenge 1: Soccer matches @ Barcelona, Rome – Challenge 2: Events @ Paradiso (Amsterdam) and Parc del Forum (Barcelona)• Alternative formulation: – For each photo of the collection answer the questions: Q1. Is this photo related to a social event of the given types? Q2. If yes, to which event is it related? – Points to classification and clustering as methods to address the problem. 2
  3. 3. Approach Q1 Q2 Q1 / Q2 3
  4. 4. Photo Filtering (1)• City classification – If geo-tagging available (~20%), use it  simple nearest-neighbour classifier – If not, match against city-specific tag models: • Created from processing independent geo-tagged photo collections TAG MODEL SAMPLES Amsterdam (74) Barcelona (57) London (89) Paris (51) Rome (42) amsterdam barcelona london paris rome netherlands catalunya uk france italy holland catalonia united kingdom francia vaticano nederland españa great britain versailles italia …. …. …. …. …. 4
  5. 5. Photo Filtering (2)• Soccer/Venue classification – In the case of venue classification, use geo-tagging information if available. – Match against soccer/venue tag model: • Parameter (cf. evaluation) TAG MODEL SAMPLES (baseline)Soccer (53, m1,b) Paradiso (6, m2,b) Parc del Forum (8, m2,b)soccer paradiso parc del forumfootball names of Spanish FCs concert primavera sound +goal names of Italian FCs festival concertgoalkeeper gig festival… live music … + domain names of scheduled bands (last.fm) knowledge 5
  6. 6. Event Partitioning• Very simple implementation: – Find all unique dates of photos that “passed” the first filtering step. – For each date, find all associated photos and split them into groups based on the city they are classified (same classifier as in Step 1). – Consider the resulting groups of photos, as the set of events. 6
  7. 7. Event Expansion• Expand in three ways: – Photos having the same owner as one of the owners in the event & captured at the same date. – Photos captured at the same location (<200m) with the event center & at the same date (only for geo-tagged photos) – Photos belonging to the same cluster (by use of method [1]) & having the same owner as one of the owners in the event (parameter: cluster type) [1] S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, A. Vakali. “Cluster-based Landmark and Event Detection on Tagged Photo Collections”. In IEEE Multimedia Magazine 18(1), pp. 52-63, 2011 7
  8. 8. Evaluation (1) Challenge 1NotationParameter 1 (p1): m1,b (baseline tag model), m1,+ (extended soccer tag model)Parameter 2 (p2): tt (use photo title + tags), ttd (use photo description + tt)Parameter 3 (p3): ∅ (no clustering), T (tag-based clustering), V (visual clustering) 8
  9. 9. Evaluation (2) Challenge 2NotationParameter 1 (p1): m2,b (baseline tag model), m2,+ (extended venue tag model)Parameter 3 (p3): ∅ (no clustering), T (tag-based clustering), V (visual clustering), H (hybrid clustering)m2,+ was created by adding to baseline the names of the bands that played in these venues in the same month (collected from last.fm API) 9
  10. 10. Failure examples (1)C1 - Run1 / False positives3559542192 3618132279 3580841609Title: AVUÍ SOM 77.331 Title: Sant Pere Title: roma 09.Tags: …, Campions, Trophy, Tags: Barcelone, Barcelona, Tags: rome, italy coliseum,campnou, soccer, football, Night Ambiance, Light palatino, chuch, soccer,caosasuna, barça, fiesta, … statues, artMany of the photo tags Captured at the same Just one of the tagsare related to soccer and date and in the vicinity of (soccer) is related toeven to a soccer event the event. soccer.(fiesta, champions). Most of the false positives were due to the expansion step (i.e. same day + close by, or same day + same user) 10
  11. 11. Failure examples (2)C1 - Run2 / False negatives3559542192 3571654936 3583033760Title: near Tor di Quinto, Title: Barcelona v.Latium, Italy Title: DSC_0029 Manchester UnitedTags: N/A Tags: FC Barcelona Fiesta Tri Tags: Sigma 10-20mm, F4-5.6 CampionsDescription: s.s. lazio wins EX DC HSM, barcelona, spain,the coppa italia moo2Here the event The information could be Event information isinformation is only inferred from title if our tag encoded in a single tag,present in the photo model contained FC names but we don’t tokenizedescription. from different countries. tags, so we miss it. Most of the false negatives were due to failure in matching the textual metadata of photos to the soccer tag model. 11
  12. 12. Discussion (1)• Most important factor: – a good tag model to be used for classification• Marginal contribution of clustering: – expansion by spatio-temporal metadata already captures most related photos – tag-based clusters tend to include many of the photos of the same user at the same date – visual clusters did not yield further improvements as one would hope (at least with employed visual similarity measure: 500 feature vector from clustering SIFT features) 12
  13. 13. Discussion (2)• Future action: study in detail failure cases and make necessary modifications to approach• Ways to improve: – better topic/entity classification methods • better/richer tag models + text matching methods • more sophisticated methods: e.g. SVMs, relational learning + more discriminative features (text, visual, social) – more elaborate city classification methods or even precise geo-tagging methods 13
  14. 14. Questions 14
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.