The document proposes a method for detecting social events from photo metadata by applying watershed segmentation techniques to cluster images into events based on similarity of user annotations, with images assigned to the same event if they share common markers like username, date, tags or nearby GPS locations. The watershed transform treats each unique annotation as a marker and floods image regions from these markers using merging conditions to group images into homogeneous event segments, with boundaries between non-overlapping user involvement preventing images from being assigned to multiple events. External data sources on keywords, locations and semantic relatedness could help prune markers and improve precision and recall of identified social events.

1. Minh S. Dao, Truc-Vien Nguyen, Giulia
Boato, Francesco G.B. De Natale
University of Trento, Italy
@MediaEval 2012-SED task

2. Watershed transform Social Event Detection
Idea:
- People tend to introduce similar
annotations for all images associated to
the same event -> homogeneous
regions
- People cannot be involved in more than
one event at the same time -> borders
among events, stop condition
http://cmm.ensmp.fr/~beucher/wtshed.html
Target: Target:
- Markers and all pixels sharing the - Events and all images associated
same characteristic (segmented with these events
homogeneous regions)
Need: Image that needs to be segmented, Markers, Flood progress (stop-condition)

3. time (col)
... ...
username = «procsilas»
dateTaken = «2009-01-17»
username = «procsilas»
dateTaken = «2009-01-17» ...
... ...
... ... ...
username = «sarahamina» username = «sarahamina» username = «sarahamina»
dateTaken = «2009-01-13» dateTaken = «2009-01-13» ... dateTaken = «2009-01-12»
... ... ...
... ...
username = «Xaf»
dateTaken = «2009-01-10»
username = «Xaf»
dateTaken = «2009-01-10»
...
... ...
. . . .
. . . .
users(row)
. . . .
... ... ...
username = «sarahamina» username = ... username = ...
dateTaken = «2009-01-12» dateTaken = ... dateTaken = ...
... ... ...

4. photo_url
username
dateTaken
title
description
tags
locations
UT image

5. Watershed transform
Hint 1: left- and right- neighbor pixels: http://cmm.ensmp.fr/~beucher/wtshed.html
can be flooded from markers
Hint 2: one segment of one
Why?
user can be merged to another
- People tend to introduce similar
segment of another user as
annotations for all images associated
long as they share the same
to the same event.
MARKERs
- People cannot be involved in more
than one event at the same time

6. www.oxforddicti
onaries.com
www.macmillan Tf-idf (keywords
dictionary.com + semantic
relatedness)
Tf-idf (locations
+ geodistance)
- Synonym
- Semantic (keywords,
relatedness locations)
- Fixed terms
Geodistance
www.froscon.de
www.cebit.de
-----------------
www.allconferences.com Wiki
index.conferencesite.eu – list of cities of a country
www.tradeshowalert.com - List of public place of a
----------------- city
www.ieee.org http://www.infoplease.co
www.acm.org m/ipa/A0001769.html
(cities and GPS)

7. Step 1: (time direction)
- left- and right- neighbor
pixels: be flooded from
markers with mergin-
condition(time, tags, [locatio
n])
- Markers also can be merged if
they satisfy merging-
condition
Step 2: (user direction)
- Each chunk of each marker of
one user will be merged with
UT image other chunks of another users

8. Discussion
- Fixed terms (i.e. technical events that for sure are
organized only in Germany, for example CeBIT, FrosCon)
can improve Precision
- Semantic Relatedness could help to get rid of or
decrease influences of «irrelevant» or «low sematic
relatedness relevant»
- Merging-condition should be improved to increase Recall
- Geodistance sphere should be defined for each
city/public places
- Roles of external data sources are important to prune
(keywords, location) markers.
- Only English (semantic relatedness, keywords, cities
names)

9. - Visual information
- Individual Geodistance sphere
- Merging-condition w.r.t features’
influences
- Various threshold for tf-idf
- More languages (now only
English)