This paper presents our approach and results in the Search and Anchoring in Video Archives task at MediaEval 2015.The Search part aims at returning a ranked list of video segments that are relevant to a textual user query. The Anchoring part focuses on the automatic selection of video segments, from a list of videos, that can be used as anchors to encourage further exploration within the archive. Our approach consists in structuring each video into a hierarchy of topically focused fragments, to extract salient segments in the videos at different levels of details with precise jump-in points for them. These segments will be leveraged both to answer the queries and to create anchor segments, relying on content based analysis and comparisons. The algorithm deriving the hierarchical structure relies on the burstiness phenomenon in word occurrences which gives an advantage over the classical bag-of-words model. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

1. IRISA at MediaEval 2015:
Search and Anchoring in Video
Archives Task
Anca Simon, Pascale Sébillot, Guillaume Gravier

2. System overview
Textual representation of a video cos
similarity
W1 W2 … Wn
V1 V2 … Vm
visual query
textual query
search
Rank all segments
based on the cohesion
score:
anchor
Goal
 Segments of variable length, topically focused at various levels of
details with precise jump-in points
214/09/2015 MediaEval

3. Leverage the burstiness phenomenon in word occurrences
•Bursty words: characterized by long inter-arrival times followed by short inter-
arrival times;
• Non-bursty words: exhibit inter-arrival times with smaller variance.
Starting point: Kleinberg’s algorithm (Kleinberg, 2002)
Hierarchy of topically focused
fragments (HTFF)
314/09/2015 MediaEval

4. Hierarchy of topically focused
fragments (HTFF)
Agglomerative
clustering of burst
intervals
For each term in the textual representation of a video compute the hierarchy
of burst intervals.
414/09/2015 MediaEval

5. Search sub-task
0.01 -> 10.5
(place, ballroom, color, king, royal, rock, tweed, palace, etc.)
0.01 -> 0.10
(ballroom)
0.01- > 1.50
(royal, ballroom, palace, etc.)
0.18 -> 1.50
(royal, site, etc.)
1.33 -> 1.50
(royal, palace)
2.29- > 3.21
(build, prosperous)
7.43- > 8.35
(friend, Picasso)
… …
…
Automatic transcript: Castle in the country (bbctwo)
[start time: 0.01 -> end time: 29.23]
Text query:
history, palace, tudor
(I am looking for documentaries
on British history)
514/09/2015 MediaEval

6. Search sub-task (contd.)
34.54 -> 39.52
(woman, store, train, roof, racer, mirror, route, cowboy hat, etc.)
36.38- > 36.42
(race car)
36.49 - > 37.1
(train, store, route)
39.3 -> 39.52
(girl, cowboy hat)
…
Visual query:
train
(I am looking for footage
from the history of the
British railways)
…
…
relevant
Visual concepts representation: Comedy map of Britain (bbctwo)
[start time: 0.01 -> end time: 59.55]
614/09/2015 MediaEval

7. Search results
SEARCH Binned relevance Tolerance to irrelevance
P@5 P@10 P@20 P@5 P@10 P@20 P@5 P@10 P@20
Textual query 0.34 0.31 0.19 0.34 0.31 0.19 0.34 0.31 0.19
Visual query 0.12 0.11 0.06 0.12 0.11 0.06 0.12 0.11 0.06
714/09/2015 MediaEval

8. Anchoring results
ANCHORING
Precision Recall MRR
LIMSI 0.557 0.435 0.773
MANUAL 0.469 0.38 0.735
- average number of anchors/video: 19.03 (CI at 95% [18.4,19.65] )
814/09/2015 MediaEval

9. Conclusion and future work
Leveraged a new topical structure for search and anchoring:
Hierarchy of topically focused fragments
-relies on the burstiness phenomenon in word reoccurrences
Future work:
• Study the impact of the granularity levels in the HTFF;
• Combine visual and textual bursts;
• Integrate semantic relations in the burst detection
algorithm.
914/09/2015 MediaEval

10. Kleinberg’s algorithm
1014/09/2015 MediaEval