This paper describes the work done by the JRS team for the linking sub-task. We submitted eight pairs of runs: four with dierent textual resources only, two using reranking based on visual similarity, and two using concept detection results. Each of the pairs contains of one run using the anchor segment only, and one using a longer context seg- ment. The results show higher variance between anchors than for the 2013 task, also the dierences between runs us- ing dierent textual resources are more salient. The use of the context does not generally improve results, and visual reranking provides small improvements

1. JOANNEUM RESEARCH
Forschungsgesellschaft mbH
DIGITAL
Institute for Information and
Communication Technologies
Steyrergasse 17
8010 Graz, Austria
Tel. +43 316 876-5000
Fax +43 316 876 - 5010
digital@joanneum.at
www.joanneum.at/digital
www.icosole.eu
Acknowledgements:
The ICoSOLE project receives funding
from the European Union’s Seventh
Framework Programme
(FP7/2007-2013) under grant
agreement n° 610370.
digpos322 014
Approach
JRS at Hyperlinking of Television Content Task
Generate basic links based on text and metadata
q Inputs: subtitles or ASR transcripts, metadata (synopsis)
q Text preprocessing: remove punctuation, normalize capitalization,
remove stop words/very short words
q Obtain set T of terms found in DBpedia
q Add synonyms ST for terms T from WordNet
q Select connected entities CT for the terms in T from FreeBase
q For geographic locations Tg T (as identified by FreeBase), select
related geographic entities GTg from GeoNames
q Weight for matching as follows:
Experiments
Werner Bailer, Harald Stiegler
q For multiple occurrences per segment, determine total weight as
Total similarity between segments v1, v2 determined as
q Use fixed segments of 20 seconds
Use of visual concepts (in some runs)
q Treat as text annotations
q Concepts only or subtitles/ASR plus concepts
Rerank based of visual matches
q Use VLAT signatures of SIFT descriptors (from DOG points)
q Every 5th frame (using one field when interlacing artifacts
are present)
q Rerank top 50 segments
Fig. 1: MAP of submitted runs
Eight pairs of runs
q Pair: one with anchor segment, one with context segment
(+/- 3 minutes)
q 4 runs: subtitles and the different text resources
q Visual reranking for subtitles and LIMSI
q Visual concepts with subtitles and visual concepts only
Results
q Text only methods are very similar to those used in 2014:
results are overall worse
ICoSOLE
q More differences between anchors
q More differences between different text resources
q Large difference between anchor only and context for
some runs
q Overall, context is only slightly better
q Visual reranking adds slight improvement
q Visual concepts: cannot be treated like other annotations,
not sufficient on their own, no improvement when
added to text