Metadata for video search
Trouvaille
Karel Braeckman
6 maart 2009

Overview
Challenges
Example situation
Search and annotation
 Search: Trouvaille
– Sources
– Demo
 Annotation
– Problems
– Demo
2 VRT-medialab: onderzoek en innovatie

Challenges
Indexing and retrieving video material
 Growing database of video material
 But video is not self-descriptive → metadata
 Annotation: assist archivist with modern tools
– Feature extraction
– Timecoded attributes
 Search: User-friendly application to search through the archive
3 VRT-medialab: onderzoek en innovatie

Example: situation at VRT
Search And Annotation
Search results
4 VRT-medialab: onderzoek en innovatie

Example: situation at VRT
Annotation and search with Basisplus
 Basisplus database contains 625 000 items
 65 fields (tfw, dienst, formaat, personen, …)
 Some fields should be chosen from controlled list:
the VRT video thesaurus
– Some hierarchy:
‘lead’, ‘broad’ / ‘narrow’ and ‘use for’ terms
– More than 300 000 lead terms
5 VRT-medialab: onderzoek en innovatie

Basisplus limitations
Annotation by archivist
- Knowledge of all fields and thesaurus
- Familiar with annotation process
But a program maker
- Does not know in what field to type
- Might use a different term, not contained in the thesaurus
- Textual results → limited representation of video
6 VRT-medialab: onderzoek en innovatie

The ideal search application
We want all relevant search results
Without any false positives
In practice a balance between the two
User friendly
Clear list of results
Find desired result fast
Jump to relevant part in video
 Time coded metadata
7 VRT-medialab: onderzoek en innovatie

Sources used for Trouvaille
Current metadata archive
(Basisplus)
Media Asset
Management
News feed
System
(Ardome)
News broadcasts
(Ardome)
+ VRT video-Thesaurus
+ List of Belgian cities
8 VRT-medialab: onderzoek en innovatie

Trouvaille Demo
9 VRT-medialab: onderzoek en innovatie

Annotation
Manual annotation is a lot of work assist archivist with
modern tools
– Annotate efficiently
– As much automation as possible
Need for time coded metadata
 Metadata describes the complete video item
 Time codes could mark the time range on which the metadata
applies.
A lot of metadata is lost during production process
 Teletext subtitles
 News rundown
 Scripts with shots / scenes and their contents
10 VRT-medialab: onderzoek en innovatie

annotation
11 VRT-medialab: onderzoek en innovatie

annotation
Assist archivist by partial automation
 Split video in shots, audio segments, scenes
– Archivist can easily add time coded metadata to a scene
– Relevant thumbnail for every shot
 Detection, even recognition of faces
12 VRT-medialab: onderzoek en innovatie

annotation
Assist archivist by partial automation
 Split video in shots, audio segments, scenes
 Detection, even recognition of faces
 Detect reuse and duplicates
13 VRT-medialab: onderzoek en innovatie

Conclusions
New search interface Trouvaille
 User friendly
 Get desired result fast
 Range of views
Result is a complete item in Trouvaille
 Annotation with time coded metadata (per scene)
 Search results should use this time codes
14 VRT-medialab: onderzoek en innovatie

Conclusions
Annotation application
 Partial automation → efficient annotation
 Time coded metadata
Too often reverse engineering of metadata that was
readily available at some stage during production process
15 VRT-medialab: onderzoek en innovatie