This document discusses challenges in searching large video archives due to lack of descriptive metadata. It introduces Trouvaille, a new search interface that sources metadata from existing archives and annotations. Key features include searching across different metadata fields, returning time-coded results to allow jumping to relevant video segments, and tools to assist annotators by automatically detecting shots, audio segments and faces to add metadata more efficiently. The goal is to make the most of existing metadata through improved search and annotation tools.

1. Metadata for video search
Trouvaille
Karel Braeckman
6 maart 2009

2. Overview
Challenges
Example situation
Search and annotation
 Search: Trouvaille
– Sources
– Demo
 Annotation
– Problems
– Demo
2 VRT-medialab: onderzoek en innovatie

3. Challenges
Indexing and retrieving video material
 Growing database of video material
 But video is not self-descriptive → metadata
 Annotation: assist archivist with modern tools
– Feature extraction
– Timecoded attributes
 Search: User-friendly application to search through the archive
3 VRT-medialab: onderzoek en innovatie

4. Example: situation at VRT
Search And Annotation
Search results
4 VRT-medialab: onderzoek en innovatie

5. Example: situation at VRT
Annotation and search with Basisplus
 Basisplus database contains 625 000 items
 65 fields (tfw, dienst, formaat, personen, …)
 Some fields should be chosen from controlled list:
the VRT video thesaurus
– Some hierarchy:
‘lead’, ‘broad’ / ‘narrow’ and ‘use for’ terms
– More than 300 000 lead terms
5 VRT-medialab: onderzoek en innovatie

6. Basisplus limitations
Annotation by archivist
- Knowledge of all fields and thesaurus
- Familiar with annotation process
But a program maker
- Does not know in what field to type
- Might use a different term, not contained in the thesaurus
- Textual results → limited representation of video
6 VRT-medialab: onderzoek en innovatie

7. The ideal search application
We want all relevant search results
Without any false positives
In practice a balance between the two
User friendly
Clear list of results
Find desired result fast
Jump to relevant part in video
 Time coded metadata
7 VRT-medialab: onderzoek en innovatie

8. Sources used for Trouvaille
Current metadata archive
(Basisplus)
Media Asset
Management
News feed
System
(Ardome)
News broadcasts
(Ardome)
+ VRT video-Thesaurus
+ List of Belgian cities
8 VRT-medialab: onderzoek en innovatie

9. Trouvaille Demo
9 VRT-medialab: onderzoek en innovatie

10. Annotation
Manual annotation is a lot of work assist archivist with
modern tools
– Annotate efficiently
– As much automation as possible
Need for time coded metadata
 Metadata describes the complete video item
 Time codes could mark the time range on which the metadata
applies.
A lot of metadata is lost during production process
 Teletext subtitles
 News rundown
 Scripts with shots / scenes and their contents
10 VRT-medialab: onderzoek en innovatie

11. annotation
11 VRT-medialab: onderzoek en innovatie

12. annotation
Assist archivist by partial automation
 Split video in shots, audio segments, scenes
– Archivist can easily add time coded metadata to a scene
– Relevant thumbnail for every shot
 Detection, even recognition of faces
12 VRT-medialab: onderzoek en innovatie

13. annotation
Assist archivist by partial automation
 Split video in shots, audio segments, scenes
 Detection, even recognition of faces
 Detect reuse and duplicates
13 VRT-medialab: onderzoek en innovatie

14. Conclusions
New search interface Trouvaille
 User friendly
 Get desired result fast
 Range of views
Result is a complete item in Trouvaille
 Annotation with time coded metadata (per scene)
 Search results should use this time codes
14 VRT-medialab: onderzoek en innovatie

15. Conclusions
Annotation application
 Partial automation → efficient annotation
 Time coded metadata
Too often reverse engineering of metadata that was
readily available at some stage during production process
15 VRT-medialab: onderzoek en innovatie