Digital Media Production

Digital Media Production
Gastcollege – Nieuwe Media en Maatschappij

Quid VRT-Medialab?
+/- 22 experts – research and development
Member of international associations
EBU (P/SCAIE, P/MAG, P/NP, P/HDTV,…)
AMWA, SMPTE, IPTC, W3C
Deliverables
Expertise
Proof of Concept
Technology transfer

http://medialab.vrt.be

Table of Content
Background – Digital Media Production
Anno 2004 - The Digital Media Factory
Reconciliation of requirements
Creative staff
Case 1 – Media Asset Management
The consumer
Case 2 – Non-linear Distribution
The near future
Case 3 – Computer Integrated Manufacturing

2004...

Background – Proof of Concept

Objectives anno 2004
Objectives anno 2004

Objectives
Tape-less production
Re-purposing of material
Integrated system for radio,TV and online
Strategy - Generic if possible, specialised if required
Central storage services are the core of the architecture
The MAM system implements the central warehouse processes
State-of-the-art production systems are integrated with the MAM system
Legacy archive system provides search and retrieval functionality

Go Live - 17 June 2007
Concurrent
engineering
Processing
during
ingest/transfer
Browse Clients
Craft editors
Draft
Processing
POST
Production
Media
Asset Mgmt
Subtitling
Analysis
INGEST
PLAY-OUT
IP network
Archiving
Annotation
8 GB P2 card
= 32 min video
DV25 (news)
Central
Storage
Follow-up
continuity
Post-
sonorisation
Editor-in-Chief
Central storage

Lessons Learned
There is a mismatch between business requirements and state-of-the-art technology;
Media technology suppliers don’t acknowledge the limitations of IT technology;
IT suppliers don’t sufficiently understand the requirements of (media) production;
Re-using applications in a context which they have not originally been designed for is usually a sub-optimal solution;
Proper reasearch and development are quintessential.

Online searching, re-viewing and cutting

One-click re-purposing of content

2009 - Reconciliation of Requirements

State-of-the-Art
Single system for radio, tv and online
File-based production
Re-use of content
But – Needle in the haystack
Unstructured information
10 times more items
No native file-based tools available
Media Asset Mgnt. - Effective re-use
Harvesting and Annotation
Search and Retrieval
Non-linear distribution
on demand
Personalisation
Computer Integrated Manufacturing
CAD – Computer Assisted Design
CAM – Computer Assisted Manufacturing

Lessons Learned – Principles of System Integration
Everything should be made as simple as possible, but not one bit simpler.

Media Asset Management
Production and distribution
Infrastructure –
Networks and Storage
Production Platform

The problem (1) – Production Context
Correspondents
(email)
Archives
Documentation
Services
Web Sites
Eurovision (Superpop– NMS)
Reuters
(NewsML1.1)
EBU popserver
(NMS)
MAM system
‘Wires’ - (Telex Services)
(IPTC7901)
Lack of an effective standard

Unstructured or semi-structured information
Redundant information – Overflow
Manualprocessing

Newsroom Computer System

The problem (2) – videoprocessing
« Prosperity in life istransitory and dependant on luck »
The semantic gap
-> interpretationis unique to audiovisual media
-> The source of valuablemetadataisProduct Engineering
-> Computer vision techniques canonlyrecogniselower-levelfeatures
Allegory of Fortune, DossoDossi (Courtesy of the J.Paul Getty museum)

For example…
Master Data Management
Infrastructure –
Production Platform

The State of the Art
archiefnummer : ALG 20010813 1
fragmentnummer : 1
reeks : 1000 ZONNEN EN GARNALEN
bandnummer : E03024404
formaat : DBCM
fragmenttitel : 1000 ZONNEN & GARNALEN
beeld : KL/PALPLUS
fragmentduur : 18 20
tekst : 0'00" TOERISTISCH REPORTAGEMAGAZINE OVERZICHT
ONDERWERPEN GENERIEK TOERISTISCH REPORTAGEMAGAZINE,
OVERZICHT ONDERWERPEN
0'50" VANDAAG : KUNSTENAAR LUC HOFKENS ONTWIERP EEN OASE
OP ZIJN DAKTERRAS IN BORGERHOUT DIE DOET DENKEN AAN DE
GRAND CANYON INTERVIEW MET LUC EN ZIJN VROUW
MARILOU BUITENBEELD DAK MET OMGEVING BUITENKANT
ARBEIDERSWONING, PANO OVER ROTSWANDEN, KRATEN MET WATER,
BEPANTING, FOTOALBUM MET VERLOOP WERKEN
4'00" JUNIOR : KLAARTJE ALAERTS, 13 JAAR WIL ASTRONAUTEN
WORDEN ZE BEZOEKT HETEUROSPACE CENTER METRUIMTEVEREN,
RAKETTEN SIMULATIE IN RUIMTEVEER, INTERVIEW, HEEFT EEN
UFO GEZIEN MAAKT ZELF KLEIN RAKETJE, SCHIET HET AF
7'50" DE SCHEURKALENDER : ARCHIEF RECLAMEFILM IBM
INTERVIEW MAURICE DE WILDE, EERSTE PERSOONLIJKECOMPUTER
trefwoorden : BELGIE; BORGERHOUT; ARTIEST; OASE; KUNST; GRAND
CANYON (NATUURGEBIED); DAK; TERRAS; INTERVIEW; EURO
SPACE CENTER; RUIMTEVAART; PC; BOOTTOCHT; RIJKDOM;
PASSAGIER; GASTRONOMIE; RESTAURANT; PERSONEEL;
VAKANTIE; BINNENBEELD; SCHIP; BECKERS LEEN; VRT;
LOTTO; RADIOOMROEPSTER; KLANKSTUDIO; UITVINDING;
BARBECUE; BETONMOLEN; IBM; RECLAMESPOT
rechthebbende : VRT
Opzoekscherm FILM Set: 16 Aantal: 1
blz 1 van 3
trefwoorden: ibm and vrt
archiefnummer: -
uitzendjaar: maand: dag:
fragmentnummer: fragmentduur:
reeks:
formaat: bandnummer:
aflevering: afleveringsnummer:
programma: uitzenddatum:
fragmenttitel:
tekst:
kategorie:
opnamedatum: opnamenummer:
journalist: rechthebbende:
SETS
The strings required for the operation are not defined
F11 F12 F13 F14 F17 F18 F19 F20 Ent
Eindigen Sets Refset Toon Vorige Volg/Leeg Thesaurus Commando Opzoeken

Proof of Concept

Shot Segmentation and Scene Recognition

Face recognition and identification

Video copy detection

Identify dupplicates– Trace dependencies
Linking stories over time
Copy detection - Intellectual Property Protection

Media Asset Management Issues

Production Environment
No upfront selection - information overload;
Multiple sources of more or less structured metadata;
Parallel sources cause dupplicates;
International context - multilingual

Video Processing
80% of the context is in the speech
Machines may recognise more objects
Machine processing is more robust

Infrastructure –

Professional Requirements
An item is relevant or it is not;
Retrieve all relevant items;
No false positives

Production Platform

Conclusions

Scripts contain the essential master data – integration with creative processes is a must;
In professional context, it is key to trace dependencies between objects. Hence, as opposed to any conventional approach, a search engine must integrate information rather than to index multiple sources;
Computer Vision does not replace formal annotation, but substantially accelerates the process and increases the accuracy

!
A consistent application framework and proper user-interfaces are the key to effective re-use and supply chain integration in general.
(≠ Digitizing analogue and disintegrated workflows)

Case 2 – Non-linear Distribution
Data
Video
Audio
Communication
(Information)
Meta
Data
Meta
Data
Infrastructure –
Digital Broadcasting via DVB, IP or other
Production Platform

1914 – First radio broadcast

1937 – Die Volksempfaenger

1955 - Mobile radio...

Digital Broadcasting – Que?

2004 – Live Streaming

Porting the radioplayer...

Stats

Launch feb 2009
Currently over 60.000 installations
2500 users per day
Average duration 30”

Business Case
IP Distributie (radio)
+/- 5.000.000 luisteruren per dag
5.000.000 uur à 128 kbps, 250 TB
90 PB per jaar à 70.000€ per PB
Totaal – 6.300.000€ per jaar
Vgl. deredactie.be

Homepage 2MB
128 seconden high-def audio(!)

Broadcast...
A customer can have a car painted any colour that he wants so long as it is black.(Henry Ford)

Watch anytime - BBC iPlayer

Good, better,...
Portable - Anywhere

Radio-player application
Quick win – “in series” live transcoding

Broadband distribution - Anytime

“Podcast”, “iPlayer”
On demand complements live offering
Major impact – master data power-lifting

?
The Perfect Storm…

Mobile – Portable and impulse-driven
Internet - Seamless integration of live and on demand
An excellent user interface…

Why it might work this time...

Why it might work this time...
Radio and Telephone

Radio and telecommunication are on the cold side
Short attention span, high level of attention
Impulse-driven
Mobile was the killer app…

Proof of Concept - The Ultimate Radio Exp(eriment)
myRadio @time of writing

Broadband connection enables non-linear distribution
Adaptive streaming enables bandwidth-friendly distribution
Excellent quality where possible, lower bit-rate if necessary
Open distribution framework

http://myradio.vrt.com

Proof of Concept (Ongoing)
Seamless integration of Live and On demand

“Bookmark” by tagging
Pause, Back and Fast Forward

Work in Process – Track each individual item

Audio segmentation
Speech recognition and indexing
Metadata harvesting (!!!)

Personalisation

Radio search engine
Assemble myRadio

Personalisation by Interactivity
User profile
Personalisation
A customizable product and tracking the user profile based on data mining techniques will allow us to create the illusion of personalisation

Trend - From « metadata » to CAD/CAM
?

Case 3 – Computer Integrated Manufacturing

State of the Art

Use Case – Drama Production

Proof of Concept - Scoop
http://www.youtube.com
(search for “scoop”, “ibbt”, “CAD”)

54
http://medialab.vrt.be
http://myradio.vrt.be
Maarten.verwaest@vrt.be