Iasa Presentatie

Hidden treasures lost forever? Speech technology for the disclosure of Dutch audiovisual archives Mies Langelaar and Willemijn Heeren

Contents
Introduction & Problem statement
Digitization/standardization in the E-repository
Speech technology for AV archives
System demonstration

Introduction
Hidden treasures of audiovisual archives lost forever?
Backlog
Data stored on deteriorating analogue carriers
Digitized and digital born data in non-standardized formats
 Digitization and international standardization needed
Often global level of description
A few keywords data unit (hour, tape, interview)
Often no content description at all, because annotation is very (time-)costly
 Reduce human effort through use of speech technology?

The approach
NWO CATCH project CHoral (2006-2010)
Goal:
investigate and develop automatic annotation and search technology for spoken word archives
Cooperation between
speech technology researchers, University of Twente
archivists, Rotterdam Municipal Archives

The test case
‘ Radio Rijnmond’ (RR) archives
city of Rotterdam's regional radio channel
initial broadcast in 1983
broadcast recordings, amounting to over 60.000 hours
partially digitized, mostly analog
partially disclosed, mostly waiting for annotation
typical of A/V archives in cultural heritage (CH)

Searching the RR archives I Minimal content descriptions per hour data

Searching the RR archives II ? ? ? ? ?

Main problems
The main problems with this example collection are:
1. a large backlog of undisclosed material  data are inaccessible for third parties
2. fairly unspecific annotations, if available
 restricted use for answering information needs
3. audio is being kept on analog data carriers or on CDs
 interactive or online search cannot be supported

Towards solutions …
Digitization/standardization in the E-repository

Digitization/standardization
in the E-repository

AV Collection of Rotterdam Municipal Archives
About 15.000 AV objects in collection
Most of this collection is on analog data
carriers
Part of the collection is on CD’s, dating form the 1980’s onwards
No standardisation in storage formats
No or minimal metadata and description of content available

Work in progress
Digitisation of the analogue audio material is done in company
Standard formats that are used are:
.WAV for uncompressed PCM audio
44.1 KHZ 16 bit stereo for audio CDs
that are already digitised, but need preservation
48 KHZ 24 bit stereo for old recordings
Digitally produced audio is accepted in its own format
Access to the objects is granted by audio CD or MP3

Work in progress (2)
Digitisation of Video and Film is done partly in company, partly by external partners
The standards that are used are:
Minimal data rate of 50 Mb for conservational purposes
Digital Betacam for VHS and Umatic tapes
Digital Video is accepted in its original recording format (miniDV, DVCam, XDCam etc)
Digibeta for 8mm, 16mm and 35mm film (processed by external partners)
DV25 for 16 mm film (processed in company)
Digital Betacam is stored in 10 bit uncompressed

How to ensure long term sustainability
Set up a trusted digital repository, consisting of hardware, software, procedures, methods, knowledge and experience

Trusted Digital Repository Feeder System Workflow Job Queue File Storage Characterisation Preservation Planning Migration Technical Registry Active Preservation Data Management Access Reporting Storage Adaptor Passive Preservation Ingest Toolkit Preservation Controller Workflow Controller User Administrator Archivist Metadata Store

Adding a minimal set of metadata, necessary for management, preservation and access
Using standard archival formats
Making agreements with producers of AV material about acceptable formats
Disclosure of content through Automatic Speech Recognition (ASR)
How to ensure long term access to data?

Disclosure through speech technology
Disclosure: automatically generate a time-stamped content description
 Allows online retrieval of fragments of AV records
Method depends on:
Available metadata
Availability of context documents
When a transcript is available:
Speech and transcript can be aligned,
i.e. Automatically couple what was said in the transcript, to where it was said in the audio
When there is no transcript:
Use automatic speech recognition to generate hypotheses of what was said where in the audio
Word Error Rates under 40% allow automatically generated content descriptions to be used as search index

AV archiving workflow CHoral Content production Indexing
Research topics
ASR : Automatic Indexing
IR: Information Retrieval
UI: User Interface Development
End user ASR IR UI

Research
Automatic indexing through speech technology:
Development of robust automatic speech recognition and audio classification tools
Information Retrieval:
Retrieval of spoken documents based on ASR output
Bridging the semantic gap between user queries and spoken content
User Interface development:
Support search and browsing in audio documents
(Re)presentation of audio content

Alignment + Speech signal Typed transcript Landgenooten waar ik enkele Begin frame # End frame # Word 00000 54400 -silence- 54400 65280 Landgenooten 65280 69120 Waar 69120 73600 Ik 73600 79520 Enkele … … …

Automatic speech recognition Acoustic model Language model Pronunciation dictionary Speech recognition 50+ hour audio 250-500 M words Pre-processing Classification speech/nonspeech Segmentation of speakers 2 nd recognition with adapted models Word level index

Types of word level indexes
Most probable words:
Lattice structures:
ASR: Er is een bekend beeld voor veel ouders de grote show in onveilige situatie voor de school TXT: ‘t is een bekend beeld voor veel ouders. De chaotische en onveilige situatie voor de school “ D’66 is z’n ene zetel kwijt”

Discussion ASR
For successful automatic annotation:
Audio should be digitally available, preferably on a server
To optimize ASR models for high-quality output,
part of the speech should be transcribed,
or related documents should be available
? How to validate automatic indexes?

User interface development
Challenges
Understand users’ requirements and information needs
Support selection and browsing of spoken content
Representation of spoken content via ‘surrogate’
Cross-linking to related content within the same or from another collection
IPR issues

CHoral speech technology for GAR
Alignment: Brandgrens interviews Rotterdam
Speech recognition: RR archives

Disc ussion
Development is ongoing in the work-flow and daily practice at audiovisual archives, and speech technology
Careful tuning of processes is needed for mutual benefit
Examples demonstrate envisioned benefits:
Potential reduction of human effort for annotation of undisclosed materials
Online access to fragments of spoken heritage

For more information, see http :// hmi.ewi.utwente.nl /project/CHoral
Questions?

Iasa Presentatie

by Mies Langelaar

Accessibility

Upload Details

Usage Rights

Statistics

Iasa Presentatie Presentation Transcript