The two-hour summary just about audio and video digitisation

1. 30 October 2014 SIPAD Workshop
Basic Principles of
Digitisation – of audio,
video and film
Richard Wright
SOIMA-LATAM, Mexico City, October 2014

2. Review
Audio and Video are signals
Audiovisual Archives hold recordings of signals
A signal is described by two attributes:
• Frequency range = bandwidth
• Dynamic range = signal-to-noise ratio
Pure tones = only one frequency (tuning fork)
All other sounds have a spectrum of frequencies
Speech: 100 Hz to 10 000 Hz (= 10 kHz)
30 October 2014 SIPAD Workshop

3. Review
All other sounds have a spectrum of frequencies
• Speech: 100 Hz to 10 000 Hz (= 10 kHz)
• Music: 30 Hz to above 18 000 Hz
• Human hearing (normal): 50 Hz to 18 kHz
• Telephone bandwidth: 300 Hz to 3.3 kHz (old)
• 78 rpm recording (shellac): up to 8 kHz
• 33 1/3 (vinyl; LP): up to 15 kHz (hi-fi)
• Audio tape (1/4”): up to 18 kHz (at 15 ips; many
different kinds of tape, and bandwidth varies with
recording speed)
• CD: theoretical max= 22 kHz = (44.1)/2;
practical max = 17.6 kHz = (44.1)/(2.5)
30 October 2014 SIPAD Workshop

4. Dynamic Range
• Speech: typically 30 to 50 dB; max about 70 dB
• Music: could be 70 dB in concert hall
– Could be 90 dB in a very quiet studio
• Human hearing: 100 dB without risk of damage; 130
dB max
• Old telephones: 20 to 30 dB
• 78 rpm shellac recording: 40 to 50 dB
• Sound on film: typically 40 to 50 dB
• 33 1/3 rpm vinyl LP: 60 to 70 dB
• ¼” audio tape: just over 70 dB (best = 73 dB)
• CD: capable of over 90 dB range (96 dB in theory)
30 October 2014 SIPAD Workshop

5. 5
The need for digitisation
Preservation planning
mapping your collections;
setting priorities;
making a collection strategy,
a preservation strategy
and a preservation plan
The Preservation Factory approach
Conservation, Digitisation
and Preservation

6. Terminology

Preservation – “Everything needed to ensure
permanent access” = maintenance

Conservation: keeping what you have (for as long
as you can) = safe storage and handling

Preservation actions: interventions. Changing what
you have = repair and replace
− Making a new negative or interneg
− Copying from an old carrier to a new carrier
− Digitising

Which separates content from carrier = liberation
6

7. Conservation

packaging, handling and shelving

environmental conditions: temperature and
humidity; protection from pollution, dirt etc

protecting the masters; and

condition monitoring = checking the stock
Full description on Preservation Guide wiki:
http://wiki.prestospace.org/pmwiki.php?
n=Main.PreservationStrategy#Conservation
7

8. 8
The need for digitisation
Only three ways to preserve:
1.Keep what you have
2.Copy using same technology
3.Copy using new technology
Keep what you have: only works for film
•
Audio, Video: analogue technology obsolete
Copy on same technology: only works for film
•
Audio, Video: analogue technology obsolete

9. 9
What about vinyl?

Gramophone
records: lacquer,
shellac and vinyl

Lacquer = a master
recording (acetate,
instantaneous disc);
very fragile !

10. 10
More on gramophone records

Shellac was used for 78
rpm commercial
recordings

Also fragile – the main
risk is handling; the
shellac itself is stable
(compared to laquer)

11. 11
But, what about vinyl?

Used for 45 and 33 1/3 rpm recordings

They are fragile (though less fragile than
shellac)

Warp from heat (can be fixed with care)

Easily scratched, and can get very dirty

The groove can be damaged in playback (if the
needle and tone arm adjustment is not right)

Vinyl is soft !!!

Vinyl and shellac can suffer chemical damage

12. Vietnam Film Institute Workshops 12
Gramophone
damage

Dropping a
needle onto a
vinyl disc

Oil coming to
the surface
on a lacquer
disc

13. 13
Where was I?
Only three ways to preserve:
Keep what you have: only works for film
•
Audio, Video: analogue technology obsolete
Copy on same technology:only works for film
•
Audio, Video: analogue technology obsolete
So we are left with only one option (for audio and
video):
Copy using new technology

14. 14
Preservation of Film

Many excellent example of conservation

Film technology still exists (but in decline)

BUT – the situation is rapidly changing

Kodak in severe economic trouble

Commercial cinemas changing to digital projection
− Norway changed completely in 2011-2012

Commercial cinema will change or go out of
business

Commercial cinema will not keep old projectors

15. 15
The end of film projection?
“ We're About to Lose 1,000 Small
Theaters That Can't Convert to Digital.
Does It Matter?”
Indiewire, USA
2012

16. 16
Digitisation of Film

Now: needed for access

Internet

Digital cinema

Soon: needed for preservation

When manufacture of film stock is stopped

Result: ALL media needs to be digitised, audio
and video and film

That's a lot of content that has to be digitised

There isn't enough time

There isn't enough money

17. 17
PRESTO Digitisation
“Better Faster Cheaper” Daniel Teruggi, Institut
National de l'Audiovisuel (INA), Paris
Basic concepts:
1) Dealing with the whole collection
2) Developing a strategy

For the institution and its collection

For the preservation work
3) Developing a preservation plan

18. 18
Digitisation Planning

All the Presto information on a wiki:
www.preservationguide.co.uk/RDWiki/pmwiki.php

StrategyStrategy – what does your institution do?

What does the collection do?

What can digitisation do?

PlanningPlanning – making a preservation plan

How to estimate a budget

Building a business case

19. 19
Collection Strategy

Long-term purpose of the digitisation

Preserving the collection

Maintaining services (old business)

Creating new services (new business)

Physical Outcomes:

Digital files

Mass storage

Cheaper, better maintenance

Cheaper, better digital access copies

Web access

20. 20
Access Outcomes

Collection access from computer stations

Mediatheque approach

Collection access via the Internet

With restricted access
− British Film Institute: higher education, public libraries …
and on YouTube
− British Library Sound Archive: higher education
institutions only

Or even unrestricted access
− INA has 30 000 hours of broadcast content online !

21. 21
Restrictions on Access

Attitudes: we don't do that

Fear: somebody might complain

Rights: we don't hold the rights
A solution:
1) public institutions create public value by
opening their collections as widely as possible
2) non-fiction content has the most information
and the least rights problems
We have a public service obligation to create
access to our collections !

22. 22
Access:
Technical Requirements

A usable catalogue

Online support services: questions, linking to
online sales, dealing with faults

Logins, access control, authentication, data
protection …

Computing power to support what could be
large numbers of people requiring access

UK National Archive crashed; Europeana crashed

Managing lots of information technology

23. 23
Digital Archive: workflow

Everything changes when shelves disappear:

Acquisition: digital ingest

Cataloguing: complex metadata issues: embedded
metadata, preservation metadata, mapping

Curation: huge opportunity to create online
collections

Research: self-research, fast scanning of 'hits' –
changes the requirements of cataloguing

Access: requirements need to be built into the
digitisation workflow (access copies, public
metadata, rights clearance, censorship?)

24. 24
Getting Started

Map – The collection needs to be divided by
format, condition, purpose ...

Priorities – arrange the “areas on the map”
according to what needs to be done first
wiki: Make a Map of your Collection
Divide the collection by physical formats, and
collect the following information on each format:
age range
storage history
genre or value
physical condition
Getting Started

25. 25
BBC film examples from wiki
Map:
http://www.preservationguide.co.uk/RDWiki/pmwiki.php
?n=Main.GettingStarted
Strategy:
http://www.preservationguide.co.uk/RDWiki/pmwiki.php
?n=Main.PreservationStrategy
Plan:
http://www.preservationguide.co.uk/RDWiki/pmwiki.php
?n=Main.PresPlan
Also: tutorials on PrestoCentre website
collection strategy and preservation plans_V1.01_0.pdf
mapping your collection_V1.01_0.pdf

26. Collection Strategy & Plan
• PDFs of wiki ‘chapters’
– Collection Strategy
– Preservation Strategy
– Preservation Plan and Budget
– Collection Strategy Preservation Strategy
– Preservation Plan and Budget
• Tutorials on PrestoCentre
– collection strategy and preservation plans_V1.01_0.pdf
– mapping your collection_V1.01_0.pdf
30 October 2014 SIPAD Workshop

27. 27
The Factory Approach
The division of labour
in pin manufacturing:
(and the great increase
in the quantity of work
that results)

28. 
The division of labour in pin manufacturing:
(and the great increase in the quantity of work
that results)

Approach: NOT about cutting corners or
reducing quality

Instead: about cutting wasted time and wasted
effort

Batches: doing one thing at a time, and then
doing it again (and again, and again)

Problem: how to keep the work interesting!
28
The Factory Approach

29. 29
A BBC Preservation Factory

30. 30
The Factory Approach

“Division of labour” requires having specialists,
which means having a team

Which means having a middle-sized or larger
project

But: even a single person can use a batch
approach, and get some efficiencies

31. BBC Preservation:
£5M per year (for 20 years)
Audio: 7000 hours per year of ¼ inch tape
Also digitised 63 000 “sepmag” elements:
separate magnetic sound tracks for film;
− Transferred to CD (good!) and to polyester (bad!)
Also 35 000 shellac and vinyl recordings
Also DAT and MD digital recordings
2” videotape: 46 000 tapes, transferred to D3
or Digibeta, done over 7 years
1” videotape: 80 000 in 5 yrs, to D3/Digibeta
 U-matic: 40 000 in 3 yrs, to MPEG-2 files

32. 32

News film: 43 000 items in 3 years

Ektachrome; digitised to SD video (Digibeta)

10% of total cost is cataloguing

20 to 30% is quality control (checking)

Since 2008: transfer 40k D3 videotape to files:

Uncompressed; MXF wrapper = INGEX

Now: transfer of BetaSP and Digibeta to files

Major confusion over High Definition formats !

33. 33
Discussion

Digitisation projects of delegates

Audio, video, film

Which physical formats (eg ¼-inch audio tape,
1-inch video tape, 16mm B&W film … ) ?

Size: how much content, how many people,
how many years ?

34. 34
Time for a break
After the break – a choice !
1)Low-level: standards and best practice for
audio, video and film
2)Higher level: case studies for preservation
and access
3) Your questions on digitisation

your long-term plan?

your immediate problems?

35. 30 October 2014 SIPAD Workshop

36. Digitisation Issues:
• Audio and Video: no problems at all with the
digitisation technology – the problems are with:
– Playback of originals
– Standards: compression, formats

37. Audio holdings (TAPE data)
Carrier %
wax cylinders 0.03
coarse groove replicated disks
(‘78s’,‘shellacs’)
1.9
instantaneous disks of any kind 0.2
microgroove disks (LPs) 17.5
open reel magnetic tape 38.6
compact cassettes 20.6
R-DAT 2.5
replicated CDs, DVDs 16.5
recordable and rewritable CDs, DVDs 1.4
Minidisks 0.4
Other 0.3
Total 100

38. Audio: major problems
• Wax cylinders: condition, equipment
• Lacquer discs: condition
• Sticky shed on 6mm tape playback
• DAT recordings: technical standards, condition,
equipment
• CD ‘ripping’ – common issue for digital-digital
transfer: error concealment (and what to do about it)
• IASA TC-04: Guidelines on the Production and
Preservation of Digital Audio Objects, March 2009.
– BWAV, 96kHz, “24bit”; BBC do 44.1kHz, 16bit

39. How to digitise audio
• IASA TC-04
• 48 kHz, “24 bit”
– 96 kHz recommended
• Save as a wav file
• Broadcast Wave Format recommended

40. Film: major problems
• No problem with playback equipment !
• Some film can last centuries
• Some film already destroyed
vinegar syndrome; colour fade; mold
• Telecine out, datacine in

41. Datacine for film digitisation
• Batch process
• HD or better for 16mm
• Many archives doing 2K scanning
• No grading shot-by-shot
– so a higher dynamic range is needed: 14 bits or better
• Needs sprocketless transport to cope with fragile film,
shrinkage, splices
• Infrared technology for scratch detection
• Wet-gate for scratch concealment (on one side)
• Several new sprocketless systems, €80 to 240k

42. How to Digitise Film
B&G document on PrestoCentre
http://www.prestocentre.eu/sites/dev.prestocentre.
antenna.nl/files/filmscanning_v1.03.pdf
White Paper: Film scanning considerations
By Tom de Smet and Harm Jan Triemstra

43. Sound on Film
• Successful digital processing of optical sound
tracks now very well established by Chase
Audio, Hollywood
– But expensive
• Analogue capture most cost-effective if sound
track in reasonable condition
• Much TV archive material has separate sound
tracks: sepopt, sepmag (beware vinegar!)

44. Video: major problems
• As for audio: equipment, operators, fragile and
decaying media; sticky-shed
• Plus: time base correction and dropout ‘concealment’
not built into old equipment
• Colour: composite signals and their decoding;
– BBC: a composite original is PAL-decoded and
saved as component, even though “the original” is
composite
• Head wear, head life, head replacement – alternative
technology for reading videotape has been propose

45. Video digitisation issues
• What codec, what file format?
– Uncompressed, lossless, lossy
– Lossless: JPEG2000, FFV1 (many others)
– MOV, AVI, MXF ... (cf LOC FADGI case studies)
• Endgame: uncompressed (no codec !)
• BBC: saving uncompressed from D3 transfers
to files: INGEX open-source software (as used
by Tate Galleries, London)

46. How to digitise video
1. Save the original
2. Digitise @ SDI = 4:2:2 = 200 megabits/sec
3. Save exactly as digitised = uncompressed
4. Use an open source file format (MXF or ?)
3b – or lossless = lossless JPEG2000,
3c – or ‘mezzanine’ = high-quality lossy
(but only for low quality originals)
(digibeta is a ‘mezzanine’ format)

47. Common issue: quality control
• Computer processing exists
– in professional audio equipment: Quadrega, NOA …
• ‘fault detection’ technology from restoration
work
• Joanneum video problem detector = VidiCert
• Audio Inspector
https://www.audioinspector.com/
• New QC software from Dave Rice, BAVC
http://www.bavc.org/qctools
• Key issues: workflow integration, statistics

48. How to Digitise Film
• B&G document on PrestoCentre
• http://www.prestocentre.eu/sites/dev.prestocentr
e.antenna.nl/files/filmscanning_v1.03.pdf
White Paper: Film scanning considerations
By Tom de Smet and Harm Jan Triemstra