How to keep file-based content usable !Includes material from the PRESTO projects andfrom Presto Partners ITI at Universit...
1/3 of material has deterioration1/4 of material cannot be released as it is too easily damaged              Vietnam Film ...
At least 2/3 of the material cannot be easily used                Vietnam Film Archive Workshops
So we are digitising !– or keeping film in cold storage                              Vietnam Film Archive Workshops
There is a three-way juggling act, and weneed to understand the trade-offs           Vietnam Film Archive Workshops
   What   are the risks?   What   are the mitigation techniques?   What   will it cost?   What   might you expect to l...
   Technical obsolescence, e.g. formats and players   Hardware failures, e.g. digital storage systems   Loss of staff, ...
Doubles every 18 months100 times every decade1 million times every 30 years                                 Vietnam Film A...
Medium            Storage Density bits/cm²              Life, yearsStone             10                                   ...
   TCO is a multiplier of unit media cost   HDD storage is 2-3 times cost of data tape   Real world costs reduce 50% ev...
Vietnam Film Archive Workshops
6 years2 to 3 years               Ultrium LTO roadmap                         Vietnam Film Archive Workshops
Encoding formats  Media formats   Storage hardware     Operating systems       Management software         Networking     ...
   1000 times more HDD capacity over last 15 years   Only 10 times lower Bit Error Rates (BER)   HDD Bit Error Rate = 1...
Google study of Annual Failure Rates in HDD servers                              Vietnam Film Archive Workshops
Volker Heydegger                                                         study on file format                             ...
   The „industry‟ is                              going „tapeless‟                             Networked sharing        ...
Vietnam Film Archive Workshops
(1) START    HERE                    Is the format a                   (2) problem?                                       ...
   Longer lived storage technology   More reliable storage technology   More copies   Encode so content is more resili...
Access                            Repair                                                                                  ...
   Preservation grade DVDs   Magneto Optical disks   Digital Film   Rosetta Discs   Expensive, Vendor Lock in, Not ma...
   Relatively „safe‟ technology (compared to HDD)   Typical „problem rates‟ are 0.1 – 1% of tapes   New strategies need...
   File checksums common approach   AV files can be very big (e.g. 1 TB)   Corruption can be very small (e.g. few bytes...
All on tape (2 copies), hard disk only for staging                                                             Increasing ...
Vietnam Film Archive Workshops
National Centre of Expertise in FilmCould put information online, in Vietnamese language, to help other institutions     ...
   Needs a Centre of Expertise in Audio   Needs a Centre of Expertise in Video   They work with VFI on a common website...
   SEAPAVAA is the organisation for audiovisual    professionals   Could have a website with practical information in a ...
   FIAF = International Federation of Film Archives   FIAT-IFTA = International Federation of Television    Archives   ...
   A non-profit organisation set up to support    audiovisual preservation and access   Based in The Netherlands (Hollan...
Vietnam Film Archive Workshops
Upcoming SlideShare
Loading in...5
×

Workshop 5 digital audiovisual collections

198

Published on

How to hold, manage and preserve a file-based audiovisual archive

Published in: Technology, Travel
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
198
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Workshop 5 digital audiovisual collections

  1. 1. How to keep file-based content usable !Includes material from the PRESTO projects andfrom Presto Partners ITI at University ofSouthampton
  2. 2. 1/3 of material has deterioration1/4 of material cannot be released as it is too easily damaged Vietnam Film Archive Workshops
  3. 3. At least 2/3 of the material cannot be easily used Vietnam Film Archive Workshops
  4. 4. So we are digitising !– or keeping film in cold storage Vietnam Film Archive Workshops
  5. 5. There is a three-way juggling act, and weneed to understand the trade-offs Vietnam Film Archive Workshops
  6. 6.  What are the risks? What are the mitigation techniques? What will it cost? What might you expect to lose? Is there a systematic approach? Vietnam Film Archive Workshops
  7. 7.  Technical obsolescence, e.g. formats and players Hardware failures, e.g. digital storage systems Loss of staff, e.g. skilled transfer operators Insufficient budget, e.g. digitisation too expensive Accidental loss, e.g. human error during QC Stakeholders, e.g. preservation no longer a priority Underestimation of resources or effort Fire, flood, meteors, aliens… Vietnam Film Archive Workshops
  8. 8. Doubles every 18 months100 times every decade1 million times every 30 years Vietnam Film Archive Workshops
  9. 9. Medium Storage Density bits/cm² Life, yearsStone 10 10000Paper 104 1000Film 107 100Disc 1010 10  Each change in „technology‟ is 1000 times denser  But the media lasts 0.1 times as long Vietnam Film Archive Workshops
  10. 10.  TCO is a multiplier of unit media cost HDD storage is 2-3 times cost of data tape Real world costs reduce 50% every 2-3 years !!! 1 + ½ + ¼ + 1/8 + 1/16 + ... = 2 Take today‟s raw media cost and x10 for annual rate 1TB online for 1 year costs $1000 Multiply again by 4 for lifetime cost £1 per hour of audio on data tape forever £10k per hour of 4k film on hard disk forever Vietnam Film Archive Workshops
  11. 11. Vietnam Film Archive Workshops
  12. 12. 6 years2 to 3 years Ultrium LTO roadmap Vietnam Film Archive Workshops
  13. 13. Encoding formats Media formats Storage hardware Operating systems Management software Networking Vietnam Film Archive Workshops
  14. 14.  1000 times more HDD capacity over last 15 years Only 10 times lower Bit Error Rates (BER) HDD Bit Error Rate = 10-14 1 TB = 1013 bits 10% chance of an error when reading all of a HDD Within a few years, you are more likely than not to get a read error when copying a HDD Vietnam Film Archive Workshops
  15. 15. Google study of Annual Failure Rates in HDD servers Vietnam Film Archive Workshops
  16. 16. Volker Heydegger study on file format sensitivity to corruption JPEG2000 with one error per 100KBCompression = Corruption amplifier Corrupting 0.001% of encoded image results in 30% of pixels affected in decoded image Vietnam Film Archive Workshops
  17. 17.  The „industry‟ is going „tapeless‟  Networked sharing of file-based contentVietnam Film Archive Workshops
  18. 18. Vietnam Film Archive Workshops
  19. 19. (1) START HERE Is the format a (2) problem? YESArchive for a few years NO (3) (4) (5c) What cost/quality/risk option can you afford Compress lossy (5b) (5a) Compress Uncompress lossless END HERE Vietnam Film Archive Workshops
  20. 20.  Longer lived storage technology More reliable storage technology More copies Encode so content is more resilient Use concealment Check often and fix quickly Vietnam Film Archive Workshops
  21. 21. Access Repair Both 1 Copy 2 Good Problem Copies Corrupted Copies detected Corrupted or Lost or Lost Failure Failure DetectionMigration Failure Vietnam Film Archive Workshops
  22. 22.  Preservation grade DVDs Magneto Optical disks Digital Film Rosetta Discs Expensive, Vendor Lock in, Not mainstream Risk: the vendor, not the technology Vietnam Film Archive Workshops
  23. 23.  Relatively „safe‟ technology (compared to HDD) Typical „problem rates‟ are 0.1 – 1% of tapes New strategies needed for many files on one tape Most problems from data tape come from drives Field studies show no data lost where multiple copies have been made and integrity checked Vietnam Film Archive Workshops
  24. 24.  File checksums common approach AV files can be very big (e.g. 1 TB) Corruption can be very small (e.g. few bytes) Scrubbing takes time and resources Moving big files is „expensive‟ So, make big files into small files! Vietnam Film Archive Workshops
  25. 25. All on tape (2 copies), hard disk only for staging Increasing archiveFrequently used on hard disk, two copies tapeAll on hard disk (1 copy), safety copy on tape sizeAll on hard disk (2 or more copies)All on flash (2 copies, e.g. USB sticks) Vietnam Film Archive Workshops
  26. 26. Vietnam Film Archive Workshops
  27. 27. National Centre of Expertise in FilmCould put information online, in Vietnamese language, to help other institutions Vietnam Film Archive Workshops
  28. 28.  Needs a Centre of Expertise in Audio Needs a Centre of Expertise in Video They work with VFI on a common website to provide information, in Vietnamese language, to help all other institutions in Vietnam Vietnam Film Archive Workshops
  29. 29.  SEAPAVAA is the organisation for audiovisual professionals Could have a website with practical information in a range of languages Already runs training programmes and conferences Vietnam Film Archive Workshops
  30. 30.  FIAF = International Federation of Film Archives FIAT-IFTA = International Federation of Television Archives AMIA = Association of Moving Image Archives ◦ Has excellent email list (in English, mainly) IASA = International Association of Sound (and Audiovisual) Archives ◦ Has excellent technical documentation Vietnam Film Archive Workshops
  31. 31.  A non-profit organisation set up to support audiovisual preservation and access Based in The Netherlands (Holland) Supported by INA, BBC and other major institutions (hosted by Dutch national audiovisual archive) Lots of online information (in English, mainly) Plans to run training and other services www.prestocentre.eu “www.prestocentre.eu” Vietnam Film Archive Workshops
  32. 32. Vietnam Film Archive Workshops

×