Film Archive headquarters, 84 Taranaki St, Wellington

Wednesday, 4 April 2012
Film Archive cafe and reception

Wednesday, 4 April 2012
Film Archive Medialibrary

Wednesday, 4 April 2012
The New Zealand Film Archive
                                120,000 titles
                                300,000 items
                                  40 + staff
            Collect - acquisition, accessioning, cataloguing,
                          research and commercial access
            Protect - storage, preservation, digitisation
            Connect - screenings, exhibitions and website
Wednesday, 4 April 2012
How will we store our films?


                          How long will they last?



                            Film Archive storage at Plimmerton
Wednesday, 4 April 2012
Wednesday, 4 April 2012
MWA Flashscan HD for 8mm, Super8mm and 9.5mm
Wednesday, 4 April 2012
Wednesday, 4 April 2012
First 1 GB hard drive, the
                          IBM 3380 introduced June 1980
                          weight - 250kg, capacity 2.52GB


Wednesday, 4 April 2012
Film Archive data storage

                                Generation     total in TB
                                  LTO 3            164
                                  LTO 4            889
                                  LTO 5            189
                                  Server           40
                                   total          1282

                          LTO is the acronym for Linear Tape-Open,
                            an open standard for digital data tape

Wednesday, 4 April 2012
UNIVAC I, March 1951 stored data using liquid mercury,
                  the next model used magnetic tape
Wednesday, 4 April 2012
IBM729V from early 1960’s

Wednesday, 4 April 2012
Film was Easy (fear of the Black Box)


Wednesday, 4 April 2012
Last Film Search,
                           Dunedin 1998




Wednesday, 4 April 2012
Videotape formats
    Analog tape formats                             Digital tape formats

    1" Type B video tape (Robert Bosch GmbH)        Betacam IMX (Sony)
    1" Type C videotape (Ampex, Marconi and Sony)   D-VHS (JVC)
    2" Quadruplex videotape (Ampex)                 D-Theater
    2" Helical Scan Videotape (Rank Cintel)         D1 (Sony)
    Betacam (Sony)                                  D2 (Sony)
    Betacam SP (Sony)                               D3
    Betamax (Sony)                                  D5 HD
    S-VHS (JVC) (1987)                              Digital-S D9 (JVC)
    W-VHS (JVC) (1994)                              Digital Betacam (Sony)
    U-matic 3/4" (Sony)                             Digital8 (Sony)
    VCR, VCR-LP, SVR                                DV
    VERA (BBC experimental format ca. 1958)         HDV
    VHS (JVC)                                       ProHD (JVC)
    VHS-C (JVC)                                     MicroMV
    Video 2000 (Philips)                            MiniDV

Wednesday, 4 April 2012
Video codecs (from wikipedia)

    Lossless compression
        Alpary                                     Lagarith - Lagarith Codec (VfW codec),
        Animation (qtrle) - QuickTime, FFmpeg           FFmpeg
        ArithYuv                                   LOCO - FFmpeg (decoder only)
        AVIzlib - LCL (VfW codec), FFmpeg          LZO
    ! ! CamStudio GZIP/LZO                         MSU Lossless Video Codec
        Dirac lossless - libdirac, FFmpeg          PNG - CorePNG, FFmpeg
        (decoder only)                        !!   SheerVideo
        FastCodec                                  Snow lossless - FFmpeg
        Fraps - FFmpeg (decoder only)              TechSmith Screen Capture Codec (TSCC)
        FFV1 - FFmpeg                                   - FFmpeg
        H264 lossless - x264 (encoder only),       Ut Video[5] - FFmpeg (decoder only)
        FFmpeg                                     YULS
        Huffyuv - FFmpeg                           ZMBV (Zip Motion Block Video) Codec
        JPEG 2000 lossless                              - FFmpeg
                                                   ZRLE used by VNC




Wednesday, 4 April 2012
Lossy compression                                 More video codecs

        Audio Video Standard (AVS) OpenAVS -          FFmpeg
        FFmpeg                                        InterVideo Video Decoder
        Blackbird FORscene video codec                Ligos LSX MPEG-2
        Cineform                                      MainConcept MPEG-2
        Cinepak - FFmpeg (decoder only)               TMPGEnc
        Dirac - Schrödinger, dirac-research, FFmpeg
                                                 !!    H.263 - FFmpeg H.263 (libavcodec)
        DV - FFmpeg                                           (decoder only)
        Firebird [1] Original FORscene video codec     MPEG-4 Part 2 (MPEG-4 Advanced
        H.261 - FFmpeg H.261 (libavcodec)                     Simple Profile)
        MPEG-1 Part 2 (MPEG-1 Video)                  3ivx
            Cinema Craft Encoder                      DivX
            Elecard MPEG-1 Decoder/Encoder            Elecard MPEG-4 Decoder/Encoder
            FFmpeg                                    libavcodec
            Ligos LSX MPEG-1                          HDX4
            MainConcept MPEG-1                        Nero Digital
            TMPGEnc                                   Xvid
    ! ! H.262/MPEG-2 Part 2 (MPEG-2 Video)
            Canopus ProCoder
            Cinema Craft Encoder
            Elecard MPEG-2 Video Decoder/Encoder

Wednesday, 4 April 2012
Lossy compression (continued)
        Indeo 3/4/5 - FFmpeg (decoder only)
        MJPEG
           FFmpeg                                Pixlet
           Morgan Multimedia M-JPEG              Apple ProRes 422 - FFmpeg
           Pegasus PICVideo M-JPEG               RealVideo - FFmpeg
           PICVideo                              Snow Wavelet Codec
        JPEG 2000 intra frame video codec        Sorenson Video, Sorenson Spark - FFmpeg
        H.264/MPEG-4 AVC or MPEG-4 Part 10,      Tarkin
             CoreAVC                             Theora - FFmpeg (decoding only), libtheora
             Elecard AVC Decoder/Encoder         VC-1 (SMPTE standard, subset of Windows
             MainConcept                         Media Video)
             Nero Digital                           FFmpeg (decoder only)
             QuickTime H.264                     VC-2 SMPTE standard (a.k.a. Dirac Pro)
             Sorenson AVC Pro codec,                Schrödinger, dirac-research
             Vanguard Software Solutions         VC-3 SMPTE standard - Avid DNxHD,
             x264 (encoder only)                 FFmpeg
             FFmpeg (decoder only                Windows Media Video (WMV) WAX
        OMS Video
        On2 Technologies TrueMotion VP3/VP4, VP5, VP6, VP7, VP8;
        ! under the name The Duck Corporation: TrueMotion S,
        ! TrueMotion 2, FFmpeg (decoder only)

Wednesday, 4 April 2012
Wednesday, 4 April 2012
JPEG2000

       Created in 2000 by the Joint Photographic Experts Group

              - Flexible codestream, scalable compression
              - provides for both Lossless and Lossy compression
              - complex encoders/decoders
              - ISO standard
              - Digital Cinema standard
              - initially slow to gain acceptance
              - licensed but coding system available free




Wednesday, 4 April 2012
[New Zealand infantry] - 1914
                          recently scanned at Weta Digital
Wednesday, 4 April 2012
Upstream, John Ford, 1928
Wednesday, 4 April 2012
IBM 1401 Data Processing System from 1959.
                 From the left: 1402 Card Read-Punch,
                 1401 Processing Unit, 1403 Printer.
Wednesday, 4 April 2012
Risks



                     earthquake fire tornado tsunami
                      arson vandalism theft litigation
                  ignorance war depression indifference




Wednesday, 4 April 2012
Technical Risks

    Obsolescence - hardware & software have a
                   finite lifetime

    Data Integrity - latent errors, ingest errors, transfer errors

    Single Point Loss - single copy vulnerable to machine error,
                         human error or natural disaster

    Mono Culture - singular technologies can suffer

    Compression - mathematically lossless, perceptually
                  lossless, lossy compression

Wednesday, 4 April 2012
Jedi Archives, Coruscant.
                            Star Wars. Episode II


Wednesday, 4 April 2012
Keanu Reeves as
                          Johnny Mnemonic
                               1995




Wednesday, 4 April 2012
Wednesday, 4 April 2012
Wednesday, 4 April 2012
Wednesday, 4 April 2012
Dreamtime (or how Goodbye Pork Pie becomes a legend)


Wednesday, 4 April 2012
Wednesday, 4 April 2012
Wednesday, 4 April 2012
Wednesday, 4 April 2012
The Automated Archive
                 no more boring tasks in your paperless office
            Digital deposit, accessioning and cataloguing through
                        metadata and crowd sourcing
             But getting everything digitised is hard on humans
                                The Distributed Archive
                        safety in numbers
     Compact storage and many copies means less risk of loss
        More reliant on complex technology and material
                     can still burn and drown
                          The Everything for Everyone Archive
                        quantity vs quality
       We can keep everything and have it instantly available
     We may lose control of the content and will that devalue it?
Wednesday, 4 April 2012
The Digital Archive promises
                                  a bright future
                          We just have to keep up with it


                               jamielean@nzfa.org.nz

                               www.filmarchive.org.nz




Wednesday, 4 April 2012

Bigger Hard Drive Jamie Lean

  • 1.
    Film Archive headquarters,84 Taranaki St, Wellington Wednesday, 4 April 2012
  • 2.
    Film Archive cafeand reception Wednesday, 4 April 2012
  • 3.
  • 4.
    The New ZealandFilm Archive 120,000 titles 300,000 items 40 + staff Collect - acquisition, accessioning, cataloguing, research and commercial access Protect - storage, preservation, digitisation Connect - screenings, exhibitions and website Wednesday, 4 April 2012
  • 5.
    How will westore our films? How long will they last? Film Archive storage at Plimmerton Wednesday, 4 April 2012
  • 6.
  • 7.
    MWA Flashscan HDfor 8mm, Super8mm and 9.5mm Wednesday, 4 April 2012
  • 8.
  • 9.
    First 1 GBhard drive, the IBM 3380 introduced June 1980 weight - 250kg, capacity 2.52GB Wednesday, 4 April 2012
  • 10.
    Film Archive datastorage Generation total in TB LTO 3 164 LTO 4 889 LTO 5 189 Server 40 total 1282 LTO is the acronym for Linear Tape-Open, an open standard for digital data tape Wednesday, 4 April 2012
  • 11.
    UNIVAC I, March1951 stored data using liquid mercury, the next model used magnetic tape Wednesday, 4 April 2012
  • 12.
    IBM729V from early1960’s Wednesday, 4 April 2012
  • 13.
    Film was Easy(fear of the Black Box) Wednesday, 4 April 2012
  • 14.
    Last Film Search, Dunedin 1998 Wednesday, 4 April 2012
  • 15.
    Videotape formats Analog tape formats Digital tape formats 1" Type B video tape (Robert Bosch GmbH) Betacam IMX (Sony) 1" Type C videotape (Ampex, Marconi and Sony) D-VHS (JVC) 2" Quadruplex videotape (Ampex) D-Theater 2" Helical Scan Videotape (Rank Cintel) D1 (Sony) Betacam (Sony) D2 (Sony) Betacam SP (Sony) D3 Betamax (Sony) D5 HD S-VHS (JVC) (1987) Digital-S D9 (JVC) W-VHS (JVC) (1994) Digital Betacam (Sony) U-matic 3/4" (Sony) Digital8 (Sony) VCR, VCR-LP, SVR DV VERA (BBC experimental format ca. 1958) HDV VHS (JVC) ProHD (JVC) VHS-C (JVC) MicroMV Video 2000 (Philips) MiniDV Wednesday, 4 April 2012
  • 16.
    Video codecs (fromwikipedia) Lossless compression Alpary Lagarith - Lagarith Codec (VfW codec), Animation (qtrle) - QuickTime, FFmpeg FFmpeg ArithYuv LOCO - FFmpeg (decoder only) AVIzlib - LCL (VfW codec), FFmpeg LZO ! ! CamStudio GZIP/LZO MSU Lossless Video Codec Dirac lossless - libdirac, FFmpeg PNG - CorePNG, FFmpeg (decoder only) !! SheerVideo FastCodec Snow lossless - FFmpeg Fraps - FFmpeg (decoder only) TechSmith Screen Capture Codec (TSCC) FFV1 - FFmpeg - FFmpeg H264 lossless - x264 (encoder only), Ut Video[5] - FFmpeg (decoder only) FFmpeg YULS Huffyuv - FFmpeg ZMBV (Zip Motion Block Video) Codec JPEG 2000 lossless - FFmpeg ZRLE used by VNC Wednesday, 4 April 2012
  • 17.
    Lossy compression More video codecs Audio Video Standard (AVS) OpenAVS - FFmpeg FFmpeg InterVideo Video Decoder Blackbird FORscene video codec Ligos LSX MPEG-2 Cineform MainConcept MPEG-2 Cinepak - FFmpeg (decoder only) TMPGEnc Dirac - Schrödinger, dirac-research, FFmpeg !! H.263 - FFmpeg H.263 (libavcodec) DV - FFmpeg (decoder only) Firebird [1] Original FORscene video codec MPEG-4 Part 2 (MPEG-4 Advanced H.261 - FFmpeg H.261 (libavcodec) Simple Profile) MPEG-1 Part 2 (MPEG-1 Video) 3ivx Cinema Craft Encoder DivX Elecard MPEG-1 Decoder/Encoder Elecard MPEG-4 Decoder/Encoder FFmpeg libavcodec Ligos LSX MPEG-1 HDX4 MainConcept MPEG-1 Nero Digital TMPGEnc Xvid ! ! H.262/MPEG-2 Part 2 (MPEG-2 Video) Canopus ProCoder Cinema Craft Encoder Elecard MPEG-2 Video Decoder/Encoder Wednesday, 4 April 2012
  • 18.
    Lossy compression (continued) Indeo 3/4/5 - FFmpeg (decoder only) MJPEG FFmpeg Pixlet Morgan Multimedia M-JPEG Apple ProRes 422 - FFmpeg Pegasus PICVideo M-JPEG RealVideo - FFmpeg PICVideo Snow Wavelet Codec JPEG 2000 intra frame video codec Sorenson Video, Sorenson Spark - FFmpeg H.264/MPEG-4 AVC or MPEG-4 Part 10, Tarkin CoreAVC Theora - FFmpeg (decoding only), libtheora Elecard AVC Decoder/Encoder VC-1 (SMPTE standard, subset of Windows MainConcept Media Video) Nero Digital FFmpeg (decoder only) QuickTime H.264 VC-2 SMPTE standard (a.k.a. Dirac Pro) Sorenson AVC Pro codec, Schrödinger, dirac-research Vanguard Software Solutions VC-3 SMPTE standard - Avid DNxHD, x264 (encoder only) FFmpeg FFmpeg (decoder only Windows Media Video (WMV) WAX OMS Video On2 Technologies TrueMotion VP3/VP4, VP5, VP6, VP7, VP8; ! under the name The Duck Corporation: TrueMotion S, ! TrueMotion 2, FFmpeg (decoder only) Wednesday, 4 April 2012
  • 19.
  • 20.
    JPEG2000 Created in 2000 by the Joint Photographic Experts Group - Flexible codestream, scalable compression - provides for both Lossless and Lossy compression - complex encoders/decoders - ISO standard - Digital Cinema standard - initially slow to gain acceptance - licensed but coding system available free Wednesday, 4 April 2012
  • 21.
    [New Zealand infantry]- 1914 recently scanned at Weta Digital Wednesday, 4 April 2012
  • 22.
    Upstream, John Ford,1928 Wednesday, 4 April 2012
  • 23.
    IBM 1401 DataProcessing System from 1959. From the left: 1402 Card Read-Punch, 1401 Processing Unit, 1403 Printer. Wednesday, 4 April 2012
  • 24.
    Risks earthquake fire tornado tsunami arson vandalism theft litigation ignorance war depression indifference Wednesday, 4 April 2012
  • 25.
    Technical Risks Obsolescence - hardware & software have a finite lifetime Data Integrity - latent errors, ingest errors, transfer errors Single Point Loss - single copy vulnerable to machine error, human error or natural disaster Mono Culture - singular technologies can suffer Compression - mathematically lossless, perceptually lossless, lossy compression Wednesday, 4 April 2012
  • 26.
    Jedi Archives, Coruscant. Star Wars. Episode II Wednesday, 4 April 2012
  • 27.
    Keanu Reeves as Johnny Mnemonic 1995 Wednesday, 4 April 2012
  • 28.
  • 29.
  • 30.
  • 31.
    Dreamtime (or howGoodbye Pork Pie becomes a legend) Wednesday, 4 April 2012
  • 32.
  • 33.
  • 34.
  • 35.
    The Automated Archive no more boring tasks in your paperless office Digital deposit, accessioning and cataloguing through metadata and crowd sourcing But getting everything digitised is hard on humans The Distributed Archive safety in numbers Compact storage and many copies means less risk of loss More reliant on complex technology and material can still burn and drown The Everything for Everyone Archive quantity vs quality We can keep everything and have it instantly available We may lose control of the content and will that devalue it? Wednesday, 4 April 2012
  • 36.
    The Digital Archivepromises a bright future We just have to keep up with it jamielean@nzfa.org.nz www.filmarchive.org.nz Wednesday, 4 April 2012