Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evolution of the technical and professional skill sets and roles within the digital audiovisual archive sector, Jean Varra, INA


Published on

Presentation made at the FIAT/IFTA World Conference 2014 in Amsterdam.

  • Be the first to comment

  • Be the first to like this

Evolution of the technical and professional skill sets and roles within the digital audiovisual archive sector, Jean Varra, INA

  1. 1. Evolution of the technical and professional skill sets & roles within the digital audiovisual archive sector FIAT /IFTA Conference Amsterdam 2014
  2. 2. 01 Introduction: archiving scene and jobs in transfer
  3. 3. Introduction :archiving scene and jobs in transfer… Digital technology entered the television domain about 20 years ago replacing progressively and unavoidably analogue television. Digital has introduced lot of changes in the audiovisual domain and therefore archiving: new technical systems, new uses, new delivery modes…. Today: •Production and post production systems have totally shifted from tape based workflows to tapeless workflows •Files have replaced analogue and digital video tape formats •Servers and networks have replaced video tape recorders •A high percentage of content produced today is digital and volumes grow faster and faster •In a IT world, automated systems play an important role data management and data control have become key factors. technical skills and roles | 6 octobre 2014 3
  4. 4. Introduction :archiving scene and jobs in transfer… Though, fundamental functions of Archives remain the same: acquisition, preservation, access, documentation…roles and skills change. Some traditional skills and roles are disappearing, some have evolved, new ones have appeared or are still appearing. The dilemma is that archivists ,while trying to follow and assimilate/digest the faster and faster changes in digital technology, must still continue to keep alive some traditional skills and roles. technical skills and roles | 6 octobre 2014 4
  5. 5. Introduction:archiving scene and jobs in transfer… The following presentation will try to answer the following questions: • What are the major evolutions of the technical side of archiving : what has already changed ? What is changing? • How current skills and jobs are(will be) affected by those changes ? Which main topics archivists must take care of ? • What are the required new roles and skills? technical skills and roles | 6 octobre 2014 5
  6. 6. 02 deliver Archiving functions in a digital world Ingest Store document
  7. 7. Archiving functions in a digital world In order to define the technical skills and roles in a digital archiving system, it is interesting to use a diagram based on the OAIS - Open Archive Information System model which details the main functions and, map on it the required skills and roles. technical skills and roles | 6 octobre 2014 7
  8. 8. 8 Preservation planning/management Ingest Access /delivery Data management Archival storage Producers/ sources Consumers / users Queries Query responses Descriptive orders Descriptive information information Master/ viewing file formats Master/ viewing file formats Administration film V/A tapes files Delivery formats and media/ browsing formats technical skills and roles | 6 octobre 2014
  9. 9. BWF MP3 03 Ingest entity JPEG 2000 TIF MPEG 1 MPEG2 H264 HEVC JPEG DV DVC pro MXF MOV Betacam SP/SX AVI AVC intra WAV AAC AC3 DIVX 2 inch 1 inch B 1 inch C IMX film
  10. 10. From analogue to digital… Analogue to digital processes can be considered as part of the ingest entity. There are still many film and analogue video formats over the world to preserve and digitize but also early digital video formats now obsolete. To ensure a successful film digitization 2 types of work are required: > Film restoration or mechanical repairmanual work > Film transfer and color correction  Evolving from telecine to scanning operation To ensure a successful video (audio) tape digitization are required: > A good preparation of the tape is the first important criteria to preserve the quality of a content: cleaning, baking and repair…if needed > a good playback of the original tape is the second criteria to preserve the quality of a content. requires tape players in good condition technical skills and roles | 6 octobre 2014 10
  11. 11. From analogue to digital: roles at risk Concerning these activities ,there are several traditional roles at risk: Film restoration: > Less and less people are interested in this job > Film laboratories are closing and disappearing less and less opportunities to work on film media. > Only large digitization of film could make this job last > More generally skills and knowledge on film media are disappearing. Video playback: > Main problem to face is to maintain old obsolete machines. > maintenance skills and knowledge on old tape machines 2 inch 1 inch are rapidly disappearing combined with lack of spare parts. > Almost impossible to motivate young people on these old technologies > To play more recent tape formats (Beta, Beta SP,DVCpro… ) there are still possibilities to find machines and spare parts. how long ? > More generally, video tape recording (even digital) and associated role (video tape recorder operators and maintenance technicians) are inevitably disappearing in the broadcast and post production field and therefore in archives. technical skills and roles | 6 octobre 2014 11
  12. 12. From video to file… Encoding is the function allowing conversion from either analogue or digital video signals to file. A file is a combination set of : > Essence ( image, sound) ie MPEG2, DVCpro, JPEG2000, WAV… > Metadata: technical metadata, identification metadata… > Wrapper:/container MXF, MOV, BWF,AVI… To preserve quality of the original content, the choice of the destination file format is essential. A file is defined and set by parameters, and technical metadata which are essential to make it usable and understandable. > spatial resolution ,bit rate, GOP, field order, interlaced/ progressive scanning, aspect ratio, audio tracks…. A file must be at least standardized but this is not enough to ensure interoperability. technical skills and roles | 6 octobre 2014 12
  13. 13. File format conversion/transcoding File format conversion/transcoding system is a central function of the ingest entity but also for the access entity. > In the ingest entity, the transcoding system is used to convert various file formats (ephemeral, experimental, old, proprietary…) into others file formats (for example into a master/mezzanine file format or a viewing format) > In the access entity, it is used to generate file formats compatible with diverse receiving platforms for diverse uses. File format conversion is required to solve interoperability problems. But, transcoding is neither a “transparent” or a “so easy” task: > It may introduce some quality problems or impairments on the content. > Even if a file is “well defined” , some interoperability problems may occur: > A file may be corrupted > Some parameters of the file may not be taken into account by a specific device or tool > A file may be accepted/perfectly used by one tool (transcoder, editing system, players…and not by another according trade marks and standard implementation…) > Unexpected problems How to face these problems? technical skills and roles | 6 octobre 2014 13
  14. 14. A new role: « file specialist » Encoding and transcoding in the digital process require very skilled and trained people , “file specialists” : technical skills and roles | 6 octobre 2014 14 Main roles To determine and set the best parameters to ensure quality and sustainability of the file. to test/check interoperability with tools of the market. to analyze, understand and solve all potential interoperability problems and set corrections if required Set the parameters of quality control systems Main skills He/she must perfectly understand the deep structure of the various files and the role of each parameter the characteristics of various coders/transcoders, file players … He/she must be able to understand, analyze specific problems and bring a solution. ( tools are required) . 
  15. 15. A validation function: quality control (QC) In the past, QC consisted in checking video and audio on video tapes. Because processes are more and more automated, most important work for AV technicians has become “monitoring systems and performing QC” Before storing and archiving files, quality control is required in order to make sure that : > During digitization, the play back of the original media has been performed successfully ( checking levels, clogging, drop outs sound presence….) > The file structure, essence or wrapper is OK / not corrupted > The parameters of the generated file are respected. > The files are correctly named ( identification) > The coding and/or the transcoding systems have not introduced new impairments. > The content is complete (beginning and end) but also that received files correspond to the expected ones. There are now automated tools able to check content quality and syntax of files delivering time coded reports that can help an operator and make him focusing on the detected problems > Baton (Interra systems), Cerify (Tektronix)… If a QC operator cannot analyze a problem, he/she can ask for help from file specialists technical skills and roles | 6 octobre 2014 15
  16. 16. QC operator technical skills and roles | 6 octobre 2014 16 Main roles Detect what are the artifacts due to the original and/or its playback ( general or specific) Detect what are the artifacts specific to coding or transcoding Detect syntax/structure problems Analyze reports from automated QC Propose corrective actions Main skills Ability to analyze image and sound Know and understand what are the artifacts due to the original and/or its playback ( general or specific) Understand file syntax problems know what are the artifacts specific to coding or transcoding Understand reports from automated tools 
  17. 17. 04 LTO digital library-Ina Archival storage entity
  18. 18. Archival storage entity Because of the ever faster growth of data volumes and diversity of uses , a hierarchy of storage ( off line, on line, near on line) is generally implemented. A hierarchy of storage requires some management rules to set and follow. Archival storage management requires a broad IT knowledge in servers and networks :hardware/software and data administration, hard drive, tape robotics, and corresponding software (HSM….) The required skills of an archival storage manager are more or less those of an IT administrator In large companies, tasks may be split between several IT specialized roles and responsibilities: hardware engineer, network engineer, security manager, help desk… technical skills and roles | 6 octobre 2014 18
  19. 19. Role : storage manager/engineer technical skills and roles | 6 octobre 2014 19 Main roles Common/usual tasks performed by a digital storage manager are to: Receive, provide/deliver , to delete, replace, back up data  organize data, collect technical metadata (produced at each step of the process) Warranty operation :acquisition and delivery… He/she also monitors permanently storage activity status ( streams, capacity, and performance… ) Objectives are to prevent system from failure and data losses use of software to monitor data tape libraries for example RVA ( read Verification appliance) in data tape libraries. Other important tasks are : To check/ensure data integrity ( prevent data from corruption, calculate check sums…) and security ( backup data and setting disaster recovery function and processes) To plan/set strategies ( according to quantities) and perform data migration from one type/generation of storage to another ( for example from LT04 to LTO6) Assume scalability of the system Prevent system from obsolescence: technological survey and planning evolution Main skills Understanding networking Understanding operating systems and related files systems. Understanding storage architectures (NAS, SAN, HSM….) Undersanding file formats Software /hardware installation Development/Programming 
  20. 20. AC3 05 Access entity H264 HEVC JPEG MOV AVC intra DnX WAV MP3 BWF JPEG 2000 Pro Res MXF BWF TIF
  21. 21. Access entity The objectives of the access/delivery entity is to provide technical services as “simple data formatting” : > file format conversion to deliver the expected file format. > Partial restore / extract creation: to deliver a sequence, > Logo or burnt in time code insertion But also “advanced” formatting functions like: (when required) > publishing, editing functions, digital restoration… Access activity: queries request and queries response, orders Delivery on various modes: > copy on digital video tape (less and less) > Files on IT media: removable hard drives, data tapes, memory sticks and cards… > File over networks (more and more) Ensure quality of what is delivered Protect data. technical skills and roles | 6 octobre 2014 21
  22. 22. Acess entity roles Simple delivery can be implemented in an automated process/workflow. (Request through tools allowing viewing/ selection/setting time code in and out/order, buy, deliver) : technical skills and roles | 6 octobre 2014 22 Main roles Monitoring the activity: check if customers orders have been satisfied on time (for example, if it is linked to an urgency news) Monitor if the technical system is OK. If not, alert concerned department( for example IT department if a server a network has failed e or the file specialist) Monitoring the delivered quality and apply corrections if needed of what is delivered (has been delivered) with the help of automated tools , Analyse the problem and correct if possible or ask to a file specialist Secure the delivery services In case of urgency or if the technical problem lasts, use alternative delivery modes to deliver the expected content. Main skills Understanding installed processes Understanding files and their problems and possible solutions see ingest entity skills Understanding delivery modes 
  23. 23. Access/delivery entity roles Sometimes, the delivery of a content requires some specific formatting actions ( ie. which cannot be automated) Editing, post production ( subtitling, dubbing…) « Advanced » Up conversion--< reframing the pictures Color grading Authoring for DVD, Blu ray, publishing… Digital restoration of image or sound … All these jobs require specific «technico-artistic skills ». Digital restoration is an increasing need in order to match today quality needs and give new opportunities to archive contents. In order to respect the original work, digital restoration requires not only to master the tools but also a to have real knowledge of history of production, the way it has been produced … technical skills and roles | 6 octobre 2014 23
  24. 24. Other access skills and roles Data protection management according to the adopted protection system ( signature, water marking…) > Marking content > Monitoring use of content > Detect illegal use Publishing over multi platforms: Web masters… > Web > VOD > Mobile > I tunes > … Data mining technical skills and roles | 6 octobre 2014 24
  25. 25. 06 Temporary conclusion
  26. 26. Summary: roles technical skills and roles | 6 octobre 2014 26 Technical roles Traditional roles: handling old media Still required in archives but regularly decreasing File specialist New essential role in the jungle of files Monitoring activity and QC control main new roles for AV technicians Data /storage management Central technical role of a digital archive Linked roles Publishing, data protection, data mining, digital restoration, post production…..
  27. 27. technical skills and roles | 6 octobre 2014 27 Ingest includes knowledge of: Old media use and behaviour file formats and their transformation record keeping systems programing languages database administration validation tools including QC Identification Preservation needs knowledge of : Technological history of the archive original record/ media either analogue or digital Original tools (ie tape player or original operating system) Migration between operating systems ( for born digital records) Storage requires to understand: Operating systems and related files systems Management actions like Software installation issues Security Back up Data mining Access needs to understand: Networks and distribution infrastructures and modes (internet, VOD… file formats and their transformation Validation tools (QC) Protection tools Management requires: To understand broad technical and IT concepts and their application To plan and evaluate solutions . Bring together a team of experts of various disciplines Make people speak the same language  Digital archivists? Summary: knowledge
  28. 28. Some axes of technical training… General Knowledge managers + all Practice of file formats and AV compression: > Fundamental knowledge of standards , formats and uses. Quality parameters and comparisons Understanding video ad audio for computer people More specialized trainings: file specialists Optimization of compression: > Setting parameters of different formats and tools ( encoders/transcoders) to get the best results, automation of tools, analysis tools… Specialized trainings on each format ( MPEG2, MPEG4JPEG2000, HEVC, MXF, DCP.. > Deep knowledge of parameters and related problems, performance analysis. Quality control: Basics of quality control of picture and sound Practice of quality control tools: of Baton, Cerify…. Media server administration: storage managers Server technologies and architectures, exchanges , security of data, users profiles…. technical skills and roles | 6 octobre 2014 28
  29. 29. Thank you for your attention technical skills and roles | 6 octobre 2014 29