Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From EDRMS to TDR: A Wish-list for Ways to Preserve Digital Records


Published on

The slides from Ed's talk at #IRMS15 outlining his ‘dream’ of how to incorporate digital preservation strategies and methodologies into the records management life-cycle.

There will be a business need to keep certain classes of records for long periods, or even permanently; these periods may exceed the life of the
EDRMS itself. Simply “exporting everything” may not be enough; there needs to be a strategy supporting a long-term repository, and a method for getting selected records into it.

An ‘ideal’ EDRMS would allow the automated migration of selected records into suitable preservation formats; and enhance the process of preservation by creating a comprehensive audit trail, and supporting metadata, thus embedding the selection decision within the records management workflow.

The importance of metadata for preservation purposes, how it might be managed and the potential merits of mapping selected MoReq fields into METS will also be investigated.

The talk doesn't propose a finished solution, but aims to start a conversation about how we can integrate digital preservation into records management and information practices.

Published in: Technology
  • Be the first to comment

From EDRMS to TDR: A Wish-list for Ways to Preserve Digital Records

  1. 1. @ulcc IRMS Conference May 2015 From EDRMS to TDR: a wish-list for ways to preserve digital records
  2. 2. @ulcc About me • Ed Pinsent, Digital Archivist at ULCC since 2004 • Teaches digital preservation on the DPTP • Background as archivist / records manager • Experience in web-archiving, repository management, metadata projects, migration, digitisation, project management, etc. • See more at digital archives blog
  3. 3. @ulcc Imagineering: Themes & Dreams for Tomorrow • Do we need to preserve born-digital records? • If so, how can we do it? • Will refer to EDRMS (Electronic Document Records Management Systems)… • And to a TDR (Trusted Digital Repository)… • That conforms with Open Archival Information System (OAIS) Reference Model
  4. 4. @ulcc Some issues 1. Need for keeping records may exceed life of the EDRMS 2. Need for keeping records may exceed life of file formats they are stored in 3. Can we get content / metadata out of the EDRMS? 4. If we need preservation, where do we do it?
  5. 5. @ulcc #1: The EDRMS problem • You may have a long-term business need for retention • May only affect a small percentage of records (e.g. Pension records, contracts) • Will your EDRMS last that long?
  6. 6. @ulcc #2: The file format problem • Digital file formats may become obsolete / unsupported / unreadable • Common response in digital preservation is to migrate • But it’s extra work: picking a suitable target format, finding a conversion tool • Doing this creates even more metadata • We’d want to automate it, ideally
  7. 7. @ulcc #3: Can we export from EDRMS? • Can we get the digital objects / files out? • Can we get the metadata out? • Do we want to? – Presumably MOREQ conformant… – Reports on the system itself (audit trail) – Reports on use of the records – Both add authenticity / legal admissibility to records
  8. 8. @ulcc #4. Where can we do preservation? • In the EDRMS itself? • Or in a dedicated archival repository system?
  9. 9. @ulcc EDRMS might not work for preservation • According to DIRKS (2012)… • Q: Will the EDRMS automatically manage the long term needs of records? • A: No. There are a number of factors that can still threaten the long term accessibility and use of records. • Vendor support • Software upgrades • Move to new EDRMS must be handled carefully • Data loss, corruption, alteration risks
  10. 10. @ulcc EDRMS might not work for preservation • Archives New Zealand: • Archives New Zealand is now in a position to accept digital records exported into its custody where the records have been appraised as having ongoing archival value. Requirements for digital transfer remain flexible to incorporate existing digital records not accompanied by recordkeeping metadata to the standard.
  11. 11. @ulcc Possible Gaps (Technical) 1. No “bridge” from EDRMS to TDR 2. Not always clear if we can export out of EDRMS, or in what format 3. Do we want to capture the EDRMS audit trail? 4. Can’t easily change file formats in the EDRMS 5. Additional metadata needed for preservation
  12. 12. @ulcc Possible Gaps (Human) Records Manager Archivist IT Manager All I care about is current records and the next five years Our selection policy means I have to preserve a history of this organisation forever I don’t care about either of you
  13. 13. @ulcc Life Cycle of Record Creation Maintenance and Use Preservation Destruction Disposition EDRMS Functions OAIS Functions
  14. 14. @ulcc What is OAIS / TDR? • ISO standard(s) • Proposes a dedicated space for storing archival copies • Allows us to protect important content in archival packages • Can carry out migration in a controlled and professional way • Uses tools which EDRMS doesn’t have • Can create preservation metadata
  15. 15. @ulcc Some proposed steps towards preservation 1. Survey 2. Define use cases 3. Do preservation planning 4. Define ingest requirements 5. Define data management requirements 6. Define access requirements 7. Think about migration
  16. 16. @ulcc Terms… • Migration = transforming digital objects from one file format to another • Moving = taking electronic records out of the EDRMS • Export = will use this term to apply only to metadata
  17. 17. @ulcc 1. Survey • Survey record holdings • Create an inventory • Identify record types with long retention needs (e.g. 30-50 years) • Note file format types in use • May be a small percentage
  18. 18. @ulcc 2. Use cases • Identify types of users • Find out what these users expect to be able to do with the records • Make decisions about how much you can support technically
  19. 19. @ulcc Example A • Internal users might want… – To find documents – To read documents – To amend or edit them (unlikely, but possible) – To delete them – To operate a disposal rule (if they are the records manager)
  20. 20. @ulcc Example B • External users might want access, but… – Documents might be closed to the public – When can they be opened? – Can they be released in redacted form? – What about an auditor or other legal official? – What are your legal obligations?
  21. 21. @ulcc And what about… • Individual digital objects – Will you need to retain formatting? Colours? Fonts? – Editability and processability? – Formulae in spreadsheets? – Email attachments? • Provenance and original order – Will you retain original file names? Folder names? References? History of use?
  22. 22. @ulcc The value of use cases • Have implications for what kind of preservation system you are going to build – Functionality – Search facilities – Access mechanisms (security marking, redaction)
  23. 23. @ulcc The value of use cases • Have implications for managing digital objects – When and how will you migrate objects? – What can you afford to lose in migration? – How much do you need to support?
  24. 24. @ulcc 3. Preservation planning • When will you move? • What will you be moving? • What about keeping metadata from the EDRMS, e.g.: – File plan scheme? – Retention scheduling rules? – Use history? – Access / security markings? • Will you migrate file formats? When? How?
  25. 25. @ulcc Can we move / export? • TNA have noted: • Most systems “have not been designed for easy import and export of information.” • “It is rare that an EDRMS export captures everything held in the system”. • (Migrating Information between EDRMS, Digital Continuity Project, 2010)
  26. 26. @ulcc 4. Define ingest rules • Move objects from EDRMS • Export EDRMS metadata • Extract technical metadata from objects • Identify access / closure requirements • Build preservable packages
  27. 27. @ulcc 5. Define data management • How will you store and manage metadata? – Keep it as a chunk of XML – Incorporate metadata with your database – Map it to METS • What about preservation metadata?
  28. 28. @ulcc 6. Define Access • Devise access rules for records, e.g.: – Security markings – Authentication procedures – Redaction processes • Apply them to your preserved objects
  29. 29. @ulcc 7. Migration • Convert old file formats into supported file formats • Devise acceptance criteria (part of preservation planning); what essential properties must not be lost through migration? • Learn from your use cases; you cannot support “everything” • Choose conversion tools • Identify formats that may cause problems • Decide when you will migrate • Keep records when you do it (preservation metadata)
  30. 30. @ulcc Possible result OAIS compliant digital archive Access function Internal user External user Auditor Search Read Edit Audit Dispose - Preserves - Holds metadata - Keeps some object functionality - Replicates some functions of EDRMS - Builds accessible DIPs - Redacts - Migrates
  31. 31. @ulcc Wishlist: Technical • Automation • More bridge-building between systems • Keeping as much original metadata as possible • Can we map MoREQ to METS? • Move towards agreement on the ideal sharing format (e.g. XML)
  32. 32. @ulcc Wishlist: Planning • An integrated digital records-to-digital archives lifecycle • Do migration early, where appropriate • Authenticity and integrity of records retained • Someone in organisation taking ownership of preservation
  33. 33. @ulcc Wishlist: Human Records Manager Archivist IT Manager We need long- term retention of these records, but I can’t do it in my EDRMS I’ll learn more about practical digital preservation You both have credible business plans, so we’ll invest in the IT you need