Hans Hofman - European Perspectives on Digital Preservation


Published on

During the last decade several projects with respect to digital preservation have been funded in Europe by the European Commission and have delivered interesting results. Such projects include community building projects or coordination actions such as ERPANET, Delos2, and Digital Preservation Europe (DPE), but also research projects such as Planets, CASPAR, Shaman, Protage. In December 2009 a new call for digital preservation will be closed, so new projects may start in 2010.

One result of all these projects and all the work done is that there is a growing community involved, more organizations and people are aware of the issues, definitely has enhanced the collaboration amongst institutions and universities in Europe, and with the last research projects some potential practical solutions are emerging that could be applied by institutions. How it all will work out in the end is still one of the big questions. For one thing it may have helped to create a good foundation for further collaboration, perhaps even without funding from the European Commission.

This presentation will provide a brief overview of the main results of some of these projects, especially Planets, and what issues they try to resolve, and a brief outlook on possible future developments.

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hans Hofman - European Perspectives on Digital Preservation

  1. 1. European Perspectives on Digital Preservation Hans Hofman (virtual) Nationaal Archief Netherlands National Digital Forum New Zealand 24 November 2009
  2. 2. Overview <ul><li>The challenge of digital preservation </li></ul><ul><ul><li>Repositories and preservation activities </li></ul></ul><ul><li>European collaboration and research </li></ul><ul><ul><li>Some European projects and their aims </li></ul></ul><ul><ul><li>Planets (preservation planning, developing ‘open and shared environments’) </li></ul></ul><ul><li>Some observations </li></ul>
  3. 3. The challenge of long term preservation <ul><li>The enormous and rapidly increasing amount of digital information </li></ul><ul><ul><li>Fragile resources </li></ul></ul><ul><li>The rapid evolution in technology </li></ul><ul><li>The risk of obsolescence and therefore corruption and/or loss of valuable information </li></ul><ul><li>Not only in governments or archives </li></ul><ul><ul><li>libraries, data centers, business companies, … </li></ul></ul><ul><li>(Pro-)active and ongoing attention / maintenance required </li></ul><ul><li>Potential solutions still fragmented </li></ul><ul><ul><li>infrastructure </li></ul></ul><ul><ul><li>not comprehensive </li></ul></ul>
  4. 4. Digital preservation <ul><li>What are the basic questions/ issues? </li></ul><ul><ul><li>Object: what are the essential characteristics? </li></ul></ul><ul><ul><ul><li>what to preserve? </li></ul></ul></ul><ul><ul><li>Capture: be sure that the material is technically sound and preservable </li></ul></ul><ul><ul><li>Storage: basic, but access control, disaster recovery </li></ul></ul><ul><ul><li>Maintenance: when at risk, how to know what the best suitable action tool is, how to validate migration/emulation? </li></ul></ul><ul><ul><ul><li>across technology </li></ul></ul></ul><ul><ul><li>Retrieval: where to find? </li></ul></ul><ul><ul><li>Representation: how to re-constitute the digital object as it was when ingested? </li></ul></ul><ul><ul><ul><li>what is the human understandable object? </li></ul></ul></ul><ul><ul><ul><li>different representations? </li></ul></ul></ul><ul><ul><ul><li>authenticity? </li></ul></ul></ul>
  5. 5. The reference model of OAIS ‘ Repository model’ for long term preservation
  6. 6. Digital Repository Metadata database data management ingest access Planning and control transfer end user creator Digital Depot system storage link representation migration-on-demand virtualisation and emulation characterisation migration conversion to standards Preservation system
  7. 7. Models, components and tools <ul><li>Repository </li></ul><ul><ul><li>OAIS reference model </li></ul></ul><ul><ul><li>Checklist for building one: PLATTER (see DPE) </li></ul></ul><ul><ul><li>Trustworthiness </li></ul></ul><ul><ul><ul><li>DRAMBORA (based on risk analysis) </li></ul></ul></ul><ul><ul><ul><li>Checklist for audit (TRAC) </li></ul></ul></ul><ul><li>Preservation planning and action </li></ul><ul><ul><li>distributed infrastructure/interoperability framework + tools </li></ul></ul><ul><ul><li>software agents using a defined platform (Protage) </li></ul></ul>
  8. 8. Some projects in Europe <ul><li>Research programmes </li></ul><ul><ul><li>FP6/IST, FP7/IST </li></ul></ul><ul><ul><li>networks of excellence, integrated projects, research projects </li></ul></ul><ul><li>Repository level: </li></ul><ul><ul><li>DPE, CASPAR, DRIVER, Parse.insight, … </li></ul></ul><ul><li>Preservation planning/action: </li></ul><ul><ul><li>Planets, Protage, Shaman </li></ul></ul><ul><li>Networking/community building: </li></ul><ul><ul><li>Former projects: Delos, ERPANET, DigiCult, DPE </li></ul></ul><ul><ul><li>Alliance for permanent access </li></ul></ul><ul><li>National initiatives (networking) </li></ul><ul><ul><li>DCC, DPE, NCDD, </li></ul></ul>
  9. 10. Digital Preservation Europe <ul><li>Finished, 2006-2009 </li></ul><ul><li>DRAMBORA ( www.repositoryaudit.eu ) </li></ul><ul><li>PLATTER </li></ul><ul><ul><li>Planning Tool for Trusted Electronic Repositories </li></ul></ul><ul><ul><li>provides a basis for a digital repository to plan the development of its goals, objectives and performance targets </li></ul></ul><ul><li>Training courses in collaboration with Planets, CASPAR (under the banner of WePreserve) </li></ul><ul><li>Research agenda </li></ul>
  10. 11. CASPAR <ul><li>Cultural, Artistic and Scientific knowledge for Preservation, Access and Retrieval </li></ul><ul><li>Heavily based on OAIS reference model (ISO 14721:2003) </li></ul><ul><li>Focus on science data and capturing representation information </li></ul><ul><li>Integrated research project </li></ul><ul><ul><li>‘ Developing technology neutral solutions’ </li></ul></ul><ul><ul><li>‘ Preservers knowledge and intelligibility’ </li></ul></ul><ul><ul><li>‘ Guarantees integrity and identity’ </li></ul></ul><ul><li>Just finished (2009) </li></ul>
  11. 12. <ul><li>Preservation Organizations using Tools in Agent Environments </li></ul><ul><li>using existing knowledge by applying software agents and to support digital preservation activities </li></ul><ul><li>agents can check out the objects to see whether they need preservation action </li></ul><ul><ul><li>based on available agent technology and on available knowledge from experts </li></ul></ul><ul><ul><li>1 st iteration about the workflow preparing and transferring records (video on website) </li></ul></ul><ul><ul><li>2d iteration (in progress): identify problem, search for solutions, recommend most suitable tools </li></ul></ul>Other projects: Protage
  12. 13. SHAMAN <ul><li>Sustaining Heritage Access through Multivalent ArchiviNg </li></ul><ul><ul><li>develop comprehensive theory of preservation </li></ul></ul><ul><ul><ul><li>next generation preservation framework (for assessment) </li></ul></ul></ul><ul><ul><ul><li>automating policy requirements and asses their effectiveness in practice </li></ul></ul></ul><ul><ul><li>supply an infrastructure; enable automation </li></ul></ul><ul><ul><ul><li>develop and implement a grid-based production system </li></ul></ul></ul><ul><ul><ul><ul><li>use of iRODS, rule based (not metadata-driven) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>focus on characterization of services that can be applied to data </li></ul></ul></ul></ul><ul><ul><li>demonstrate the applicability to various domains </li></ul></ul><ul><ul><li>4 year project started 1-12-2007 </li></ul></ul>
  13. 14. Planets <ul><li>A 4-year research and technology development project co-funded by the European Union to address core digital preservation challenges. </li></ul><ul><li>Started June 2006 with €15m budget </li></ul><ul><ul><li>Coordinated by the British Library </li></ul></ul><ul><ul><li>Involves 16 partners </li></ul></ul><ul><ul><ul><li>national libraries and archives, </li></ul></ul></ul><ul><ul><ul><li>leading technology companies and </li></ul></ul></ul><ul><ul><ul><li>research universities </li></ul></ul></ul><ul><li>Builds on strong digital archiving and preservation programmes </li></ul>
  14. 15. Planets Functional Model
  15. 16. Objectives of Preservation Planning <ul><li>Identify and analyse the organisational context </li></ul><ul><ul><li>including a risk assessment </li></ul></ul><ul><ul><li>define a framework for preservation / policy </li></ul></ul><ul><li>Support decision-making about digital preservation including </li></ul><ul><ul><li>Identifying criteria for preservation within that context </li></ul></ul><ul><ul><li>Defining workflow for evaluating/ defining preservation plans </li></ul></ul><ul><ul><li>Developing methodologies for assessing the risks of applying different preservation strategies for different types of digital objects </li></ul></ul><ul><li>Enable formulation, evaluation and execution of high-quality and cost-effective preservation plans that suit the organisational (e.g. repository) needs </li></ul><ul><li>Support the on-going evaluation of the results of executing preservation plans and provide a feedback mechanism </li></ul><ul><li>Document the planning process carefully </li></ul>
  16. 17. Essential characteristics of ‘digital objects’ <ul><li>What needs to be preserved? </li></ul><ul><ul><li>content </li></ul></ul><ul><ul><li>context </li></ul></ul><ul><ul><li>structure </li></ul></ul><ul><ul><li>form / appearance </li></ul></ul><ul><ul><li>(sometimes) behaviour </li></ul></ul><ul><li>What criteria for determining these essential characteristics? </li></ul><ul><li>Authenticity, reliability, integrity and usability </li></ul>
  17. 18. Collection profile <ul><li>What types of objects (both technical and intellectual aspects)? </li></ul><ul><li>Technical: file formats </li></ul><ul><ul><li>registries (e.g. PRONOM, UDFR, …) </li></ul></ul><ul><li>Intellectual: for instance documentary form, structure, look and feel, ‘behaviour ’ </li></ul><ul><ul><li>objective tree ‘templates’ </li></ul></ul><ul><ul><li>an (intellectual) object may consist of different computer files </li></ul></ul><ul><ul><ul><li>what strategy then? </li></ul></ul></ul>
  18. 19. Organisational policy Preservation Planning Planets: Preservation Planning Object (type) Technical environment Regulatory environment Information technology Standards Tools Strategies User requirements P-Plan P-Plan P-Plan Object (type) Technical environment Action Constraints Guidelines significant properties ?
  19. 21. Preservation Planning and OAIS
  20. 22. Characteristics of a P-plan <ul><li>It is a concrete translation of a preservation policy how to handle/treat a certain type of digital objects in a given institutional setting </li></ul><ul><li>New plans will be needed over time due to </li></ul><ul><ul><li>changes in technology </li></ul></ul><ul><ul><li>changes in organisational setting </li></ul></ul><ul><ul><li>changes in user requirements </li></ul></ul><ul><ul><li>changes in available tools </li></ul></ul><ul><ul><li>changes in preservation methods </li></ul></ul><ul><li>It also specifies a series of steps or actions along with responsibilities and rules and conditions for execution. </li></ul><ul><ul><li>This is called preservation action plan. It is in the form of an executable workflow definition, detailing the actions and the required technical environment </li></ul></ul><ul><ul><li>Relationship with a specific action </li></ul></ul><ul><ul><li>The preservation plan provides the context/ background of the preservation action plan </li></ul></ul>
  21. 23. The issue of trustworthiness <ul><li>Practical approaches aimed at ensuring long-term authenticity , reliability, integrity and usability of digital materials are emerging at a similar pace </li></ul><ul><li>The discipline remains immature though: </li></ul><ul><ul><li>Are adopted approaches successful ? </li></ul></ul><ul><ul><li>What is the metric for defining success? </li></ul></ul><ul><ul><li>Which approaches are appropriate for particular digital preservation challenges? </li></ul></ul><ul><ul><li>Which preservation services and/or service providers can be trusted ? </li></ul></ul>
  22. 24. Summary - observations <ul><li>Focus shifting from repository to preservation action? </li></ul><ul><ul><li>what to preserve (appraisal)? </li></ul></ul><ul><ul><li>assumption that creation can not be influenced? </li></ul></ul><ul><ul><li>how to validate actions? </li></ul></ul><ul><li>Issue of trustworthiness </li></ul><ul><ul><li>organisational perspective </li></ul></ul><ul><li>Models for repository infrastructure </li></ul><ul><ul><li>sharing </li></ul></ul><ul><ul><li>national, European, wider? </li></ul></ul><ul><li>Preservation actions as separate (web)services </li></ul><ul><ul><li>sharing knowledge and experiences </li></ul></ul><ul><li>Sustainability </li></ul>
  23. 25. References <ul><li>www.planets-project.eu </li></ul><ul><li>www.digitalpreservationeurope.eu </li></ul><ul><li>www.casparpreserves.eu </li></ul><ul><li>www.shaman-ip.eu </li></ul><ul><li>www.protage.eu </li></ul><ul><li>www.repositoryaudit.eu </li></ul><ul><li>www.wepreserve.eu </li></ul><ul><li>www.driver-repository.eu </li></ul>
  24. 26. Thank you for your attention! Questions? [email_address]