Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tdr Overview Pres Advocates


Published on

Presentation on the TDR - as of 10th October 2007. Made to the Faculty of Advocates

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Tdr Overview Pres Advocates

  1. 1. Overview of the NLS TDR For the Faculty of Advocates James Toon TDR Project Manager Lawnmarket – Room 208, Ext 3791 [email_address]
  2. 2. Outline <ul><li>A – Introduction and Background </li></ul><ul><li>B – What does it comprise </li></ul><ul><li>C - Policies </li></ul><ul><li>D – Strategic Approach </li></ul><ul><li>E – Schedule and Happenings </li></ul><ul><li>F - Questions </li></ul>
  3. 3. Section A – Introduction and Background
  4. 4. <ul><li>Part of the NLS Digital Vision </li></ul><ul><li>A Trusted Digital Repository must be at the heart of the Digital NLS to ensure long-term digital preservation and access.” </li></ul><ul><li>Strategic Objectives </li></ul><ul><li>Develop and implement a trusted digital repository (TDR) infrastructure based around international standards and best practices (build and acquire) </li></ul><ul><li>Provide necessary storage capacity for the TDR (store) </li></ul><ul><li>Implement discovery and access strategies to deliver data stored in the TDR (make available) </li></ul>Background
  5. 5. <ul><li>The 2003 Legal Deposit Libraries Act, which extended our legal deposit privilege to non-print materials </li></ul><ul><li>The need to preserve our growing collection of digitised material </li></ul><ul><li>The need to preserve digital collections purchased by NLS </li></ul><ul><li>The potential to host and preserve digital content for partner institutions in Scotland </li></ul>What is the TDR? The Trusted Digital Repository is a repository system that will allow the National Library of Scotland to preserve and manage digital content of enduring value to the Nation. Its core drivers are;
  6. 6. TDR Mission <ul><li>In this context, the mission of the NLS TDR is as follows: </li></ul><ul><li>Collect and store digital 'stuff' (be it 'born digital' or a 'surrogate' of print material) </li></ul><ul><li>Make this material easily accessible to users (although subject to some constraints) </li></ul><ul><li>Provide a platform for the development of significant digital collections (of all types) </li></ul><ul><li>Ensure what is available today remains available many years to come through the application of digital preservation standards and technologies </li></ul>
  7. 7. TDR Project overview <ul><li>Options appraisal undertaking in 2005 compared the build vs. buy options. Decision to build based on open source platform. </li></ul><ul><li>Grant of £1.8M Awarded by Scottish Executive Aug 2006 over 2 years </li></ul><ul><li>TDR Project concentrating on web archiving as primary objective. </li></ul><ul><li>Setting up an in house development team of full time and contract developers. </li></ul><ul><li>Procure and install significant storage system (up to 200Tb) </li></ul><ul><li>Building the software in 4 key release cycles over the development of the product. Release 4 due end Sept 2008 </li></ul><ul><li>Setting out a 5 year strategic plan for the TDR as a key part of the library, with associated implementation costs and roadmap </li></ul>
  8. 8. Section B – What does it comprise
  9. 9. Software components <ul><li>Method of delivering content into the system from the ‘outside’ </li></ul><ul><li>Method or harvesting content from the internet </li></ul><ul><li>Methods of organising and managing the content objects and their metadata </li></ul><ul><li>A repository system to enforce and service the content objects and their metadata </li></ul><ul><li>A method of discovering and retrieving the content stored in the repository </li></ul><ul><li>A method of carrying out services for users based on the content in the repository </li></ul>
  10. 10. Hardware component <ul><li>Commenced full EU procurement process in August 2006 </li></ul><ul><li>Contracts signed end December 2006 </li></ul><ul><li>First Storage Area Network (SAN) system installed March 2007 </li></ul><ul><li>Fibre optic line installed between NLS and data centre July 2007 </li></ul><ul><li>Second mirror SAN installed August 2007 </li></ul><ul><li>Final storage network testing carried out Sept 2007 </li></ul>
  11. 11. What is a repository? <ul><li>Make up of a repository; </li></ul><ul><li>1 – Digital Object . A package of information that includes the content of the work and data about the work object, including policies which aid discovery and that may dictate the usage and availability of the work. </li></ul><ul><li>2 – Repository . A location where digital objects are stored and responsible for enforcing policies bound to these objects </li></ul><ul><li>3 - Service . Software products that provide services based on the contents of the repositories and their policies, such as dissemination, transformation, rights management and preservation </li></ul>
  12. 12. Section C – Policies
  13. 13. Integrated policy decisions <ul><li>The following are examples of many policies that must be developed and integrated into the library operational structure; </li></ul><ul><li>Digital Preservation </li></ul><ul><li>Rights Management and security </li></ul><ul><li>Web Archiving </li></ul><ul><li>Cataloguing and metadata </li></ul><ul><li>Acquisition and permissions </li></ul>
  14. 14. Digital Preservation Policies Policies based around simple three stage approach to preservation 1 – Archive . The objects cannot be preserved if they are not in the archive. 2 – Risk Management . Assessment of the likely preservation risk at ingest. Allow for pragmatic but proactive approach to individual preservation cases. i.e. file normalisation as part of ingest as prevention (e.g. Adobe PDF conversion) 3. Preserve . Provide method for migration/emulation on a per object basis. Original bit stream is main archive, and we are maintaining the method of preservation rather than carrying out transformation en masse
  15. 15. Rights Management Policies <ul><li>Security and rights enforcement based around; </li></ul><ul><li>User security associating individuals to access groups (such as creating a set of users who are just Advocates) </li></ul><ul><li>User security applicable down to individual object level </li></ul><ul><li>Controlled availability for delivery (such as per IP address, or book in/book out methods) within Legal Deposit boundaries </li></ul><ul><li>Association of individual object security policies </li></ul><ul><li>Management of individual or group digital signatures </li></ul><ul><li>Full logging and audit trail </li></ul><ul><li>Enforcement of publisher DRM requirements – not victim of DRM requirements </li></ul>
  16. 16. Web Archiving Policies <ul><li>Emphasis on high quality collections based on; </li></ul><ul><li>Selective, thematic collections </li></ul><ul><ul><li>Collection areas such as Scottish music, sport, politics etc </li></ul></ul><ul><ul><li>Collection sub areas such as, bagpipe music, Scottish rugby, political party sites – all with inherited collection metadata </li></ul></ul><ul><li>Event Based collections </li></ul><ul><ul><li>Scottish elections, special events, disasters (such as flood in NLS building!) </li></ul></ul><ul><li>Domain level collections </li></ul><ul><ul><li>Yearly/twice yearly broad brush collection of all Scottish and/or UK websites </li></ul></ul>
  17. 17. Cataloguing and Metadata Policies <ul><li>Not dissimilar to NLS standard practices, but with a few additions; </li></ul><ul><li>Descriptive with MARC using AACR2 </li></ul><ul><li>Use of METS for data wrappers </li></ul><ul><li>Automatic technical metadata extraction through DROID/PRONOM database for The National Archive </li></ul><ul><li>Use of PREMIS metadata for technical details (amongst others) </li></ul><ul><li>Metadata mapping to allow bulk ingest of objects from other databases </li></ul>
  18. 18. Acquisition and permissions policies <ul><li>Significant administrative overhead </li></ul><ul><li>Open archive suggestions (i.e. from Curators and users alike) for web collections </li></ul><ul><li>Ability for curators to deposit selected digital objects (such as PDF etc) to collections </li></ul><ul><li>Ability for users of hosted repository to self archive </li></ul><ul><li>Permissions carefully managed, even post Legal Deposit implementation </li></ul><ul><li>Copyright and DRM management </li></ul><ul><li>Watching brief on new DRM/Legal deposit issues such as ACAP (Automated content access protocol) </li></ul>
  19. 19. Section D – Strategic Approach
  20. 20. Strategic Approach <ul><li>Key focus on sustainability </li></ul><ul><li>In parallel with the development activity is the construction of a 5 year strategy for the TDR, looking at where we want to be; </li></ul><ul><li>International Standards. OAIS ISO standard, RLG OCLC TDR Checklist </li></ul><ul><li>Plan for auditing the ongoing compliance </li></ul><ul><li>Risk management strategy to manage processes </li></ul><ul><li>Benefits management realisation plan to guarantee success </li></ul><ul><li>Operational integration into NLS either as centralised or decentralised model </li></ul><ul><li>Facilitator of information management for Scotland </li></ul>
  21. 21. Section E – Schedule and Happenings
  22. 22. Development Milestone plan <ul><li>Milestone release 1 - End June 2007 . Implementation of pilot standalone web archiving system using IIPC tool set </li></ul><ul><li>Milestone release 2 – End Dec 2007 . Release of hosted repository system, including use of Fedora with associated ingest mechanism and metadata manager to meet OAIS requirements </li></ul><ul><li>Milestone release 3 – End March 2008 . Replacement of pilot web archiving system with NLS own workflow management tool </li></ul><ul><li>Milestone release 4 – End Sept 2008 . Implementation of resource discovery/delivery tool for TDR/Digital Library to provide access </li></ul>
  23. 23. What’s happening now? <ul><li>System development (planning, design, team recruitment etc) </li></ul><ul><li>Developing collections policy, default criteria for collection and specific selective and event collection plans </li></ul><ul><li>Working with national partners on full domain web archiving </li></ul><ul><li>Undertaking benchmarking activity for strategy plan </li></ul><ul><li>Writing a PESTLE (Political, Environmental, Social, Technological, Legal and Ethical) analysis of international repository effort </li></ul><ul><li>Communication with national and international stakeholders via International Internet Preservation Coalition (IIPC) </li></ul><ul><li>Starting to better understand the pros and cons of web archiving through practice! </li></ul>
  24. 24. Any questions?