Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Planning for digitisation and conservation
- 1. Planning for digitisation and the life-cycle approach Tom Derrick Digital Curator, British Library
- 2. www.bl.uk 2 National Newspaper Building, Boston Spa, Yorkshire • 24m high bay storage • Automated retrieval by cranes and conveyors • Low oxygen, temperature, and humidity
- 3. www.bl.uk 3 • Two Centuries of Indian Print • Hebrew Manuscripts Digitisation • King’s Topographical Collection • England and France, 700-1200 • Qatar Foundation Digitisation Partnership • Google Books Digitisation • Public Private Partnerships Current major digitisation initiatives
- 4. www.bl.uk 4 Endangered Archives Programme Through an annual competition, EAP grants provide funding to preserve social and cultural archival material that is in danger of destruction, neglect or physical deterioration world-wide. To date, the EAP has awarded more than 290 grants in 80 countries, preserving cultural and social archives across Africa, Asia, Europe, Americas and Oceania. In 2017/2018 EAP hosted Chevening fellow to strengthen its activities in South America and the Middle East. EAP266: History of Bolama, the first capital of Portuguese Guinea (1879-1941), as reflected in the Guinean National Historical Archives
- 5. www.bl.uk 5
- 6. www.bl.uk 6 Save Our Sounds Programme #saveoursounds www.bl.uk/save-our-sounds
- 7. www.bl.uk 7 Thinking of a digitisation project?
- 8. www.bl.uk 8 Start digitisation Plan / schedule project conservation assessment schedule work with imaging services Order storage space Present project to DIAMs Get approval Contact Access & Reuse, Copyright Intellectual Property Arrange for clearance Contact Imaging Services Supply list of items to be digitised agree cost per image Contact Conservation Supply list of items to be digitised Arrange for conservation assessment Idea of Project - scope contact Digitisation Team Digitisation Project Outline Form
- 9. www.bl.uk 9 How do Heritage Made Digital support digitisation? • Putting into practice a digitisation workflow • Ingesting new and existing material • Documenting gaps and making recommendations 9
- 10. www.bl.uk 10 Initial consultation about your project proposal 10 • Number of items • Format • Data range of material • Provenance of collection • Project timeline • Funding source, internal or partnership • What is delivered by this project? • Where will the digital objects be made available? How will they be used?
- 11. www.bl.uk 11
- 12. www.bl.uk 12 Project management • Structure • Timelines/Targets • Data for planning and reporting • Budget • Training • Storage
- 13. www.bl.uk 13
- 14. www.bl.uk 14 Legal agreements and contracts • Ensure agreements are in place between the BL, partners and funders • Understand what type of agreement is required • DIAM should be informed of the agreements before they are signed • Contracts should reflect the selected content (rights, sensitivity, provenance) • BL can accept any restrictions imposed on future use of the digitised content • Restrictions are communicated at the right time to staff who will implement them • Drafting contracts is resource intensive
- 15. www.bl.uk 15 Two Centuries of Indian Print Digitisation Workflow Diagram
- 16. www.bl.uk 16 Conservation for Digitisation Responsible for conservation and treatment of collection items which is the key that unlocks access • Condition assessments • In-depth conservation treatments • Training for item handling • Collaboration with curators and imaging
- 17. www.bl.uk 17 Useful for conservation to know… • Is funding available for hands-on conservation? • Where can we assess the items?
- 18. www.bl.uk 18 Why carry out pre-digitisation assessment? • Identify problems and issues in advance • Ensure adequate Collection Care resources available • Make workflow a smoother process • Reduce risks during transit • Exclude unsuitable items
- 19. www.bl.uk 19 Assessing condition • We are essentially assessing whether item(s): • Assessment not of item’s general condition but suitability for digitisation • Items in poor condition may still be suitable candidates for digitisation
- 20. www.bl.uk 20 Why something is unsuitable Condition
- 21. www.bl.uk 21 Why something is unsuitable Size
- 22. www.bl.uk 22 Why something is unsuitable Format
- 25. www.bl.uk 25 Imaging Studios 4 Imaging studios at the BL o 30 overhead scanners o 9 camera bays o 4 microfilm scanners Imaging technicians o Proficient on all scanning devices and pre and post processing functions o Competent in image manipulation, colour management and file generation Senior Imaging technicians o Use high value high quality photographic capture equipment o Imaging staff regularly trained on collection item handling skills by BL conservation department
- 26. www.bl.uk 26 Imaging Scoping • Collection type and format determines equipment used • Handling advice from conservation specialist • Overhead scanners - flat, unbound material • Digital Cameras - mss, difficult to handle, oversized, 3D objects + valuable/precious material • Test imaging process to determine effort, time, cost, impact on item • In-house or outsource, from original or microfilm
- 28. www.bl.uk 28
- 29. www.bl.uk 29 Digitisation Technical Standards • Set of baseline requirements governing digitisation at the Library • Master - uncompressed TIFF or JPEG2000 • Define optimal JPEG2000 profiles - lossless and lossy (e.g. newspapers) • Resolution – 300-400ppi and Digital colour depth - 8bit or 24bit • Colour management to RGB, embed ICC colour profile • File size – affects image quality, storage requirements, portability • OCR in ALTO xml; metadata in METS; • IIIF for presentation and access, ODRL for specifying rights
- 30. www.bl.uk 30 Digitised Image Quality B CA
- 31. www.bl.uk 31 Digitised Image Quality D E
- 32. www.bl.uk 32 Strategic Ingest • Ingest spreadsheet captures metadata about object • pSIP production as METS captures structural metadata • Ingest workflow • Publication
- 33. www.bl.uk 33 IIIF Viewer o IIIF compliant (http://iiif.io/) o Downloads o Citations o Search within text o License and usage terms
- 34. www.bl.uk 34 Accessible through online ‘exhibition’ www.bl.uk/early-indian-printed-books
- 35. www.bl.uk 35 Finding Digitised Collections: Online Exhibitions Online Catalogue
- 36. www.bl.uk 36 Digital Curators in Digitisation Projects Digital Curator, Hebrew Manuscripts, experimented with 3D modelling Digital Curator, Two Centuries Indian Print, exploring OCR technologies for Bengali Print
- 38. www.bl.uk 38 OCR for Bangla Workshop Promoting exchange of digital literacy Practical experience using open source OCR to Google Cloud Vision worked best!
- 39. www.bl.uk 39 Digital Scholarship Blog: britishlibrary.typepad.co.uk/digital- scholarship # @BL_DigiSchol @BL_IndianPrint digitalresearch@bl.uk or tom.derrick@bl.uk
Editor's Notes
- “Digitisation” describes a process where physical (or “analogue”) materials are converted into digital versions which can be understood and stored by computers. Analogue materials can be any item which conveys text, image, sound or video – drawings, manuscripts or recordings for instance. In a digital format their information can be preserved, shared or accessed more easily and effectively. Consequently, digitisation is being adopted by institutions the world over, particularly for use in maintaining records of culturally and historically significant information.
- https://www.bl.uk/aboutus/quickinfo/facts/ Low oxygen, robotic retrieval. 600,000 bound volumes, wrapped parcels and boxes of newspapers 150,000 periodicals 740,000 reels of microfilm 750 million newspaper pages in the collection now Collections growing at up to 450 lin metres per year
- Hebrew Manuscripts – in partnership with National Library of Israel. Scrolls spanning a thousand years which reveal Jewish social and cultural life. King’s Topographical Collection - 30-40,000 maps (still raising funding to catalogue and digitise), 2,500 watercolours, drawings and prints from the vast collection amassed by King George III for whom geography was a personal passion as well as a professional necessity. 1555-1824. SP Lohia Foundation website. England and France 700-1200 - In partnership the Bibliothèque nationale de France, funded by the Polonsky Foundation. Illuminated manuscripts. Bilingual website allowing for side-by-side comparison – online by Nov 2018. Qatar Project – 500,000 images phase 1 from India Office records plus 20,000 images of Arabic Scientic Manuscripts & selected maps through Phase 2 - target 1,125,000 pages. Qatar Foundation (a private, non-profit – development of scientific education) Google Books – 250,000 out of copyright books. Google pays for the costs of scanning the books, which will be available on both Google and BL’s websites. Google benefits from being able to use the content as data to mine for their search engine, translation and other services. Public Private Partnerships – Academic Publishers such as Gale Cengage, Adam Matthew.
- EAP – annual competition for applications. The grants provide funding to enable successful applicants to locate relevant endangered archival collections, to arrange their transfer to a suitable local archival home where possible, to create digital copies of the material and to deposit the copies with local institutions and the British Library. Equipment funded by project remains in the country – scanners, digital cameras, computers etc. The grants provide funding to enable successful applicants to locate relevant endangered archival collections, to arrange their transfer to a suitable local archival home where possible, to create digital copies of the material and to deposit the copies with local institutions and the British Library. Equipment funded by project remains in the country – scanners, digital cameras, computers etc. Major projects - individual researchers may apply for grants to locate relevant collections, to arrange their transfer to a suitable local archival home where possible, and to deliver copies to the British Library and a local institution for the benefit of researchers worldwide. Pilot projects - the Programme would particularly welcome applications for pilot projects to investigate the survival of archival collections on a particular subject, in a discrete region, or in a specific format, and the feasibility of their recovery. Grants will be made annually. The funds requested for the project should be fully justified in terms of the work that is proposed. It is envisaged that a major project will be in the region of £50,000 and a pilot project, £10,000.
- GiA and Heritage Lottery application Part of British Library’s Living Knowledge vision for 2023 the programme covers eight years, addressing most at-risk items and creating infrastructure for new digital acquisitions. Preserving the UK’s national sound archive with over 6.5 million recordings from 1880s onwards. Second largest sound archive in the world (after Library of Congress). Only 2% preserved digitally Bid to the HLF for the strands re sound heritage collections and access and engagement Sound Heritage: Safeguarding the nation’s sound collections (unique recordings and legacy published content) By 2023: Digital preservation of 60% of sound collection Digitise ca 100,000 items Digital Acquisition: Digitally acquiring the UK’s published sound and music By 2023: 90% of all new sound recordings published in the UK that fall within the British Library’s acquisition policy will be acquired and preserved Radio Archive: Developing an extensive radio archive for the UK By 2023: Capture and deliver access to radio content from majority of UK licensed radio stations (c.700) estimated 3million hours of radio broadcast per year IT Infrastructure & Workflows: Developing end-to-end workflows and modernising technical capability for large scale upload Access & Engagement: Transforming access to the nation’s sound heritage and delivering learning and professional training programmes
- Clear objectives and benefits Careful pre-planning and scoping (define selection criteria) Accurate budgeting and resourcing Ensure legal compliance and due diligence (copyright, data protection) Assign user terms (human and machine readable licences) Sustainability (post-project maintenance and user engagement) Good governance (quality control, standards, preservation plan) Active engagement with partners and users Number of items Format Data range of material Provenance of collection Project timeline Funding source, internal or partnership What is delivered by this project? Where will the digital objects be made available? How will they be used?
- HMD workflow team help with ingesting material. Currently into the DLS and in the future the new DAMPS system. HMD do not fund completion of partially digitised objects Living Knowledge Vision: We make our intellectual heritage accessible to everyone, for research, inspiration and enjoyment. Heritage Made Digital Programme to look strategically at digitisation and access to the collections at strategy (commercial vs philanthropic balance), put in place a central digitisation workflow team to coordinate and schedule digitisation operations all the way through to publishing content, a range of new digitisation – newspapers, Ethiopic MSS of poor condition or high-priority content
- Understand approach, deliverables, deviations from BL process, partner/funder expectations These are questions to help the DIAM team direct you towards the most appropriate approvals process/documentation Contract template, ‘short’ DIAM form, BC, Research project justification/Web Steering Group
- Web based workflow management tool. Microsoft product integrates with Microsoft Office. BL piloting Multiple collaborators/editors Customised viewing and exporting Pricing based on how much amount of content used and number of users – usually one time licences. Not a substitute for a relational database. We are not using to host images.
- Approval based on completing documentation and presenting to DIAM. Complete project outline form to reflect pre-planning. Templates including budget, file size calculator, resources Budget calculator for projects with less 50K budget – current imaging prices Timing of presentation to DIAM – as early as possible to get feedback, consider options, communicate with external parties, plan for due diligence. Come for initial discussion and bring formal proposal later. When project approved, it is issued a Digital Asset Register number (DAR) which tracks it through the end to end digitisation process (used internally to identify ingested and published content) and used for Management Information reporting.
- What agreement is needed/templates to use– full (funding, digitisation, may or may not include A&R), letter of agreement/email from funder, end-user agreement regarding access and reuse. DIAM should have a sight of draft agreements before they are signed. If Legal and Contracts is involved with drafting an agreement, DIAM should see it Agreements should reflect the copyright status of the content and objectives can be met. Takes time and resource Legal/Licensing.
- Example of an end to end workflow. BL holds over 500,000 printed books in all South Asian regional languages (classical and modern). Plus over 200,000 relating to South Asia, in English and other Western languages - See more at: https://www.bl.uk/collection-guides/south-asia-printed-books-and-periodicals#sthash.of0bqw6Y.dpuf Earliest imprints of mission presses through to unparalleled runs of publications acquired systematically under the Indian Press and Registration of Books Act of 1867, which mandated that a copy of every published work in British controlled India be submitted, to research-level publications of the present day, which continue to be acquired through purchase and donation. In 2015 we embarked a 2 year pilot funded by AHRC and UK government to digitise and make available 1,000 early Bengali printed books, many of which are uniquely held by the British Library and therefore can only be explored by researchers visiting the library. A small amount (about 100 of the 4,000) from SOAS. They date from 1770s-1914. Thematically broad content, with a strong representation of religious tracts (from earliest missionary presses), poetry, farces, fiction, academic texts in sciences, medicine etc. We’ve recently received further funding to digitise 600 more printed books from the SA collection, predominantly 19th century Sylheti texts. Also digitised English language catalogues recording bibliographic details of all books published in India between 1867 through to late 1960s. Great resource for researchers of early publishing in India. Available through data.bl.uk, a repository where the BL makes available datasets from digitised collections. Being digitised internally. My role to support computational research of the collection by digital humanists, through the creation and promotion of the collection’s datasets. OCR for Bengali Print, digital research approaches to Book History, and datasets for use in computational research. [The Newton Fund has also made it possible for us to digitise the South Asian Vernacular Tracts series, of which the Library holds c. 6000 volumes. These are small publications, including pamphlets, bound together into volumes, which cover a wide range of South Asian languages and subjects. They are often rare and fragile, and disproportionately do not survive in other library collections, meaning they are much in demand by researchers in South Asia.]
- Ensures Library is able to make collections available to all Take appropriate steps to preserve them for future generations With more and more of the Library’s content being opened up by digitisation, the demands on the preventive team for its expertise has never been greater. We make initial condition assessments of items to be digitised; recommend any treatments to stabilise or repair items; advise on handling and transport, camera equipment, lighting and cradles. We work with our conservators, imaging specialists, (internally and externally) curators, project managers and procurement managers to ensure successful and safe digitisation of our collections within an agreed schedule. Estimates and assessments We assess all prioritised items for treatment at the start of each year and provide a detailed estimate of the proposed treatment and how long it will take to complete. This enables us to maximise the resources and specialisms that we have in-house and plan an effective programme of work over a 12-month period. We also undertake internal consultations to develop long-term strategies for collections. In-depth conservation treatments and research Our teams of conservators specialise in a variety of book- and paper-based disciplines as well as parchment and vellum, photographs and modern materials. We undertake a wide range of treatments as required, keeping intervention to a minimum where possible. As part of this process, we also carry out research into formats, structures, sewing and binding styles, as well as the use and deterioration patterns of substrates and media.
- Important to remember assessment is not of an item’s general condition but its suitability for digitisation. Traffic light system to grade material. This system can also be used for the digitised images during quality control and for risk analysis. Amber - Preparation – removing staples or fixings, often rusty. Or repairing tears (but not little tears that don’t interfere with text) Out of 500 items for Gale project 20 were classed as red.
- [top left: may be able to open for digitisation (i.e. minimum required conservation)] [centre: red rot – common in Indian archives] [bottom left + top right: mould]
- Size affects digitisation equipment. The small item’s content is less valuable than the binding and the physical form. Therefore we only photographed the object and didn’t risk opening the book to digitise the content as the content was a duplicate of another item. Therefore this was flagged red and not sent for digitisation.
- Use a Japanese tissue called Kozo – a re-moistable tissue coated with gelatin or wheat starch. We provide the coating then let dry and need then only apply a small amount of moisture.
- Difference between preservation copy and reference copy. The former requires more rigorous set up in terms of imaging, i.e. would need to be an exact representation of the original – so colour spec needs to be accurate.
- Overhead scanners for flat unbound i.e. maps and posters and printed material (books, journals, newspapers) that can be opened flat. Depends on the size and format of the material. to assess quality and quantity of items. This also enables Imaging Services to produce a guideline cost for digitisation. If necessary an imaging trial may be carried out and could include digitisation of items at either end of the quality scale as well as the majority of average items. This will enable a more accurate cost to be provided based on time and equipment required to digitise. Based on the above a schedule will be prepared detailing total to be digitised, number of imaging technicians required to carry out the digitisation and equipment to be used. Most important thing to note is that there will always be exceptions to any estimates no matter how accurately it is assumed the scoping exercise is, therefore it is wise to build into your budget a contingency in terms of time and budget. It is also important to note that Imaging Services do not provide any conservation resources and instead utilise the services of the BL conservation department on a consultative basis. We are required to consult with conservation when in doubt and in fact our processes include an escalation procedure in the event that an Imaging Technician has any doubts with regard to collection items. As a result of the above an SLA document is created as part of the project scheduling and added to any project documentation that already exists.
- Some scanners have photographic sensors whilst others have linear (the band of light). Overhead scanners. Photographic sensors Reduction type linear sensors Sizes vary from A4 to A0 Suitable for flat unbound material i.e. maps and posters and printed books that can be opened flat. Some manuscript material can be scanned but usually with consultation from conservation. Cameras with digital backs - Capture One (cropping, straightening Foliating etc) now on Apple Macs Scanners 11 x Zeutschel OS12000 6 x Standard Fujitsu – bitonal 5 x Standard Fujitsu – colour 2 x Fujitsu 5750 1 x Kirtas Book Scanner 1 x Zeutschel OS10000 1 x Zeutschel OS14000 A0 1 x Zeutschel V Scanner 4 x Zeutschel OS15000 1 x Book2net Kiosk 1 x Ricoh MPC75000 Multi Machine 1 x Sunrise MicroFilm/Fiche 2 x Epson flat bed (V500 & V750 Pro) 10 x Post Processing PC’s Software Photoshop & Acrobat software Scanner Manufacturers Bespoke.
- Capturing a large item Conservation note: here using a plasier zote foam board to rest the item secured with velcro strap. Hybrid device – X71 camera Can be used to digitise any material following consultation with conservation and a trial to review product. No moving parts. Driven by the scan system. Digital Cameras Analogue camera body and Lens system Digital back Suitable for manuscript material, material considered difficult to handle, oversized material, 3D objects and valuable/precious material. Please refer to BL technical standards at https://wiki.bl.uk:8443/pages/viewpage.action?pageId=134677065
- OCR generally performs slightly better with single column text than double. With newspapers over historical print, and the quality of scanning can make a difference. In our case we are dealing with scanned documents which have been scanned using flatbed scanners, scanned at 400ppi, 8-bit greyscale lossless.. We are producing scans in TIFF, JPEG2000 and PDF formats. What do you think the potential obstacles are to successfully OCR’ing these images? A – focus issue/blurred – should be rescanned B – Ink transfer (see through) – can have detrimental effect - should have blank paper backing C – text running into binding – if book can be pressed flat without risk of damage, it should. Consider type of scanner – flat-bed in this case.
- D – over-exposed – may or may not present an issue for OCR. Nevertheless, needs to be rescanned because of poor aesthetic quality. E – RGB mis-registration – caused by the scanning software error – not common but watch out for.
- METS used to capture structural metadata about digital object Metadata Extension Repository (rights, preservation, admin, structural) catalogues are updated with a link to the digital object Content files are ingested into the DLS Metadata Aggregator publishes IIIF manifests containing descriptive metadata from the catalogues Universal Viewer loads a IIIF manifest published by the Aggregator, and loads the related images direct from the DLS
- iiiF is a standard for serving and displaying high-quality images; iiiF community of practice – BL member of the iiif consortium; developing compatible software that is easy to install and provides a great user experience Image API – Image data Presentation API – title, Structure (TOC), Sequence (iiif xml manifest) Developments – BL project for Universal Player http://accesssit.ad.bl.uk/item/viewer/ark:/81055/vdc_0000000000C0 User actions will depend on copyright status, location, permission (BL/licence) Single interface for mss, printed books, born digital + sound Calls JPEG2000s stored in the Digital Library System and metadata stored elsewhere
- Dedicated Conservators and Digital Curators support research and innovation
- We’re engaged in a number of other digital research initiatives. We’ve run a series skills sharing workshops to foster digital literacy in Indian institutions and share best practices. We have provided training covering: digitisation workflows and standards how information technology is transforming humanities research today, and in turn Library services. - latest developments for OCR of Bangla within the context of archival collections. Researchers, librarians and information professionals attending used and compared different open source OCR systems. Across the workshops we have delivered training to more than 130 information professionals and academic researchers from Indian institutions. The workshops have been very popular and the team are planning more for the coming year that again will be held in India. Other events 2CiP have organised include an Academic Symposium which featured a Digital Humanities panel. The talks focussed on ‘Using GIS and Gephi to analysing print networks in northern India, focussing on the Hindi and Urdu press’, ‘Textual transmissions of Tagore’s plays’ (Gephi is visualisation and exploration software (Java) for creating graphs and network analysis). The team also hosted at the BL mid-career librarians from the National Library of India. They received a week of sessions with different teams from the library including digital scholarship.