Supporting Significant Properties in a Working Archive (SPs part 5), by Stephen Grace and Gareth Knight


Published on

This presentation, the fifth of six parts on the practical analysis of significant properties of digital objects, seeks out tools capable of extracting and maintaining properties for an archival-scale number of objects. The presentation was given as part of module 3 of a 5-module course on digital preservation tools for repository managers, presented by the JISC KeepIt project. For more on this and other presentations in this course look for the tag 'KeepIt course' in the project blog

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 1. Format Identification 1. "I have an object; what format is it?" 2. Format Validation 1. "I have an object that purports to of format F; is it?" 2. "I have an object of format F; does it meet profile P of F?" 3. "I have an object of format F and external metadata about F in schema S; are they consistent?" 3. Characterization 1. "I have an object of format F; what are its salient properties (given in schema S)?" JHOVE2 alpha prototype available Documents are analyzed and checked for being well-formed (consistent with the basic requirements of the format) and valid (generally signifying internal consistency).
  • (such as format, width, height, size, or the number of colours) through eXtensible Characterisation Extraction Language (
  • Supporting Significant Properties in a Working Archive (SPs part 5), by Stephen Grace and Gareth Knight

    1. 1. Supporting SPs in a working archive: Software Tools
    2. 2. Challenge <ul><li>Reality: Infeasible to perform manual maintenance of large number of objects. Require software capable of extracting & maintaining SPs for large of objects </li></ul><ul><li>Requirements: </li></ul><ul><li>Object analysis tools </li></ul><ul><ul><li>Support requisite formats </li></ul></ul><ul><ul><li>Identify all/some SPs </li></ul></ul><ul><ul><li>Support batch analysis </li></ul></ul><ul><ul><li>Ideally well supported and documented </li></ul></ul><ul><li>Description schemas to record SPs </li></ul><ul><ul><li>Flexible </li></ul></ul><ul><ul><li>Machine and format idependent </li></ul></ul><ul><li>Conversion/emulation tools capable of maintaining SPs </li></ul>
    3. 3. Format identification <ul><li>File identification through Magic Number and ‘light touch’ scan of encoding structure. </li></ul><ul><li>Recognise 100s (potentially 1000s) of formats </li></ul><ul><li>Provide basic encoding info, but not detailed structure </li></ul><ul><li>Examples: </li></ul><ul><ul><li>File (1): Free version created in 1986 & available for all operating systems. </li></ul></ul><ul><ul><li> (Windows) </li></ul></ul><ul><ul><li>DROID: Java app developed by TNA. Integration with PRONOM. Format ID & assignment of PUID, which can be linked to preservation planning. . </li></ul></ul><ul><ul><li>FFIdent: Java library to ID and extract basic information. Recognizes 27 encoding formats using header information (magic number & common structural information) </li></ul></ul>
    4. 4.
    5. 5. Detailed Analysis <ul><li>Email: </li></ul><ul><ul><li>Aperture - Java framework able to decode structured text and convert to other format </li></ul></ul><ul><ul><li>ReadPST: Open source tool for processing Outlook PSTs </li></ul></ul><ul><ul><li>XENA - Java tool developed by NAA </li></ul></ul><ul><li>Audio: </li></ul><ul><ul><li>MP3Info - technical info viewer and ID3 1.x tag editor that supports the MP3 file format. </li></ul></ul><ul><ul><li>SoX/SOXI (Sound eXchange): extracts descriptive MD and technical info </li></ul></ul><ul><ul><li>MetaFlac: Extractor tool for FLAC audio. </li></ul></ul><ul><li>Images: </li></ul><ul><ul><li>TiffInfo </li></ul></ul><ul><ul><li>ImageMagick </li></ul></ul><ul><ul><li>JHOVE </li></ul></ul>Perform detailed analysis of internal structure of one or more files. See InSPECT Testing Reports available at for further info on these tools
    6. 6. JHOVE 1/2 <ul><li>JHOVE ( </li></ul><ul><li>Format-specific digital object validation API written in Java </li></ul><ul><li>Functionality: Format identification, Format validation, Format Characterisation </li></ul><ul><li>Supports: AIFF, ASCII, Bytestream, GIF, HTML, JPEG, JPEG 2000, PDF, TIFF, UTF-8, WAV, and XML. </li></ul><ul><li>JHOVE2 ( </li></ul><ul><li>Supports: JPEG 2000, PDF, SGML, Shapefile, TIFF, ASCII & UTF-8 encoded text, WAVE, XML, ICC color profile </li></ul><ul><li>Functionality: Format identification, validation, feature extraction & policy-based assessment </li></ul>
    7. 7. JHOVE Demo
    8. 8. XCL (eXtensible Characterization Language) <ul><li>Content extraction </li></ul><ul><ul><li>Extracts content & tech properties through use of XCEL and saved as XCDL. </li></ul></ul><ul><li>Format support: </li></ul><ul><ul><li>PNG, TIFF, GIF, BMP, JPEG, JP2, PBM, PCD, PCX, PICT, PPM, PSD, SVG, TGA, XBM and XPM, MS DOC, DocX, PDF </li></ul></ul><ul><li>Content comparison </li></ul><ul><ul><li>Compare 2 objects e.g. TIFF & PNG, PDF & Doc </li></ul></ul>
    9. 9. XCL Extract & compare
    10. 10. XCL Demo
    11. 11. Final thoughts <ul><li>Analysis tools useful, but have problems: </li></ul><ul><ul><li>Limited format support </li></ul></ul><ul><ul><li>Variable access methods (GUI, CLI, APIs) </li></ul></ul><ul><ul><li>Inconsistent reporting process </li></ul></ul><ul><ul><li>Different metrics (e.g. text vs. no.) </li></ul></ul><ul><ul><li>Metric variations (e.g. milliseconds) </li></ul></ul><ul><li>Partial solution: Wrap tools into services </li></ul><ul><ul><li>PLANETS Interoperability Framework </li></ul></ul>