Microfilm or Digitize: Which is Right for You?

Microform or Digitize: Which is Right For You? Brad Houston Records Manager, University of Wisconsin-Milwaukee

Reformatting
v. 1. To create a copy with a format or structure different from the original, especially for preservation or access. 2. To migrate information from one carrier to another.
Source: Richard Pearce-Moses, A Glossary of Archival and Records Terminology, Society of American Archivists, 2005
Encompasses microfilming, digitization, photocopying, redevelopment, you name it!

Why reformat your records?
Save space in the office or records center
Improve access to frequently-requested records
Move records to format more suitable for long-term preservation
Alternately: Provide duplication for redundancy/off-site storage

Goals of this presentation
Discuss general issues to consider before embarking on reformatting project
Examine the two main forms of reformatting practical for most municipalities
Digital and Microfilming, of course!
Describe scenarios in which either, both, or neither type of reformatting is appropriate

PART I: GENERAL CONSIDERATIONS
Legal
Operational
Administrative

Before you start reformatting…
Why is my office thinking about this in the first place?
Space concerns?
Vital Records Protection?
Increased Public Access?
Better Discoverability?
Improved Security?
Do these records NEED to be reformatted?

How are these records used?
Preservation Concerns
Vital records
Deeds, Birth Certificates, etc.
Private-use records
Archival Documents
Policies, reports, minutes
Infrequently accessed
Unlikely to be subpoenaed/discovered
Access Concerns
Administrative records
Notarization, budgets, etc.
Public-use records
Short-retention documents
Procedures, expense reports
Frequently accessed
Increased likelihood of subpoena/discovery

What are the startup costs?
Time/Labor needed to reformat
Specialized labor/skills needed for project
Equipment outlays
Public notices (as needed)
Other buy-in costs (appeals to officials?)

What are the maintenance costs?
Cost of physical equipment maintenance
Time/labor needed to maintain
Time/labor needed to provide access
Media stability offsets
Is the maintenance cost offset by return on investment?

What are the security implications?
Does the information in the reformatted records require protection?
If so:
Reformatting a net increase or decrease in security?
Initial security measure outlays
Staff information security training
Secure access to public (as applicable)

What are the legal implications?
Federal Rules of Civil Procedure Sec. 34 (December 2006)
Discovery applies equally to reformatted records!
Public Records Laws in various states
Example: Wis. Stat. 19.32: Open Records Act
Other federal/state disclosure laws
Sarbanes-Oxley, Gramm-Leach-Bliley, etc.
May apply if working with private contractors

Should I reformat at all?
Do reformatting candidates have high retention periods (>10 years)?
Is my estimated ROI (in either time or cost) more than the time or cost to reformat?
Do I have the infrastructure (or can I build it) to adequately support and maintain system?
If no to any of the above, consider postponing reformatting program!

PART II: MICROFILM AND MICROFICHE
An Oldie But a Goodie

The least you need to know: Microfilm
Preservation : Well-suited
Access : Requires time investment
Initial outlay : Medium equipment/low training costs
Continued Maintenance : Low Cost
Security Impact : Neutral
Legal impact: Potentially high

First steps: Needs planning
Active vs. Inactive records for reformatting?
Storage and use copies? One or both?
Need for long-term preservation?
Microfilming from source documents or data from a computer?
Large or small amount of documents to be filmed?

Film Selection: Physical Medium
Diazo Film
Best for use copies; less stable for permanent preservation
Vesicular Film
Flips master image polarity; good for non-printing films. Non-ideal for preservation
Silver Halide Film
Best for long-term preservation; easily damaged with repeated use

Film Selection: Format
Microfilm roll (16mm or 35mm)
Inexpensive and compact, but cumbersome to access
Best format for storage of vital or archival documents
Microfiche (105mm, rows/columns)
Easier to organize/access, but more expensive to produce and store
Best format for case files, semi-active records

Initial Outlay: Equipment
Microfilm Camera
Rotary—best for initial reformatting project
Planetary—better image quality, diverse sizes
Microform reader
Different designs for film and fiche– some do both, for a price
Built in printer?
Planetary microfilm camera

Initial Outlay: Document Prep
Remove documents from folders, remove fasteners, etc.
William Saffady: 1000 pages/hour (2.5 drawers/hour) as a benchmark
This figure increases for non-standard file sizes
Do you need a target ? Find out before you film!

Reduction Parameters
Reduction: Number of times a page is reduced through microphotography
24x reduction for standard-sized documents
30x reduction for large documents
48x for COM reduction
Remember to make sure your microform reader can read the specified reduction!
Variable-zoom lens, as appropriate

Quality Control
Should be done for each slide to check for:
Legibility (naked eye overview)
Resolution: ability of reader to distinguish fine detail
Specialized resolution chart
Density: Contrast between info/non-info areas
Densitometer
For standards, see: ANSI/AIIM MS23-1998

Systems Maintenance
Low cost of providing access, training staff/public on use
Cameras may need to be replaced periodically
Readers may last decades, depending on complexity
If all else fails, records readable with a light table and magnifying glass!

Environmental/Security
On-site records=your responsibility to maintain good preservation environment!
Windowless room if possible
No more than 50% relative humidity
No more than 70 degrees Fahrenheit
Acid-free/chemically stable sleeves (microfiche)
Same security implications as with paper records
Maintain secure room, locked cabinets, etc. as needed

Access concerns
Microfiche may be indexed/organized same way as “regular” case files
Microfilm: roll format means special index should be created
Consider primary access point of records
Entries should include: reel, frame, secondary access
Provide cross-references as needed
Indicate in index when filming took place, and by whom the records were filmed

Legal Issues and Records Retention
Will you be able to produce the documents in a timely manner?
Uniform Photographic Copies Act (28 USC 1732)
If records have 10 year retention period or less, off-site paper storage may be more cost-effective

Final Verdict:
Microfilm remains useful for its relative lack of expense, easy reproduction, and extreme longevity (>500 years!). However, the issue of accessibility, particularly with microfilm rolls, must be taken into account when deciding to film records for preservation or access.

PART III: DIGITIZATION
New Paradigms, New Problems

The Least You Need to Know: Digitization
Preservation: Potentially problematic
Access: Very good, with the right tools
Initial Outlay: Low equipment/medium software/high staff training
Continued Maintenance : high level of staff time/effort
Security Impact: Potentially high
Legal Impact: High, but manageable

Digitization is NOT a panacea!
Potentially very powerful productivity tool and space saver
At minimum, the following are required for an effective program:
Content Management System
Digital Preservation Plan
Disaster Recovery Plan

Needs Planning: Digitization
Active vs. Inactive records
For digital records, this is a function of retrieval speed
Public vs. Private access to the records
What are your preservation needs?
File-sharing needs?

Initial Outlay: Hardware
Scanner, preferably with a document feed
Resolution required depends on type of scan
Digital Camera/Overhead Scanner
Importing/editing computer
Public Access computer (as needed)
Storage Media

Digital Storage Media
Good: Removable storage media (CD-ROMs, DVD-ROMs, External Hard Drive)
Good for quick access; low longevity
Better: Disc Array or Server Array
Medium longevity and access speed; better for redundancy of data
Best: Distributed Storage Space
Maximum redundancy; slowest retrieval time

Initial Outlay: Software
Scanning/Import software
Usually comes with scanner
Image Manipulation software
For document reformatting, include Optical Character Recognition (OCR) capability
Content Management System (CMS)
The most important (and most expensive) component of digitization management

Content Management Systems
Should include robust metadata at point of entry:
Subject
Author
Recipient
Date
Other keywords/tags
Some CMSs extract this automatically– ask your vendor!

Content Management, Records Management
Consider a CMS with integrated Records Management:
Facilitated indexing
Automated or semi-automated disposition
Improved information security
Legal protection under “Safe Harbor” rule (FRCP 37)

DoD 5015.2 RM standard
Standards for ERMS– often advertised!
A compliant system will include:
Metadata entry and file plans (including retention scheduling)
Records declaration and classification
Records protection and/or change auditing
Designation of official ERMS administrator(s)

Imaging Parameters
Pixels per inch/dots per inch (ppi/dpi)
Capture mode
Bit Depth
File Format
Compression
n.b. Higher values increase image quality AND file size/storage requirements

PPI/Resolution
Should be as high as possible, but consider display and space needs!
To adjust resolution of already-imported image, adjust image size (smaller=sharper)
Document Type PPI (master) PPI (access) Text 400 150 Photographs 400-800 200-300 Maps 600 (400 if oversized) 150-300 Graphics (including arch. Drawings) 600-800 200-400

Capture Mode/Bit Depth
A measure of the amount of colors able to be displayed in the image
USUALLY an easy call to make– with some exceptions
Document Type Capture Mode Bit Depth Text Bitonal 1 bit B/W photos/drawings Grayscale 8 bit Color photos/drawings RGB 24-48 bit (access v. master) Maps Grayscale/RGB 24-48 bit (access v. master)

File Format/Compression
Higher compression=greater degradation of data over time
Lossy vs. Lossless compression
Proprietary vs. Non-Proprietary formats
Document Type File Format (Preservation) File Format (Access) Text TIFF or PDF/A JPEG or PDF Photographs TIFF or JPEG 2000 JPEG Graphics TIFF or JPEG 2000 JPEG Maps TIFF or JPEG 2000 JPEG

What’s in a name?
Don’t rely on the CMS to find files for you!
File names should be easily-browsable:
Date of imaging or use
Type of file (e.g. dept. acronym?)
Unique identifier (e.g. project name, serial #)
File naming not a substitute for proper metadata in the CMS

Quality Control: Physical Aspects
Looking for similar defects as with microfilm QC
Legible images
Correct orientation/resolution
Density, color balance, etc.
Targets and color bars assist with this check
Some QC defects are digitization-specific:
Correct size, name, file format, etc.
Digital artifacts (lines on screen, etc.)

Quality Control: Content Aspects
Has the OCR read the content properly?
Usually a cursor drag determines this
Most OCR programs allow for manual corrections
Is the metadata complete and appropriate?
Dublin Core: search and use metadata
PREMIS: Preservation metadata
Administrative metadata (rights, statutes, etc.)
Structural metadata (how records tie together)

Systems Maintenance
Maintenance of a digitization system often comprises up to two-thirds of total budget!
Cataloging, description, indexing
Ongoing staff training
Physical data storage
Data migration
Consider cost-sharing with neighboring municipalities

Access to Digitized Records
The key upside to digitization!
Public Access Terminal/Web Access/Admin use only?
Rights management?
Will you allow printing?
Watermarks
Electronic file structure
Searchability?

Preservation Concerns
IT department usually keeps data storage secure, environmentally stable
Data itself may have stability problems!
Software obsolescence
Hardware obsolescence
Data integrity loss
File format obsolescence

Preservation Solutions?
Non-proprietary formats, lossless compression
Usually implies access copy creation
REGULAR (5 years/3 version) reformatting:
Migration to new systems/file formats
Emulation of programs to read files on old systems
Encapsulation in XML “wrappers”
Work with IT to see which is right for your organization

Information Security
Risk of security breach is MUCH higher because of ease of reproducibility
Potential Solutions:
Versioning/Logging of imaged documents
Declaration of imaged documents as “records” in DoD 5015.2 system
Visible or metadata-based watermarks
Embargo on portable storage for imaged documents

Digitization Legal Issues
Zubulake Case (2004): If you have the records, you must produce them or pay penalties
Federal Rules of Civil Procedure
Electronic records are discoverable
Safe Harbor rule DOES NOT APPLY if there is reasonable expectation of litigation (Broccoli)
Plaintiff may request format for production

Legal Issues: Responses
Make sure your CMS has a robust search feature
Semantic web /fuzzy logic?
Be prepared to convert formats as needed
Talk to legal team about Litigation Hold procedures
Formalize records retention/destruction schedules and procedures

Final Verdict
Digitization offers organizations excellent options for saving space and making reformatted records accessible to staff and to the public. However, great care must be taken to ensure that preservation, security, and legal issues are addressed before a digitization solution is implemented.

PART IV: CONCLUSION
Summation, scenarios, resources, questions

In Summary:
Microfilm and Microfiche
Excellent preservation medium
Low-to-medium startup costs
Low maintenance and staff training requirements
Potential accessibility issues
High security, reduced legal response ability
Digitization
Superb accessibility if managed within CMS
Increased ability to respond to legal requirements
Medium-to-high startup costs
High maintenance and staff training requirements
Security and preservation issues

Scenarios: Office A
Reformatting target: Birth Records
Considered a “vital record” by statute
Medium-to-high usage
High security needs
Medium-to-low subpoena likelihood
What would you suggest?
My answer: Microfilm

Scenarios: Office B
Reformatting Target: Financial Records
Records Schedule states 7-year retention time
Subject to public access under state records law
Very high use– referred to on daily basis
High likelihood of audit/subpoena
Medium security requirements
My answer: Do not reformat OR digitize as requested

Scenarios: Office C
Reformatting Target: Child Welfare Case Files
Records Schedule states 20-year retention time
High use while clients active; low use thereafter
Very high security requirements
Medium-to-high potential for subpoena
My answer: Microfiche case files and/or digitize

Scenarios: Office D
Reformatting Target: Committee Minutes
Permanent Retention/Archival
Subject to public access under state records law
High use by both public and staff users
Low security requirements
Medium likelihood of audit/subpoena
My answer: Digitize with web access

Microfilm or Digitize: Which is Right for You?

by Brad Houston

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 4

Statistics