Taming the Monster: Digital Preservation Planning and Implementation Tools

Taming
the Monster
Digital Preservation Planning
and Implementation Tools

Dorothea Salo
Photo: “Happy Easter, to my Peeps”
http://www.ﬂickr.com/photos/76074333@N00/449028423/ One System, One Library
WorldIslandInfo.com / CC-BY 2.0
2 June 2011

Why is this
so scary?

http://www.ﬂickr.com/photos/76074333@N00/449028423/

Isn’t this just
as scary?

Photo: “News Paper Origami Dragon Monster”
http://www.ﬂickr.com/photos/epsos/3777343342/
epSos.de / CC-BY 2.0

Yet we
persevere.

Photo: “News Paper Origami Dragon Monster”
http://www.ﬂickr.com/photos/epsos/3777343342/
epSos.de / CC-BY 2.0

DIGITAL IS NO
DIFFERENT.

Photo: “559 - The Matrix - Seamless Texture”
http://www.ﬂickr.com/photos/zooboing/4335531915/
Patrick Hoesly / CC-BY 2.0

Many of the same ideas apply...
• Planning and policy
• Risk assessment
• Risk management
• (knowing that we can’t save everything)
• Materials quality matters!
• Problem discovery and remediation
• Crisis management
• Chief problems: staﬀ, $$$, organizational
commitment
Photo: “Where I Teach”
http://www.ﬂickr.com/photos/eklektikos/2541408630/
Todd Ehlers / CC-BY 2.0

Planning and
assessment
tools


Scene-setting

• Rosenthal, David. “Requirements for Digital
Preservation: a Bottom-Up Approach.”
• http://www.dlib.org/dlib/november05/rosenthal/
11rosenthal.html
• If you’re new to this, or trying to ﬁnd your
feet, this is the best short introduction I
know.
• The list of threats is outstanding.

Photo: “Bottoms Up! - Duck; San Anton Gardens, Malta”
http://www.ﬂickr.com/photos/foxypar4/3123113762/
John Haslam / CC-BY 2.0

TRAC
• “Trusted Repository Audit Checklist”
• Despite the name, covers a LOT more than
the technology!

!
• Budget
• Staﬃng
• “designated communities”
• CRL will audit you, if you like
• (don’t, unless you’re really serious!)
• http://catalog.crl.edu/record=b2212602~S1

DRAMBORA
• Digital Repository Audit Method Based on
Risk Assessment
• A “self-test,” if you will.
• DRAMBORA is equally good as a pre- or post-test.
• Personally, I prefer DRAMBORA to TRAC,

!
especially for those just starting out.
• http://www.repositoryaudit.eu/
• (registration required for toolkit access)

Coping with
ﬁle formats


The one acronym you
need to know: FITS
• “File Information Tool Set”
• (you need to know this; otherwise it’s hard to Google)
• Wrapper for several ﬁle-format detector
software packages
• Intended to be baked into other software
• It’s early days yet!
• (This means you can’t always trust what the tools tell
you, especially when they’re telling you about errors.)

What’s this ﬁle?

• wotsit.org “The Programmer’s File and
Data Resource”
• Directory of ﬁle extensions
• When in doubt: open in a browser or text
editor and see what you get.
• N.b.: Microsoft Word is NOT a text editor!

Solving the
geographic
distribution
problem


What problem, now?
• The “all your eggs in one basket” problem.
• If all your bits are on one server, and the server room
is ﬂooded, or your town is nuked—oops.
• Not the same as backups!
• Don’t get me wrong, backups are important!
• Backups are SHORT-TERM, and usually LOCAL.
Geographic distribution (plus associated auditing) is
intended for the long term.
• Don’t forget auditing!
Photo: “Nido”
http://www.ﬂickr.com/photos/italintheheart/3679974298/
Jorge Elías / CC-BY 2.0

LOCKSS
• Lots of Copies Keeps Stuﬀ Safe!
• (There is also Portico, but Portico only works with
e‑journal content.)
• Open-source software that handles replication and
(some) auditing.
• “Private LOCKSS network”
• A group of institutions agrees to build a LOCKSS
network just for the stuﬀ they’re interested in.
• ASERL does this for ETDs. Many institutions
(including UW-Madison) participate in a PLN for
govdocs.

“The cloud”
• Typical cloud-based storage services make
NO promises they won’t lose your stuff.
• And for large quantities of data, bandwidth can become
an issue.
• And can they look at your stuff? Should they be able to?
• Some early movers in this market fading
• Iron Mountain had to kill their service.
• DuraCloud
• trying to finesse this issue by negotiating tougher SLAs
with cloud-storage providers
Photo: “Sky View From Humboldt Park”
http://www.flickr.com/photos/purpleslog/2589612577/
Purple Slog / CC-BY 2.0

Repository
and digital-library
platforms


Friendly word
of advice:

PICK
SOFTWARE
LAST. Photo: “Briana Calderon; future educator of america.”
Arielle Calderon / CC-BY 2.0

Another friendly word of
advice:

DON’T CHASE
THE SHINY.

Photo: “Sparkle Texture”
http://www.ﬂickr.com/photos/abbylanes/3214921616/
Abby Lane / CC-BY 2.0

Digital-library software
• Is almost always VERY BAD at digital
preservation!
• (most packages don’t even try!)
• So if a ﬁle gets corrupted on the server, or whatever...
no warnings, no restore, nothing. Also, provenance?
Who needs provenance? Event tracking? What’s that?
• I’m not saying don’t use it. I’m saying that
it doesn’t solve this problem.
• In fact, if you’re using this software, you need to solve
this problem FOR IT.
Photo: “National DIGITAL Library”
http://www.ﬂickr.com/photos/schex/193912573/
Jesse Schexnayder / CC-BY 2.0

Examples

• ContentDM: http://contentdm.com/
• Omeka: http://omeka.org/
• Greenstone: http://greenstone.org/

Institutional-repository
software

• Is SHOCKINGLY bad at digital preservation!
• (Though sometimes better than most DL software.)
• Examples
• Hosted/commercial: Digital Commons (BePress),
ContentDM, DigiTool
• If you go hosted, you’d better ask about their digital-
preservation practices!
• Open-source: EPrints, DSpace, Fedora
Photo: “IMG_0668”
http://www.ﬂickr.com/photos/12967790@N00/66531124
Robert / CC-BY 2.0

A new approach:
curation
microservices


Do we really need

Photo: “giant crystal blob”
http://www.ﬂickr.com/photos/a_of_doom/527905701/
A of DooM / CC-BY 2.0
THE BLOB?

How about a jigsaw
puzzle instead?
• Break the digital-preservation problem
down into parts.
• Code up each part, making sure that it
plays nicely with other parts.
• lots of nice APIs!
• which means other software can adopt/adapt
microservices as well!
• Put parts together as you need them.
Photo: “Lapsana Apogonoides Puzzle”
http://www.ﬂickr.com/photos/gdesigneralex/2313092112/
gdesigneralex / CC-BY 2.0

California Digital Library

• Pioneering this approach
• Has open-sourced code for microservices
• Has added microservices together to build
its “Merritt” storage/repository service

Escaping the silos:
Fedora Commons


What is Fedora Commons?
• Blueprints and foundation, not the whole
house (analogy credit to Peter Gorman)
• You build the house you want!
• Or you build condominiums on the same
foundation.
• Need different user interfaces for different materials?
• Need different structures and behaviors?
• No problem! Fedora can handle that.
• (have I run this analogy into the ground yet?)

We had this...

Diagram courtesy of Peter Gorman.

We are building this.

Diagram courtesy of Peter Gorman.

E-records
management


Axioms
• Records management is
about policy and
procedures.
• If your policy doesn’t ﬁt with
their procedures, guess what
wins? Choose battles wisely.
• There is never enough
storage space.
• Nobody cares until
there’s a crisis.
• Software will not save
you... but it might help!
Photo: “The Never Ending Math Problem”
http://www.ﬂickr.com/photos/acidwashphotography/2967752733/
d3 Dan / CC-BY 2.0

Duke Data Accessioner

• Accessioning tool for digital data
• use case: J. Important Scholar dumps her hard drive
on your desk, expects you to cope
• File migrator, metadata manager, GUI,
plugins (e.g. for ﬁle-format detection)
• Bit rough, but in production use.
• http://library.duke.edu/uarchives/about/tools/data-
accessioner.html

Archivematica

• Soup-to-nuts records management and
digital preservation tool.
• Evaluation and accessioning all the way through
preservation actions. (Oddly, they seem to be
missing disposal... but they’re in alpha, so...)
• Open source
• Runs on a Linux server; RMs and archivists log in to
GUI application remotely.
• Normally I hate and fear silos, but this one
is smartly built on microservices.

Practical E-Records
• Weblog by Chris Prom and protegés
• Tool evaluations, conference-session
writeups, essays on praxis
• Best reading out there for the do-it-
yourselfer
• If you’re not reading it, why not?
• http://e-records.chrisprom.com/

Last thoughts


If you can’t do everything...

Image: “Confused”
http://www.ﬂickr.com/photos/kristiand/3223044657/
Kristian D. / CC-BY 2.0

that’s okay. Who can?

DO SOMETHING.

Photo: “Came hame háááá!”
http://www.ﬂickr.com/photos/kristiand/3223044657/
Guirí R. Reyes / CC-BY 2.0

The worst threat?

INACTION. Photo: “Fatty’s role model”
http://www.ﬂickr.com/photos/cloudzilla/4910616774/
cloudzilla / CC-BY 2.0

Thank you!
This presentation is available
under a Creative Commons 3.0
United States license.

Taming the Monster: Digital Preservation Planning and Implementation Tools

Recommended

Recommended

More Related Content

Similar to Taming the Monster: Digital Preservation Planning and Implementation Tools

Similar to Taming the Monster: Digital Preservation Planning and Implementation Tools (20)

More from Dorothea Salo

More from Dorothea Salo (20)

Recently uploaded

Recently uploaded (20)

Taming the Monster: Digital Preservation Planning and Implementation Tools