Uploaded on

Presented by Lisa Gregory at Best Practices Exchange, December 2012. This presentation compared OCLC's Digital Archive to DuraCloud, and the process of migrating storage from one to the other.

Presented by Lisa Gregory at Best Practices Exchange, December 2012. This presentation compared OCLC's Digital Archive to DuraCloud, and the process of migrating storage from one to the other.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
259
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
9
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. State Library • Part of the North Carolina Departmentof North of Cultural ResourcesCarolina • Work closely/pool resources with the State Archives • Digital Information Management Program
  • 2. CONTENT STAFF State Publications Genealogy Research ~ 4.75 FTE North Caroliniana Local server CONTENTdm (state-supported)Connexion Digital Import Offsite storage (vendor)SYSTEMS STORAGE
  • 3. Born-Digital 3.25 Digitized .75CONTENTdm CONTENTdm Connexion Project Client Local Storage .75 Remote Storage
  • 4. CONTENT• We preserve access and master copies• 1.27 TB, 162,000+ files• Mostly .tif, .pdf, .jpg, .txt
  • 5. CONTENTFile structure by “project” admindocs fulltext images_access images_master images_processed metadataNaming convention pubs_serial_annualreportclean2005.pdf gen_statefair_lifecharacterthomasruffin1871_0001.tif
  • 6. Local storage • managed by department-wide IT • includes working & preservation content • server is shared, but our directory is restricted • daily incremental backups STORAGE
  • 7. OCLC’s • Began using in 2008Digital • Web interface for accessArchive • FTP or automatic uploads • Integrated with CONTENTdm • Detailed reporting, broken out by CONTENTdm collection • Fixity checks, virus checks
  • 8. +• Integration with • Integration with CONTENTdm CONTENTdm• Fixity checks and • Finding and retrieving items virus scans • Manifest/batch• Responsive upload requirement support • Vendor-side error• Extensive reports reporting • Verifying storage contents
  • 9. DuraSpace’s • Began using in 2012 • Web interface for access • Web interface or client-side tools for upload • Content Management System-agnostic • Fixity checks
  • 10. +• Presentation is like a traditional gui file manager • Searching• Can designate spaces, • Sorting permissions• Can make a space public • Verifying storage• Powerful upload tools contents• Fixity scans • Overwriting isn’t• Robust reporting• Easy to get content out hard to do• Choice of storage services • Batch delete• VERY collaborative support• Non-profit • MD5
  • 11. PREPARATION CONTENTdm ? Local OCLC DA server
  • 12. CONTENTdm Local server1. Exported metadata from CONTENTdm2. Exported file names from local server3. Bashed preservation file names, checksums4. Identified and recovered missing files
  • 13. 1. Exported metadata Onerous to from CONTENTdm impossible2. Exported file names Easy from local server3. Bashed preservation Easy but time file names, checksums consuming4. Identified and Easy-ish recovered missing files
  • 14. 1. Exported metadata Onerous to from CONTENTdm impossible• OCLC had to provide export for largest & most critical collection• 363 MB tar file -> 18 x 100+ MB csv files• Added frustration: metadata for compound objects v. multi-page pdfs
  • 15. 2. Exported file names Easy from local server1. Bashed preservation Easy but time file names, checksums consuming• Spreadsheet gymnastics• Manual review for filename/checksum inconsistencies
  • 16. 4. Identified and Easy-ish recovered missing files• Missing from CONTENTdm? Added by librarians• Missing from local server? Request to OCLC or re-download from CONTENTdm
  • 17. THE MOVE Local server DuraCloud1. Tested sync and upload tools2. Discussed spaces3. Ran sync tool on local preservation storage4. Ongoing maintenance: upload tool
  • 18. 1. Tested sync and upload tools Easy• Helped determine flags to manage computer resources during sync• Verified logging output, permissions• Helped flesh out local workflow
  • 19. 2. Discussed spaces Easy, and Interesting• Many spaces or few, to accommodate different workflows?• Assignment of permissions
  • 20. 3. Ran sync tool on local Easy preservation storage• Ran continuously for 5 2/3 days• 94,177 items
  • 21. 4. Ongoing maintenance: upload tool Easy-ish• Uploads done weekly and monthly• Upload tool used to avoid accidental overwriting• Have to create “mock” file structure
  • 22. Working Staging – Limited Accessdirectory Local server StagingWorkingdirectory DuraCloudWorkingdirectory
  • 23. Insights • Room for preservation metadata improvement • Working with full metadata dumps is problematic • Need for more automated monitoring for local storage • Integration with CMS not helpful unless FULL integration in other words: • Streamlined ingest = streamlined preservation
  • 24. Still more • No, really: manual management and auditing is getting less feasiblethoughts • What is acceptable content loss? • What is acceptable preservation metadata error rate? • Responsiveness to enhancement requests should be figured into vendor choice • At 5 years out, PREMIS lite is just fine