DEVELOPING A STAFF-ONLY
SAMVERA APPLICATION
Struggles & Successes
Trey Pendragon
James Griffin
Princeton University Library
PUDL (Princeton University Digital Library)
PUDL
Summary
• Store files on disk in a special format
• Scripts to generate MODS/METS/JPEG-2000 Derivatives
• Collection pages for information about each collection.
• One collection per item.
PUDL
Successes
• 51 collections ingested
• Millions of pages ingested
• Support for complex internationalized materials in a
variety of formats.
PUDL
Struggles
• Time from “Imaged” to “Public” was often months.
• Training was difficult, and a lot of the work for ingest was
left to very few individuals.
• Only a few people knew how to do any work on it, and a
lot of specialty knowledge was necessary.
• Metadata was forked – now it was in the catalog and
PUDL.
Latin American Ephemera
Latin American Ephemera
Summary
• First foray into Samvera at Princeton University
• Fedora 3
• Staff ingest into one application, whose Solr records are
indexed by a front-end Blacklight application.
• Workflow for managing when and how things are ingested
and made public.
Latin American Ephemera
Successes
• Controlled Vocabularies
• Ingest works!
• Separated front-end worked!
Latin American Ephemera
Struggles
• Ingest eventually broke – the synchronization method we
had didn’t scale to the number of items.
• Code needed help – first time we’d written a Samvera
application, or even used Ruby!
Plum (Hyrax)
Plum
Summary
• Originally developed using CurationConcerns
• Compliance with the PCDM using Fedora Commons 4 for persistence
• Migrated to Hyrax
• Curatorial Workflow
• Items are ingested into Plum
• Spotlight instances are synchronized for digital exhibits
• Blacklight instances are synchronized for collection discovery
• This includes GeoBlacklight (https://maps.princeton.edu)
Plum
Summary
• Material Types Supported
• Latin American Ephemera
• Books (Multi-Volume Works)
• Visual Resources (i. e. Images)
• Geospatial Material (Scanned maps and vector datasets)
Plum
Successes
✓Ingest and management of items could be handled by anyone!
• Imaging to publishing works online required only days!
✓Metadata can be imported (No forking of data!)
✓Automation of derivative generation (No additional scripting!)
✓IIIF Manifests can be generated for all works
✓Support for geospatial material
✓Spotlight as a configurable, separate platform
Plum
Struggles
• Migration
• Fedora 4 required a restart every six hours
• Bulk ingest was generally fragile and very slow
• Saving in the UI for lengthy book works was also slow
• Users became increasingly frustrated
• Features and Scope
• Many Hyrax features were introduced by the community
• Not all were a priority for Plum
Figgy (Plum Burned and Rebuilt)
Figgy
How Plum was Burned
• Uses Valkyrie
• Metadata are now persisted in PostgreSQL
• Files are persisted on disk
• Features kept in parity with Plum
• IIIF synchronization was preserved
• Not written on Hyrax…
• ...but uses Samvera Gems
• (hydra-editor, Hydra::AccessControls, etc.)
Figgy pudding with flaming brandy
Figgy
Successes
• Digital Exhibits and Discovery
• Synchronization with Blacklight/Spotlight using message brokerage
• Code bases for digital exhibits and discovery were unchanged!
• Performance and Persistence
• Migration over 60x faster (nearly every interaction below 200ms)
• Application code is all our code - we can control feature priorities
• Increased stability (no more restarting services)
• This also simplifies ingestion processes
Figgy
Struggles
• Figgy
• Application code is all our code - we have to support this now!
• How best to continue using Samvera Gems without Hyrax
• Valkyrie
• Valkyrie storage adapters also require maintenance
• PostgreSQL will remain our primary store for metadata
• Do we now approach Rubyists using PostgreSQL?
Acknowledgements
• Jon Stroop
• Esmé Cowles
• Anna Headley
• Eliot Jordan
• Nikitas Tampakis
• Kevin Reiss
• Shaun Ellis
• Francis Kayiwa
• John Kazmierski
• Axa Liauw
• Philippe Menos
• Joyce Bell
• Roel Munoz
• James Griffin
• Trey Pendragon
• ...more
• Samvera Community!
Thank You!
tpendragon@princeton.edu - Trey Pendragon
jrg5@princeton.edu - James Griffin
Princeton University Library

Developing a Staff-Only Samvera Application

  • 1.
    DEVELOPING A STAFF-ONLY SAMVERAAPPLICATION Struggles & Successes Trey Pendragon James Griffin Princeton University Library
  • 2.
  • 3.
    PUDL Summary • Store fileson disk in a special format • Scripts to generate MODS/METS/JPEG-2000 Derivatives • Collection pages for information about each collection. • One collection per item.
  • 4.
    PUDL Successes • 51 collectionsingested • Millions of pages ingested • Support for complex internationalized materials in a variety of formats.
  • 5.
    PUDL Struggles • Time from“Imaged” to “Public” was often months. • Training was difficult, and a lot of the work for ingest was left to very few individuals. • Only a few people knew how to do any work on it, and a lot of specialty knowledge was necessary. • Metadata was forked – now it was in the catalog and PUDL.
  • 6.
  • 7.
    Latin American Ephemera Summary •First foray into Samvera at Princeton University • Fedora 3 • Staff ingest into one application, whose Solr records are indexed by a front-end Blacklight application. • Workflow for managing when and how things are ingested and made public.
  • 8.
    Latin American Ephemera Successes •Controlled Vocabularies • Ingest works! • Separated front-end worked!
  • 9.
    Latin American Ephemera Struggles •Ingest eventually broke – the synchronization method we had didn’t scale to the number of items. • Code needed help – first time we’d written a Samvera application, or even used Ruby!
  • 10.
  • 11.
    Plum Summary • Originally developedusing CurationConcerns • Compliance with the PCDM using Fedora Commons 4 for persistence • Migrated to Hyrax • Curatorial Workflow • Items are ingested into Plum • Spotlight instances are synchronized for digital exhibits • Blacklight instances are synchronized for collection discovery • This includes GeoBlacklight (https://maps.princeton.edu)
  • 12.
    Plum Summary • Material TypesSupported • Latin American Ephemera • Books (Multi-Volume Works) • Visual Resources (i. e. Images) • Geospatial Material (Scanned maps and vector datasets)
  • 13.
    Plum Successes ✓Ingest and managementof items could be handled by anyone! • Imaging to publishing works online required only days! ✓Metadata can be imported (No forking of data!) ✓Automation of derivative generation (No additional scripting!) ✓IIIF Manifests can be generated for all works ✓Support for geospatial material ✓Spotlight as a configurable, separate platform
  • 14.
    Plum Struggles • Migration • Fedora4 required a restart every six hours • Bulk ingest was generally fragile and very slow • Saving in the UI for lengthy book works was also slow • Users became increasingly frustrated • Features and Scope • Many Hyrax features were introduced by the community • Not all were a priority for Plum
  • 15.
    Figgy (Plum Burnedand Rebuilt)
  • 16.
    Figgy How Plum wasBurned • Uses Valkyrie • Metadata are now persisted in PostgreSQL • Files are persisted on disk • Features kept in parity with Plum • IIIF synchronization was preserved • Not written on Hyrax… • ...but uses Samvera Gems • (hydra-editor, Hydra::AccessControls, etc.) Figgy pudding with flaming brandy
  • 17.
    Figgy Successes • Digital Exhibitsand Discovery • Synchronization with Blacklight/Spotlight using message brokerage • Code bases for digital exhibits and discovery were unchanged! • Performance and Persistence • Migration over 60x faster (nearly every interaction below 200ms) • Application code is all our code - we can control feature priorities • Increased stability (no more restarting services) • This also simplifies ingestion processes
  • 18.
    Figgy Struggles • Figgy • Applicationcode is all our code - we have to support this now! • How best to continue using Samvera Gems without Hyrax • Valkyrie • Valkyrie storage adapters also require maintenance • PostgreSQL will remain our primary store for metadata • Do we now approach Rubyists using PostgreSQL?
  • 19.
    Acknowledgements • Jon Stroop •Esmé Cowles • Anna Headley • Eliot Jordan • Nikitas Tampakis • Kevin Reiss • Shaun Ellis • Francis Kayiwa • John Kazmierski • Axa Liauw • Philippe Menos • Joyce Bell • Roel Munoz • James Griffin • Trey Pendragon • ...more • Samvera Community!
  • 20.
    Thank You! tpendragon@princeton.edu -Trey Pendragon jrg5@princeton.edu - James Griffin Princeton University Library