Harvard University IT's Digital Repository Service redesign

1. A Generational Refresh:
Continuing Harvard's Digital Repository Service
legacy of innovation
Stefano Cossu, Harvard University IT stefano_cossu@harvard.edu
Open Repositories 2023, Stellenbosch University
Cape Town, South Africa

2. Harvard's Digital Repository Service
(DRS)
Established in 2000
No viable DP solutions at the time
Needed to address specific, complex needs of HUIT
2

3. DRS today
10 million objects
900 million replicated files
2Pb replicated data
63 departments
Actively supported
3

4. DRS challenges today
Technical debt
Inflexible content model
Inefficient UI/UX
4

5. DRS Futures
3-year capital-funded R&D and implementation project
Phase 1: discovery (Jul 2022 – Jun 2023)
Phase 2: planning (Jul – Dec 2023)
Phase 3: implementation (Jan 2024 – Jun 2025)
Open to different options (commercial, open source, home built)
Opportunity to re-envision digital preservation
5

6. DRS Futures team
Leaderless group
HUIT (central IT) and Library members
Made up of diverse skills, seniority levels, and backgrounds
Connected with many involved departments
Highly collaborative with specialized task forces
6

7. Dual approach to the problem
Inductive (bottom-up)
Department-specific interviews (focused on workflows)
Cross-department focus groups (focused on specific areas)
Office hours (free format)
Deductive (top-down)
Build up from previous experience
Set DP theoretical foundations & long-term vision
Anticipate challenges of a more capable system
7

8. DRS Futures tenets
Separation of storage and services
Separation of archive and workspace
Task automation
Re-envision digital preservation
Revolving feedback
Build for future scale
8

9. Separation of storage and services
Data store migrated to OCFL in 2022 (>1 year timeline)
Plan to keep storage fabric intact
Replace services on top of OCFL
9

10. Separation of archive and workspace
Provide users with a mid-term workspace services & store
Keep OCFL focused on preservation
Keep solution search focused on discrete functional areas
Multiple products can fulfill parts of the complete solution
10

11. Task automation
Remove repetitive tasks from staff duties
Set up event-driven architecture
Move preservation actions to the background
Added complexity paid off by volume
11

12. Re-envision digital preservation
Preserving the semantic context as well as the content
Archival resources are live materials, changing over time
Facilitate reuse and cumulative evolution of information
12

13. Revolving feedback
From Reference Model for an Open Archival Information System:
The Monitor Designated Community function interacts with Archive
Consumers and Producers to track changes in their service
requirements and available product technologies. [...] This function may
be accomplished via surveys, via a periodic formal review process, via
community workshops [...]. It provides reports, requirements alerts
and emerging standards to the Develop Preservation Strategies and
Standards function. It sends preservation requirements to Develop
Packaging Designs.
“
“
13

14. Revolving feedback
Build upon relationships acquired during discovery phase
Develop processes for continuous & iterative improvement
Support communication workflows along with production
workflows
14

15. Build for future scale
Exponential growth and problems related to it
Large untapped sources
A/V materials
Research data
Whole major schools
Unexpected usage patterns and needs
15

16. Where we are at
Gathered and summarized all feedback
Working on reconciling top-down and bottom-up approaches
Preparing RFP
Hiring developers and change manager
16

17. Key artifacts for Phase 1
User requirements catalog from stakeholder input
Technical foundational principles & requirements
Weight matrix of requirements (MoSCoW notation)
Persona profiles
Abstract reference content model
RFP
17

18. Conclusions
Allocating time and budget for long discovery phase paid off
Approaching the project with an unbiased, fact-driven mindset
Unexpected priorities and direction emerged
Having an open mind requires an open mind from our partners' end
18

19. Thank you
https://sites.harvard.edu/drs-futures/
19