Integrating ArchivesSpace and Archivematica at the Bentley Historical Library
•Download as PPTX, PDF•
0 likes•37 views
Max Eckard, Lead Archivist for Digital Initiatives at the Bentley Historical Library, will cover the Bentley's integration of ArchivesSpace and Archivematica to streamline digital archiving workflows. He will highlight the decision-making process behind integrating both systems, things he wishes he’d known then that he knows now, goals for the future, and other tips and tricks. In his role at the Bentley Historical Library, Max oversees the digitization program, digital curation activities, web archives, and associated infrastructure.
![[^1]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Integrating ArchivesSpace and Archivematica at the Bentley Historical Library
Similar to Integrating ArchivesSpace and Archivematica at the Bentley Historical Library (20)
Recently uploaded
Recently uploaded (20)
Integrating ArchivesSpace and Archivematica at the Bentley Historical Library
- 1. Integrating ArchivesSpace and Archivematica at the Bentley Historical Library Max Eckard Lead Archivist for Digital Initiatives Integrations with ArchivesSpace | February 19, 2020
- 2. [^1]
- 4. Facilitate the creation/reuse of descriptive and administrative metadata across preservation and management systems.
- 8. Why Integrate ArchivesSpace and Archivematica? ● Shortcomings with previous workflow for digital processing ○ FileMaker Pro: Metadata was not in a widely used, open system ○ Microsoft Word → EAD: Workflow for EAD generation too localized and complicated ○ AutoPro: Not intended to be a long-term solution ○ Lack of system(s) of record: Lots (and lots… and lots…) of duplicate metadata entry ○ Scale: Scale was an issue! ● ArchivesSpace ○ System of record for descriptive and administrative metadata ○ Gradual migration of metadata from disparate systems into ArchivesSpace ● Archivematica ○ System of record for digital processing/Archival Information Package (AIP) creation ○ Gradual migration of digital backlog into Archivematica Both systems (DSpace, too!) play nicely with others!
- 9. ● Worked with Artefactual ● Developed a new “Appraisal and Arrangement” tab in Archivematica ● Integrated Archivematica-ArchivesSpace that permits archivists to: ○ After initial transfer… ■ Display resource records in a treeview ■ Create and edit descriptive metadata for new or existing archival objects ■ Drag and drop digital content onto archival description, creating digital objects associated with those archival objects ○ After AIP creation and deposit… ■ Write newly-minted DSpace handles back to ArchivesSpace Sponsoring New Features in Archivematica
- 12. How well has this systems integration aged? ● Pretty well! But... ● We use the particular workflow outlined here less and less ○ Works well for relatively small, heterogeneous transfers ○ Doesn’t work as well for: ■ Relatively large or homogenous transfers ■ Transfers destined for digital repositories besides DSpace ● That said, we use integration more and more (in more loosely- and tightly- coupled gradations)
- 13. Lessons Learned ● Systems integration is where it’s at! Side note: The integration of people is as challenging as the integration of systems. ● More important than any particular workflow is… ○ Having systems that are designed to play nicely with others ○ The technical upskilling we did as a team while implementing, migrating to, and integrating ArchivesSpace and Archivematica ■ bentley-historical-library/aip-repackaging: Scripts to support repackaging and depositing Archivematica AIPs to DSpace ■ bentley-historical-library/dappr: A client to communicate with a remote DSpace installation using its backend API ■ Lots of other ArchivesSpace repositories... We are now much more flexible in how we approach digital processing!
- 15. References ● [1]: Dallas Pillen, “Integrating Archive-It and ArchivesSpace at the Bentley Historical Library,” Integrations with ArchivesSpace, December 4, 2019 Resources ● Archival Integration (blog) ● ArchivesSpace-Archivematica-DSpace Workflow Integration ○ Part 1: Configuring ArchivesSpace and DSpace Integration within Archivematica ○ Part 2: Appraisal, Arrangement to ArchivesSpace and Deposit to DSpace ● BHL Archival Curation: Digital Processing
Editor's Notes
- Thought I’d start with an overview of our institutional context/technical ecosystem. I actually borrowed this slide from my colleague, Dallas Pillen, who recently gave a webinar in this very Integrations with ArchivesSpace series on ArchivesSpace - Archive-It integration. So, we’ve got a lot going on here. We’ve got… (And I’m not even mentioning all of the other more localized database and spreadsheet systems we use here as well.) It’s REALLY IMPORTANT to note that these systems are used by a wide variety of stakeholders within the Bentley (including the “back of the house” Curation team and the “front of the house” Reference and Academic Programs team) as well as beyond (including novice researchers like U-M undergrads and more advanced researchers from both inside and outside the U-M community, as well as the general public). They are likewise hosted and supported by a wide variety of stakeholders. For some, we rely on U-M Library LIT, for others, we rely on U-M ITS, and for still others, we’re experimenting with it being ourselves. Managing all of these systems--especially the handoffs of data and metadata between them--can get overwhelming. So we actively look for ways to integrate them with one another, creating a kind of functional coupling between them so that they act as a coordinated whole to fulfill a number of archival workflows. And actually the integration I’ll be talking about today was part of a larger project that’s kind of the one that kicked this whole thing off for us.
- So, yeah, the Archivematica-ArchivesSpace integration that I’ll talk about today was actually part of larger, Mellon Foundation-funded ArchivesSpace-Archivematica-DSpace Workflow Integration project (2014-2016) that united three Open-Source Software platforms.
- The point of the ArchivesSpace-Archivematica portion of the integration was to... But, for additional context, as part of the overall workflow, we also wanted to: Streamline the ingest and deposit of content in a preservation repository. Find solutions that met the Bentley’s local needs, but which were also flexible and scalable for other institutions; modular, so that institutions may adopt some, none, or all of the development features; and based upon open standards so that other tools and/or repository platforms could be integrated. Share all code and documentation with the archives and digital preservation communities. And, just to give you a sense of why we were interested in this, let me show you where we were coming from.
- We had been doing digital processing with a bunch of localized, disparate, silos of data and metadata that didn’t really work together at all. So for example, here’s how we tracked accessions in a FileMaker Pro database (affectionately called BEAL).
- We also used to do arrangement and description work of both physical and digital archives in Microsoft Word documents, generating Encoded Archival Description (EAD) using macros applied to various Microsoft Word styles.
- And, we did digital processing with the AutomatedProcessor (AutoPro), a homegrown digital preservation tool written in Windows shell scripts.
- While the use of FileMaker Pro, Microsoft Word, and AutoPro for digital processing lowered technical barriers and introduced efficiencies into the our digital processing initiatives, there were numerous shortcomings. The use of a custom FileMaker Pro database, for example, limited our ability to take advantage of the affordances of more widely-used systems (e.g., Archon and Archivists’ Toolkit and later ArchivesSpace), such as the ability to integrate with other tools. Using Microsoft Word to generate EADs was certainly easier than hand-encoding XML, but training processors in Microsoft Word styles and macros made the process very localised and more complicated than, say, ArchivesSpace, for entering descriptive information. AutoPro had limited error handling, a poor user interface and various support issues, and was never really intended to be a long-term solution In general, there was also a lack of well-defined system(s) of record: This meant lots of duplicate/redundant metadata entry in various platforms, and also meant we had a really hard time managing this metadata over time. None of these tools, I’ll add, really helped us work at scale. At all. Meanwhile, ArchivesSpace and Archivematica had emerged as two of the most exciting open source platforms for working with digital archives. We were adopting ArchivesSpace to…and Archivematica to…and, best of all... [animation]... that is, both use common metadata standards, have APIs, are open-source (although that’s not necessarily a prerequisite for systems integration, but it helps), etc.
- And so, for this grant, we sponsored some development in Archivematica, essentially paying Artefactual to develop a new “Appraisal and Arrangement” tab in Archivematica. In addition to being the spot where this ArchivesSpace-Archivematica integration would happen, this introduced functionality to appraise and review digital content from within Archivematica. Not going to talk about this much but feel free to ask any questions... As it pertains to this webinar, however, it integrated Archivematica and ArchivesSpace via the introduction of an ArchivesSpace ‘pane’ within the Appraisal and Arrangement tab. This feature utilizes the ArchivesSpace API to: Display resource records in a tree view depicting the intellectual hierarchy of archival objects in ArchivesSpace Create and edit descriptive metadata for new or existing archival objects; authored in Archivematica and written ArchivesSpace It also permits archivists to drag and drop digital content onto archival description to create ArchivesSpace digital object records. Afterlaunching the ingest of Submission Information Packages (SIPs) in Archivematica, a related integration of Archivematica and DSpace automatically uploads a fully ingested AIP and associated descriptive metadata as a unique item in DSpace, the persistent URL of that item (its “handle”) will in turn be written back to ArchivesSpace so that it may serve as a link to either the digital content in ASpace or when archival description is exported to an EAD finding aid.
- Screencast
- All in all, this systems integration, when mapped to a digital preservation workflow like the Digital Curation Centre Curation Lifecycle model, looks something like this. As you can maybe see, Archivematica’s the one doing the driving, data is flowing both bidirectionally and unidirectionally between systems at various stages of the workflow, and Archivematica is using various integration methodologies like the ArchivesSpace API to integrate with ArchivesSpace (as well as SWORD v2 and the DSpace API to integrate with DSpace, even though I didn’t really touch on those).
- So, this was four years ago. You might be asking, how well has this systems integration aged? Pretty well, actually! We still use it, to this day, just this morning, in fact! It has survived upgrades to both ArchivesSpace (started with 2.2 and are now on 2.5x), and Archivematica (1.6 to 1.9), as well as the latter’s migration to a new server. I will stay that we use the particular workflow I outlined here less and less. It works really well for relatively small, heterogenous transfers, but that doesn’t exactly match the kind of accessions or transfers we usually get, which, more often, are small and homogenous or, regardless of whether their homogeneous or heterogeneous, are definitely trending on the bigger and bigger side (whether you measure that by number of files or size). As I showed earlier, we also have some more specialized platforms for digitized images and streaming audiovisual material, and this particular workflow obviously connects to DSpace but not to them. That said, we do use this strategy of integration, and in particular integrations with systems, like ArchivesSpace and Archivematica and various other repository systems, that are designed to play nicely with others, MORE and MORE.
- Which leads me to lessons learned. We’ve definitely drank the Kool-Aid and think systems integration is where it’s at. Side note: It turns out the integration of people is as challenging--if not more challenging--as the integration of systems (the proxies for those people), which is why I mentioned all of those stakeholders at the beginning. Looking back, I think we’d now say that more important than any particular workflow is… Just simply having systems that are designed to play nicely with others Really, the technical upskilling we did as as a team while implementing, migrating to, and integration ArchivesSpace and Archivematica. As some evidence of that, here are a couple of GitHub repositories with code we’ve developed on our own based off of these integrations and the workflow they support: AIP Repackaging scripts to… essentially these help us to replicate the integration part of what I just showed without locking us in to a particular workflow Similarly, DAPPr, a Python-based API wrapper for DSpace that, again, allows us to get data and metadata into and out of DSpace in a programmatic way without being forced to use a particular workflow All of this results in the fact that… [animation]... We now much prefer to sit down with a digital processing problem, think about what we want the end product to look like and all the different ways we might get there, and go from there.
- Actually I think we’re saving questions for the end.