3. National Digital Stewardship Residency
“...to build a dedicated community of
professionals who will advance our nation’s
capabilities in managing, preserving, and
making accessible the digital record of
human achievement.”
Washington DC
10 Residents; 2013-2014
New York City
5 Residents; 2014-2015
Boston
5 Residents; 2014-2015
25. Results: Outline of Challenges
Theme Participant Responses
Management Structure ● “I think only the division chair knows how much server space we
have in total.”
Personnel Practices ● “My research assistants manage all my data.”
Interdepartmental Relationships ● “I don’t know where exactly to get support for my database. I don’t
know if its IT’s jurisdiction or job.”
Workflow Structure ● “I have no time to standardize my data management.”
Technical Infrastructure ● “There are not enough computer terminals in the imaging lab.”
39. ➔ Ensure that physical collections are
stable and preserved
➔ Increase access through digital surrogate
➔ Create a sustainable central repository of
digital assets
Susan and Elihu Rose Foundation
DIGITAL
ARCHIVES
PROJECT
(a.k.a. DAP)
2012-2015
Hello everyone! Thank you so much for having us. My name is Vicky Steeves and this is Shira Peltzman, and we are happy to be in sunny San Diego as opposed to snowy NYC.
Seriously, SO HAPPY.
We’re here to talk to you about the National Digital Stewardship Residency program, and about the work we’re each doing at our respective host institutions. We’ll begin with a quick overview of the NDSR program at large, and then I will discuss my project at the American Museum of Natural History, which focuses on surveying scientific research and collections data. I’ll then hand over the reigns to Shira, who will talk about her work preserving born-digital assets and implementing sustainable preservation policies at Carnegie Hall.
The NDSR program is designed to provide graduates who have recently finished a relevant master’s degree with practical, hands-on experience in digital stewardship. Each resident is embedded at a different host institution, which is typically an academic, federal, non-profit, or cultural heritage organization to work on a digital stewardship project. Funded by the Institute of Museum and Library and Services, NDSR began its pilot year in 2013 with support from the Library of Congress through the National Digital Stewardship Alliance. In 2014 the program won funding to expand to Boston and New York, each of which will host 5 residents per year for a total of two years each.
In New York, the NDSR program is supported by the Metropolitan NY Library Council and the Brooklyn Historical Society, with Margo Padilla, the program director, and project manager Annie Tummino both situated at METRO. Here are the five NY residents pictured at the beginning of our orientation period during immersion week, meeting for the first time. The hallmark of the NDSR program is that it is built on the cohort model, which gives us a network of support, personally and...
professionally. We’ve used the opportunity for a regular space given to us by METRO, to meet twice a month to discuss our projects--anything going wrong, anything we’re doing that we think could be of use to someone else, or just updates.
These gatherings have really brought us together as a team and allow us to explore the diversity between ourselves and our projects--XXXXwe see different solutions we could bring home to our host institutions in each meeting.
So here we are during the first week of our residency. Because we all began the program with different levels of expertise or experience in digital preservation, the first couple weeks of the NDSR program functioned as an intensive training session, or "boot camp" that was designed to put us on more or less an even playing field.
The NDSR-NY cohort regularly present and participate in conferences as well as workshops. This is designed to buttress the hands-on experience we gain through our day-to-day.
METRO also allows us to attend their workshops free of charge, and both METRO and BHS offer us the space and resources to run our own. On the left, you can see the fine folks of AVPreserve giving us a rundown on the new ISO standard for auditing TDRs, and on the right, you can see a presentation on metadata standards by Peggy Griesinger the resident at the MoMA.
Another aspect of the Residency has been maintaining and contributing to our NDSR blog. The experience of blogging and expanding our social media presence has not only increased awareness about our projects, but also has put us in contact with a network of professionals that extends beyond the city New York City itself.
Applications are currently open for host institutions as well as residents for the 2015-2016 iteration. We would strongly encourage you to apply to be a host institution if you have a project in mind. Aside from having the opportunity to mentor an emerging professional, NDSR provides an excellent opportunity to get a project off the ground or to receive additional support and funding for ongoing projects.
If you’re not located on the East coast but are interested in finding out how to get an NDSR program started in your region, please be in touch with us and we can give you more information about how New York and Boston got off the ground. I will now pass it on to Vicky who will discuss her project dealing with digital scientific research and collections data at the American Museum of Natural History.
Hello again! I’m very delighted to share my experiences working on an NDSR project at the American Museum of Natural History with you.
This will be the basic structure of this presentation--I will give you some context and describe the AMNH and the role Science plays here, then delve into my project with considerations made for methodology, preliminary results, and my first forays into recommendations for the Museum.
Science is the heart of the AMNH--it makes everything in the Museum tick, from exhibitions to education initiatives. I’m here to try and preserve the legacy of that research, which has taken an increasingly digital and complex shape since the 90s.
The Museum has five research science divisions, some with nested departments. These divisions are: Anthropology, Paleontology, Invertebrate Zoology, Vertebrate Zoology, and Earth & Planetary Science, all supported by the AMNH Research Library, where I am located. The AMNH is in the unique position of being both a cultural heritage organization and a research institution, so the Library is in an equally unique position of supporting traditional archives as well as research data management.
Currently at the AMNH, we have over 200 scientists in our employ with over 33 million specimens in our collections. Luckily for me, I only had to speak to 41 curators and their immediate scientific staff. Many of the other scientists are members of their “labs” or “teams,” so speaking the curators was more than sufficient.
However, for all the unique and rare scientific data the Museum produces, there is no institution-wide plan to store, manage, or preserve this data. Each curator is left to their own devices, literally, to store their research. This is a function of the relationship each scientist has with the Museum: they are considered “free agents” and received little funding for research from the Museum. The Museum pays their salaries but leaves it up to the curators to find their own funding for research initiatives, including data and storage management.
Becuase it is such a timesuck, most curators at the Museum don’t manage their data with regularity. There are backups made, but no real plan for the data after publication. I am here to provide the recommendations that would enable the AMNH to remedy that situation.
My project at the AMNH has three phases:
SURVEY: To develop and implement a survey of existing digital assets at the AMNH
this includes interviewing those 41 curators about their digital data with regard to questions that would inform choices made for digital preservation. The three main question categories were “storage” i.e. do they have enough and what was their rate of growth, “management” i.e. do they have any management practices and what are they, and “preservation,” which are essentially questions about how long they think their data will be useful for others and what needs to be preserved alongside it to make it useful to others.
You can see here some excerpts from the survey I developed. I formed a lot of the interview questions from the great tool from Purdue called the “Digital Curation Profiles” and the toolkit they provide--they gave a really great framework for the interview, so if you need to do something like this at your institution, I’d really recommend it.
I would sit down with each curator or staff member and start by asking them to describe their research cycle, which often gave me answers about their data’s lifecycle. This helps me understand how much and what kind of data is generated at each stage so I can account for that in my recommendations.
for example: if a curator tells me it is only important for them to store processed gene sequences, which are aligned and tend to be annotated, and not the raw sequence data, there are significantly different storage and preservation concerns than if they wanted to preserve it all. Most data have an expiration date, and these questions help me identify that.
I am currently on the ANALYSIS phase: this means I take the results of the survey and translate them into functional requirements to be met at the Museum.
I have transcribed all the audio interviews with a free transcription software, and used analysis methodologies from sociology, as all of my data are qualitative.
To analyze these materials, I am using called NVivo, which allows me to import transcriptions of audio interviews or interview notes for those who didn’t want to be recorded, and code it. This means I read through each interview and looked for major themes or ideas that came across in each one. I then read through the interviews again and “coded” or “marked” them to denote where these themes were expressed. It was difficult to find something that wasn’t too abstract it couldn’t be empirically proved--after all, I work primarily with scientists. Through reading sociological sources I found it relatively simple to identify these things in NVivo, which have to be mutually exclusive and all-inclusive to be worthwhile.
This is an example of what my semi-finalized data will look like--obviously with more participant responses as I continue the work. The analysis phase began in late February and is still in process, as the dataset is quite large. You can see on the left side of this table are the specific overlapping themes I coded for, and on the right the corresponding participant quote.
I have two outstanding curators to meet with who are out on fieldwork, but should not impact my results too too much.
Though the majority of my data is qualitative, I have been able to discover how much digital data is currently stored by AMNH scientists, and the projected annual growth of digital data. The numbers for fiscal year 2014 are based on my interviews with curators and their staff, as well as the projected numbers for FY 2015. Using both these data points, I was able to construct 5 year projections for the growth of digital data at the AMNH. You can see its exponential.
Here are what I believe are the data at greatest risk--CT data (click), genomics data (click), and digital images (click), both moving and still. These are not only some of the largest files at the AMNH, but the ones most subject to data loss. To preserve CT data, it is important to store the raw CT data--from this, other scientists can reconstruct whatever part of the taxa they’d like. The same raw skull data can be used by many scientists: those focusing on inner ear development, or brain casing, or facial morphology. Similarly to CT data, it is important to keep the raw genomic sequence data as well, because different scientists use different algorithms to align them. For digital images, well, the digital preservation world has a suite of recommendations for those, but the base line is: keep the file formats open source.
For these data, the usual means--file format or software obsolescence--sometimes take a backseat to data simply walking out of the door. Because the Museum has its own gene sequencing and imaging facilities, visiting scientists, students, and research collaborators will often come use AMNH specimens on AMNH machines, and take the resulting digital files with them because the Museum lacks the storage capacity and infrastructure to take a copy. This puts our specimens at risk, which is of extreme importance because many are unique.
The last phase of my project is to ADVISE: I will use that baseline metric discovered in my analysis phase to compare long-term preservation options and make recommendations to the Museum.
This is my final deliverable: a report that contextualizes the AMNH with the rest of the research and cultural heritage institutions initiatives and provides recommendations as to what the Museum should do with all their digital data. Centralized storage, management, and curation of said data by the Library or data managers in each division would allow the Museum to strategically place itself in a position to begin systematically preserving its digital scientific data, and the legacy of the groundbreaking scientific work done there.
Though my findings are still preliminary, I believe that the Museum should build a trustworthy digital repository based on open-source software to house its scientific data throughout its whole lifecycle, based out of the AMNH Research Library. Without a central repository that provides storage and management of scientific research data, the risk of data loss only increases as the amount of digital data increases. A repository would answer the needs of the curators, which are primarily centralized and additional storage, while putting the Library in a position to begin preserving the data so intrinsically important to the Museum, as is its job.
Thank you so much. You can find me on Twitter at Vicky Steeves and you can email me at my AMNH email listed there. I am going to pass the mic to Shira, who will be discussing her project working with born digital materials at Carnegie Hall.
Thanks Vicky.
My project at Carnegie Hall is focused on creating and implementing policies, procedures, and best practices for the preservation of our born-digital assets.
I’m going to begin this talk by describing the kinds of digital content that I work with at Carnegie Hall, followed by a quick overview of our Digital Archives Project, of which my work is a key part. I’m then going to describe all the components of my NDSR project, followed by an overview of where I am now and what remains to be done before the project wraps up in late May of this year.
Carnegie Hall produces born-digital content on a daily basis. In addition to presenting close to 200 performances each year on our three stages, we’re also home to over 500 independently produced events annually. As you might imagine, a significant portion of our born-digital material relates directly to these shows, including recordings of performances that take place at the Hall, interviews with musicians, blog posts about upcoming concerts, and live webcasts and radio broadcasts of the performances themselves.
But, perhaps surprisingly, the majority of our born-digital material actually has nothing to do with the shows we present.
That honor goes to the Weill Music Institute, or WMI, which produces an extraordinary range of music education and community programs each year that extend far beyond the physical walls of our concert halls. In support of Carnegie Hall’s mission to “bring the transformative power of music to the widest possible audience,” WMI’s programs are designed to allow people of all ages and backgrounds in the New York City metropolitan region and around the world to engage with music, both as performers and audience members.
I want to briefly describe just a few of these programs to give you some idea of what they entail, and also so that you have a basic understanding of the digital content that comes out of them:
One of WMI’s hallmark initiatives is a series of free neighborhood and community concerts that take place throughout all five boroughs of New York City, which are filmed as they take place and made available online.
WMI also runs series of Professional Training Workshops, which are designed to give young artists the chance to learn from world-renowned conductors, instrumentalists, and vocalists. These master classes are recorded and made available for research to both scholars and musicians alike.
Another ongoing WMI project is the Musical Connections series, which provides free interactive performances, creative projects, and artist residencies in homeless shelters, correctional facilities, and healthcare settings for those who have limited access to live music,
and finally I also wanted to mentioned a series called Musical Exchange, which enables teens to connect with their peers from around the world and share their musical performances. All of these projects are filmed and made available to the public via WMI’s Online Resource Center, which contains an extensive collection of multimedia and interactive materials for educators, students, and teaching artists.
I could go on, but I think you guys get the idea.
For years this material was recorded principally on videotape, but over the past decade as file-based workflows became commonplace, the number of digital production materials at Carnegie Hall rose substantially. For a while this was OK, but as both the size and variety of these assets began to increase, it became clear that Carnegie Hall would need to develop a strategy to better manage this material. This realization ultimately lead to the creation of a 3 year-long undertaking at Carnegie Hall called the Digital Archives Project.
The Digital Archives Project was jointly funded by the Carnegie Corporation, the Andrew W. Mellon Foundation, the Susan and Elihu Rose Foundation, and the Grammy Foundation. The DAP has three main objectives: to conserve and preserve our physical collections, to increase access to archival material online, and to create a central, sustainable repository of our digital assets, and the last of these objectives is where my NDSR project enters the equation.
My NDSR project was designed to focus on three distinct areas,
the first of which is configuring and implementing Cortex, our new Digital Asset Management System. This requires working closely with our consultants, AVPreserve, and with Orange Logic, Cortex’s developers, to ensure that the DAMS is tailored to fit Carnegie Hall’s unique set of needs. This process also involves developing a set of ingest procedures for all of the born-digital material produced within the organization, alongside documentation describing these.
The second area of focus is on designing workflows for the creation and management of born-digital assets. This entails the creation of file naming conventions and other pre-ingest workflows, tracking down all extant born-digital assets from both vendors and internal staff, and developing a set of requirements for the long-term preservation and sustainability of digital files.
The third area of focus is on developing policies and procedures for digital preservation, which includes tasks like improving our documentation practices, updating the Archive’s mission statement, writing a disaster preparedness plan for the Archives, and developing selection and acquisition criteria for the DAMS.
As you may have gathered, the scope of my project is fairly large, and so when I began my Residency it quickly became clear to me that the very first step would need to be reaching out to all departments within Carnegie Hall to find out what kinds of digital content they work with and how this material is created, used, and stored. So actually in this regard, my project has several similarities with Vicky’s.
Conducting these interviews was vital because aside from helping me gain a better understanding of how each department fits into the ‘bigger picture’ at Carnegie Hall, they’ve also helped me understand exactly how my colleagues do their jobs and what their day-to-day interactions with digital media are like.
I structured these interviews to take place in two halves, the first of which was geared toward developing an overall understanding of what my colleagues’ actually do, what kinds of media they work with, and what problems they typically run into.
After I had a chance to type up some notes from our initial discussion, I scheduled a follow-up interview with each department. This half of the interview was much more “hands-on”.
Before our second meeting I asked every department to gather any hard drives, thumb drives, optical media etc. that they thought might contain digital assets, and during our meeting I had them actually go through review their contents with me. I also took this opportunity to have them walk me through their departmental file directories, so that I could assess their storage needs and determine how they organize their files (or in some cases how they don’t organize their files).
This was helpful for not only for me but for my colleagues as well, who in almost every case ended up discovering copies of files that they had long since assumed to be lost, which helped to underscore this project’s value.
This last point actually speaks to what I’ve found to be perhaps the single greatest challenge of my project, which is that up to now, there hasn’t been a streamlined process at Carnegie Hall that has allowed staff to push digital assets and their associated metadata upstream.
The overarching goal of the Digital Archives Project is to help fix this problem going forward, so that staff are able to gather metadata at every stage of a digital file’s lifecycle rather than relying on Archives or IT to collect metadata long after the media was originally created.
The extended conversations that I had early on with creators and users of content at Carnegie Hall put my in a somewhat unique position to make a case to each department about what they stand to gain from a paradigm in which our digital assets are actively managed from the point of production onward.
After conducting these interviews, the next step was to type up and review all of my notes so that I could eventually synthesize their content into an early draft of my final deliverable, which is a Digital Preservation and Sustainability document that will serve as the foundation for Carnegie Hall’s digital preservation policy. This document will ultimately contain selection and acquisition criteria, a disaster preparedness plan, an updated Archives mission statement that includes a remit specifically for digital preservation, file naming conventions, and ingest procedures for all of our born-digital material, and collections access policies, in addition to a series of recommendations that I would like to see Carnegie Hall implement in the future.
Right now I’m midway through writing this document, and I’m on track to have a draft of this complete by early May, which will leave me with roughly a month to revise it before delivering the final draft at the end of that month.
I know we have a limited amount of time remaining here today and Vicky and I were hoping to be able to answer a few questions if there are any, so I’ll wrap up by saying that I’m happy to talk more about my project with any of you later today or by email after the fact.
Thanks!
I know we have a limited amount of time remaining here today and Vicky and I were hoping to be able to answer a few questions if there are any, so I’ll wrap up by saying that I’m happy to talk more about my project with any of you later today or by email after the fact.
Thanks!
