Recommended
More Related Content
What's hot
What's hot (20)
Similar to Crowdsourcing Metadata Practices at USU
Similar to Crowdsourcing Metadata Practices at USU (20)
More from Andrea Payant
More from Andrea Payant (15)
Recently uploaded
Recently uploaded (20)
Crowdsourcing Metadata Practices at USU
- 1. Crowdsourcing Metadata Practices at USU __________________________________ Utah Library Association May 16, 2019
- 2. Name Here Programmer You can simply impress your audience and add a unique zing. Name Here Programmer You can simply impress your audience and add a unique zing. Becky Skeen Andrea Payant Randy Williams Anna-Maria Arnljots Cataloging Assistant anna-maria.arnljots@usu.edu Metadata Librarian andrea.payant@usu.edu SCA Cataloging Librarian becky.skeen@usu.edu SCA Librarian/Folklore Faculty randy.williams@usu.edu
- 3. Outline 1 Crowdsourcing: Engaging the Public in Metadata Creation 3 Beyond Crowdsourcing: Partnering for Discoverability, Equity & Inclusion 2 Adding Value to Collections: Highlighting Examples From Folklore
- 4. Wisdom of the Crowd Engaging the Public in Metadata Creation
- 5. Crowdsourcing Methods Gathering information from students/donors and using online webforms Outsourcing Working one-on-one with community members to gather information Interviews Organizing events that engage the community and encourage participation Community Events
- 6. Outsourcing
- 7. Background • Coordinating efforts for new collections o Donors Metadata spreadsheets Booklets o Student groups • Reviewing online feedback for all collections o Webforms / Comments
- 8. Coordinating Efforts - Donors Donors SCA Curator CMS Metadata Spreadsheet Donated Materials
- 9. Coordinating Efforts - Students SCA Curator CMS Students Metadata Spreadsheet Collection Materials
- 10. Digital Collection Users Webform / Comments Additional Metadata Webforms / Comments
- 11. Examples • Folklore collections o Upcoming FOLK COLL 66 (Donor: Hal Cannon) o Meeting with Hal and Folklore Curator Basic primer on metadata and how to use the metadata spreadsheet • Student groups o Jackson Hole Dude Ranching Tradition o The Climate Challenge: USU Voices • Online feedback o Compton Studio Photographs, 1884-1994
- 12. Challenges • Metadata education o Dublin Core Excel spreadsheet metadata templates • Webform o Accuracy of information
- 13. Successes • Improved collection information • Simplified processes o General education on importance of metadata o Excel spreadsheet metadata templates Who, what, where, when etc. instead of Dublin Core fields • Increased library interaction with campus and community
- 14. Interviews
- 15. Background • Community members o 1 liaison community member o 1 family member o 2 cataloging and metadata representatives One rep to interact directly with interviewee One rep to record audio/take notes o Packets Printouts of collection items (2 copies)
- 17. Challenges • Scheduling • Storytelling • Accuracy of information • Return on investment
- 18. Successes • Improved collection information • Investment given by the community led to higher collection usage • The interviewees really enjoyed themselves and reminiscing about the past • Extraordinary experience for catalogers • Flexible Administration
- 19. Community Events
- 20. Planning Guidelines • Determine objective • Research relevant community organizations/events • Choose community liaison(s) • Establish strategy to reach goal Where (location of event) What (Resources needed) Staff Equipment Forms Who & How (Publicity) o Flyers o Postcards o Word-of-mouth
- 21. Publicity
- 22. Community Event
- 23. Challenges • Logistics of large-scale event • Balancing staff with community members (supply = demand) • Spending appropriate amount of time with each person • Publicity - what forms would be most useful? • Getting staff and equipment to/from event • After event metadata gathering (know limitations of your event)
- 24. Successes • Positive relationships with community – future donations • Good turnout • Digital collections with robust metadata • Improved metadata collection workflows
- 25. Lessons Learned • Have a plan and follow it • Need to evaluate process and make necessary changes • Community members are eager to contribute information and share their stories • Collective knowledge is invaluable for gathering robust metadata • Consider audience when creating strategies for collecting metadata – Be Flexible • Important to form strong relationships with community leaders and organizations • Location of community events important, need to make community members feel ownership for the project • What may be seen as a lost opportunity may actually open up new possibilities = referrals
- 26. Highlighting Examples from Folklore Adding Value to Collections
- 27. Barre Toelken Fieldwork Image Collection Personal / Intimate Interviews • Renowned USU Folklorist • Metadata Specialist with background in Folklore
- 28. Metadata specialists can serve as conduits between researchers seeking data and informants with first-hand knowledge
- 29. Navajo wedding basket with traditional motif
- 30. Krampus close-up, Neu Gotzens, Austria, 2001
- 31. Stone mouse sculpted on a building in Hamelin, Germany
- 32. St. Anne’s Retreat Student Fieldwork • Informant Data • Metadata Specialist with background in Folklore
- 34. Community Scholars / Students / Extension Faculty Partnering for Diversity, Equity, & Inclusion
- 35. Community Partners In an effort to make USU Special Collections and Archives holdings more diverse, equitable, and inclusive, we collaborate on our ethnographic endeavors to include community partners in all stages of the planning, from concept through completion This includes input regarding description/metadata Latinx Voices Symposium, 2008
- 36. Community Partners • Andrew Dupree, Community Scholar • Brock Alder, Bear River Health Department Division of Substance Abuse • Judge Thomas Willmore, First District Court
- 37. • Andrew was a co-project lead, project originator, interviewer, • Helped with collection description/metadata • Judge Willmore approved the project • Andrew and Brock assisted with • interview questions • competency training
- 39. Community Events and Scholarly Endeavors promote DEI Effort Leading to Future Partnerships Journal of Western Archives (left) article co-authored by Williams and community partners Eduardo Ortiz and Maria Lisa Spicer-Escalante; Utah Historical Quarterly (right) article coauthored by Williams and Jennifer Duncan, SCA Librarian.
- 40. Student Partners (and Community Partners) • Utah State University Folklore Graduate Students • University of Wyoming American Studies Graduate Students • Turner Family • American Folklife Center at the Library of Congress
- 43. Student Exhibits & Description
- 44. USU Extension & Community Partnership
- 45. Engaged Learning
- 46. Informing the National Narrative: Stories of Utah’s Opioid Crisis
- 47. THANK YOU
Editor's Notes
- Introduce our group – mention adding the voices of our colleagues who made contributions to todays presentation but couldn’t be here to talk to you themselves
- First, we are going to talk about our general crowdsourcing practices that are focused on engaging the public in metadata creation – this section will focus on perspectives from Becky and I from our own experience and involvement. Then we are going to move on highlight examples of folklore collections that have been improved through unique crowdsourcing opportunities, which we will discuss from the perspective of our unit member and colleague Anna, who serves as a sort of hybrid between our unit and Special Collections and Archives - Anna works primarily with folklore collections and has a degree in folklore as well that improves her ability to describe these types of things. And Third, we will talk about taking the concept of crowdsourcing further to demonstrate our practices for improving equitable representation and inclusivity by facilitating community partnerships – and this perspective is given by our Folklore curator from Special Collections and Archives, Randy Williams.
- Alright, let's talk about how we have been able to engage people outside cataloging and metadata services in metadata creation Just a quick clarifier before I begin – as we talk about our crowdsourcing efforts – I just want to point out that all the metadata that folks outside our unit create and supply along with our collections is always filtered and checked for quality control within our cataloging and metadata unit and we make edits or necessary additions to make our metadata compliant with our application profiles, and to follow local, regional, and national best practices for metadata
- Ok – so we have grouped our efforts into a few categories. First, is straight up outsourcing, when core metadata is created by those collecting, donating, or using materials. Second, is conducting interviews – when catalogers sit one on one with community members and gather information. And third, is community events, when we make a concerted effort to create a venue where groups with the potential for collective knowledge can meet together and contribute information to improve the relevancy of metadata for our collections
- Ok so first let's talk about outsourcing
- In a general sense, we outsource metadata creation by either coordinating our efforts with other units that connect us with participants like donors or student groups or we utilize web tools to gather information
- We are fortunate to be able to have a seat at the table so to speak as collections are donated to the library's special collections. One Special Collections and Archives curator at USU in particular, Randy Williams over folklore has invited us into her conversations with donors and allowed us to facilitate the production of descriptive metadata by donors themselves as they prepare to gift us their materials. We are able to give them basic metadata training and explain the importance that is has to the discoverability of their items as well as the value of their added input to the descriptions. Our primary tool for gathering this information is a metadata template spreadsheet that works as a sort of fill in the blank system and, in some cases, especially for anyone who may be less technologically inclined, we provide a physical booklet that allows for hand written descriptions of items
- We have also been able to similarly work with student groups in coordination with Special Collections Curators and/or Class instructors to have them produce metadata for collections they create as part of their class that will eventually become one of our digital history collections. We have actually been given class time to discuss the importance of metadata before they begin their projects and to introduce them to the metadata template spreadsheet tool that they will use when recording their information
- Additionally, we gather crowdsourced metadata by using web tools for collections that have already been made available to the public but lack robust description. Our digital asset management system is currently CONTENTdm and the platform has a built-in comment feature which we periodically review to check for valuable information. We incorporate as much commentary as appropriate and leave a thank you comment in reply to let the user know that their content is appreciated (assuming they come back to the page to check) and also to let potential contributors know that we welcome their input too. But the most effective way we have been able to gather online feedback is by using a webform created by our systems unit. We can use it for any item in a collection. When a user views an item the first field they see in the metadata reads "What do you know about this item? Click this link to tell us more" the user clicks the link which opens to the form and when they submit the contents are emailed to us. The email contains ID information for the specific digital object so the metadata cataloger can just click a link to the exact item. Respondents are able to choose whether or not they would like to be credited for the information they provide, if they opt to do that we include their name in the contents of the description field. Once changes are made we email the respondent with a personalized message thanking them for their contribution. We include a link to the item in the email so the user can quickly see the results of their efforts. In some cases, emails have encouraged conversations with the respondents and they contribute more information that they have gathered from family and friends.
- So, most recently I have been able to work with Randy for an upcoming folklore collection for Hal Cannon. Hal is a prominent folklorist and the founder of the western folklife center and he is planning to donate some of his life's work to the library. As planning for this donation progressed, I was invited to meet with Hal a couple times to give him a primer on the principles of metadata and familiarize him with the template spreadsheet and to answer any question he had regarding metadata for his items. Hal was willing and eager to provide descriptions for his collection, as it made sense that he would be able to do it better than anyone else. He is currently going through the process of adding metadata. Some of the student groups that our unit has been able to work with were also involved with creating new folklore collections. The two examples here Jackson Hole Dude Ranching and the Climate Challenge involved students doing fieldwork and conducting oral history interviews. Each class was given different levels of metadata related instruction. For the climate change collection our unit was able to work with the students in class to help supervise metadata creation and we were much more involved in the planning process than with the Jackson Hole group and there was a marked difference in the quality of metadata that they provided for the items For the webform example I wanted to bring up the Compton Studio Photographs collection – the Compton studio operated in Brigham City, Utah for over a hundred years and the collections gives a pictorial history of the city and its residents and is of particular interest to genealogists. It is our most popular collection and we get more webform feedback for this collection than any other and the data provided has given us information for previously unidentified people and places
- The main challenges we faced with outsourcing methods we have employed have been the use of the metadata spreadsheet. When we first began using this as a tool we included Dublin Core elements and this made instruction and use overly complicated Another challenge, particularly with webforms has been the difficulty we have judging the accuracy of the information that people provide. So far, we generally assume that their best guess is better than our unknown so we go ahead and add what people contribute unless it is obviously inaccurate or just not relevant or beneficial to include
- Largely we consider our efforts for outsourcing a success due to the improved collection information and the increased library interaction with the campus and community. We have also been able to address our challenges by simplifying the metadata template spreadsheet from Dublin core to a more user friendly question and answer format which has decreased the learning curve for those using it and lessened the need for us to give unnecessarily complicated metadata instruction
- Next I will talk about some of our experience with interviewing to gather metadata
- The interviews we have conducted thus far have stemmed from a community event involving the Mendon, Utah digital history collection. The interviews followed this pattern: We would go to the community members homes to make the process easier for the participants. One Liaison community member agreed to serve as a point of contact because of their close ties to the city and its residents. They gave us suggestions for who to interview and were also willing to coordinate meeting times and attend them to further facilitate our interactions with the community. We requested that at least one family member be present if possible to help interviewees feel more comfortable. Then there were two representatives from our unit present – one person primarily interacted with the interviewee and took notes and the other would monitor the audio recording and assist as needed. The interviewee was given printout versions of digital items in the Mendon collection that we were hoping to get information for and the metadata rep had identical printouts in matching sequential order that they would take notes on – We would go through each item together to gather whatever information they knew
- Here is a picture of one of our interviews in progress – you can see the interviewee reviewing one of the printouts as I take notes and you see their family member assisting as well
- There were some challenges we faced with in-person interviews. First, finding a meeting time that worked for everyone was sometimes difficult. Storytelling, while enjoyable, would sometimes draw out the length of a meeting, which we wanted to avoid so as to not be burdensome to the participants. Sometimes the interviewees were not sure about the information they were giving so the accuracy of information was occasionally in question. Also, due to the nature of what we were doing we would sometimes go through the planning and interview but it would yield little benefit with regards to making more robust descriptions
- That being said, overall the interviews were successful in that they did give us more metadata for previously unidentified people and places. The practice also resulted in higher collection usage at least around the time that the interviews were conducted as families would check out what they had helped to develop. Participants were very appreciative and enjoyed the opportunity to stroll down memory lane. It was also a great experience for us as catalogers as we were able to do some out of the box work that catalogers don't usually get to do. We also benefited from a flexible and accepting administration that enables us to do this type of work.
- Now we will give you some highlights from collections that have been improved through unique crowdsourcing opportunities from the perspective of Anna-Maria – who couldn’t be here today…
- The Barre Toelken Fieldwork Image Collection created by renowned USU folklorist, serves as an example of where a personal interview with the creator is most beneficial in producing metadata. In this case, Toelken provided an intimate view and understanding of the multi-themed collection, which resulted in detailed and accurate data in which to create the best possible metadata. Under the direction of Fife Folklore Archives Curator Randy Williams, student intern Dusty Decker, and later Special Collections and Archives student worker Andrew Izatt, gathered information from Toelken about each of his fieldwork images. The students added the data to a metadata spreadsheet. This information went through item by item quality control by the folklore curator and metadata specialist. Following this initial collection, a second grouping of images were donated. Again, Toelken and this time his wife Miiko Toelken, worked with Library personnel, with Williams and FFA student worker Heidi Williams. Williams and Williams worked with Toelken over months to gather and enter metadata.
- The metadata specialist working with this collection was our colleague, Anna-Maria Arnljots, who has a background in folklore who also had direct contact and access to Toelken and his family--who provided a wealth of information needed to create robust metadata and a descriptive narrative for the collection. In this case, the metadata creator essentially becomes a conduit between the researcher seeking data and the informant with first-hand knowledge of the items in the collection. After initial efforts by the people that were mentioned before to gather metadata, Anna made numerous visits to the Toelken household, and even to the nursing home as Professor Toelken’s health was failing, in order to gather as much data as possible to bolster the metadata for the nearly 1,300 items featuring half a century of folklore fieldwork. Without these personal interviews by the metadata team with the primary informants who had intimate knowledge of the event surrounding the images, it would have been impossible to correctly identify the important details necessary for rich metadata. Not only the date and time, names, locations, and cultural significance, but the nuance that only Toelken, and those intimately connected, would know. Without the transmission of this knowledge from the creator to the metadata preparers for this digital collection, the collection would not be as detailed, potentially making it harder for a researcher to get the desired search results. Barre Toelken, wife Miiko Toelken, as well as daughter Kaz Toelken, at times, an intimate level of crowdsourcing. To illustrate the value this work has added to collections we will show you some items in the collection…
- The next three slides just show the robust descriptions that items possess in the Barre Toelken collection that would very likely, if not for crowdsourcing effort, be much less detailed (for instance if someone without knowledge or expertise may provide)
- St. Anne’s Retreat in Logan Canyon has been a source of a contemporary oral tradition and legend-tripping for the last fifty years. Local legends tell of a haunted place with stories laced in ancient themes telling of nuns and priests secretly engaging in romantic encounters, nuns getting pregnant by priests, giving birth and drowning their babies in the legendary pool. The informants/creators of these legendary accounts consist of legend-trippers themselves, and/or the friend of a friend (FOAF), who recount their experiences. These can be found in student fieldwork collections in the USU Fife Folklore Archives, and depict various themes and versions of legend-tripping at St. Anne’s Retreat.
- These examples of “crowdsourcing” from students, using their informant data required as part of their class assignments, are clearly different from the informant/creator dynamic discussed in the first example with Professor Toelken. This form of crowdsourcing is solicited from individuals as the students conducting folklore fieldwork assignments essentially collect the data. In this case, the informant presents their own version of how they perceived their own experience, making a visit to a haunted place, or by re-telling a story they may have heard from a friend. One thing both examples share is that the conduit, the metadata creator, remains the expert on how to best translate the content of the various accounts and create the proper metadata. The following is a sample story from the folklore archives: A long time ago there used to be a nunnery at Saint Anne's. One of the nuns got pregnant by a young priest. She hid the fact that she was pregnant for a long time. When she had the baby she was told she had to leave the nunnery. She was grieved at what had happened and went out and drowned her baby in the swimming pool, then hung herself. Her spirit haunts the place in the form of a dog. Sometimes people can hear dogs howling at Saint Anne's. Nobody has ever seen the dogs In conclusion, the idea is to create a hybrid—to basically synthesize the data from the informant of the collection with the expertise of the metadata creator. In this case, a master’s degree in folklore allows her to recognize areas of cultural sensitivity, terminology, along with other nuances. Most importantly, this hybrid structure adds value to the project, particularly important as it aids the discoverability of student fieldwork efforts in folklore as well as the lifelong work of Barre Toelken, someone so well respected in his field and whose work should be represented in the best way possible.
- While many archivists have evolved their professional scope to bring diversity, equity, and inclusion into their collections, we posit that much can still be done. One area for growth is greater work by archival professionals to partner with communities to help them tell and preserve their own stories, incorporating a community’s own perspective and goals and description. This work often takes more time, but the payoffs are immeasurable. One way to add DEI to a project is through description which often comes from detailed fieldnotes, transcripts, conversations with community scholars, and being imbedded in the project community events and other activities
- In an effort to make USU Special Collections and Archives holdings more diverse, equitable, and inclusive, we collaborate on our ethnographic endeavors to include community partners in all stages of the planning, from concept through completion. This includes input regarding description and metadata. USU has quite a few digital collections that were created while making efforts to collaborate and take advantage of community knowledge, input, and expertise. Some examples include: Latinx Voices, Cache Valley Refugee Project, Ranch Family Documentation Project, Great Salt Lake Wetlands, and others.
- So, I will just go into a bit more detail and talk about the Voices from Drug Court Oral History Project. The efforts involved in this collection have given a voice to an historically excluded community. For this project the community partners included: Andrew Dupree, Community Scholar Brock Alder, Bear River Health Department Division of Substance Abuse Judge Thomas Willmore, First District Court
- The project was conceived and co-managed by Andrew Dupree a participant and now graduate of the Cache Valley Drug Court. Andrew also assisted with interviews and aided in collection description and metadata gathering. Judge Willmore approved the project. Andrew and Brock assisted with interview questions and competency training. In addition to the community partnerships, all of our ethnographic endeavors have USU Institutional Review Board approval (and other approvals as necessary, including from Native Nations and/or Governmental agencies)
- We also include deep collaborations within the library Cataloging and Metadata Services as well as Digital Initiatives. A side note on this – the CMS unit is super grateful to Randy for including metadata in the these efforts so we can also work to improve our own practices and to be more mindful of under-represented groups and diverse communities and to be more equitable and inclusive with our metadata As Randy worked closely with Andrew, and other drug court interviewees, we learned terminology that was previously unknown to us and we worked to include these into the metadata as subject keywords which added tremendous value and improved discoverability. For example: Intensive outpatient program or IOP and Recovery Café. This information comes from the partner, the interview transcript, and the ethnographer’s fieldnotes
- For each ethnographic project we also host community events to introduce new collections. And, it was actually at such an event that Andrew Dupree learned about USU’s oral history efforts, which led to him approaching the library about working on a project involving drug court. We have also worked to produce scholarly outputs and present publicly on what we have done to make sure that we inform others of this important work. This has led to and continues to lead and influence further efforts.
- Another example collection to highlight is the Jackson Hole Dude Ranching Tradition focusing on the Triangle X Ranch For this project the partners included: Utah State University Folklore Graduate Students University of Wyoming American Studies Graduate Students The Turner Family (who own and run the ranch) American Folklife Center at the Library of Congress
- During the Summer semester of 2017, all the participating partners conducted interviews, took photographs, created fieldnotes, and created metadata using a metadata spreadsheet and a predetermined file naming protocol to assign to all of the digital objects they were working to produce
- Robust naming protocols help students, community scholars, and donors name a file in such a way that it allows for easier interpretation of the origin of a file, it increases organization, consistency, and serves as a match point and unique identifier for digital objects and their accompanying metadata. As curators and metadata specialists take the time to train students, the description efforts and quality are vastly improved
- Ethnographic fieldwork students use their metadata skills while creating digital exhibits from the project products. This not only introduces a new generation to metadata, but it also utilizes the “crowdsourcing” idea by including the community partners, in this case the Turner Family, who helped describe the information (audio, images) in the exhibit.
- Similar to the drug court oral history project, our upcoming effort continues the important work of giving a voice to an excluded community as we are partnering with USU Extension faculty and ten community partners in ten Utah Counties to collect and preserve stories of the Utah opioid epidemic with the hope of bringing hope and healing to people affected.
- Training sessions that include instruction on how to identify themes in interview data (coding) and the importance of following best practices with metadata and file naming are a necessary piece this project and to all ethnographic endeavors. As you can see Randy and I just this week conducted a training session at USU's Kaysville campus
- This particular project is a large undertaking and stakeholders have submitted an National Endowment for the Humanities grant to produce a scholarly book that will inform the national narrative about Utah’s Opioid crisis
- That brings us to the end of our presentation – thank you for joining us today - now we have time to answer any questions you have for us or we would love to just hear your thoughts, suggestions, or your own experiences if you would like to share