Here’s the basic plan for this talk: a little introduction, what my methods were, what I found and what they mean.
In this study, I wanted to find out what happens when the average user – someone who’s not an archivist, not a subject specialist, not otherwise trained in searching – uses the information in a typical digital image collection. I’m looking at image collections in particular because, in image collections, users are reliant on the metadata we can give them to be able to find and make sense of the objects they are looking at. There’s not really an equivalent to full-text searching, at least not yet, so when you go into a collection online, you have to deal with the kind of information shown here. Problem is, this information is not easy to come up with – so much of archival description is at the collection or folder level, this information isn’t readily harvestable when objects are digitized and it needs to be created by hand.So to compensate, things like Dublin Core have been developed – a simplified descriptive metadata standard. Which is great, in that it reduces the burden on archivists or catalogers, but it is a limited set of information and a specific vocabulary. My big question in this study is whether DC metadata is useful: do users understand the vocabulary? Is the information sufficient?
First I’ll talk about my users, and then what I did to them.
I worked with 78 subjects total, 72 of them undergrads representing a range of majors – I didn’t want to study just scientists or English majors or whatever. They all did a demographic questionnaire to start with, and here are the results:
So these are experienced searchers: confident but not overly so, used to working with both text and multimedia, but their experience is narrow – a whole lot of Google but not much else.
The survey was completed by 50 subjects and looked to answer these research questions:
Participants did two tasks
I also ran two focus groups, with similar research questions to the survey, but with the added goal of finding out why people said what they had to say.
They did the same two tasks as the survey participants, plus a card sorting task (putting cards with DC elements on them in order) and I asked them to come up with definitions for the elements.
The last bit was search testing in an actual image collection.
I used the Claremont Colleges Digital Library because it uses simple Dublin Core for the most part.
What did I find? Here I’ll break down the results into two sections: the kinds of information users said they wanted, and what they thought of Dublin Core.
User Evaluation of Dublin Core Metadata in Image Collections
User Understanding of Dublin Core Metadata in Digital Image Collections<br />Kathleen Fear<br />July 9, 2009<br />
Introduction<br />Methodology<br />Findings<br />Discussion and Conclusions<br />
Is the Dublin Core metadata provided in a digital collection perceived to be useful?<br />
Discussion and Conclusions</li></li></ul><li>Methodology: Study population<br />78 subjects (72 undergraduates; 6 graduate students)<br />41 unique majors<br />Recruited randomly in the undergraduate library; from SI110; and using Ex-Lab<br />
How often do you use Google or another search engine? (n=78)<br />“Almost every day” (72)<br />
How often do you search for images using Google Images, Flickr or another service? (n=78)<br />“A few times a week” (38)<br />
In the past year, how often did you use an online library catalog?(n=78)<br />“A few times a year” (37)<br />
Search Expertise (n=78)<br />“It is rarely hard for me to find what I’m looking for” (32)<br />
Survey (n=50)<br />What information do users <br />think is useful? <br />How do they express that information?<br />
Methodology: Survey<br />Task 1: What information is useful?<br />“Imagine you are searching for images to put in a PowerPoint for class…what information would be useful when deciding whether an image meets your criteria or not?”<br />
Methodology: Survey<br />Task 2: Ranking DC elements<br />
Focus Groups (n=18)<br />What information do users think is useful and why? <br />How do they interpret the DC elements?<br />
Methodology: Focus group<br />Task 1: What information is useful?<br />Discussion<br />Task 2: Rating DC elements<br />Task 3: Card sorting<br />Task 4: Definitions<br />
Search Testing (n=10)<br />Do users behave in ways that align<br /> with what they say? <br />What information do users find useful in an actual collection?<br />
Methodology: Search testing<br />Environment: Claremont Colleges Digital Library<br />Training task: Find a bullfighter and the date associated with the image<br />Task: Find 5 images relating to ‘pioneer life’ in California at the end of the 19th century<br />Reflection questions and exit interview<br />
Discussion and Conclusions</li></li></ul><li>Description<br />Listed by survey and focus group participants as potentially useful?<br />Identified by search testing participants as useful?<br />YES<br />YES<br />
Publisher<br />An entity responsible for making the resource available.<br />FG1: The rights-holder (the creator or whoever bought the image from the creator).<br />FG2: The name of the publisher if it was a professional picture and was in the newspaper, etc.<br />
Source<br />Listed by survey and focus group participants as potentially useful?<br />Identified by search testing participants as useful?<br />NO<br />NO<br />
Relation<br />A related resource.<br />FG1: Relevance to your search terms.<br />FG2: How the image is relevant.<br />
But what about context?<br />“And relation, like, that would seem like how it relates to my search, but that's nothing to do with it actually.” (U01)<br />“… I don't really know what that means, I guess. Like, relation to what?” (U09). <br />“I don't really know what relation means, at least in this context” (U07)<br />“Relation, is that like related images? Because I was kind of looking for that.” (U05)<br />
Conclusions<br />Users are not blank slates<br />Dublin Core provides useful information<br />Dublin Core vocabulary can be misleading or incomprehensible<br />
Thank you!<br />Questions?<br />This work was supported by a Rackham Graduate Student Research Grant.<br />Thank you to Beth Yakel and Soo Young Rieh for their advice, feedback and support!<br />Kathleen Fear<br />School of Information, University of Michigan<br />Kathleen.firstname.lastname@example.org<br />umich.edu/~kfear<br />