Measuring Anonymity in Academic Virtual Reference


Published on

Kristin Grabarek Roper and I presented this information as a poster session at the American Library Association's Annual Conference 2012. We hope that other librarians, academic and otherwise, may choose to adapt our methods to measure anonymity and assess identity at their institutions.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Measuring Anonymity in Academic Virtual Reference

  1. 1. Measuring Anonymity in Academic Virtual Reference Karen Sobel – University of Colorado Denver & Kristin Grabarek Roper – Denver Public Library
  2. 2. IntroductionLibrarians who provide instant message-based (IM) virtual reference services often perceive that their patrons self-identify to some degree, even whentransactions are anonymous. Over time, they begin to form conclusions about their patrons’ characteristics as a group.Sobel and Roper decided to investigate the realities behind these perceptions. They analyzed two years’ worth of IM reference transcripts to see whatdemographic information their patrons really provided. This work resulted in two major products:1) Detailed and concrete information about their IM reference users’ characteristics2) A method for analyzing transcripts to identify demographics of patron groups over timeThe researchers believe that other libraries will benefit from using the methodology they have developed to investigate their users’ demographicsthrough identifiers provided during individual conversations. MethodologyThe following methodology can be easily adapted to assess patron group demographics at any library. While Sobel and Roper performed their researchat an academic library, public and other libraries providing IM reference service can investigate the patron characteristics that matter most to theirwork.1) The researchers selected representative time periods to study (Fall semester 2009 and Fall semester 2010).2) The researchers created a random sample scheme for selecting transcripts to study. They chose to do this by identifying peak times of service: 10AM-noon and 2PM-4PM. They then selected the transcript closest to the middle of each of these time periods every day. This created the group of samples from which they drew their conclusions.3) The researchers identified the demographic characteristics most affecting individual reference transactions at their institution. Most reflected demographics; some data on academic needs was also gathered. Choices were based on their personal expertise and judgment. Characteristics selected were: origin of message (for example, through a chat box embedded on a course page), gender, age group, school (the library serves three institutions of higher education), full-time or part-time enrollment, year in school, and English as a Second Language status. Information on academic needs included the subject area of the question (when applicable), and major category of reference need (such as assistance with a database or basic library information).4) The researchers wrote detailed criteria for evaluating each characteristic. For example, transcripts identified as coming from a speaker of English as a Second Language might include an explicit explanation “Hi, I am an international student from Japan and I need help with….” It might also include language or spellings pointing to other languages (such as “jaja” instead of “haha,” or syntax pointing to non-native speech.5) Roper carefully reviewed all the selected transcripts and reviewed each characteristic for every transcript.6) The researchers compiled the data in a spreadsheet using Microsoft Excel. They had decided prior to performing the study that they were most interested in raw numbers and simple frequencies. (More robust statistical software such as R or SPSS could calculate correlations or evaluate statistical validity for libraries interested in exploring these factors.)
  3. 3. Types of Data The following types of data were analyzed: • Origin: Users can initiate an IM reference transaction from a variety of locations on the Auraria Library website. Transcript analysis began by determining where the user was on the website when s/he initiated the IM reference transaction. • Gender: While IM reference service users are not required to provide any specific information about themselves before beginning a transaction, some do choose to state their name. Often, a name will suggest the gender of the user. In cases where the gender was reasonably assumed, it was recorded in the analysis. In cases where a name did not clearly indicate gender, the field was marked as unknown. • Age: Because of the diversity of the campus population, researchers thought it might be useful to determine if there is a difference in IM reference service use based on age. However, age is challenging to determine even in face-to-face reference interviews; attempting to glean information from IM transcripts proved futile and this category was abandoned. • School: Auraria Library users may be affiliated with one of three schools: CCD, MSCD, or UCD. Additionally, Colorado residents may use certain materials at the Auraria Library. The Auraria Library web page and specifically the IM reference service may be accessed from literally any place in the world and used by any person regardless of affiliation; therefore, the researchers proceeded with caution when recording affiliation information. • Status: This category was used to indicate whether the user was a student, an instructor, or an alum. The term “Other” was included to reflect users who specifically identified as something other than student, instructor, or alum. Due to the large number of “Other” users who identified as “Parent,” that category was added. Students, instructors, and alum from institutions other than CCD, MSCD, and UCD were identified as “Other.” • Year: This category was used to identify graduate students, undergraduate students, instructors, or “not applicable.” “Not applicable” refers to instances in which the user self-identified specifically as someone other than a student and therefore it was clear that the category of “year” was not applicable. Graduate, undergraduate, or instructor status was determined only if the user specifically identified as one of the three, either by stating so or by asking a question about a class that could easily be identified as a grad or undergrad class. In all other cases, this field was marked unknown. • English as a Second Language (ESL): This category was designed to capture the number of ESL IM reference service users through language nuances. This category was approached with caution as well; it is difficult to determine whether poor grammar on IM results from lack of English language proficiency due to ESL status, to lack of proficiency in a native speaker, or even due to the user employing chat-style conversation techniques which are less grammatically correct. • Type of question & Subject area: Roper examined the first question each user asked in order to determine the classification into which that reference transaction fell. These transactions were coded as one of the following: University information, Library information, Reserves, Databases, Catalog, and Reference. In addition to identifying specific areas of study, other subject areas were included, such as Textbooks, Policy, and Interlibrary Loan. ApplicationsThese categories were selected because they closely reflected the curiosity of Auraria librarians interested in learning what patron groups use IMreference service, and because they match the categories used in previous research of this type. These categories also reflect the unique nature of theAuraria Campus, where three institutions share the same library resources.• What types of data might your institution code for in a transcript analysis of this nature?• Has your institution undertaken an IM reference service satisfaction survey? If so, how would your transcript analysis markers compare to those used to compile your survey?• What elements would a transcript analysis capture that might be missed in satisfaction survey results?
  4. 4. Transcript ExamplesTranscript examples provide insight as to how Sobel and Roper coded the transcripts. Because of the complexity of the category, examples of Types ofQuestions are presented here. The researchers examined the first question each user asked in order to determine the classification into which that reference transaction fell. These transactions were coded as one of the following: University information, Library information, Reserves, Databases, Catalog, and Reference. University information refers to questions related to general information about one of the three colleges, or about the Auraria Campus: “Can you send me the ISBN number for a book for a course?” (August 12, 2009) Library information refers to questions related to the location of materials within the library and library policies, including inter-library loan policies: “How many times can I renew a book I have checked out?” (August 12, 2009) Reserves refers to questions regarding materials instructors had placed on reserve for a particular course, or how to place items on reserve for a course: “How do I e-reserve something?” (September 3, 2009) Databases refers to technical questions regarding database use and how to locate a specific article within a database: Graphic Representation of “How do you access a database?” (September 3, 2009) or “How do I find the "Expanded Academic Index ASAP?" Ive Types of Questions been looking under Article Databases & Indexes but cant seem to find it.” (September 4, 2009) Catalog refers to questions regarding the availability of a specific resource: “ I am not sure if your copy of "Planet In Peril" (video) is available on your website or if you have a copy to rent out at your library?” (October 14, 2009) Reference refers to questions on a particular topic, where the user has not yet selected a resource: “What is Boolean logic?” (October 1, 2009) Reference also includes reference questions that result in database use instruction: “Hi, I am doing a research paper on (Forthcoming) Roper, Kristin Grabarek and Karen Sobel. disney and trying to show its monopolistic ways I am having “Anonymity versus Perceived Patron Identity in Virtual trouble researching what would be the best way to do Reference Transcripts.” Public Services Quarterly8.3(Summer this.” (November 2, 2009) 2012).