Email
Like
Liked ×
Save
Private Content
Embed
Loading embed code…
We have emailed the verification/download link to "".
Login to your email and click the link to download the file directly.

To request the link at a different email address, update it here. Close
Validation messages. Success message. Fail message.

Check your bulk/spam folders if you can't find our mail.
Loading

+ Follow

“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together”

by Amanda Dinscore on Apr 09, 2010

1,619 views

Ethnography, Libraries, Library, Web_Design, Usability

Accessibility

View text version

Upload Details

Uploaded via SlideShare as Microsoft PowerPoint

Usage Rights

File a copyright complaint

1 Embed 12

http://www.slideshare.net

Statistics

Likes: 2
Downloads: 11
Comments: 0
Embed Views: 12
Views on SlideShare: 1,607
Total Views: 1,619

Allison: Rochester Study…10 minutes Amanda: Fresno State Study… 10 minutes Amanda: Usability Study…20 minutes Allison: Talking Points (develop and lead discussion)…30 minutes Allison: Conclusion…5minutes
Focus on design anthropology in which ethnographic study serves to aid in the design of products and services. Sought to engage students in their natural contexts. Library was closed and only offering limited services for the first half of the Library Study at Fresno State. New building opened in February, 2009. Study included observations of student life. Examples of some of the methods used were: “Floating Reference”—in which students acted out the role of the student and librarian at the reference desk in the Free Speech Area“Visions of Student Scholarship”—asked students to photograph student scholarship wherever they found it“Day Mapping” – with campus maps, disposable cameras, and notebooks. Ethnographic interviews—includinghome interviews (picture of lives beyond campus)“Bootlegging Workshops”—students brainstormed ideas about library services and acted out various scenes “Student Theatre Workshops”—research team devised a plot around a library service situation and directed the actors through the first half of a skit“Interior Design Workshops” –allowed students to design their own library space“Web Design Workshops”—from scratch or using “patchwork prototyping” (existing elements in new combinations)
Images: Set up a large poster board in the Free Speech Area in front of the library and asked students to draw how they feel when they write a paper. Mostly negative responses indicating a great deal of stress and frustration. Negative impressions of the library and librarians were demonstrated in several of the skits and other activities that students performed. Students expect to have difficulty when doing research and writing assignments and generally don’t see the library or librarians as being very helpful.
Some of the recommendations, particularly from the web design workshops, were useful for informing our redesign, including: separate search boxes for different types of holdings (doing this now) less institutional, more youthful and content rich feel move to a 2.0 model—general interest in personalization, class tie-ins, information about required texts for classes. Also requested online collaboration spaces like Google Docs--not practical. Integration with My Fresno State and FacebookGenerally—more “open-ended” observations and interactions. Did NOT seek statistical significance of findings. Observations about student life. Still needed to know…(read slide)
Decided to form a subcommittee of the main library web group to investigate what we could do to get more concrete data.Decided that a usability test was the best option. At first, only comprised of library faculty but we knew we wanted to build upon what was discovered in the Library Study. This led to further discussion and involvement with Professor Delcore, one of the main authors of the Library Study.Volunteered his students.
At first, we considered the use of focus groups, surveys, analysis of site usage logs—provided only limited information. Decided on usability testing. Previous decisions based on mostly anecdotal information, assumptions about user behavior and what students “should” know, and staff and faculty personal preferences. Needed data to support change
Purpose:Data rather than assumptions about user behavior. User group: Juniors (not experts yet, but may have some familiarity with the site) Pre-Survey/Consent form: consent to be videotaped Tasks: 10 tasks requiring basic research (finding a book, article, interlibrary loan, etc…); 30-45 minutes Script/Moderator Instructions: had to be very precise, offer very specific instruction to moderators. Demonstrate speak aloud method“Please let me know what you’re thinking.” How to move on, if necessary. “Thank you for trying. Some of these questions are difficult to answer. Would you like to move on to the next question.” What to say if they went off site Really stressed that it was not a test of them but of the site Balance between establishing rapport and consistencyTest design: what data do we want? (audio & video? Screen capture?), what software to use? how participants would be recruited, where to test, what software to use to record data, what incentives to provide… Pre-test: (10) Very important. Learned a lot and changed the wording and our approach to several things based on this. Recruit & Test: where to find the participants? Who would recruit? Where will they go and when? Analyze the results: working on this now. Think big at first, then streamline to what is practical
Subjects: Fresno State Juniors FTE: 4,081 95% Confidence LevelConfidence Interval of 15 Methods: On-site recruitment --$10 Starbucks gift cards—everyone recruited (but mostly Anthro. students). Did not take much time and was much easier than we thought it would be. Pre-Survey/Consent Form—Had to be over 18, a Junior, and had to give consent to be videotaped. Also asked about how often they used the library web site and how often, their major. Introduction Script—created a draft and then tweaked it throughout the process and through the pre-testing phase. This was read by one of 5 anthropology students who were recruited and trained as moderators. Usability Test Activity –consisted of 10 tasks that could be completed using the library’s web site. Consisted of such things as finding books, using interlibrary loan, searching for journal articles, etc… All were tasks that we expect most users to be able to accomplish with very little difficulty. This also took several meetings to draft and revise.
“Lostness” calculated using: Number of different pages visited Total number of pages visited Minimum (or optimum) number of pages that must be visited to complete the taskFrom “Measuring the User Experience” Tullis and Albert

2 Likes

Amanda Dinscore, Librarian at CSU Fresno 2 years ago
Margot , Librarian at Golden Gate University 3 years ago

“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together” Presentation Transcript

Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together
Allison Cowgill & Amanda Dinscore
Henry Madden Library
California State University, Fresno
California Association of Research Libraries
April, 2010
Sacramento, CA
University of Rochester Study
Ethnographic approach to studying campus libraries and their users
The Library Study at Fresno State
September, 2008 - May, 2009 Goal: “to discover information about student life that campus librarians could use to both increase library usage and improve student users’ experience of library services.”
“Draw How You Feel When You Write a Paper.”
Library Study Recommendations
Provide different search boxes for different kinds of holdings.
Attempt a redesign that aims for a less institutional, more youthful and content-rich feel.
Move toward a Web 2.0 model.
Still Needed to know…
How do students use the site?
What tasks can students accomplish?
What assumptions are we making?
What can we discover that will inform our re-design?
Web Usability Workgroup (WUW)
Decision to conduct a usability study and the formation of the Web Usability Workgroup (WUW)
Involvement of Anthropology professor, Hank Delcore, and his students in ANTH 111B (Intermediate Ethnographic Fieldwork)
“How people develop and use technological solutions (like the library's website) is one of the most complicated and important areas of modern life, so it's a natural anthropological attention.” Delcore, 2010
Usability Testing
Usability Testing demonstrates how users interact with the system.
Participants are asked to complete a series of pre-defined tasks and to “think aloud” while completing them.
Benefits:
See what real users actually do (rather than what they say they do)
Provides quantitative data rather than opinion or conjecture
Participants complete common tasks (finding a book, article, etc…)
Process
Develop a plan
Purpose
Determine the user group to be tested
Pre-survey/Consent form
Tasks to be performed (number & time)
Script & Moderator Instructions—ensures consistency
Test Design—how it will be conducted and how results are to be recorded
Pre-test
Recruit & test
Analyze the results
Nuts & Bolts
Subjects: Juniors
Sample Size: 42 (plus 10 pre-tests)
Recruitment: On-site
People: 5 student moderators, 5 WUW members (Anthropology and library faculty)
Facilities: 4 small rooms for testing; welcome area
Equipment: 4 Macs
Software: Screenium
Test Documents:
Pre-Survey/Consent Form
Introduction Script
Moderator Instructions
Usability Test Activity
Cost: $636 ($520 for Starbucks gift cards; Screenium software site licenses $29 per computer)
Benefitted from pre-testing and multiple run-throughs with team members and moderators
Challenges with script, wording of questions, issues that we hadn’t previously considered (e.g. using Google)
Benefits of using student moderators (objective; offered valuable insight)
Talk aloud method was sometimes challenging
Testing took less time than we thought
On-site recruitment worked better than expected
The $10 Starbucks card incentive worked well
Subjects took it seriously
Software (Screenium) worked well
Reflections
Coding & Data Entry
Subject Number, time stamp, task number (1-10)
Link Label (words clicked) & URL
Notes (including quotes from subjects)
“Lostness”
Codes for task completion
Codes for positive & negative comments
Codes for non-verbal behaviors such as smiling, frowning, laughing
Data Analysis (in process)
Discussion

“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together”

by Amanda Dinscore on Apr 09, 2010

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 12

Statistics

“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together” Presentation Transcript