“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together”
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

“Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together”

  • 1,955 views
Uploaded on

Ethnography, Libraries, Library, Web_Design, Usability

Ethnography, Libraries, Library, Web_Design, Usability

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,955
On Slideshare
1,942
From Embeds
13
Number of Embeds
2

Actions

Shares
Downloads
12
Comments
0
Likes
2

Embeds 13

http://www.slideshare.net 12
http://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Allison: Rochester Study…10 minutes Amanda: Fresno State Study… 10 minutes Amanda: Usability Study…20 minutes Allison: Talking Points (develop and lead discussion)…30 minutes Allison: Conclusion…5minutes
  • Focus on design anthropology in which ethnographic study serves to aid in the design of products and services. Sought to engage students in their natural contexts. Library was closed and only offering limited services for the first half of the Library Study at Fresno State. New building opened in February, 2009. Study included observations of student life. Examples of some of the methods used were: “Floating Reference”—in which students acted out the role of the student and librarian at the reference desk in the Free Speech Area“Visions of Student Scholarship”—asked students to photograph student scholarship wherever they found it“Day Mapping” – with campus maps, disposable cameras, and notebooks. Ethnographic interviews—includinghome interviews (picture of lives beyond campus)“Bootlegging Workshops”—students brainstormed ideas about library services and acted out various scenes “Student Theatre Workshops”—research team devised a plot around a library service situation and directed the actors through the first half of a skit“Interior Design Workshops” –allowed students to design their own library space“Web Design Workshops”—from scratch or using “patchwork prototyping” (existing elements in new combinations)
  • Images: Set up a large poster board in the Free Speech Area in front of the library and asked students to draw how they feel when they write a paper. Mostly negative responses indicating a great deal of stress and frustration. Negative impressions of the library and librarians were demonstrated in several of the skits and other activities that students performed. Students expect to have difficulty when doing research and writing assignments and generally don’t see the library or librarians as being very helpful.
  • Some of the recommendations, particularly from the web design workshops, were useful for informing our redesign, including: separate search boxes for different types of holdings (doing this now) less institutional, more youthful and content rich feel move to a 2.0 model—general interest in personalization, class tie-ins, information about required texts for classes. Also requested online collaboration spaces like Google Docs--not practical. Integration with My Fresno State and FacebookGenerally—more “open-ended” observations and interactions. Did NOT seek statistical significance of findings. Observations about student life. Still needed to know…(read slide)
  • Decided to form a subcommittee of the main library web group to investigate what we could do to get more concrete data.Decided that a usability test was the best option. At first, only comprised of library faculty but we knew we wanted to build upon what was discovered in the Library Study. This led to further discussion and involvement with Professor Delcore, one of the main authors of the Library Study.Volunteered his students.
  • At first, we considered the use of focus groups, surveys, analysis of site usage logs—provided only limited information. Decided on usability testing. Previous decisions based on mostly anecdotal information, assumptions about user behavior and what students “should” know, and staff and faculty personal preferences. Needed data to support change
  • Purpose:Data rather than assumptions about user behavior. User group: Juniors (not experts yet, but may have some familiarity with the site) Pre-Survey/Consent form: consent to be videotaped Tasks: 10 tasks requiring basic research (finding a book, article, interlibrary loan, etc…); 30-45 minutes Script/Moderator Instructions: had to be very precise, offer very specific instruction to moderators. Demonstrate speak aloud method“Please let me know what you’re thinking.” How to move on, if necessary. “Thank you for trying. Some of these questions are difficult to answer. Would you like to move on to the next question.” What to say if they went off site Really stressed that it was not a test of them but of the site Balance between establishing rapport and consistencyTest design: what data do we want? (audio & video? Screen capture?), what software to use? how participants would be recruited, where to test, what software to use to record data, what incentives to provide… Pre-test: (10) Very important. Learned a lot and changed the wording and our approach to several things based on this. Recruit & Test: where to find the participants? Who would recruit? Where will they go and when? Analyze the results: working on this now. Think big at first, then streamline to what is practical
  • Subjects: Fresno State Juniors FTE: 4,081 95% Confidence LevelConfidence Interval of 15 Methods: On-site recruitment --$10 Starbucks gift cards—everyone recruited (but mostly Anthro. students). Did not take much time and was much easier than we thought it would be. Pre-Survey/Consent Form—Had to be over 18, a Junior, and had to give consent to be videotaped. Also asked about how often they used the library web site and how often, their major. Introduction Script—created a draft and then tweaked it throughout the process and through the pre-testing phase. This was read by one of 5 anthropology students who were recruited and trained as moderators. Usability Test Activity –consisted of 10 tasks that could be completed using the library’s web site. Consisted of such things as finding books, using interlibrary loan, searching for journal articles, etc… All were tasks that we expect most users to be able to accomplish with very little difficulty. This also took several meetings to draft and revise.
  • “Lostness” calculated using: Number of different pages visited Total number of pages visited Minimum (or optimum) number of pages that must be visited to complete the taskFrom “Measuring the User Experience” Tullis and Albert

Transcript

  • 1. Usability Testing of a Library Web Site: Librarians and Anthropologists Working Together
    Allison Cowgill & Amanda Dinscore
    Henry Madden Library
    California State University, Fresno
    California Association of Research Libraries
    April, 2010
    Sacramento, CA
  • 2. University of Rochester Study
    Ethnographic approach to studying campus libraries and their users
  • 3. The Library Study at Fresno State
    September, 2008 - May, 2009 Goal: “to discover information about student life that campus librarians could use to both increase library usage and improve student users’ experience of library services.”
  • 4. “Draw How You Feel When You Write a Paper.”
  • 5. Library Study Recommendations
    Provide different search boxes for different kinds of holdings.
    Attempt a redesign that aims for a less institutional, more youthful and content-rich feel.
    Move toward a Web 2.0 model.
    Still Needed to know…
    How do students use the site?
    What tasks can students accomplish?
    What assumptions are we making?
    What can we discover that will inform our re-design?
  • 6. Web Usability Workgroup (WUW)
    Decision to conduct a usability study and the formation of the Web Usability Workgroup (WUW)
    Involvement of Anthropology professor, Hank Delcore, and his students in ANTH 111B (Intermediate Ethnographic Fieldwork)
    “How people develop and use technological solutions (like the library's website) is one of the most complicated and important areas of modern life, so it's a natural anthropological attention.” Delcore, 2010
  • 7. Usability Testing
    Usability Testing demonstrates how users interact with the system.
    Participants are asked to complete a series of pre-defined tasks and to “think aloud” while completing them.
    Benefits:
    See what real users actually do (rather than what they say they do)
    Provides quantitative data rather than opinion or conjecture
    Participants complete common tasks (finding a book, article, etc…)
  • 8. Process
    Develop a plan
    Purpose
    Determine the user group to be tested
    Pre-survey/Consent form
    Tasks to be performed (number & time)
    Script & Moderator Instructions—ensures consistency
    Test Design—how it will be conducted and how results are to be recorded
    Pre-test
    Recruit & test
    Analyze the results
  • 9. Nuts & Bolts
    Subjects: Juniors
    Sample Size: 42 (plus 10 pre-tests)
    Recruitment: On-site
    People: 5 student moderators, 5 WUW members (Anthropology and library faculty)
    Facilities: 4 small rooms for testing; welcome area
    Equipment: 4 Macs
    Software: Screenium
    Test Documents:
    Pre-Survey/Consent Form
    Introduction Script
    Moderator Instructions
    Usability Test Activity
    Cost: $636 ($520 for Starbucks gift cards; Screenium software site licenses $29 per computer)
  • 10. Benefitted from pre-testing and multiple run-throughs with team members and moderators
    Challenges with script, wording of questions, issues that we hadn’t previously considered (e.g. using Google)
    Benefits of using student moderators (objective; offered valuable insight)
    Talk aloud method was sometimes challenging
    Testing took less time than we thought
    On-site recruitment worked better than expected
    The $10 Starbucks card incentive worked well
    Subjects took it seriously
    Software (Screenium) worked well
    Reflections
  • 11. Coding & Data Entry
    Subject Number, time stamp, task number (1-10)
    Link Label (words clicked) & URL
    Notes (including quotes from subjects)
    “Lostness”
    Codes for task completion
    Codes for positive & negative comments
    Codes for non-verbal behaviors such as smiling, frowning, laughing
    Data Analysis (in process)
  • 12. Discussion