1. Usability Testing
Allison Bailey
LIS 4804
The purpose of this test is to evaluate the performance of the website with regard to usability,
efficiency, aesthetics and functionality. Our goal is to evaluate the website by observing test subjects
while they complete scenarios which will model the anticipated natural use of the collection.
Methodology
Participants/user profile: Our users are those people active in the disability rights, woman’s rights and
LGBT rights movements, as well as researchers, educators, scholars and students.
Our test participants are fellow library and information science students who are also working on
creating a digital collection.
The setting of the usability test: Classroom 305 in Ruffato Hall. Subjects will be provided access to a
computer or they may use one of their own.
Testing procedures and instruments: The test will be administrated to five classmates, one at a time.
Subjects will fill out a consent form and be asked complete the predetermined test scenarios one at a
time, using think aloud protocol while being observed by all members of the Hershey Collection Team.
After they have completed the scenarios, subjects will be asked to complete a post-test questionnaire
asking for their subjective evaluation of the website based on specific criteria, as well as any of their
own thoughts or suggestions they wish to add.
Roles of usability facilitator and observers: Prior to the start of the test, a facilitator will provide brief
instructions to complete the test. The facilitator will provide the subject with a written copy of the
scenarios one at a time and in the same order. The facilitator and observers will watch the subject
complete the test scenario without providing any help in completing the objective. The observers will
take notes regarding the test subject's use of the website as needed. The facilitator will monitor how
long each task takes the user to complete. After all subjects have completed the test observers and
moderators will meet to debrief and compare observations.
Tasks:
Browse through the collection for an image
Contact site administrators for more information.
Search the site for specific information
Access metadata information
Browse through the collection for a text document
Scenarios
Scenario 1: Contact, picture browse
The user wishes to use an image from the Laura Hershey Collection in an article he/she is
writing for publication. The user must find an appropriate image to accompany the article and needs to
contact someone through the website to inquire about the rights and permissions for republication of
the image. Estimated time: 4 minutes
Scenario 2: Text document, browse, social bookmarking
The user wishes to find a poem by Laura Hershey and post it to their Facebook time line. The
user does not have a specific poem in mind, but wishes to find one that appeals to him/her.

2. Estimated time: 5 minutes
Scenario 3: Text document, search, metadata
A student is researching the Disability Caucus and its role in the 1988 presidential elections and
wishes to find something written by Laura Hershey at the time. The student will need to find
information about the original publication of the document in order to cite the source. Estimated time: 3
minutes.
Performance evaluation measures:
 Whether the user is able to complete the task without using site documentation or help from
members of the Laura Hershey Team. Can they complete it fully, partially or not at all?
 Whether the user is able to complete the tasks and scenarios in the amount of time estimated for
each. A timer will be used to measure completion time.
 Whether the user is able to complete the tasks with less than 5 mistakes.
Subjective evaluation measures will include a post-test questionnaire completed by the subject asking
them to rate the website on a number of criteria.
Subjective evaluation measures will also include notes and thoughts of the observers and facilitators
after watching subjects use the website.
Indicators of problem severity:
 A task takes over two minutes longer than the estimated time to complete.
 A user verbally expresses confusion, frustration or asks for help.
 A user makes over four mistakes while trying to complete the task/scenario.
Report contents and presentation:
The report contents will include:
 Whether any test subjects met the criteria for problem severity.
 Whether any test subjects completed the task or scenario in the estimated time and without
making mistakes
 Answers to the post-test questionnaire
 Observations made by members of the Laura Hershey group during the test
 Our recommendations for any changes to be made based on our findings.