Website User Studies

PENN STATE SPECIAL COLLECTIONS
WEB SITE USER STUDIES

Michelle Belden
Spring 2011

GETTING STARTED

 Read two books available on our shelves:
 Web Site Design with the Patron in Mind,
by Susanna Davidsen & Everyl Yankee
 Design talk : understanding the roles of usability
practitioners, web designers, and web developers in
user-centered web design, by Brenda Reeb

 Reviewed mission statement of PSU Special Collections
(“…to provide primary source materials for researchers from
Penn State and around the world”)

GETTING STARTED, CONTINUED

• Brainstormed user groups
• Brainstormed user tasks
• Got input/approval from colleagues
• Met with Usability Team
• Became certified for IRB
• Secured funding

STUMBLING BLOCKS
 IRB (Institutional Review Board) process
 Funding/reimbursement process

 Recruitment

OTHER NOTES
o Not testing finding aids or digital projects
o Specific eye towards administrative organization
o Having deadline was good to force IT cooperation

STUDY PROTOCOL
 Introductions

 Explanation of the study:

 We are testing the usability of the website, NOT your skills as a searcher.
 Please think aloud while performing the tasks,
about how and why you make choices,
and what confuses or frustrates you.
 We can clarify the tasks, but cannot help you perform them.
 The session will be recorded, and notes will be taken, but your involvement will be
anonymous. Recordings will only ever be shown within the department for internal
purposes. Your name will never be associated with the recordings or the notes.
 After you have done the tasks we will ask you to complete a brief survey. The entire
session should last about 1 hour.
 Participation in this study is voluntary, and you can choose to end this session any time.

 Do you have any questions?
 Consent forms
 Set up computer and give participant list of tasks
 Take notes
 Questionnaire
 Ask again if any questions
 Thank them, provide compensation, get receipt

TASK LIST
 Warm-up:
 Use our website to answer the following questions:
 What hours is the Special Collections Library open to the public?
 Where is the Special Collections library located?

 PSU students (undergrad and grad):
 You have been assigned to write a paper using historical documents from the Penn State
Special Collections library. Your professor has told you that there are special procedures for
using Special Collections. Using our website, find two ways using a special collections library is
different from using a general academic library.
 You have found sources within Special Collections and written your paper. Use our website to
find the correct format for citing those materials.

 PSU staff:
 Your filing cabinets are overflowing, and your supervisor has asked that you transfer old “stuff”
to the University Archives to make room for current “stuff”. Locate information on what records
we collect on our website.

 General:
Can you locate the following on our website: (Each subject asked 2)
 Digital images of Pennsylvania railroads?
 Information about special collections materials related to the 8th air force?
 Information about special collections materials related to Amish settlements in Pennsylvania?
 Information about special collections materials related to African American history?

Can you use our website to:
 Figure out who you would ask a question about the United Steelworkers archives?
 Order a Fred Waring Christmas CD?
 Find a yearbook photo of David Arthur Valentine, who graduated in 1961?
 Find out if we have video of the October 1940 football game with Temple?

QUESTIONNAIRE
1) How much time do you spend using the Internet for homework or work, per week?
___ Never ___ 1-5 hours ___ 5-10 hours ___More than 10 hours

2) Do you consider yourself an expert computer/Internet user? Yes/No

3) Have you ever been oriented to special collections (generally or at Penn State)
through a class, tour, online tutorial, or other outreach activity? Yes/No

4) In the last year, how many times have you used special collections materials?
At Penn State: Never, Once or twice, Between 2 and 5 times, More than 5 times
Anywhere Else: Never, Once or twice, Between 2 and 5 times, More than 5 times

5) The Special Collections Library Web Site is easy to use:
__ Strongly Agree __Agree __Disagree __Strongly Disagree

6) If you selected Disagree or Strongly Disagree please tell us what causes difficulties:

7) What changes would you make to improve the Special Collections Library’s website?

OBSERVATIONS – TASK CONSTRUCTION
 Terms matter
 When phrase “historical materials” in
question, they go to “Historical Collections” unit
– same for “archival materials” and University
Archives
 “Reference” confused for citation
 Along these lines, subject knowledge matters –
“Amish” in question, “Pennsylvania German” in
collection title
 Order of questions can influence approach
 May need to change order from study to study

OBSERVATIONS – SEARCH (GENERAL)

 About half didn’t even look at the side bar
navigation
 Everyone likes to use the site search
 But it is unclear that this searches all the
University Libraries.
 They also want to search on format (and can’t)

OBSERVATIONS – SEARCHING SUBJECTS
 None of those looking for Amish materials
found our largest Amish collection
 Only one person looking for African-American
materials found our largest A/A collection
 Most commented they wanted collections
listed by topic

OBSERVATIONS – THE USELESSNESS OF
LONG EXPLANATIONS ON WEBSITES

 Half of the subjects commented there was
too much text on any given page
 Several could be on the right page but not
see a link right in front of them, if there were
too many “ other things” to look at

Yadda yadda yadda yadda yadda

IMMEDIATE CHANGES
 Add curator contact info to all collection pages
 Add “(yearbook)” to LaVie link
 Add citation info to tutorial & online reference pages
 Add link to digital collections from search collections page
 Hopefully, organize by subject rather than unit
 Hopefully, further edit long textual descriptions on top-
level pages

NEXT STEPS
 First round not perfect, but we’ve learned a
lot, not just about the web site but about the
user study process.
 For the next round, we may use a more
ethnographic approach.
 Google Analytics.

 Site search just for Special Collections?

 Better finding aids & FA search (then test).

 I will be asking to join the Usability Team.

AREAS FOR FUTURE
RESEARCH/COLLABORATION/BRAINSTORMING

 Apply Kulthau?
 Standardized list of questions for

archival website user studies?
 Explore ideas for “Research Help”:
 Diagrams/visual dictionary of common terms
 Possible to do mouseover definitions in CMS?
 FAQ since looks simpler/quicker than tutorial?
 "Search Tips“

 Thinking about how to illustrate
concepts/collections with just one image

KULTHAU’S INFORMATION SEARCH PROCESS

http://comminfo.rutgers.edu/~kuhlthau/information_search_process.htm

ONLINE TUTORIAL & TOUR
 http://www.libraries.psu.edu/psul/speccolls/tutorial.html
 http://www.libraries.psu.edu/psul/speccolls/tour.html
 http://animoto.com/play/M2PuJCs1K062vokzVTvb0w?utm_content=main_link

MORE PICTURES TO EXPLAIN THINGS
 What do we mean by “collection”

 Records of an organization

 Personal papers

Website User Studies

More Related Content

What's hot

Viewers also liked

Similar to Website User Studies

Recently uploaded

Website User Studies