We used TechSmithMoraeto conduct usability testing of the new WVU Libraries website. The first round of usability testing was internal (undergraduate student employees), from which we compiled and implemented suggested changes based on test results. This is the results of the second round of usability testing, with randomly selected undergraduate students, and focusing on testing the changes that were implemented.

1. USABILITY TESTING
WVU Libraries Website

2. WVU LIBRARIES WEBSITE STUDY
Problem:
We used TechSmith Morae to conduct usability testing of the new WVU Libraries website. The first round of usability testing was internal (undergraduate
student employees), from which we compiled and implemented suggested changes based on test results. This is the results of the second round of
usability testing, with randomly selected undergraduate students, and focusing on testing the changes that were implemented.
Study Description:
A study to explore the usability issue with the WVU Libraries website.
Study Instructions:
Thanks for participating in this study!
You will be asked to do tasks to help WVU Libraries understand where we need to improve our website https://lib.wvu.edu.
As you perform each task, please talk out loud about what you are doing and thinking.

3. 1ST ROUND OF TESTING
Task 1: Go to the WVU Libraries website.
Task 2: Find today’s hours for Eliza’s Coffee Shop.
Task 3: Where can you find information about the kinds of cameras you can check out from the library?
Task 4: Find the database Project Muse.
Task 5: What is the loan period for books for undergraduate students?
Task 6: On which floor of the Downtown Library can you find a book with the call number D810.S7L4 1954a?
Task 7: Find out how to renew a book online
Task 8: See if study room 1028 is available at 5pm tomorrow.
Task 9: Find a peer-reviewed article on global warming.
Task 10: Does the library have a book written by Mark Twain?
Task 11: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library.
Task 12: Where can you find information about how to print in color?
Task 13: I need help writing citations in APA.
Task 14: Find directions to the Health Sciences Center Library.
Task 15: Is there a way to ask a question about the library or your research from the library website?
Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the
course. Where should you go on the website to find it?
Task 17: Can you find Martin Dunlap’s phone number on the WVU Libraries website?

4. ROUND 1 IMPLEMENTS
Search Box
• Add color tabs
• Possibly add icons
• Talk to research services about indexing the webpages
• Use 'Site Search' instead of 'Website'
• Add 'Call Numbers' or 'Call#' next to MountainLynx
WVU Header
• Eliminate 'Contact Us' Looked At Analytics
• Make 'Library Hours' instead 'WVU Home'
• Change 'Directory' to go to WVU Directory
• 'Get Help' becomes 'Contact Us' page containing:
– Contact Us form
– Ask A Librarian
– Research Consultation form
– Suggest A Purchase form
– Hours possibly?
Design
• Refining Print CSS
• Make Ask A Librarian tab more dominant
• Look at adding color to homepage quicklinks
• Increase contrast of backpage search container
Services
• Add/Restructure to list the following:
– Printing, Copying, Scanning
– Computers and Laptops
– Study Room Reservations
• Reorganize Borrowing page:
– Make navigation reflect priority of undergraduate students
– Redesign to make information more accessible
• Reduce extraneous information
• Identify standardized links that should live on every page
Web Application
• Include Google Analytics code into applications that end with
.lib.wvu.edu (eReserves, rooms, and hydra)
• Have Call Number/Floor Maps link directly from the catalog
• Change 'Directory' to 'Faculty & Staff'
• Redesign/Rework Interface
– Room Reservations
– Directory
– Hours
Not implemented

5. 2ND ROUND OF TESTING
Task 1: Go to the WVU Libraries website.
Task 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library?
Task 3: Find today’s hours for Eliza’s Coffee Shop.
Task 4: Where can you find information about the kinds of cameras you can check out from the library?
Task 5: Find the database Project Muse.
Task 6: What is the loan period for books for undergraduate students?
Task 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D?
Task 8: Find out how to renew a book online
Task 9: See if study room 1028 is available at 5pm tomorrow.
Task 10: Find a peer-reviewed article on global warming.
Does the library have a book written by Mark Twain?
Task 11: How would you submit a request for an appointment for with a librarian for research assistance?
Task 12: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library.
Task 13: Where can you find information about how to print in color?
Task 14: I need help writing citations in APA.
Find directions to the Health Sciences Center Library.
Task 15: Find the business database “MRI Mediamark Reporter.”
Is there a way to ask a question about the library or your research from the library website?
Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the
course. Where should you go on the website to find it?
Task 17: Can you use the website to find the journal of Aircraft?
Task 18: Can you find Martin Dunlap’s phone number on the WVU Libraries website?
Task 19: Find a call number for a copy of “Gone With the Wind” on video.
Additions Removed

6. TEST PARTICIPANTS
Sam
Electrical Eng., 2 yrs.
Stephanie
Animal/Nutrition, 1 yrs.
Taylor
Biochemistry, 4 yrs.
Vanessa
Industrial Rel., 5+ yrs.
Zela
Industrial Eng., 1 yrs.
Andrew
History, 5+ yrs.
Hartley
Secondary Ed., 2 yrs.
Jesse
Speech Path., 4 yrs.
Morgan
Mathematics, 3 yrs.
Robert
Finance & Accnt., 3 yrs.

7. 1ST OVERALL TASK:
AVERAGE TIME
* Average time includes delivery of tasks.

8. 2ND OVERALL TASK:
AVERAGE TIME
* Average time includes delivery of tasks.

9. AVERAGE TIME: COMPARISON * Average time includes delivery of tasks.
ORDER TASK Test 1 - Overall Time Test 2 - Overall Time
Task 1: Go to the WVU Libraries website. 0.59 0.44
Task 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library? 1.4
Task 3: Find today’s hours for Eliza’s Coffee Shop. 1.47 0.9
Task 4: Where can you find information about the kinds of cameras you can check out from the library? 1.58 1.24
Task 5: Find the database Project Muse. 1.39 0.57
Task 6: What is the loan period for books for undergraduate students? 1.33 0.94
Task 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D? 2.6 1.42
Task 8: Find out how to renew a book online 0.81 0.87
Task 9: See if study room 1028 is available at 5pm tomorrow. 1.21 0.9
Task 10: Find a peer-reviewed article on global warming. 1.3 0.71
Eliminated 1: Does the library have a book written by Mark Twain? 0.75
Task 11: How would you submit a request for an appointment for with a librarian for research assistance? 0.94
Task 12: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library. 2.11 0.58
Task 13: Where can you find information about how to print in color? 1.33 0.64
Task 14: I need help writing citations in APA. 1.26 0.9
Eliminated 2: Find directions to the Health Sciences Center Library. 1.04
Task 15: Find the business database “MRI Mediamark Reporter.” 0.87
Eliminated 3: Is there a way to ask a question about the library or your research from the library website? 0.56
Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? 1.31 1.68
Task 17: Can you use the website to find the journal of Aircraft? 0.67
Task 18: Can you find Martin Dunlap’s phone number on the WVU Libraries website? 1.28 0.78
Task 19: Find a call number for a copy of “Gone With the Wind” on video. 1.4

10. 0 0.5 1 1.5 2 2.5 3
Go to the WVU Libraries website.
Find today’s hours for Eliza’s Coffee Shop.
Where can you find information about the kinds of cameras you can check…
Find the database Project Muse.
What is the loan period for books for undergraduate students?
Where would you find a book at the Downtown Campus Library with a call…
See if study room 1028 is available at 5pm tomorrow.
Find a peer-reviewed article on global warming.
Find information about borrowing a laptop from the Downtown Library and…
Where can you find information about how to print in color?
I need help writing citations in APA.
Can you find Martin Dunlap’s phone number on the WVU Libraries website?
Usability Test 2 - Overall Time
Usability Test 1 - Overall Time
CROSSOVER TASKS: TIME COMPARISON (-) * Average time includes delivery of tasks.

11. CROSSOVER TASKS: TIME COMPARISON (+/=)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
Find out how to renew a book online
You’re in English 102. Your instructor said to go to the library website and use
the library guide for the course. Where should you go on the website to find it?
Usability Test 2 - Overall Time Usability Test 1 - Overall Time
* Average time includes delivery of tasks.

12. NEW TASKS: AVERAGE TIME
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
If WVU Libraries doesn’t have a specific book, where would you go on the
website to request it from another library?
How would you submit a request for an appointment for with a librarian for
research assistance?
Find the business database “MRI Mediamark Reporter.”
Can you use the website to find the journal of Aircraft?
Find a call number for a copy of “Gone With the Wind” on video.
* Average time includes delivery of tasks.

13. 1ST OVERALL
TASK:
SUCCESS
Completed
Completed with difficulty
Completed with ease
Almost Didn’t Completed
* Task 1 is not included
on this graph. Individual
solutions to Tasks do
not qualify as failure.
See individual tasks for
more details.

14. 2ND OVERALL
TASK:
SUCCESS
* Task 1 is not included
on this graph. Individual
solutions to Tasks do
not qualify as failure.
See individual tasks for
more details.
Completed
Completed with difficulty
Completed with ease
Almost Didn’t Completed

15. TASK 2: If WVU Libraries doesn’t have a specific book, where would you
go on the website to request it from another library?
Other Destinations:
- 30% went to Collections
Student Observations:
- To students Collections is vague and encompasses what the
library has. Exposition on this page as to what the library has, and
what to do if we don’t have something is expected.
- A large percentage of students wouldn’t borrow from another
library, but would simply get the book from Amazon.
Video:
https://testing.lib.wvu.edu/usability2/Task2.wmv

16. TASK 3: Find today’s hours for Eliza’s Coffee Shop.
Other Destination:
- Services
Student Observations:
- A lot of students skipped over Hours in About Us because there
are too many items. Seven items would be more appropriate.
Video:
https://testing.lib.wvu.edu/usability2/Task3.wmv

17. TASK 4: Where can you find information about the kinds of cameras you
can check out from the library?
Other Destinations:
- About Us
- Instruction
Student Observations:
- Multimedia Equipment may be too vague. Framing this as ‘what you
can check out’ , or ‘equipment rental’ might be more appropriate.
- 40% of weren’t aware that you could check out a camera.
Video:
https://testing.lib.wvu.edu/usability2/Task4.wmv

18. TASK 6: What is the loan period for books for undergraduate students?
Other Destinations:
- About Us
Student Observations:
- 20% of students expect to find loan information under
Policies. Perhaps cross-listing would be beneficial.
Video:
https://testing.lib.wvu.edu/usability2/Task6.wmv

19. TASK 7: Where would you find a book at the Downtown Campus Library
with a call number that starts with the letter D?
Other Destinations:
- 20% went to Services
- 40% went to Collections
Student Observations:
- A lot of students don’t know what Call# are. The way maps and
Call#s are rendered could be better framed.
- 20% of students expect the Call#s link on the search box to be call
number search.
Video:
https://testing.lib.wvu.edu/usability2/Task7.wmv

20. TASK 14: I need help writing citations in APA.
Other Destinations:
- 60% went to Instruction
Student Observations:
- Students are expecting to find citation help on the first page of
Instruction.
I would suggest doing so, and restructuring Instruction.
Video:
https://testing.lib.wvu.edu/usability2/Task14.wmv

21. TASK 15: Find the business database “MRI Mediamark Reporter.”
Student Observations:
- Database side navigation is very confusing. The web application
needs to be reworked.
- Misspellings for databases yield no suggestions. Showing
responses as the user searches is more preferable.
Video:
https://testing.lib.wvu.edu/usability2/Task15.wmv

22. TASK 16: You’re in English 102. Your instructor said to go to the library
website and use the library guide for the course. Where
should you go on the website to find it?
Other Destinations:
- 30% went to Instruction
- 30% went to Collections
Student Observations:
- Students don’t know the difference between Course Reserves, LibGuides,
and Study Guides… this causes a great deal of confusion.
- There doesn’t seem to be a clear and present way to get to Course Reserves,
LibGuides, and Study Guides when the student isn’t on the homepage.
In general, Instruction as a primary navigation item needs to be rethought with
a focus on what is beneficial to the primary target audience.
Video:
https://testing.lib.wvu.edu/usability2/Task16.wmv

23. SYSTEM USABILITY SCALE (SUS)
QUESTIONS
1. I think that I would like to use this system frequently
2. I found the system unnecessarily complex
3. I thought that the system was easy to use
4. I think that I would need the support of a technical person to be able to use this system
5. I found the various functions in this system were well integrated
6. I thought there was too much inconsistency in this system
7. I would imagine that most people would learn to use this system very quickly
8. I found the system very cumbersome to use
9. I felt very confident using the system
10. I needed to learn a lot of things before I could get going with this system

24. 1ST SUS: RESULTS
4
4
5
1
5
4
4
3
1
6
3
2
1
2
1
1
2
2
2
1
7
7
8
1
3
1
6
1
3
2
1
4
2
0 1 2 3 4 5 6 7 8 9 10
I think that I would like to use this system frequently
I found the system unnecessarily complex
I thought that the system was easy to use
I think that I would need the support of a technical person to be able to use this system
I found the various functions in this system were well integrated
I thought there was too much inconsistency in this system
I would imagine that most people would learn to use this system very quickly
I found the system very cumbersome to use
I felt very confident using the system
I needed to learn a lot of things before I could get going with this system
Strongly Disagree Disagree N/A Agree Strongly Agree

25. 2ND SUS: RESULTS
1
4
2
6
1
5
2
5
2
5
1
5
2
2
2
1
3
2
1
1
2
1
1
2
3
5
1
3
1
1
4
1
4
1
3
1
3
5
2
3
0 2 4 6 8 10
I think that I would like to use this system frequently
I found the system unnecessarily complex
I thought that the system was easy to use
I think that I would need the support of a technical person to be able to use this system
I found the various functions in this system were well integrated
I thought there was too much inconsistency in this system
I would imagine that most people would learn to use this system very quickly
I found the system very cumbersome to use
I felt very confident using the system
I needed to learn a lot of things before I could get going with this system
Strongly Disagree Disagree N/A Agree Strongly Agree

26. USABILITY SUGGESTIONS

27. 2ND TESTING RESULTS:
Common Usability Mistakes:
- Students by default still treat the search box as a site search
only.
- Several students treat the large search box as a page filtering
device. This is mainly due to its size and being on every page.
Redesigning this to be more like a search box, or with ‘site
search’ first might be more beneficial. Also, a large search box
on the homepage with a condensed search box and the back
page is preferable.
- Journal spelling suggestions or persistent search suggestions
would be beneficial.
Usability Suggestions:
- Collections - Exposition on this page as to what the library has,
and what to do if we don’t have something is expected.
- About Us - Compartmentalize; seven items would be more
appropriate
- Multimedia Equipment - Framing this as ‘what you can check
out’ , or ‘equipment rental’ might be more appropriate.
- Policies/Loan Periods - Cross-listing would be beneficial.
- Maps/Call#s - The way maps and Call#s are rendered could
be better framed.
- Restructuring Instruction: Students don’t know the difference
between Course Reserves, LibGuides, and Study Guides.
- Databases/Journals - Showing responses as the user
searches is more preferable.

28. FIXABLE/ IN PROGRESS (BASE CAMP):
Usability Changes 1
- Web Application - Have Call Number/Floor Maps link
directly from the catalog.
- Templates for apps (directory, hours, etc.) still have old
WVU header links.
Library Website 8.1
- Floating library header/navigation
- Search website within WVU Library website
- Homepage screen autofits to monitor size / tablet and
mobile navigation more clean
- Redesign Collections
WebApps
- Hours
- Directory
- Collections
Fixes
- Edit 404 errors or dead ends to show the search query and
provide options
- Add loading wheels to applications or media that require
significant bandwidth.
Future Testing
- Experiment with remote un-moderated usability testing
- Conduct mobile website usability testing (Mr Tappy)

29. UX ENHANCEMENTS
With more people using mobile android devices than desktop
computers, we must consider:
- How web applications, versus websites, versus libraries, versus
digital collections are all streamlined on the web through:
- New Branding
- Navigational Hierarchy
- Responsivity
- Sub-Branding
- Universal Iconography
Guides / Policies
- Content Style Guide
- Media/Event Promotion Guide
- Website Policies
- WVU Library Pattern Lab
Mobile/Tablet Experiences
- Condense center headers to one-block accordions.
- Facets and filters on mobile should not disappear/move, require
instructions, or use icons only; they need to be apparent, and there
need to be several indicators that ‘taps’ have been successful (color
blocking works too).
- Explore and use hamburger-left, logo center, and search-right,
navigation.
- WebApp tab bars
- Test character-limits on facets.
- Refining focus and functionality

30. UX ENHANCEMENTS
THOUGHTS: