Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WVU Libraries Website Study

159 views

Published on

We used TechSmithMoraeto conduct usability testing of the new WVU Libraries website. The first round of usability testing was internal (undergraduate student employees), from which we compiled and implemented suggested changes based on test results. This is the results of the second round of usability testing, with randomly selected undergraduate students, and focusing on testing the changes that were implemented.

Published in: Education
  • Be the first to comment

  • Be the first to like this

WVU Libraries Website Study

  1. 1. USABILITY TESTING WVU Libraries Website
  2. 2. WVU LIBRARIES WEBSITE STUDY Problem: We used TechSmith Morae to conduct usability testing of the new WVU Libraries website. The first round of usability testing was internal (undergraduate student employees), from which we compiled and implemented suggested changes based on test results. This is the results of the second round of usability testing, with randomly selected undergraduate students, and focusing on testing the changes that were implemented. Study Description: A study to explore the usability issue with the WVU Libraries website. Study Instructions: Thanks for participating in this study! You will be asked to do tasks to help WVU Libraries understand where we need to improve our website https://lib.wvu.edu. As you perform each task, please talk out loud about what you are doing and thinking.
  3. 3. 1ST ROUND OF TESTING Task 1: Go to the WVU Libraries website. Task 2: Find today’s hours for Eliza’s Coffee Shop. Task 3: Where can you find information about the kinds of cameras you can check out from the library? Task 4: Find the database Project Muse. Task 5: What is the loan period for books for undergraduate students? Task 6: On which floor of the Downtown Library can you find a book with the call number D810.S7L4 1954a? Task 7: Find out how to renew a book online Task 8: See if study room 1028 is available at 5pm tomorrow. Task 9: Find a peer-reviewed article on global warming. Task 10: Does the library have a book written by Mark Twain? Task 11: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library. Task 12: Where can you find information about how to print in color? Task 13: I need help writing citations in APA. Task 14: Find directions to the Health Sciences Center Library. Task 15: Is there a way to ask a question about the library or your research from the library website? Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? Task 17: Can you find Martin Dunlap’s phone number on the WVU Libraries website?
  4. 4. ROUND 1 IMPLEMENTS Search Box • Add color tabs • Possibly add icons • Talk to research services about indexing the webpages • Use 'Site Search' instead of 'Website' • Add 'Call Numbers' or 'Call#' next to MountainLynx WVU Header • Eliminate 'Contact Us' Looked At Analytics • Make 'Library Hours' instead 'WVU Home' • Change 'Directory' to go to WVU Directory • 'Get Help' becomes 'Contact Us' page containing: – Contact Us form – Ask A Librarian – Research Consultation form – Suggest A Purchase form – Hours possibly? Design • Refining Print CSS • Make Ask A Librarian tab more dominant • Look at adding color to homepage quicklinks • Increase contrast of backpage search container Services • Add/Restructure to list the following: – Printing, Copying, Scanning – Computers and Laptops – Study Room Reservations • Reorganize Borrowing page: – Make navigation reflect priority of undergraduate students – Redesign to make information more accessible • Reduce extraneous information • Identify standardized links that should live on every page Web Application • Include Google Analytics code into applications that end with .lib.wvu.edu (eReserves, rooms, and hydra) • Have Call Number/Floor Maps link directly from the catalog • Change 'Directory' to 'Faculty & Staff' • Redesign/Rework Interface – Room Reservations – Directory – Hours Not implemented
  5. 5. 2ND ROUND OF TESTING Task 1: Go to the WVU Libraries website. Task 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library? Task 3: Find today’s hours for Eliza’s Coffee Shop. Task 4: Where can you find information about the kinds of cameras you can check out from the library? Task 5: Find the database Project Muse. Task 6: What is the loan period for books for undergraduate students? Task 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D? Task 8: Find out how to renew a book online Task 9: See if study room 1028 is available at 5pm tomorrow. Task 10: Find a peer-reviewed article on global warming. Does the library have a book written by Mark Twain? Task 11: How would you submit a request for an appointment for with a librarian for research assistance? Task 12: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library. Task 13: Where can you find information about how to print in color? Task 14: I need help writing citations in APA. Find directions to the Health Sciences Center Library. Task 15: Find the business database “MRI Mediamark Reporter.” Is there a way to ask a question about the library or your research from the library website? Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? Task 17: Can you use the website to find the journal of Aircraft? Task 18: Can you find Martin Dunlap’s phone number on the WVU Libraries website? Task 19: Find a call number for a copy of “Gone With the Wind” on video. Additions Removed
  6. 6. TEST PARTICIPANTS Sam Electrical Eng., 2 yrs. Stephanie Animal/Nutrition, 1 yrs. Taylor Biochemistry, 4 yrs. Vanessa Industrial Rel., 5+ yrs. Zela Industrial Eng., 1 yrs. Andrew History, 5+ yrs. Hartley Secondary Ed., 2 yrs. Jesse Speech Path., 4 yrs. Morgan Mathematics, 3 yrs. Robert Finance & Accnt., 3 yrs.
  7. 7. 1ST OVERALL TASK: AVERAGE TIME * Average time includes delivery of tasks.
  8. 8. 2ND OVERALL TASK: AVERAGE TIME * Average time includes delivery of tasks.
  9. 9. AVERAGE TIME: COMPARISON * Average time includes delivery of tasks. ORDER TASK Test 1 - Overall Time Test 2 - Overall Time Task 1: Go to the WVU Libraries website. 0.59 0.44 Task 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library? 1.4 Task 3: Find today’s hours for Eliza’s Coffee Shop. 1.47 0.9 Task 4: Where can you find information about the kinds of cameras you can check out from the library? 1.58 1.24 Task 5: Find the database Project Muse. 1.39 0.57 Task 6: What is the loan period for books for undergraduate students? 1.33 0.94 Task 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D? 2.6 1.42 Task 8: Find out how to renew a book online 0.81 0.87 Task 9: See if study room 1028 is available at 5pm tomorrow. 1.21 0.9 Task 10: Find a peer-reviewed article on global warming. 1.3 0.71 Eliminated 1: Does the library have a book written by Mark Twain? 0.75 Task 11: How would you submit a request for an appointment for with a librarian for research assistance? 0.94 Task 12: Find information about borrowing a laptop from the Downtown Library and the Evansdale Library. 2.11 0.58 Task 13: Where can you find information about how to print in color? 1.33 0.64 Task 14: I need help writing citations in APA. 1.26 0.9 Eliminated 2: Find directions to the Health Sciences Center Library. 1.04 Task 15: Find the business database “MRI Mediamark Reporter.” 0.87 Eliminated 3: Is there a way to ask a question about the library or your research from the library website? 0.56 Task 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? 1.31 1.68 Task 17: Can you use the website to find the journal of Aircraft? 0.67 Task 18: Can you find Martin Dunlap’s phone number on the WVU Libraries website? 1.28 0.78 Task 19: Find a call number for a copy of “Gone With the Wind” on video. 1.4
  10. 10. 0 0.5 1 1.5 2 2.5 3 Go to the WVU Libraries website. Find today’s hours for Eliza’s Coffee Shop. Where can you find information about the kinds of cameras you can check… Find the database Project Muse. What is the loan period for books for undergraduate students? Where would you find a book at the Downtown Campus Library with a call… See if study room 1028 is available at 5pm tomorrow. Find a peer-reviewed article on global warming. Find information about borrowing a laptop from the Downtown Library and… Where can you find information about how to print in color? I need help writing citations in APA. Can you find Martin Dunlap’s phone number on the WVU Libraries website? Usability Test 2 - Overall Time Usability Test 1 - Overall Time CROSSOVER TASKS: TIME COMPARISON (-) * Average time includes delivery of tasks.
  11. 11. CROSSOVER TASKS: TIME COMPARISON (+/=) 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Find out how to renew a book online You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? Usability Test 2 - Overall Time Usability Test 1 - Overall Time * Average time includes delivery of tasks.
  12. 12. NEW TASKS: AVERAGE TIME 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library? How would you submit a request for an appointment for with a librarian for research assistance? Find the business database “MRI Mediamark Reporter.” Can you use the website to find the journal of Aircraft? Find a call number for a copy of “Gone With the Wind” on video. * Average time includes delivery of tasks.
  13. 13. 1ST OVERALL TASK: SUCCESS Completed Completed with difficulty Completed with ease Almost Didn’t Completed * Task 1 is not included on this graph. Individual solutions to Tasks do not qualify as failure. See individual tasks for more details.
  14. 14. 2ND OVERALL TASK: SUCCESS * Task 1 is not included on this graph. Individual solutions to Tasks do not qualify as failure. See individual tasks for more details. Completed Completed with difficulty Completed with ease Almost Didn’t Completed
  15. 15. TASK 2: If WVU Libraries doesn’t have a specific book, where would you go on the website to request it from another library? Other Destinations: - 30% went to Collections Student Observations: - To students Collections is vague and encompasses what the library has. Exposition on this page as to what the library has, and what to do if we don’t have something is expected. - A large percentage of students wouldn’t borrow from another library, but would simply get the book from Amazon. Video: https://testing.lib.wvu.edu/usability2/Task2.wmv
  16. 16. TASK 3: Find today’s hours for Eliza’s Coffee Shop. Other Destination: - Services Student Observations: - A lot of students skipped over Hours in About Us because there are too many items. Seven items would be more appropriate. Video: https://testing.lib.wvu.edu/usability2/Task3.wmv
  17. 17. TASK 4: Where can you find information about the kinds of cameras you can check out from the library? Other Destinations: - About Us - Instruction Student Observations: - Multimedia Equipment may be too vague. Framing this as ‘what you can check out’ , or ‘equipment rental’ might be more appropriate. - 40% of weren’t aware that you could check out a camera. Video: https://testing.lib.wvu.edu/usability2/Task4.wmv
  18. 18. TASK 6: What is the loan period for books for undergraduate students? Other Destinations: - About Us Student Observations: - 20% of students expect to find loan information under Policies. Perhaps cross-listing would be beneficial. Video: https://testing.lib.wvu.edu/usability2/Task6.wmv
  19. 19. TASK 7: Where would you find a book at the Downtown Campus Library with a call number that starts with the letter D? Other Destinations: - 20% went to Services - 40% went to Collections Student Observations: - A lot of students don’t know what Call# are. The way maps and Call#s are rendered could be better framed. - 20% of students expect the Call#s link on the search box to be call number search. Video: https://testing.lib.wvu.edu/usability2/Task7.wmv
  20. 20. TASK 14: I need help writing citations in APA. Other Destinations: - 60% went to Instruction Student Observations: - Students are expecting to find citation help on the first page of Instruction. I would suggest doing so, and restructuring Instruction. Video: https://testing.lib.wvu.edu/usability2/Task14.wmv
  21. 21. TASK 15: Find the business database “MRI Mediamark Reporter.” Student Observations: - Database side navigation is very confusing. The web application needs to be reworked. - Misspellings for databases yield no suggestions. Showing responses as the user searches is more preferable. Video: https://testing.lib.wvu.edu/usability2/Task15.wmv
  22. 22. TASK 16: You’re in English 102. Your instructor said to go to the library website and use the library guide for the course. Where should you go on the website to find it? Other Destinations: - 30% went to Instruction - 30% went to Collections Student Observations: - Students don’t know the difference between Course Reserves, LibGuides, and Study Guides… this causes a great deal of confusion. - There doesn’t seem to be a clear and present way to get to Course Reserves, LibGuides, and Study Guides when the student isn’t on the homepage. In general, Instruction as a primary navigation item needs to be rethought with a focus on what is beneficial to the primary target audience. Video: https://testing.lib.wvu.edu/usability2/Task16.wmv
  23. 23. SYSTEM USABILITY SCALE (SUS) QUESTIONS 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought that the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system
  24. 24. 1ST SUS: RESULTS 4 4 5 1 5 4 4 3 1 6 3 2 1 2 1 1 2 2 2 1 7 7 8 1 3 1 6 1 3 2 1 4 2 0 1 2 3 4 5 6 7 8 9 10 I think that I would like to use this system frequently I found the system unnecessarily complex I thought that the system was easy to use I think that I would need the support of a technical person to be able to use this system I found the various functions in this system were well integrated I thought there was too much inconsistency in this system I would imagine that most people would learn to use this system very quickly I found the system very cumbersome to use I felt very confident using the system I needed to learn a lot of things before I could get going with this system Strongly Disagree Disagree N/A Agree Strongly Agree
  25. 25. 2ND SUS: RESULTS 1 4 2 6 1 5 2 5 2 5 1 5 2 2 2 1 3 2 1 1 2 1 1 2 3 5 1 3 1 1 4 1 4 1 3 1 3 5 2 3 0 2 4 6 8 10 I think that I would like to use this system frequently I found the system unnecessarily complex I thought that the system was easy to use I think that I would need the support of a technical person to be able to use this system I found the various functions in this system were well integrated I thought there was too much inconsistency in this system I would imagine that most people would learn to use this system very quickly I found the system very cumbersome to use I felt very confident using the system I needed to learn a lot of things before I could get going with this system Strongly Disagree Disagree N/A Agree Strongly Agree
  26. 26. USABILITY SUGGESTIONS
  27. 27. 2ND TESTING RESULTS: Common Usability Mistakes: - Students by default still treat the search box as a site search only. - Several students treat the large search box as a page filtering device. This is mainly due to its size and being on every page. Redesigning this to be more like a search box, or with ‘site search’ first might be more beneficial. Also, a large search box on the homepage with a condensed search box and the back page is preferable. - Journal spelling suggestions or persistent search suggestions would be beneficial. Usability Suggestions: - Collections - Exposition on this page as to what the library has, and what to do if we don’t have something is expected. - About Us - Compartmentalize; seven items would be more appropriate - Multimedia Equipment - Framing this as ‘what you can check out’ , or ‘equipment rental’ might be more appropriate. - Policies/Loan Periods - Cross-listing would be beneficial. - Maps/Call#s - The way maps and Call#s are rendered could be better framed. - Restructuring Instruction: Students don’t know the difference between Course Reserves, LibGuides, and Study Guides. - Databases/Journals - Showing responses as the user searches is more preferable.
  28. 28. FIXABLE/ IN PROGRESS (BASE CAMP): Usability Changes 1 - Web Application - Have Call Number/Floor Maps link directly from the catalog. - Templates for apps (directory, hours, etc.) still have old WVU header links. Library Website 8.1 - Floating library header/navigation - Search website within WVU Library website - Homepage screen autofits to monitor size / tablet and mobile navigation more clean - Redesign Collections WebApps - Hours - Directory - Collections Fixes - Edit 404 errors or dead ends to show the search query and provide options - Add loading wheels to applications or media that require significant bandwidth. Future Testing - Experiment with remote un-moderated usability testing - Conduct mobile website usability testing (Mr Tappy)
  29. 29. UX ENHANCEMENTS With more people using mobile android devices than desktop computers, we must consider: - How web applications, versus websites, versus libraries, versus digital collections are all streamlined on the web through: - New Branding - Navigational Hierarchy - Responsivity - Sub-Branding - Universal Iconography Guides / Policies - Content Style Guide - Media/Event Promotion Guide - Website Policies - WVU Library Pattern Lab Mobile/Tablet Experiences - Condense center headers to one-block accordions. - Facets and filters on mobile should not disappear/move, require instructions, or use icons only; they need to be apparent, and there need to be several indicators that ‘taps’ have been successful (color blocking works too). - Explore and use hamburger-left, logo center, and search-right, navigation. - WebApp tab bars - Test character-limits on facets. - Refining focus and functionality
  30. 30. UX ENHANCEMENTS THOUGHTS:

×