Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How users conduct research using discovery systems: Results of a Usability Study


Published on

This presentation presents the findings of a usability study that was conducted as part of a discovery system redesign project. Recordings from the usability test are included.

Published in: Technology
  • Login to see the comments

How users conduct research using discovery systems: Results of a Usability Study

  1. 1. How Users Conduct Research Using Discovery Systems: Results of a Usability Study Kelsey Brett Discovery Systems Librarian University of Houston Libraries
  2. 2. Discovery System Redesign Began in Fall of 2014
  3. 3. Discovery Redesign Activities Primo Sandbox at time of usability testing • Comparative analysis - Fall of 2014 • Focus groups with service desk staff - Fall of 2014 • Usability Testing - Spring of 2014
  4. 4. Usability Study: Methodology • Received IRB approval prior to study – Exempt category • Pop-Up Usability Study in the library atrium • Alternated between two version of the tests, each with two tasks - averaged test took 10 minutes • Caller, Facilitator, and Note taker – Facilitator and Note taker alternated roles • Incentivized participation with library branded giveaways and snacks • Recorded the audio and screen of each session using Morae Usability Software
  5. 5. Usability Study: Test Version 1 Task 1 “You are trying to find an article that was cited in a paper you read recently. You have the following citation: Clapp, E., & Edwards, L. (2013). Expanding our vision for the arts in education. Harvard Educational Review, 83(1), 5-14. Please find this article using OneSearch” Task 2 “You are doing a research project on the effects of video games on early childhood development. Find a peer-reviewed article on this topic, using OneSearch.”
  6. 6. Usability Study: Test Version 2 Task 1 “Recently your friend recommended the book “The Lighthouse” by P.D. James. Use OneSearch to find out if you can check this book out from the library.” Task 2 “You are writing a paper about the drug cartels’ influence on Mexico’s relationship with the United States. Find a newspaper article on this topic, using OneSearch.”
  7. 7. Usability Study: Participation • 15 Participants – 15 undergraduate students • 4 Freshmen • 1 Sophomore • 7 Juniors • 2 Seniors – 1 graduate student – 1 post-baccalaureate student • Disciplines represented – 11 participants majoring in Sciences – 2 majoring in Business – 2 majoring in Communications – No participation from Humanities students
  8. 8. Results at a Glance Test 1, Task 1 Participant Time on Task Task Completion 1 1m 54s Y 2 4m 13s Y 3 1m 26s Y 4 1m 17s Y 5 1m 26s Y (required assistance) 6 1m 43s Y 7 1m 27s Y 8 1m 5s Y Test 1, Task 2 Participant Time on Task Task Completion 1 3m 44s Y 2 2m 21s Y 3 5m 23s Y (required assistance) 4 2m 5s Y 5 3m 32s Y 6 2m 45s Y 7 3m 8s Y 8 3m 1s Y (required assistance) Test 2, Task 1 Participant Time on Task Task Completion 1 1m 7s Y 2 56s Y 3 No recording Y 4 2m 21s Y 5 1m 8s Y 6 2m 14s Y 7 1m 15s Y Test 2, Task 2 Participant Time on Task Task Completion 1 4m 45s Y (required assistance) 2 59s Y 3 No recording N 4 7m 47s Y 5 2m 52s Y 6 1m 33s Y 7 1m 30s Y
  9. 9. Users do not understand library or academic jargon -Terms such as Peer-reviewed, Digital Library, and citation cause confusion
  10. 10. Pre-search filtering options -Users appreciate them, and except more of them
  11. 11. Users modify their search terms -If relevant results do not appear on the first page
  12. 12. Users do not easily distinguish between scholarly and non- scholarly resources -When asked to find a newspaper article, users did not distinguish them from other types of resources
  13. 13. Redesign Decisions • Removed ‘UH Digital Library’ from Drop-down menu and added ‘Articles +’ • Changed ‘Peer-reviewed journals’ facet to ‘Peer- reviewed articles’ • Tabs were converted into buttons and renamed • Facets were re-ordered to promote most useful limiters • Configured the title link to go directly to full-text or catalog record • More white space incorporated into display
  14. 14. Future Possibilities • Improve integration with stack’s guide for location information • Remove Newspaper articles? • Incorporate instructional information to improve searching • Create search scopes for specific collections
  15. 15. Possibilities for improvements on vendor side • Improve relevancy ranking • Improve autofill and ‘did you mean’ functionalities • Support natural language searching • Improve integration with competing ILS and ERM systems
  16. 16. Interface prior to Redesign May 2014 – May 2015
  17. 17. Redesigned Interface May 2015 – Present
  18. 18. THANK YOU! Kelsey Brett