LibGuides Usability Testing: Customizing a Product to Work for Your Users
•Download as PPT, PDF•
1 like•258 views
Presentated at the 2010 Library Assessment Conference. Baltimore, MD. Ray, Lauren and Hungerford, Rachel. (October 2010).
Recommended
More Related Content
What's hot
What's hot (9)
Viewers also liked
Viewers also liked (15)
Similar to LibGuides Usability Testing: Customizing a Product to Work for Your Users
Similar to LibGuides Usability Testing: Customizing a Product to Work for Your Users (20)
More from Lauren Ray
More from Lauren Ray (6)
Recently uploaded
Recently uploaded (20)
LibGuides Usability Testing: Customizing a Product to Work for Your Users
- 1. LibGuides Usability Testing Customizing a Product to Work for Your Users Rachel Hungerford – University of Washington Libraries Lauren Ray – University of Washington Libraries
- 2. Background University of Washington Libraries moved to LibGuides in Summer 2009 Easier authoring and dynamic content Replaced subject guides list with LibGuides Home Page Created basic guidelines and templates for UW subject guides Few requirements led to lots of variation UW LibGuides “live” in September 2009
- 3. UW Libraries Subject Guides Early 2009
- 4. UW Libraries LibGuides Template Example Summer 2009
- 5. Usability Testing Usability Testing Started in December 2009 Study Goals Identify problem areas of the newly implemented system Determine and improve guides’ usefulness Identify users’ needs and adapt a vendor solution to meet them
- 6. Method 10 Students – 7 Undergrad, 3 Grad Test format Pre and Post-Test and Post-Task Questionnaires Six Tasks X’s & O’s Exercise Varied Tasks by Subject and Format Lab & Equipment Morae
- 7. Findings
- 8. Overview Main Issues: Homepage layout Navigation from general to specific guides Search functions Findings Groupings: Provide guidance and context Provide a more consistent layout Improve navigation Follow best practices for the web
- 9. Guides Should Provide Context and Guidance Issues Perceived scope of the library catalogs Users often thought the search boxes were scoped to the subject they were currently on Questioned the difference between the catalog on the library homepage and within the guides Misuse of the top search box People used the search boxes without noticing or changing the drop down search options
- 10. Guides Should Provide Context and Guidance Students were confused about resources that lacked descriptions (research databases and search boxes) Users had trouble finding format specific information Users expected format options within the subjects’ pages
- 11. Guides Should Provide Context and Guidance User Requests More subject-specific information on the guides – less general research help
- 12. Guides Should Provide a More Consistent Layout Issues Users frequently didn’t notice the tabs located on top of the guides Inconsistent layouts among guides confused users and made it harder to find resources Users felt many guides’ homepages lacked a focal point and weren’t sure of their next steps
- 13. Guides Should Have Better Navigation Issues The path from main guide to sub-guide was inconsistent and not fully apparent The list of guides on the homepage was too long for users to read
- 14. Guides Should Follow Best Practices for the Web Issues Users didn’t read long tabs Database acronyms were confusing to students (such as JSTOR) Students missed rollover database descriptions One user mentioned she preferred static text because rollovers forced her to remember information when comparing resources
- 15. Next Steps after Usability Testing
- 16. Remodeling Data into New Guidelines Challenge of Flexibility: 100+ Subject Guides Listed and prioritized changes Patron Personas helped inform changes to guidelines Staff Forum: Presented results and asked for feedback Checklist for Librarians eased transition to new template
- 17. Excerpt from Recommendations Matrix
- 18. Excerpt from LibGuides Checklist for Librarians
- 19. Excerpt from LibGuides Checklist for Librarians
- 20. New Home page/tab Template Orients users to the guide Provides consistency across guides Shifted extraneous content
- 21. LibGuide using Initial Template, Fall 2009
- 22. LibGuide utilizing post-Usability Testing Recommendations
- 23. LibGuide utilizing post-Usability Testing Recommendations
- 24. More Description Provides Guidance and Scope Search Term Suggestions Database and Catalog Descriptions Template prompts librarians to create short descriptions and search strategies
- 26. Improved Layout and Navigation Pre-LibGuides list of subject guides worked better for users Eliminated confusion with secondary pages or “sub- guides” Related content remains in same location throughout guides
- 27. UW Libraries LibGuides Home Page
- 28. New Subject Guides List post-Usability Testing
- 29. New Guideline Implementation Checklist provided step-by-step guidance Librarians given specific time frame to make changes Drop-in workshops for in-person assistance BRDG Team reviewed guides for compliance Personal follow-up with librarians well received
- 30. Usability Limitations and Further Study Testing environment is different than the users’ natural work space Future questions to explore: How users access the guides and if that can be improved How often guides are used for research How the guides compare with other help content provided by the Libraries
- 31. Wrap Up Usability study allowed us to examine a vendor’s product to better understand how to adapt it to our users We were able to make changes to LibGuides based on data, not our opinions Continued study will further help us understand our users’ needs to best serve their research requirements
- 32. Thank You Questions? Contact: Rachel Hungerford -- rhunger@uw.edu Lauren Ray -- olray@uw.edu
Editor's Notes
- Lauren Ray – Research Commons and Educational Outreach Services Librarian, UW Libraries Rachel Hungerford – Currently a User Experience Researcher at T-Mobile, Former Usability Engineer for the Human Factors/User Experience Program at UW Libraries and grad student in Information Management and User-Centered Design
- In summer 2009, the University of Washington Libraries began subscribing to Springshare’s LibGuides system, a hosted content management system designed specifically for library research guides. The UW Libraries Subject liaison program consists of over 60 subject librarians that support the academic departments of the university Like many academic libraries around the country, we chose to work with LibGuides because it provided: Easier tools for authors to create and maintain guides And a variety of options for organizing and presenting various kinds of content. We procured LibGuides in the summer of 2009 and, along with another librarian, I trained librarians on the system in a series of in-person workshops. Librarians were required to change their guides to LibGuides over the summer, and these were launched to the public on our site at the start of Fall Quarter 2009.
- Here is a look at one of our Pre-LibGuides Subject Guides created in CSS. Very consistent look between guides, stringent guidelines. “Finding Articles”, “Finding Books”, etc…
- When we implemented this system in the summer of 2009, we set up the site to take advantage of the LibGuides system features We created an optional template that authors could choose to use in creating their guides, and that would work across a broad range of disciplines. (Here is a slide showing how the guide template looked) We also replaced our list of subject guides, linked from the Libraries home page, to the LibGuides Site homepage – configured to use default LibGuides structure. We found that Subject liaisons created guides in different ways Some depended heavily on template Others took advantage of the system’s adaptability by creating advanced research guides or “sub-guides” and developing their own look and feel to distinguish their guides from others. The usability testing process begun in Fall 2009 to determine: How users were responding to the various guide layouts Effectiveness of the guides in conveying the usefulness of subject-specific resources Inform our system design List of research questions developed collaboratively with subject liaisons and Libraries Information Technology Services staff And now I’ll turn it over to Rachel Hungerford, who will discuss the usability testing process and findings.
- After largely following LibGuides standard templates and implementing the new system, it was decided that we needed to test the usability of the new tool – to better understand how to improve it to best work for our users The goals of the study were to -- identify the problem areas as well as the aspects that worked well for the newly implemented system -- To determine and improve the guides’ usefulness -- Identify users’ needs and adapt a vendor solution to meet them
- Methodology For the study we had 10 participants - 7 undergrads and 3 grad students from multiple disciplines, varying ages and gender For the test Users were asked to complete pre and post test surveys as well as a brief questionnaire after each task ranking the ease of use The participants completed six tasks After completing the tasks users completed an X’s and O’s exercise where we provided them with printouts of various guides and asked them to circle areas of the page they felt were helpful and to cross out any area that they felt was less useful or that they didn’t see themselves using The tasks were intentionally varied by subject and format. They covered a range of topics, including: including English, History, Dance, and Psychology. Some tasks asked users to find very specific materials whereas others were more exploratory. Students were also asked to find specific formats of information including books, articles, and images. For our lab and equipment, we have a small usability lab set up with one computer and chair, as well as a chair positioned slightly behind the participants’ for the moderator. We used Morae to record the test and administer questionnaires.
- Usability testing showed that some system structures, like the LibGuides homepage layout and navigation from general guides to more specific sub-guides were confusing for users. Format-related guides were difficult to browse for when mixed with subjects/disciplines on the homepage, even if they were pulled into a separate category or onto the side of the page. The search function for the site was consistently misused. We also identified other issues related to page layout inconsistencies that could be improved with stricter content guidelines. We grouped our findings around four concepts that we found our guides should follow: provide guidance and context provide a more consistent layout improve navigation follow best practices for the web
- One of our main findings grouping was that Guides Should provide context and guidance Some of the issues we came across: We found there was a great deal of confusion surrounding the perceived scope of the library catalogs within the guides. Users often thought the World Cat search boxes on the guides were scoped to the subject they were currently on. For example, if a student was on the American Literature LibGuide, he or she would think the catalog search box was only searching within American Literature materials. We also found that the LibGuides system search box included by default on the top of each LibGuide, and in the LibGuide Index, was not understood in context to its location. This search box is intended to allow users to search the content of the given institutions’ guides, as well as the library’s catalog and site (using a drop-down menu). After testing the first five participants, it was very clear that users did not understand what the top search box on the guide was searching. Each student used the top search box incorrectly at least once and several of them used it incorrectly multiple times. It appeared that the participants would use search without reading the drop down options, thus, making assumptions about what it was searching. - Removed after 1st five users
- Students were also confused about resources that didn’t have descriptions of what they contained, such as lists of databases with no explanatory text – including research databases and search boxes for popular resources. We also found that format-specific resources (such as image and news databases) were better located when they appeared within a subject guide, rather than as separate guides. Users had a great deal of trouble finding image databases and the library’s Images LibGuide. Also -- Many users mentioned that they wouldn’t have gone to the library to find images because they did not think the library had that content, or they were used to using Google to find images.
- One request users made was that… During the post-test survey and the X’s & O’s activity several users mentioned that they wanted to see resource-oriented information tailored specifically to the subject they were on and less general information about the discipline or research help
- Moving on to our second category : Guides Should Provide a More Consistent Layout During testing it was observed that users frequently did not notice a guide’s tabs right away as a navigational option. -- Users’ eyes were drawn to the top middle of the page first and would focus on content there, especially if there was actionable content, such as links to other pages or resources. -- During the X’s&O’s exercise one user commented that she felt the tabs were visually separate from the guide and were part of the UW Libraries header. Through observation and user comments, it was also found that inconsistent layouts among the guides confused the users, making it hard for them to find resources. - This was especially apparent when the individual subject guide did not have an introductory Home page. – leaving users disoriented and unsure of what to do next Users also vocalized that they felt that the guides were lacking a focal point, -- and that they desired an obvious place to direct their attention -- and actionable content that would lead them to what they should do next.
- Another main finding category we had was that guides should have better navigation One of the issues we had was that the path from main guide to sub-guide was inconsistent and not fully apparent Broad subjects are currently broken down on the subject guides into a main guide, such as history, and various sub-guides, like History of Asia or Medieval History. -- During testing we found that participants liked that there were more specific sub-guides, which they felt would help people new to the subject understand a subject's breadth. -- However, during the study users would often go straight to the sub-guides within a subject and would miss helpful content provided on the general guide. -- An issue was that guides didn’t show or link back to categories, and that links from the sub-guides to the general guides were inconsistent among the various subjects. Users also had trouble finding what they wanted on the LibGuides Homepage, due in part to the fact that the Homepage list is very long and difficult for users to scan. Additionally, none of the users seemed to notice the information on the side boxes, such as “Featured Guides” or “Helpful Links”. It was not until users were confused or having difficulties that they noticed the side content.
- Our final findings grouping was that guides should follow best practices for the web In general, users did not have issues with the labels of pages or the tabs and seemed to understand what they meant. However, we found that users didn’t really read long tab names, which in some cases prevented them from finding the pages that they were looking for Database acronyms were confusing to students (such as JSTOR), they wanted the acronym to be spelled out It was also found that students did not read descriptions of resources when these descriptions appeared as rollovers. -- One user verbalized that she preferred static descriptions because rollovers forced her to remember what each said when making a decision on which resource to choose. We learned that important information should remain as static text on the page, rather than being hidden under a rollover. And now moving onto to our next steps….
- After compiling data from usability testing, the Libraries BRD (a group of librarians charged with creating and maintaining browsable resource discovery tools on the Libraries website) was charged with coming up with new recommendations and guidelines for librarians to create and modify their subject guides.
- This proved challenging. We were trying to find balance: better serving our users based on real data we compiled during usability testing had to work for over 100 subject guides guidelines and support for librarians who have wide ranging collections and resource types to promote We took the recommendations compiled by the usability team and worked over a few months to prioritize these into things that would be “required” or “suggested” changes. We also looked at the Patron Personas Project, another usability project that created 5 representative UW Libraries patrons – to help guide decisions. My colleague Jennifer Ward will be presenting later today on the Personas Project at UW at 1:30. Cool name: Persona Development and Use, or, How to Make Imaginary People Work for you. We also presented rough results of the usability testing to staff to explain our process and get their feedback, and alert them to changes coming.
- A Recommendations Matrix was created by the Usability Team that listed recommended changes that should be made to the LibGuide related issues that arose during testing supporting evidence
- BRD then created a detailed checklist of step-by-step instructions for making changes to the guides with rationale behind each change With Screenshots and links to screencasts Here is an excerpt from the checklist
- Here is another excerpt from the checklist, showing librarians the new required LibGuides template and pointing out rationale for the different elements of the template.
- Because we found that users were confused by LibGuides that didn’t include a home page/tab, we made a home page on each guide a requirement, and created a template that librarians had to use for this. We based this new template on a home page that participants responded favorably to in usability testing.
- Here’s an example of one of our guides before usability testing was done, utilizing the template we threw together when we first acquired LibGuides. This Civil and Environmental Engineering guide utilized many of the features we had originally suggested liaisons use: A welcome message in the center Helpful Links box on the left A Poll Profile box Find Articles, Find Books, Dissertations, etc tabs
- Same guide with new guidelines implemented post-Usability testing. This includes the strictly required: large, central box titled “What’s in the Guide,” bulleted list of each page/tab title & short description places the contents of the guide directly in the users’ line of sight
- Here is another example of how the guides look after new recommendations were made. Related guides and links are listed below “What’s in the Guide” Consistency in the Home page layout helps users viewing multiple guides On other pages of the guide, librarians were required to shift extraneous content over to the side
- Another change that was made was that when liaisons include search boxes for resources such as the catalog, they are now required to provide tips and guidance for that subject area Because users wanted more explanation and context for what they were looking at, we required librarians to provide search term suggestions, database and catalog suggestions.
- Suggesting phrases or LC subject headings helps show users how to best frame a search within the discipline. highlights the strengths or limitations of a particular resource point out useful tools or ways of limiting/sorting searches within the resource takes guides from being a list of links to actual guides to doing research within a given discipline BRDG created template boxes for both library catalogs with textual descriptions and tips. Librarians copy and edit
- In terms of layout and navigation, we found that the LibGuides system Homepage, which lists all of the guides available in a library’s system, was not working for our users. We decided to revert back to the our own 3-column layout that is hosted on our own web server. Because users found it confusing that they didn’t go straight to a page explaining resources in “history” or “art” but to a list of “sub-guides” tagged with that subject when clicking on the subject in this list, we eliminated this secondary page, a built-in feature of the LibGuides system. We instead required librarians to create specific navigation to sub-guides within their main subject guide. Because the LibGuides Homepage and secondary pages were confusing for users, we eliminated it and reverted to the 3-column layout, and is hosted on our own web server. We also standardized the layout throughout all guides by requiring that similar kinds of content remain in the same area of each page and guide – tutorials, for instance, had to remain on the right column within any guide.
- Here is a screenshot of the LibGuides system home page, which we originally used as our subject guides home page. Too long Wouldn’t necessarily take person straight into a guide
- When we reverted to our own page, we were able to list guides that wouldn’t require users to scroll. Again, clicking on a subject in this guide would go straight into the guide itself, without going to a sub-list of guides tagged with that subject.
- After distributing the checklist of required and suggested changes, liaisons had 3 months to make changes We provided workshops for authors One-on-one assistance and personalized email follow-up.
- With every research study comes some amount of limitations and areas to further explore A quick overview of some of ours As with all usability tests, the testing environment is different than the users’ natural work space – -- though we try to encourage participants to act as natural as possible, the testing environment is not the same as if they were researching in their dorm without someone looking over their shoulder Some questions we’d like to explore further: -- how are users currently accessing the guides? Can that be improved? -- How often are the guides used for research? What role do they play in students’ academic lives? -- How do the guides compare with other help content provided by the libraries? These along with many other questions would help us further improve LibGuides for our students
- And now to wrap up… The usability study allowed us to examine a vendor’s product to better understand how to adapt it to our users. Through our observations and users’ feedback we were able to make changes to LibGuides based on data, not just on our opinions -- Which we strongly believe will contribute to their overall success and adoption by our users Continued study will help us further understand our users’ needs to best serve their research requirements
- Thank you