Straight from the user's mouth: Usability testing UNC's LibGuides
•Download as PPTX, PDF•
1 like•495 views
Presentation for Empirical Librarians Conference 2016 at North Carolina A&T State Unviversity, Greensboro, NC
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Straight from the user's mouth: Usability testing UNC's LibGuides
Similar to Straight from the user's mouth: Usability testing UNC's LibGuides (20)
More from Sarah Joy Arnold
More from Sarah Joy Arnold (8)
Recently uploaded
Recently uploaded (20)
Straight from the user's mouth: Usability testing UNC's LibGuides
- 1. Straight from the User's Mouth: Usability Testing UNC's LibGuides Sarah Arnold & Stephanie Hsieh UNC Chapel Hill Libraries
- 2. Overview What is usability? What is usability testing? What is a subject guide? Background & Planning Process Applying for Institutional Review Board Survey Usability tasks Best Practices Collaboration between Librarians & LIS Students
- 3. What is usability? When a user can do what they want to do, in the way they expect to do it, without hindrance or frustration (Definition adapted from the Handbook of Usability Testing by Jeff Rubin and Dana Chisnell) *http://www.usability.gov/what-and-why/user-experience.html
- 4. What is usability testing? Evaluating how well a product meets some usability criteria Defined by the designers, tested by users Goal is to get recommendations for improving usability Produces mostly qualitative data
- 5. What does a usability test look like? 5 participants* Complete tasks using the product “Talking aloud” about their process and thoughts Can vary a lot in size, complexity, and depending on need *https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
- 6. What is a subject guide? Subject guides: “a website created to give students an overview of books and article databases related to a specific topic. Some example topics include: Middle Eastern Cinemas & Film, Resources on Endangered Species, and How to Find Book Reviews.” Made using LibGuides Underutilized resource
- 7. Background Moved from Library á la Carte to LibGuides v. 1 in 2011 Upgraded to LibGuides v. 2 in Summer 2014 Current standards* combine best practices reported by other libraries and anecdotal information from studies not focused on LibGuides In general, subject guides are underutilized *http://guides.lib.unc.edu/usinglibguides/subjectguides
- 8. Planning process Apply to UNC’s Institutional Review Board for exemption Create a survey to gather baseline data and recruit study participants Come up with representative tasks for representative users* *https://www.nngroup.com/articles/recruiting-test-participants-for-usability-studies
- 9. Institutional Review Board approval Campus MassMail When you ask someone to do something, you are not exempt according to Institutional Review Board standards Application process and timing
- 10. Survey Gauge user understanding or opinion when exploring new initiatives Methodology Recruitment tool Qualtrics
- 11. Example questions Have you ever used a subject guide? When you see the term “subject guide,” what comes to mind? What is one thing you DO/DON’T like about the design? Do you have any other suggestions or comments about the subject guides you have seen today?
- 12. Heat maps Based on this page, where would you go to find more information about Buildings and Maps in Orange County?
- 13. Usability tasks Representative users complete representative tasks while you listen 3 rounds of testing 5 users per round* Specific focus each round Top vs. side navigation 2 vs. 3 columns of information 3-5 vs. 6+ items (links, books, databases, etc.) per box
- 14. Top vs. side navigation
- 15. 2 vs. 3 columns
- 16. 3-5 vs. 6+ items
- 17. Usability tasks - Details Recording and staffing the testing sessions Set up a representative guide Observation Guide + Participant handouts Demographic and basic knowledge questions Follow-up questions
- 18. Task 1: Talk me through what you’re thinking
- 19. Tasks 2-4: Locate specific items
- 20. Task 5: Accessing resources
- 21. Best Practices Plan, plan, plan Take advantage of resources on your campus Test your test Reminder: You aren’t your user
- 22. Collaboration between Librarians & LIS Students Provide an opportunity for future librarians to explore traditional testing practices with guidance of someone with experience Even if neither of you have experience, you have a partner
- 23. Collaboration between Librarians & LIS Students Apply classroom learning to a real-world situation Learn more about how academic libraries function
- 24. Questions? For resources on user experience and usability, visit libux.web.unc.edu/resources Stephanie Hsieh Carolina Academic Library Associate (Graduate Assistant) syfh@live.unc.edu Sarah Arnold Instructional Technology Librarian, User Experience sjarnold@email.unc.edu
Editor's Notes
- Stephanie What is user experience? What is usability testing? online usability tools that assist in user testing From Daniel: Success / failure Completion time Paths Comments
- Stephanie What is user experience? What is usability testing? online usability tools that assist in user testing Notes re: Nielsen’s 5 users: Nielsen suggests it is better to run as many small tests as your budget allows with no more than 5 users per test the more users you test, the less you learn about your tool/site you will hear the same or similar comments over and over after each user end goal is to improve the design of our subject guides, so covering as much ground as possible across 3 smaller tests gives us the opportunity to slowly roll out improvements that don’t overwhelm users all at once also gives us the chance to evaluate and assess after each group
- Stephanie Purpose of study what is a subject guide? Here’s the purpose description from our IRB application: A subject guide is a website created to give students an overview of the library resources related to a specific topic. At UNC, subject guides are created using a third-party software called LibGuides. Some examples include "Resources on Endangered Species" and "How to Find Book Reviews." The subject guides at UNC are under utilized, so we would like to investigate ways to make them more usable during the research process. Subject guide definition: At UNC, a subject guide is a website created to give students an overview of books and article databases related to a specific topic. Some example topics include: Middle Eastern Cinemas & Film, Resources on Endangered Species, and How to Find Book Reviews.
- Sarah Background History of subject guides at UNC began as HTML pages and slowly have migrated to LibGuides Content Management System Standards for creating LibGuides Shared with every new LibGuide account holder Always available on a LibGuide about using/creating a LibGuide Currently aren’t strictly enforcing Underutilized issue > is it the guide or is it the librarian? Highest used guides are on Health Sciences topics and citation managers More practical skills than research starting point
- Sarah Planning process - 3 objectives Applying for IRB exemption Exemption as defined by UNC’s IRB = research places subjects at no more than minimal risk In other words, risk you would experience in daily living Developing a survey in Qualtrics Laying out usability tasks for users to complete From Nielsen: The three main rules for simplified user testing are: Get representative users Ask them to perform representative tasks with the design Shut up and let the users do the talking
- Sarah IRB Approval Initially applied for IRB exemption in order to use the campus MassMail system IRB approval is required when asking students, faculty, and staff to perform a task like taking a survey Side benefit is being able to publish/share findings though we are being very narrow in focus Human Research Ethics UNC’s Office of Human Research Ethics administers and approves all human subject testing Goal of OHRE and IRB is to ensure ethical standards are met and followed so research participants are aware of the entire process of the research study including risks and benefits By applying for exemption status, we didn’t have to fill out the entire application However, we didn’t qualify for exemption because we planned to ask users to complete tasks Scramble to complete application Ended up going through the expedited review process that is performed on an ongoing basis We split up the different application sections > example questions Brief summary that covers purpose, participants, and procedures/methods Project personnel > must all have completed ethics training which takes ~4 hours Any funding sources Background and rationale = literature review Short answer questions based on responses re: # and type of participants, etc. Study design = methods in detail Came up with a loose timeline including soft deadlines for completing the application IRB approval can take time and include some back and forth re: documents to be added, sections to adjust/expand on, etc. Application process is when you do the bulk of your planning since communications, consent forms, tasks, etc. must be submitted and approved We developed our survey questions and tasks One negative of this process is inability to see survey results before you develop the tasks > though you can submit modification to study Important to know your goals and what you’d like to find out from your users
- Sarah Survey In general, surveys allow you to get inside your user’s head and find out what they are thinking Methodology Must consider how you will analyze the results More open ended questions = more data to analyze, but can be more insightful Multiple choice/Likert scale questions = easier to analyze, but potential to overlook an option + add your own bias/limitations increases One regret for this study: Campus resources > Odum Institute Recruitment tool Surveys can commonly be used to recruit participants for studies and other testing For our purposes, our survey was a way to also recruit participants for the follow-up usability testing One caveat, good or bad, the survey respondents who provide their information are self-selected In the case of the library, tend to be supporters and heavy users already > preaching to the choir Qualtrics robust surveying tool that has many features such as question blocks, heat maps, incorporation of logic based on a user’s response to chosen questions, etc. Description/explanation/cover letter > Demographic + library use > specific subject guide questions > heat maps > specific top nav do/do not like > any other comments + follow-up study Other survey tools include Survey Monkey, Google Forms, LibWizard/LibSurveys Also, Optimal Workshop: more robust usability testing tool that includes a head map feature (other tools include online card sorting and tree testing)
- Sarah Example questions
- Sarah Heat maps Provides insight into what users are seeing on the screen Wording probably gave away the answer to this question and follow up that showed the side navigation layout
- Sarah Usability Tasks We wanted to find out which approach our users preferred in order to best help them with their research Tasks needed to be broad enough to apply to each of these focus areas, but also give us the opportunity to uniquely identify what works and what doesn’t Decided to look at 3 specific issues with our subject guides with 5 users each = 15 total usability test participants location of navigation (top vs. side), 2 vs. 3 columns of information per page/tab, and # of items per box (3-5 vs. 6+) Based on readings and research we did at the beginning of our study, we found these 3 issues came up again and again Also wanted to look into how subject guides are presented on our site vs. built in pages that LibGuides provides Plan is to make small updates after each test based on findings Next slides show examples of our focus for each round of testing
- Sarah Usability tasks Needed a quiet, private space that was easy to access, but also had the equipment we needed to record the testing sessions Using a media lab through the School of Information and Library Science Camtasia to record participants actions on screen + audio with external microphone Each session requires a moderator and a notetaker More people in the room would be intimidating for the participants Moderator follows observation guide, asking any questions or for details from the participant Notetaker is back up to video recording + allows moderator leisure to focus on the participant Copied an already existing subject guide on a generic topic (Orange County history) to be representative of what users see Observation guide + participant handout Script that allows for standard questions and actions across all tests Includes what tabs/sites to open prior to testing, basic introductory explanation for participants explaining the test, each task Provide tasks on separate sheets of paper for participant to read and follow Demographics some repetition from survey, but can work in your favor in 2 ways: relaxes participant by getting them used to talking out loud with you occasionally they will provide more insight than they previously did on the survey Follow-up questions Always try to capture any remaining thoughts the participant might have while they’re still with you in the moment Do you have any other suggestions or comments about the sites you have looked at today?
- Task 1: First/Now, I’m going to ask you to look at this page and tell me what you make of it: what strikes you about it, whose site you think it is, what you can do here, and what it’s for. Just look around and talk me through what you’re thinking. Feel free to scroll, but don’t click on anything yet.
- Task 2-1: What do you expect to find on this page? Why would you use this resource? You are starting research about the historical relationship between UNC and Chapel Hill. Using the subject guide in front of you, where would you begin to look for information on this topic? Based on this experience, have your thoughts changed on why you would use this resource and the types of information you are able to find on it? Task 3-1: Locate specific database, Biography and Genealogy Master Index Task 4-1: Locate specific item, The News of Orange County
- Task 5: Looking at this page, what strikes you about it, whose site do you think it is, what can you do here, and what is it for? Just look around and talk me through what you’re thinking. How would you use this page in your research, if at all? On the databases tab, what resource would you use to find more information on your topic? Follow-up after asking above questions for both pages: Between the 2 pages, which do you prefer?
- Sarah Best practices for planning a usability study of this scope and size/recommendations Create a timeline, or at least talk about it IRB assistance/support, colleagues, experts in usability testing and survey methodology Run through your test before you administer it on an actual test participant Work out the bugs Time to practice script You aren’t the user!
- Both Collaboration between librarians and LIS students
- Both Collaboration between librarians and LIS students