Download to read offline
![Conducting Usability Research with a Team of One [Revised: October 2009]](https://cdn.slidesharecdn.com/ss_thumbnails/kowalskiheweb2009aps7-091002163651-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
![Boost SharePoint User Adoption Through DIY Usability Testing [workshop] Share...](https://cdn.slidesharecdn.com/ss_thumbnails/boostuseradoptionthroughusabilitytesting-141010231125-conversion-gate02-thumbnail.jpg?width=640&height=640&fit=bounds)
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Don't Fear the User
- 1. Don’t Fear theUser Lauren Hirsh - @LASHirsh Jacob Geib-Rosch - @JacobGeibRosch
- 2. Who are we? LaurenHirsh - Duke Web Services Jacob Geib-Rosch - Fuqua School of Business
- 3. What are wegoing to talk about? 1) Overview of Usability Testing 2) Why it’s useful 3) How to do it Q&A
- 4. What is Usability Testing? Photoby rawpixel.com on Unsplash
- 5. What is Usability? -Intuitive Design - Ease of Learning - Efficiency of use Usability.gov https://www.usability.gov/what-and-why/usability-evaluation.html - Memorability - Error frequency and severity - Subjective satisfaction Usability = how easy something is to use. 6 factors:
- 6. Intuitive Design A nearlyeffortless understanding of the architecture and navigation of the site.
- 7. Ease of Learning Howfast a user who has never seen the user interface before can accomplish basic tasks.
- 8. Efficiency of Use Howfast an experienced user can accomplish tasks.
- 9. Memorability After visiting thesite, a user can remember enough to use it proficiently in future visits.
- 10. Error Frequency andSeverity How often users make errors while using the system, how serious the errors are, and how users recover from the errors.
- 11. Subjective Satisfaction If theuser likes using the system.
- 12.
- 13. Usability is not UserExperience
- 14.
- 15. Why Do Usability Testing? Photoby rawpixel.com on Unsplash
- 16. 8 - 10seconds
- 17. ¡adios! Some good graphic?Maybe the monkey? Gif from Giphy.com http://gph.is/1fLgeIZ
- 18. Usability is morethan a pretty face Just because your site looks great doesn’t mean people will want to use it.
- 19. I am notmy users.
- 20.
- 21. Discover Discover which aspectsof your web site are difficult or problematic for your users.
- 22. Diagnose Understand why usersare behaving a certain way on your website
- 23. Validate Ensure that thewebsite changes you made do in fact improve users’ experiences and don’t diminish them.
- 24. User Research & TestingMethods
- 25. User Research Methodologies Quantitative Qualitative Behavioral Attitudinal A/BTesting Surveys Eyetracking Guerilla Usability Testing Card Sorting Interviews Concept Testing Unmoderated UX Studies Ethnographic Field Studies Focus Groups
- 26.
- 27. How to Do GuerillaUsability Testing
- 28. What is GuerillaUsability Testing? If there’s one key requirement of user testing it’s having users test your product. Find users using whatever means necessary. Find them where they are, like at the library.
- 29. Look for Patterns You’relooking for patterns of usage, which just takes 5 - 10 participants.
- 30. Average Time 3 -4 hours of prep to figure out where to find users, write your goal, and write your script. 3 - 5 hours in the field talking to users. 4 - 6 hours analysing your results.
- 31. Use When You’re validatinga design prior to implementation. Use as often as possible, though it’s especially important for high-risk designs.
- 32.
- 33. 1.Think about yourusers.
- 34. 1. Think AboutYour Users. 2.Determine your research question.
- 35. 1. Think AboutYour Users. 2. Determine your research question. 3.Develop a testing plan.
- 36. 3. Testing Plan: WriteYour Script 1. Begin with an intro & disclaimer and get consent.
- 37. 3. Testing Plan: Intro& Consent ● Introduce yourself. ● Thank them for taking their time. ● Get consent for the type of information you’re collecting.
- 38. 3. Testing Plan: Intro& Consent ● Explain the format of the test. ● Make sure the participant knows you’re testing the site, not them. ● Discuss incentives (if any).
- 39. 3. Testing Plan: WriteYour Script 1. Begin with an intro/disclaimer and get consent. 2. Demographic questions.
- 40. 3. Testing Plan: WriteYour Script 1. Begin with an intro/disclaimer and get consent. 2. Demographic questions. 3. 3-5 tasks or questions.
- 41. 3. Testing Plan: WriteYour Script 1. Begin with an intro/disclaimer and get consent 2. Demographic questions 3. 3-5 tasks or questions 4. Exit questions
- 42. 1. Think AboutYour Users. 2. Determine your research question. 3. Develop a testing plan. 4.Conduct your test.
- 43. 4. Conduct YourTest: What you’ll need ● Go to where your users are. ● If you can’t, set up where you can find people similar to your target audience who may have spare time. ● Be sure to have: a Script, a Sign, a Computer, and Something for you to take notes on.
- 44. 4. Conduct YourTest: Put Your Participants at Ease ● Make the test comfortable, sit with them, and have a comfortable surface. ● If you need to pause to take notes, just let them know what you’re doing.
- 45. 4. Conduct YourTest: Using Your Script ● It really is just a document with text and prompts. ● How closely you stick to it is up to you. ● For beginners, I’d advise sticking to it, but if something glaring comes up, don’t be afraid to go off-script.
- 46. 1. Think AboutYour Users. 2. Determine your research question. 3. Develop a testing plan. 4. Conduct your test. 5.Analyze your data.
- 47. 5. Analyze YourData: Things to Look Out For ● Difficulty clicking targets (that are too small).
- 48. ● Difficulty clickingtargets (that are too small). ● Clicking things that aren’t actually clickable. 5. Analyze Your Data: Things to Look Out For
- 49. 5. Analyze YourData: Things to Look Out For ● Difficulty clicking targets (that are too small). ● Clicking things that aren’t actually clickable. ● People doing unexpected things.
- 50. 5. Analyze YourData: Things to Look Out For ● Difficulty clicking targets (that are too small). ● Clicking things that aren’t actually clickable. ● People doing unexpected things. ● People being surprised by where a link takes them.
- 51. 5. Analyze YourData: Things to Look Out For ● Difficulty clicking targets (that are too small). ● Clicking things that aren’t actually clickable. ● People doing unexpected things. ● People being surprised by where a link takes them. ● Difficulty locating desired information.
- 52. 1. Think AboutYour Users. 2. Determine your research question. 3. Develop a testing plan. 4. Conduct your test. 5. Analyze your data. 6.Share your insights.
- 53. 1. Think aboutyour users. 2. Determine your research question. 3. Develop a testing plan. 4. Conduct your test. 5. Analyze your data. 6. Share your insights. REPEAT
- 54.
- 55. Tips & Tricks: Doa Dry Run (Beta Test) Just do some run throughs ahead of time with people in the office.
- 56. Tips & Tricks: TakeGood Notes It’s not just about what the participant says, but also what they do.
- 57. Tips & Tricks: PokerFace Try not to bias people.
- 58. Conclusion Basic User TestingIs Easy and Worth It
- 59.
Editor's Notes
- #3 L and J
- #4 L: What exactly is usability and usability testing? J: Why do usability testing? - Usability testing will benefit you in your work L: Learn how to conduct usability tests - specifically what’s called guerilla usability testing - as well as how to use the results to improve the design of your websites J: Then we’ll finish up with some Q&A.
- #5 Usability testing refers to evaluating a product or service by testing it with representative users. The goal is to identify any usability problems and assess participant's satisfaction with the product.
- #6 But what do we mean by “Usability”. Usability refers to how easy something is to use. It consists of a combination of 6 factors: And usability testing enables you to assess these factors in combination: Intuitive Design, Ease of Learning, Efficiency of Use, Memorability, Error frequency and severity Subjective satisfaction
- #7 If something has an intuitive design, then understanding the architecture and navigation of the site is nearly effortless for users. That means your information is structured well. Your labels are clearly-worded and all your design elements stand out and relative priority is appropriate and obvious.
- #8 This is how fast a user who has never seen the user interface before can accomplish basic tasks. When someone uses your system for the first time, even never having seen it before, they can get done what they are trying to do.There’s not a steep learning curve. The available functionality on a page makes sense even without a deep understanding of the system.
- #9 This refers to how fast an experienced user can accomplish tasks. You don’t want expert users bogged down by needless steps, excessive clicking and searching or by pop up explanations of features that they’ve used a hundred times. They can get in, accomplish their task, and get out.
- #10 Memorability means that after visiting the site, a user can remember enough to use it proficiently in future visits. To what degree can a visitor who hasn’t been to your site or used your product in a long time remember what to do when they go back?
- #11 How often users make errors while using the system, how serious the errors are, and how users recover from those errors. No system is perfect. Any site or application will have technical or usability errors creep up now and then. And no user is perfect. Everyone makes mistakes sometimes, even if just accidentally clicking the wrong thing. Making sure you have a safety net that includes things like an undo option where feasible and clear and easy-to-understand error messages with obvious calls-to-action will mitigate frustration.
- #12 Subjective satisfaction just means the user likes using the system. A streamlined and functional chore is still a chore. User experience is fundamentally emotional because we’re humans, and whether we like it or not, we perceive everything through an emotional lens. You’re doing the hard work up front to make sure that the user feels powerful and capable.
- #13 If you’re designing a system that works really well and is easy to use, but it’s not something anyone needs or wants, then, well, you’ve got another issue entirely.
- #15 It is however, a critical component of user experience. This is Dan Willis’s UX umbrella and it shows how usability is just a piece of what accounts for the overall user experience.
- #16 Aside from the fact that we’re telling you you should
- #17 L: This is how long you have to demonstrate that your website is worth staying on. According to Usability Expert Jakob Nielsen: On the Web, usability is a necessary condition for survival. If a web site is difficult to use, people leave. If the homepage fails to clearly state what a company or program offers and what users can do on the site, people leave. If users get lost on a web site, they leave. If a web site's information is hard to read or doesn't answer users' key questions, they leave. Note a pattern here? There's no such thing as a user reading a web site manual or otherwise spending much time trying to figure out an interface. There are plenty of other web sites available; leaving is the first line of defense when users encounter a difficulty.
- #18 L: In fact, do you know most frequent user behavior on websites? You guessed it...leaving.
- #19 Stunning good looks and even fast load times are not enough to make a site people actually want to use and will stay on.
- #20 L: Even if you are a usability expert, if you are not building a website for people exactly like you, you can only get so far without talking to actual users you are not your users/
- #21 L: I know some people can have an aversion when they first think about testing I can test my website by myself; why would I have to test with other users I get that but am going to talk about how usability testing really can be worth your while DISCOVER - Discover which parts of your web site are difficult for users. DIAGNOSE - Understand why users do what they do (Google Analytics) VALIDATE - Ensure that the website changes you made do in fact improve users’ experience, and don’t diminish it. Even if you are a usability expert, you can only get so far without talking to actual users Finding out why can be difficult. And while web analytics are helpful for a lot of things, understanding why your website is not performing well (whether it’s not converting users)
- #22 L: Usability testing is excellent at helping you discover which aspects of your website are difficult or problematic for your users. Again, you are not your user. What you think should be easy, frequently isn’t.
- #23 Usability testing can help you understand why users are behaving the way they are on your website While website analytics tools like Google Analytics are excellent for showing you where on your website you have a high bounce or drop-off rate, but can’t tell you why or how to fix the problem. Let’s say you have designed a Wordpress site for a retailer that has launched an online store in the last year. Analytics show that customers are bailing in the checkout process once shipping fees have been calculated. How would you aim to improve the company’s conversion rate? Without doing user testing, you don’t know whether the root of the problem is the amount of the shipping fees, concerns about the security of the order form, or confusion about the form itself. [REPLACE WITH MORE RELEVANT EXAMPLE?]
- #24 If you do diagnose a usability problem and you make some design tweaks to try to fix, usability testing enables you to validate that the changes are effective before releasing. What this all amounts to is saving time and money. Small iterations, testing, and validating is better than investing a lot of time and money only to find that your website is not doing its job and you have confused and unhappy users, low conversion rates, and therefore unhappy clients and managers. And usability testing can also give you a professional edge - you can demonstrate that your designs are evidence-based, that you are truly utilizing a user-centered design process and when it’s time to launch, your site will be more likely to be successful because you’ve identified and addressed the most important usability problems up front.
- #25 There are a variety of user research and testing methods
- #26 Of which, Guerilla usability testing is one. Guerilla usability testing is a qualitative technique, It’s also behavioral, so rather than asking people what they think they would do, you’re observing what they do. It’s really good at telling you about the underlying behaviors of your users and giving you insight into why they’re doing them.
- #27 Why it’s powerful… You do not need a large number of people In fact, research suggests that as few as five people can help you identify 85% of a website’s usability problems.
- #29 If there’s one key requirement of user testing it’s having users actually test your product or prototype. You’re looking to find users using whatever means necessary. Go where they are, like a coffee shop or the library. So guerilla usability testing is a more informal method of testing. You don’t need a lab or special equipment to do this. And you don’t need to recruit participants or schedule appointments.
- #30 When you’re testing something you’ve made, you’ll want to look for patterns of behavior. Individuals are different, of course, but it’s when you see different people all do the same thing that you know something’s up. This means you need to run your test with at least 5, and probably no more than 10 people.
- #31 So how long should this take? The breakdown for this sort of basic testing is something like this: 3 - 4 hours of prep work: finding users, writing down your testing goals, and writing out your script. Another 3 - 5 hours in the field talking to your users and watching them do the tasks. 4 - 6 hours afterwards analyzing the results and prepping presentation. So you’re really just looking at 10 - 15 hours total.
- #32 You’re going to want to use this when you’re validating your design prior to implementation. It’s really a part of the development process, so you’ll want this to be a regular part of your release cycle. You’ll want to use this on high-risk areas of your product where it’s critical to get it right or you have new concepts or interactions that your users haven’t used before.
- #33 So let’s go through some steps to show how you could do it yourself.
- #34 First, and this may come as a shock, you’ll want to think a little about your actual users. How old are they? Where might they live or work? Do during their free time? Where would be a good place to find them to talk to them (if you wanted to do such a crazy thing)?
- #35 Next, you’re going to determine your research question. So take some time and decide on one single question you want to answer with your test. Again, make sure it’s something that’s critical to your project’s workflow.
- #36 Now that you know what you want to know… your known unknown, if you will… you’ll want to start planning your test. That means writing a script so you’re being consistent with each participant. A script has 3 main parts: The intro and disclaimer The questions Your exit questions Using natural language, you’re going to write down a task that you’d like to see your users try out. Then give the user questions and tasks, such as, “I’d like you to start on this page and try to find an in-network provider.”
- #37 You’ll want to start by introducing yourself, explaining the test, and making sure the participant knows what they’re getting into.
- #38 First, you’ll start by welcoming the participant, saying who you are, and thanking them for their time. Be sure to introduce yourself in a natural way. Think through how you’d normally greet people in a situation like this and write that down. This is going to get you into the mindset of writing a script in a way that feels normal to you so you’ll have an easier time sounding normal while reading through the rest of it. Then you’ll move on to the disclaimer. So, while we’re not doing formal research, it’s still important to make sure your users fully understand exactly what they’re getting into. So be sure to tell them exactly how their information will be used in reporting. For example, if you might want to record a part of the test and replay it for stakeholders later (which I would recommend), then let them know that here. Be sure to get the user’s clear consent before you get started. Having the user sign a consent form is the best way to do this. This page shows an example consent form from Duke Web Services.
- #39 After you have their permission, you’ll want to move on to explaining what it is you’re going to be doing. So you’ll want to set up your test by providing necessary context. For example, “Today you’re going to use a website to try to find a healthcare provider.” Tell them that you’ll have some questions or tasks for them, and that this information is what you’re going to be using to make decisions about the final site or application you’re going to be making. You’ll also want to let them know that you’re testing the application and NOT them. I also like to state that the participant shouldn’t worry about hurting my feelings. I’m just the tester today and I’ll take the results back to the team and we’ll see how we can make the project better. It’s also a good idea at this point to explain that as they’re going through the tasks it’s good to think out loud so that we can get feedback about what they’re thinking as they’re doing things. Finally, if you have any incentives, go ahead and review those here. For example, free coffee or cookies or—if you’re a big spender—gift cards.
- #40 When you start asking questions, go ahead and start with some basic demographics. Serves a dual purpose: Gives you a sense of who your participants are and what sorts of users they represent. It also helps make your participants feel more comfortable because they’re the sorts of things that are easy to answer.
- #41 Finally! Writing the actual questions and tasks! When writing your questions and tasks it’s important to stay focused and brief. Remember, the point of this exercise is to quickly speak with several users to try to discern patterns. To do this, you want to ask more people fewer, more focused questions. Getting people in and out quickly is going to get you more data that you can use to compare and find the patterns you’re looking for. For your tasks, you’re simply asking the participant to show you how they would accomplish a goal. For example, “show me how you would make an appointment at a clinic on this site.”
- #42 At then end you’ll wrap up with your exit questions. These are just some basic follow-ups that serve as a catch-all. You’ve seen questions like these before at the ends of surveys. I usually just ask the following two: “Is there anything else you’d like to say about the site or application?” “Is there anything you’d like to say about the test itself?”
- #43 Now that you’ve got your script, it’s time to go out and talk to users and watch them use your site or application. Watch as they do so and take good notes. Remind them to speak out loud as they think through what they’re doing.
- #44 So here’s what you’ll need to get started. First, go to wherever your users normally are if you can. So if you think you can find people like your users at the library or at a coffee shop, go there. If you can’t, then try to find people as similar as possible and do your best. You’ll want to make sure you have your Script, a Sign, a Computer, and Something to take notes on. This is what I like to call the 2Ss, 1C, and another S. You can remember it with the handy acronym “SSCS.”
- #45 Before you get started with things, just make sure the participant is comfortable so they aren’t distracted during the test. One tip is to make sure you’re sitting with them instead of standing over them. If you have a laptop, bring a mouse so they can choose whether they want to use that or the trackpad. If you need to pause to take notes, just mention that’s what you’re doing as you do so. “Give me a couple more seconds to finish taking my notes….”
- #46 Let’s touch on reading from a script. Here’s an example script from Duke Web Services. It really is just a 2 or 3 page Word document. Using a script helps you keep your tests--and thus its results--as consistent as possible. But because you’re not going to be publishing these in a peer-reviewed journal, exactness isn’t critical. At the same time, while going off script isn’t forbidden, it does mean you’ll have to use your judgement. I don’t recommend going off-script though unless you feel like you need to track something down. Remember, you’re just trying to find patterns of behavior so you can make your project better for your users.
- #47 Once you’ve conducted 5 - 10 tests it’s time to go back and compare your results. Be sure to make note of anything you didn’t expect or any unexpected patterns of behavior. This is where you’ll be really glad you took good notes!
- #48 So when you’re analyzing your data, here are some things to look for. Difficulty clicking targets. Maybe they were too small on mobile and the user had to click them a couple of times but didn’t mention it out loud.
- #49 Participants clicking things that don’t actually do anything. Yes, this will happen.
- #50 People doing unexpected things. This is the good stuff. For links to things that aren’t in the expected workflow for my prototype I will usually create a page that lets the user know the page doesn’t exist and prompts them to use their browser back button.
- #51 Similarly, people being surprised by where a link takes them.
- #52 Difficulty locating desired information. This happens more than I care to admit and it will never cease to surprise me every time a participant fumbles around missing the clearly-labeled, giant orange button that’s right in front of them.
- #53 When presenting your findings you’ll want to talk about the behaviors you saw and the conclusions you drew of course, but nothing has an impact on those you’re presenting to like direct user quotes, images, and videos. Being able to let the user or their actions speak for themselves can be truly enlightening to reluctant stakeholders. “Here is something a real user actually said when they were trying to do this,” is really hard to argue with. Imagine if that’s 2 or 3 users. This is where you’re glad you thought ahead to this point and got specific permission for the types of reporting artifacts you’re using. Great! Now you’re done! What’s next, you ask? It’s almost 5 on Friday. Margarita time!
- #54 No! You do it again! User testing should be a key part of the development process. That means doing it as much as you have bandwidth for. Having the ability to refine your project before it’s implemented will save you time and money, which is truly important for under-resourced teams where projects can live on for years after they go live before they’re touched again (but no-one here would know anything about that).
- #55 L: Finally, let’s just go over some quick tips and tricks that will help you out as you do your testing.
- #56 Just grab some people in your own office and test your test before you get started. This will get you talking through your script and help you see where you might need to change things up. For example, this is great at revealing that you might be showing the participant where the answer to a later question is ahead of time, which would bias your test. It’ll also help you work the awkwardness out of your script reading.
- #57 We touched on this a little before, but it’s important to remember that it’s not just about what the participant says, but also what they do. Note their actions and pay attention to how they’re doing things. Often users can think they’re accomplishing their task when they aren’t. Pay attention to reactions and expressions, because behavior doesn’t always match what people say. A user could be cruising along through a flow, get caught up on a radio button that takes a few clicks to activate, but they don’t consciously think much of it.
- #58 Try not to bias people with your own reactions. “What? That GIANT ORANGE BUTTON IS RIGHT IN FRONT OF YOU!”
- #59 So, in conclusion: user testing doesn’t have to be difficult or intimidating. Just get out and get people using what you’re making and watch and learn as they do it. You will be amazed, we promise.