Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Usability Testing 101 Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide  Dmitry Nekrasovski Open T...
Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Introduction <ul><ul><li>us·a·bil·i·ty  / [ yoo -z uh- bil -i-tee]  n. </li></ul></ul><ul><ul><li>the extent to which  use...
Introduction <ul><ul><li>us·a·bil·i·ty test·ing  / [ yoo -z uh- bil -i-tee  tes -ting]  n. </li></ul></ul><ul><ul><li>the ...
Introduction Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide  http://www.youtube.com/watch?v=SFw...
Introduction <ul><ul><li>What did you notice? </li></ul></ul><ul><ul><li>Any questions? </li></ul></ul>Copyright © Open Te...
Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on  tasks  that relate to us...
Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user...
Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user...
Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user...
Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Introduction <ul><ul><li>Who is involved in usability testing? </li></ul></ul><ul><ul><li>Participant  – the user helping ...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul>Copyright © Open Text Corporation ...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of...
Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of...
Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul>Copyright © Open Text Corporation 200...
Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be qui...
Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be qui...
Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be qui...
Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be qui...
Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Hands-On Exercise <ul><ul><li>You will be handed out red, yellow, and blue cards. </li></ul></ul><ul><ul><li>Facilitator  ...
Hands-On Exercise <ul><ul><li>Goal: </li></ul></ul><ul><ul><li>Evaluate how easy it is to use the alarm function on a watc...
Hands-On Exercise <ul><ul><li>Which brand of watch did you test? </li></ul></ul><ul><ul><li>How many of the 3 tasks did yo...
Hands-On Exercise <ul><ul><li>For each brand, let’s calculate: </li></ul></ul><ul><ul><li>Completion rate  (measure of eff...
Hands-On Exercise <ul><ul><li>What was your experience like in your roles? </li></ul></ul><ul><ul><li>Any questions? </li>...
Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul>Copyright © Open Text Corporation...
Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul><ul><ul><li>Ignore “kayak problem...
Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul><ul><ul><li>Ignore “kayak” proble...
Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul><ul><ul><li>Ignore “kayak” proble...
Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Summary <ul><ul><li>What you have learned </li></ul></ul><ul><ul><li>Key principles of usability testing </li></ul></ul><u...
Thank You Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
Upcoming SlideShare
Loading in …5
×

Usability Testing 101

1,014 views
921 views

Published on

Presented at Open Text Content World 2009.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,014
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Hi &amp; welcome My name is … with … Today we’ll be talking about usability testing – what it is and how to do it If that’s not what you were expecting or are interested in, you may want to check out another session Also, if you attended this session last year, there are not a lot of differences, so I would suggest checking out another session as well First, I’d like to get a show of hands Raise your hand if you have experience with usability testing OK, let’s get started
  • Here’s the agenda for the session We’ll end with a summary and Q&amp;A
  • This is the definition of usability from the international standard ISO 9241-11 Users – anyone who interacts with the system to accomplish a goal System – could be software, hardware, website, or any combination of these Goals – are what usability is ultimately measured against – can the users do what they want to do? Usability has 3 components: effectiveness, efficiency, and satisfaction Efficiency: How quickly can users accomplish their goals? Effectiveness: How accurately and completely can users accomplish their goals? Satisfaction: How satisfied are users with their experience with the system?
  • Process – structured – not free form – not just “what do you think of this” Evaluation – not just testing, but understanding Representative users – people who are either using the system or could possibly do so in the future
  • To illustrate what all this means in practice, I’d like to show you a short video of a usability test of a job site I’d like you to pay attention to The role of the user doing the test in the video The role of the person running the test
  • Things to notice User in the video is completing a specific task – filling out a job application form She is encouraged to talk about what she is doing and how she is reacting to the system The person running the test (the facilitator) is acknowledging and asking clarifying questions
  • Goals: What the users want to do with the system Tasks: How they would accomplish this Example: Alarm clock Goal: Get up in time for an 8AM conference call Task: Set the alarm for 6AM Goals are important to the user, but can’t be tested Tasks can!
  • Usability testing can be done - During the design process After the design process is complete After is good, but during is better, because the results can be incorporated in the design
  • Two kinds of usability testing: quantitative and qualitative - Quantitative: focus on metrics – task completion, error rate, satisfaction Qualitative: focus on behaviour – reactions, insights, think aloud comments
  • How many is a few? Opinions differ! Jakob Nielsen: 3-5 users sufficient for qualitative testing; 10-20 for quantitative testing depending on desired confidence in results Who are representative users: People whose goals and level of expertise with the system match those of the target audience This usually doesn’t include the designers or developers of the system!
  • What you need to do What you need to keep in mind
  • The facilitator is also sometimes referred to as the moderator
  • First, you want to determine what you want to get out of the test Understand users’ pain points Compare alternative designs Get metrics like completion times and success rates The answer will help you determine how to structure the test and how many people to test with
  • Usability tests are not complicated to run, but they do require a lot of planning in advance This includes -Finding and scheduling participants Developing appropriate tasks Creating test instructions – you need to make sure that these are clear, concise, and simple Hardware and software setup
  • The day before the test, try going through it yourself You can also run a “pilot test” with another person involved in the project Make sure that - You or your pilot tester can do the test in the time allotted - All the functions you will be testing are working and accessible from your test machine
  • Remind them to think out loud. Tell me what you see on this screen/page. Tell me what you’re going to do next. Tell me what you expect to happen when you click this button.
  • During the test, avoid hinting to participants what they should do next If they say “I’m not sure what to do next”, ask “What do you think you should do next?” Similarly, avoid influencing participants’ impressions of the system Instead, ask open ended questions like “What do you like/dislike about this aspect of the system?” You’re looking for specifics, not necessarily because they are important, but because they help you understand what the user is reacting to
  • As a facilitator, during the test you need to be patient, reassuring, and diplomatic It’s easy for test participants to blame themselves and feel bad when they run into problems Make it clear to participants that You’re testing the system, not them You know that they are not stupid
  • As an observer, you are not involved in the actual running of the usability test All you have to do is listen, pay close attention to the user’s interaction with the system, and take detailed notes Remember to be quiet and avoid distracting users!
  • As an observer, here are the types of things you are looking for: Does the user get the purpose of screens/features? Can they find their way around the interface? Does the terminology make sense to them? “ Head slappers” – sudden insights because of what the participant does or doesn’t do “ Shocks” – behaviour that challenges your assumptions Passion – what does the user love or hate about the system – important to note this and bring this back to the design team
  • Don’t pay too much attention to opinions users express during the test These are unreliable – people may exaggerate them because they think that’s what you’re looking for Instead, pay attention to what they do, and how they explain what they do
  • As an observer, you may feel disappointed or even angry if the users’ behaviour doesn’t meet your expectations You may also be tempted to jump to conclusions after the first participant Don’t focus on this – focus on keeping an open mind to what’s going on during the test Any questions so far?
  • Does anyone not have a card? Does anyone not have a group of 3? If you don’t, feel free to join another group as an observer Let me know if your group doesn’t have a pen
  • Take about 5 minutes to do the task. Facilitator: Once your participant is done, ask them to answer the post-test question Observer: Write down the number of tasks completed and the participant’s answer.
  • Let’s go around the room. For each group, I’m looking for: Which brand of watch did you test? How many of the 3 tasks did your participant complete? Your participant’s answer to the post-test question
  • Recall that usability has 3 components: effectiveness, efficiency, and satisfaction. Task completion rate is a measure of effectiveness. Subjective questions like the one you asked your participant help measure satisfaction. So we can calculate effectiveness and satisfaction ratings for each brand.
  • Any observations about your experience in your role?
  • Now I’d like to talk briefly about some best practices for interpreting test results
  • Review your notes with the project team as soon as possible after a round of tests Focus on the important things you learned – these tend to be fairly obvious Identify the problems first – you can figure out how to solve them later
  • Kayak problems – where the user encounters an issue, but quickly gets back on track Like someone who falls off a kayak, but gets back on immediately Ignore these in favour of issues that users couldn’t deal with easily
  • Resist the impulse to solve problems found in usability testing by adding explanations and instructions - Even if the participants suggest that this is the way to solve a problem, this may not necessarily be a good idea - Often, the real solution is to remove distractions and confusing elements
  • Address the “low-hanging fruit” first: issues where Both the problem and the solution are obvious, or Effort required to implement is minimal and the impact is highly visible
  • Thank you - I’ll be happy to take any questions Thanks again, and don’t forget to visit the Innovation Lab, where you can participate in usability tests and focus groups run by the Open Text UXD team! If you’re specifically interested in user adoption, there’s a focus group on that topic tomorrow at noon. Come by the Innovation Lab to sign up.
  • Usability Testing 101

    1. 1. Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    2. 2. Usability Testing 101 Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide Dmitry Nekrasovski Open Text User Experience Design
    3. 3. Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    4. 4. Introduction <ul><ul><li>us·a·bil·i·ty / [ yoo -z uh- bil -i-tee] n. </li></ul></ul><ul><ul><li>the extent to which users can </li></ul></ul><ul><ul><li>use a system to achieve their goals </li></ul></ul><ul><ul><li>with efficiency , effectiveness , and satisfaction </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    5. 5. Introduction <ul><ul><li>us·a·bil·i·ty test·ing / [ yoo -z uh- bil -i-tee tes -ting] n. </li></ul></ul><ul><ul><li>the process of evaluating a system’s usability </li></ul></ul><ul><ul><li>with representative users </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    6. 6. Introduction Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide http://www.youtube.com/watch?v=SFwU_rvMBaE&
    7. 7. Introduction <ul><ul><li>What did you notice? </li></ul></ul><ul><ul><li>Any questions? </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    8. 8. Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user goals </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    9. 9. Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user goals </li></ul></ul><ul><ul><li>Can be done at any time (but the earlier, the better!) </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    10. 10. Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user goals </li></ul></ul><ul><ul><li>Can be done at any time </li></ul></ul><ul><ul><li>Can be quantitative or qualitative </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    11. 11. Introduction <ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Based on tasks that relate to user goals </li></ul></ul><ul><ul><li>Can be done at any time </li></ul></ul><ul><ul><li>Can be qualitative or quantitative </li></ul></ul><ul><ul><li>Requires only a few representative users </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    12. 12. Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    13. 13. Introduction <ul><ul><li>Who is involved in usability testing? </li></ul></ul><ul><ul><li>Participant – the user helping to test the system </li></ul></ul><ul><ul><li>Facilitator – the person conducting the test </li></ul></ul><ul><ul><li>Observer – may take notes or just watch </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    14. 14. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    15. 15. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of the test </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    16. 16. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of the test </li></ul></ul><ul><ul><li>Plan all aspects of the test </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    17. 17. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of the test </li></ul></ul><ul><ul><li>Plan all aspects of the test </li></ul></ul><ul><ul><li>Try it yourself first </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    18. 18. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of the test </li></ul></ul><ul><ul><li>Plan all aspects of the test </li></ul></ul><ul><ul><li>Try it yourself first </li></ul></ul><ul><ul><li>Remind participants to think out loud </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    19. 19. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of the test </li></ul></ul><ul><ul><li>Plan all aspects of the test </li></ul></ul><ul><ul><li>Try it yourself first </li></ul></ul><ul><ul><li>Remind participants to think out loud </li></ul></ul><ul><ul><li>Don’t hint - probe </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    20. 20. Doing usability testing <ul><ul><li>What to do if you’re the facilitator </li></ul></ul><ul><ul><li>Determine the goals of the test </li></ul></ul><ul><ul><li>Plan all aspects of the test </li></ul></ul><ul><ul><li>Try it yourself first </li></ul></ul><ul><ul><li>Remind participants to think out loud </li></ul></ul><ul><ul><li>Don’t hint - probe </li></ul></ul><ul><ul><li>Show empathy </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    21. 21. Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    22. 22. Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be quiet </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    23. 23. Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be quiet </li></ul></ul><ul><ul><li>Know what to look for </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    24. 24. Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be quiet </li></ul></ul><ul><ul><li>Know what to look for </li></ul></ul><ul><ul><li>Pay attention to actions and explanations, not opinions </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    25. 25. Doing usability testing <ul><ul><li>What to do if you’re the observer </li></ul></ul><ul><ul><li>Listen, watch, and be quiet </li></ul></ul><ul><ul><li>Know what to look for </li></ul></ul><ul><ul><li>Pay attention to actions and explanations, not opinions </li></ul></ul><ul><ul><li>Keep an open mind </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    26. 26. Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    27. 27. Hands-On Exercise <ul><ul><li>You will be handed out red, yellow, and blue cards. </li></ul></ul><ul><ul><li>Facilitator Observer Participant </li></ul></ul><ul><ul><li>Break into groups of 3 so that each group has 1 card of each color </li></ul></ul><ul><ul><li>Once you’re in a group, go over the instructions on your card </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    28. 28. Hands-On Exercise <ul><ul><li>Goal: </li></ul></ul><ul><ul><li>Evaluate how easy it is to use the alarm function on a watch. </li></ul></ul><ul><li>Tasks: </li></ul><ul><li>Set the watch time to 5:17 PM. </li></ul><ul><li>Set the alarm so that it goes off in 2 minutes. </li></ul><ul><li>Turn off the alarm once it chimes. </li></ul><ul><li>Post-test question: </li></ul><ul><li>“ On a scale from 1 to 5, where 1 is very difficult and 5 is very </li></ul><ul><li>easy, how easy did you find using the watch?” </li></ul><ul><li>Write down # of tasks completed and participant’s answer </li></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    29. 29. Hands-On Exercise <ul><ul><li>Which brand of watch did you test? </li></ul></ul><ul><ul><li>How many of the 3 tasks did your participant complete? </li></ul></ul><ul><ul><li>What was your participant’s answer to the post-test question? </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    30. 30. Hands-On Exercise <ul><ul><li>For each brand, let’s calculate: </li></ul></ul><ul><ul><li>Completion rate (measure of effectiveness) </li></ul></ul><ul><ul><li>Perceived ease of use rating (measure of satisfaction) </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    31. 31. Hands-On Exercise <ul><ul><li>What was your experience like in your roles? </li></ul></ul><ul><ul><li>Any questions? </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    32. 32. Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    33. 33. Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    34. 34. Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul><ul><ul><li>Ignore “kayak problems” </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    35. 35. Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul><ul><ul><li>Ignore “kayak” problems </li></ul></ul><ul><ul><li>Resist the impulse to add things </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    36. 36. Interpreting test results <ul><ul><li>Review findings as soon as possible </li></ul></ul><ul><ul><li>Ignore “kayak” problems </li></ul></ul><ul><ul><li>Resist the impulse to add things </li></ul></ul><ul><ul><li>Address the “low-hanging fruit” first </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    37. 37. Agenda Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    38. 38. Summary <ul><ul><li>What you have learned </li></ul></ul><ul><ul><li>Key principles of usability testing </li></ul></ul><ul><ul><li>Importance of goals and tasks </li></ul></ul><ul><ul><li>The roles of the facilitator and the observer </li></ul></ul><ul><ul><li>How to conduct a simple usability test </li></ul></ul><ul><ul><li>How to interpret test results </li></ul></ul>Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide
    39. 39. Thank You Copyright © Open Text Corporation 2008 - 2009. All rights reserved. Slide

    ×