Usabilty Testing Methods

2,335 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,335
On SlideShare
0
From Embeds
0
Number of Embeds
1,315
Actions
Shares
0
Downloads
34
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Usabilty Testing Methods

  1. 1. User Interface Evaluation Usability Testing Methods http://jthom.best.vwh.net/usability/ http://www.cs.umd.edu/~zzj/UsabilityHome.html
  2. 2. Usability Testing Methods <ul><li>Caring out experiments to find out specific information about a design and/or product. </li></ul><ul><li>Basis comes from experimental psychology. </li></ul><ul><li>Uses statistical data methods </li></ul><ul><ul><li>Quantitative and Qualitative </li></ul></ul>
  3. 3. Usability Testing Methods <ul><li>During usability testing, users work on specific tasks using the interface/product and evaluators use the results to evaluate and modify the interface/product. </li></ul><ul><li>Widely used in practice, but not appropriately used. </li></ul><ul><li>Often abused by developers that consider themselves to be usability experts. </li></ul><ul><li>Can be very expensive and time consuming. </li></ul>
  4. 4. Usability Testing Methods <ul><li>Performance Measurement </li></ul><ul><li>Thinking-aloud Protocol </li></ul><ul><li>Question-asking Protocol </li></ul><ul><li>Coaching Method </li></ul>
  5. 5. Usability Testing Methods <ul><li>Co-discovery Learning </li></ul><ul><li>Teaching Method </li></ul><ul><li>Retrospective Testing </li></ul><ul><li>Remote Testing </li></ul>
  6. 6. Performance Measurement <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 6. </li></ul></ul>
  7. 7. Performance Measurement <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: Yes </li></ul></ul><ul><ul><li>Satisfaction: No </li></ul></ul><ul><li>Quantitative Data is collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  8. 8. Performance Measurement <ul><li>What is it? </li></ul><ul><li>Used to collect quantitative data. </li></ul><ul><li>Typically, you will be looking for benchmark data. </li></ul><ul><li>Objectives MUST be quantifiable </li></ul><ul><ul><li>75% of users shall be able to complete the basic task in less than 30 minutes. </li></ul></ul>
  9. 9. Performance Measurement <ul><li>How can I do it? </li></ul><ul><li>Define the goals that you expect users to perform </li></ul>
  10. 10. Performance Measurement <ul><li>How can I do it? </li></ul><ul><li>Quantify the goals </li></ul><ul><ul><li>The time users take to complete a specific task. </li></ul></ul><ul><ul><li>The Ratio between successful interactions and errors. </li></ul></ul><ul><ul><li>The time spent recovering from errors. </li></ul></ul><ul><ul><li>The number of user errors. </li></ul></ul><ul><ul><li>The number of commands or other features that were never used by the user. </li></ul></ul><ul><ul><li>The number of system features the user can remember during a debriefing after the test. </li></ul></ul><ul><ul><li>The proportion of users who say that they would prefer using the system over some specified competitor. </li></ul></ul>
  11. 11. Performance Measurement <ul><li>How can I do it? </li></ul><ul><li>Get participants for the experiments </li></ul><ul><li>Conduct very controlled experiments </li></ul><ul><ul><li>All variables must remain consistent across users </li></ul></ul><ul><li>http://www.cs.umd.edu/~zzj/PerfMeas.htm </li></ul>
  12. 12. Performance Measurement <ul><li>Problems With Performance Measurement </li></ul><ul><li>No qualitative data. </li></ul>
  13. 13. Thinking-aloud Protocol <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 4. </li></ul></ul>
  14. 14. Thinking-aloud Protocol <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: No </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data is NOT collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  15. 15. Thinking-aloud Protocol <ul><li>What is it? </li></ul><ul><li>Technique where the participant is asked to vocalize his or her thoughts, feelings, and opinions while interacting with the product. . </li></ul>
  16. 16. Thinking-aloud Protocol <ul><li>How can I do it? </li></ul><ul><li>Select the participants, who will be involved? </li></ul><ul><li>Select the tasks and design scenarios. </li></ul><ul><li>Ask the participant to perform a task using the software. </li></ul>
  17. 17. Thinking-aloud Protocol <ul><li>How can I do it? </li></ul><ul><li>During the task, ask the user to vocalize </li></ul><ul><ul><li>Thoughts, opinions, feelings, etc. </li></ul></ul><ul><li>http://www.cs.umd.edu/~zzj/ThinkAlo.htm </li></ul>
  18. 18. Thinking-aloud Protocol <ul><li>Problem With Thinking-Aloud Protocol </li></ul><ul><li>Cognitive Overload </li></ul><ul><ul><li>Can you walk & chew gum at the same time? </li></ul></ul><ul><ul><li>Asking the participants to do too much. </li></ul></ul>
  19. 19. Question-asking Protocol <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 4. </li></ul></ul>
  20. 20. Question-asking Protocol <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: No </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data is NOT collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  21. 21. Question-asking Protocol <ul><li>What is it? </li></ul><ul><li>Similar to Thinking-aloud protocol. </li></ul><ul><li>Instead of participant saying what they are thinking, the evaluator prompts the participant with questions while using the system. </li></ul>
  22. 22. Question-asking Protocol <ul><li>How can I do it? </li></ul><ul><li>Select the participants, who will be involved? </li></ul><ul><li>Select the tasks and design scenarios. </li></ul><ul><li>Ask the participant to perform a task using the software. </li></ul>
  23. 23. Question-asking Protocol <ul><li>How can I do it? </li></ul><ul><li>During the task, ask the user to questions about the product </li></ul><ul><ul><li>Thoughts, opinions, feelings, etc. </li></ul></ul><ul><li>http://www.cs.umd.edu/~zzj/Question.htm </li></ul>
  24. 24. Question-asking Protocol <ul><li>Problem With Thinking-Aloud Protocol </li></ul><ul><li>Cognitive Overload++ </li></ul><ul><ul><li>Can you walk, chew gum & talk at the same time? </li></ul></ul><ul><ul><li>Asking the participants to do too much. </li></ul></ul><ul><ul><li>Added pressure when the evaluator asks questions. </li></ul></ul><ul><ul><li>Can be frustrating on novice users. </li></ul></ul>
  25. 25. Coaching Method <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 4. </li></ul></ul>
  26. 26. Coaching Method <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: No </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data is NOT collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  27. 27. Coaching Method <ul><li>What is it? </li></ul><ul><li>A system expert sits with the participant and acts as a coach. </li></ul><ul><li>Expert answers the participant’s questions. </li></ul><ul><li>The evaluator observes their interaction. </li></ul>
  28. 28. Coaching Method <ul><li>How can I do it? </li></ul><ul><li>Select the participants, who will be involved? </li></ul><ul><li>Select the tasks and design scenarios. </li></ul><ul><li>Ask the participant to perform a task using the software in the presence of a coach/expert. </li></ul>
  29. 29. Coaching Method <ul><li>How can I do it? </li></ul><ul><li>During the task, the user will ask the expert questions about the product </li></ul><ul><li>http://www.cs.umd.edu/~zzj/Coaching.htm </li></ul>
  30. 30. Coaching Method <ul><li>Problem With Coaching Method </li></ul><ul><li>In reality, there will not be a coach present. </li></ul><ul><li>This is good for creating a coaching system, but not for evaluating the interface. </li></ul>
  31. 31. Co-Discovery Learning <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 6. </li></ul></ul>
  32. 32. Co-Discovery Learning <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: No </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data is NOT collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  33. 33. Co-Discovery Learning <ul><li>What is it? </li></ul><ul><li>Two test users attempt to perform tasks together while being observed. </li></ul><ul><li>They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product. </li></ul><ul><li>They are encouraged to explain what they are thinking about while working on the tasks. </li></ul><ul><ul><li>Thinking Aloud, but more natural because of partner. </li></ul></ul>
  34. 34. Co-Discovery Learning <ul><li>How can I do it? </li></ul><ul><li>Select the participants, who will be involved? </li></ul><ul><li>Select the tasks and design scenarios. </li></ul><ul><li>Ask the participants to perform a task using the software. </li></ul>
  35. 35. Co-Discovery Learning <ul><li>How can I do it? </li></ul><ul><li>During the task, the users will help each other and voice their thoughts by talking to each other. </li></ul><ul><li>http://www.cs.umd.edu/~zzj/Codiscov.htm </li></ul>
  36. 36. Co-Discovery Learning <ul><li>Problem With Co-Discovery Learning </li></ul><ul><li>Neither is an expert </li></ul><ul><ul><li>The blind leading the blind. </li></ul></ul>
  37. 37. Teaching Method <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 4. </li></ul></ul>
  38. 38. Teaching Method <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: No </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data is NOT collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  39. 39. Teaching Method <ul><li>What is it? </li></ul><ul><li>You have 1 participant use the system. </li></ul><ul><li>Ask the participant to teach a new novice participant how to use the system. </li></ul>
  40. 40. Teaching Method <ul><li>How can I do it? </li></ul><ul><li>Select the participants, who will be involved? </li></ul><ul><li>Select the tasks and design scenarios. </li></ul><ul><li>Ask the 1 st participant to perform a task using the software. </li></ul><ul><li>Ask the 1 st participant to teach a new participant. </li></ul>
  41. 41. Teaching Method <ul><li>How can I do it? </li></ul><ul><li>Observe their interactions. </li></ul><ul><li>http://www.cs.umd.edu/~zzj/Teaching.htm </li></ul>
  42. 42. Teaching Method <ul><li>Problem With Teaching Method </li></ul><ul><li>Neither is an expert </li></ul><ul><ul><li>The blind leading the blind. </li></ul></ul><ul><ul><li>Possible to discover some interesting things about the learn-ability of your interfaces. </li></ul></ul>
  43. 43. Retrospective Testing <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 4. </li></ul></ul>
  44. 44. Retrospective Testing <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: Yes </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data can be collected. </li></ul><ul><li>Can NOT be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  45. 45. Retrospective Testing <ul><li>What is it? </li></ul><ul><li>A videotape of the session is observed by the usability expert and the participants. </li></ul>
  46. 46. Retrospective Testing <ul><li>How can I do it? </li></ul><ul><li>Select the participants, who will be involved? </li></ul><ul><li>Select the tasks and design scenarios. </li></ul><ul><li>Use one of the usability testing methods that we have discussed. </li></ul><ul><li>Videotape the session. </li></ul>
  47. 47. Retrospective Testing <ul><li>How can I do it? </li></ul><ul><li>Review the videotape with the users. </li></ul><ul><li>http://www.cs.umd.edu/~zzj/Retrospe.htm </li></ul>
  48. 48. Retrospective Testing <ul><li>Problem With Retrospective Testing </li></ul><ul><li>Extremely time consuming! </li></ul>
  49. 49. Remote Testing <ul><li>Applicable Stages: </li></ul><ul><ul><li>Design, Code, Test & Deployment </li></ul></ul><ul><li>Personnel </li></ul><ul><ul><li>Usability Experts, approximately 1. </li></ul></ul><ul><ul><li>Developers, 0. </li></ul></ul><ul><ul><li>Users, 5. </li></ul></ul>
  50. 50. Remote Testing <ul><li>Usability Issues Covered </li></ul><ul><ul><li>Effectiveness: Yes </li></ul></ul><ul><ul><li>Efficiency: Yes </li></ul></ul><ul><ul><li>Satisfaction: Yes </li></ul></ul><ul><li>Quantitative Data can be collected. </li></ul><ul><li>Can be conducted remotely. </li></ul><ul><li>Can be used on any system. </li></ul>
  51. 51. Remote Testing <ul><li>What is it? </li></ul><ul><li>The participants are separated from the evaluators. </li></ul><ul><li>No formal observation. </li></ul><ul><li>No usability lab. </li></ul>
  52. 52. Remote Testing <ul><li>How can I do it? </li></ul><ul><li>Give the product/software to participants. </li></ul><ul><li>Collect information about how they use your software/product. </li></ul><ul><li>Methods </li></ul><ul><ul><li>Same-Time Different Place </li></ul></ul><ul><ul><li>Different-Time Different Place </li></ul></ul>
  53. 53. Remote Testing <ul><li>How can I do it? </li></ul><ul><li>Lotus Video Cam, Look@Me, SnagIt </li></ul><ul><li>Usability Logger </li></ul><ul><ul><li>http://www.usabletools.com/ </li></ul></ul><ul><li>Journaled Sessions </li></ul><ul><li>http://www.cs.umd.edu/~zzj/Retrospe.htm </li></ul>
  54. 54. Remote Testing <ul><li>Problem With Remote Testing </li></ul><ul><li>The evaluator is not there. </li></ul><ul><li>Can’t observe facial expressions. </li></ul><ul><li>Great for Web based systems. </li></ul>
  55. 55. Usability Testing Methods <ul><li>Select the method that works best for you. </li></ul><ul><li>Select the method that fits your implementation. </li></ul><ul><li>Be thorough during your experiments. </li></ul><ul><li>The more data, the better. </li></ul>
  56. 56. Usability Testing Methods <ul><li>Hawthorne Effect </li></ul><ul><ul><li>The tendency for people to improve their performance after any change when they know their performance is being studied. </li></ul></ul>

×