Usability testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Usability testing

  1. 1. Usability Testing Chapter 6
  2. 2. Reliability <ul><li>Can you repeat the test? </li></ul>
  3. 3. Reliability <ul><li>Huge difference between test subjects </li></ul><ul><ul><li>Therefore, only 2 subjects is not a good reliability check </li></ul></ul><ul><li>Significance levels (e.g., p = .20) </li></ul><ul><li>Confidence levels </li></ul><ul><ul><li>When should an usability problem be fixed? </li></ul></ul><ul><li>Error rates have the highest variability </li></ul><ul><ul><li>Therefore, more test subjects are needed </li></ul></ul>
  4. 4. Validity <ul><li>Does this test reflect what usability issues you want to test? </li></ul><ul><li>Reliability measured with stats but validity requires methodological understanding </li></ul><ul><li>Typical problems </li></ul><ul><ul><li>Wrong users or wrong tasks </li></ul></ul><ul><ul><li>Confounding effects – text based to GUI system </li></ul></ul>
  5. 5. Test Plans <ul><li>What’s the goal of the test? </li></ul><ul><li>What is it you are trying to prove or show? </li></ul><ul><li>Pilot tests – show the problem areas of the test. E.g., the instructions </li></ul>
  6. 6. Test Plans <ul><li>The goal of the test? </li></ul><ul><li>Where and were will the test take place? </li></ul><ul><li>How long is each test session expected to take? </li></ul><ul><li>What computer support will be needed for the test? </li></ul><ul><li>What software needs to be ready for the test? </li></ul><ul><li>Who will serve as experimenters for the test? </li></ul><ul><li>How many test users are needed? </li></ul><ul><li>… </li></ul>
  7. 7. Getting Test Users <ul><li>Test subjects (users) should be representative of who will use the system. </li></ul><ul><li>Sales (“demoability”) – is it easy to show </li></ul><ul><li>Asking managers to choose subjects </li></ul><ul><ul><li>They will pick the best or the worse </li></ul></ul>
  8. 8. <ul><li>Formative evaluation </li></ul><ul><ul><li>Help improve the interface as part of an iterative design process </li></ul></ul><ul><li>Summative </li></ul><ul><ul><li>Assessing the overall quality of an interface </li></ul></ul>
  9. 9. Novice and expert users <ul><li>Most tests should test novice users </li></ul><ul><li>If possible, test for expert users. But test separately from novice users. </li></ul><ul><li>What are the effects of training or not training before a test? </li></ul>
  10. 10. Between-Subject test <ul><li>Simplest and most valid </li></ul><ul><li>Different tests subject using different systems </li></ul><ul><li>Possible problems: </li></ul><ul><ul><li>Individual differences </li></ul></ul><ul><ul><li>Assignment to test groups (volunteer early or late) </li></ul></ul>
  11. 11. Within-Subject test <ul><li>All users are tested on all systems </li></ul><ul><li>Problem: no longer a set of novice users when testing a secondary system </li></ul>
  12. 12. Choosing experimenters <ul><li>It’s better to get an usability person to conduct the test but you, the developer, can also run a test </li></ul><ul><li>In fact, it’s good to know the system well </li></ul><ul><li>From Usability Engineering (Nielsen) </li></ul><ul><ul><li>“ It is possible for computer scientistws to learn user test methods and apply them with good results.” </li></ul></ul><ul><li>However, designers try to explain away problems. </li></ul>
  13. 13. Ethical aspect of testing human beings <ul><li>Users should feel as comfortable as possible </li></ul><ul><li>Tell the test subjects: </li></ul><ul><ul><li>No information will be revealed </li></ul></ul><ul><ul><li>Explain the testing environment. E.g., computer, etc. </li></ul></ul><ul><li>The tester should not allow observers because they tend to influence results. </li></ul><ul><li>The tester should not interfere with the user. Let the user find the solution themselves </li></ul><ul><li>Emphasize that it is the system that is being tested, not the user </li></ul>
  14. 14. Test tasks <ul><li>Based on a task analysis. What is it the user needs to do. </li></ul><ul><li>Tasks should be small enough to be completed in time that the experiment takes. </li></ul><ul><li>Test tasks should be in writing. The user may refer to the steps. </li></ul>
  15. 15. Stages of a Test <ul><li>1. Prep – is everything in working order? </li></ul><ul><li>2. Intro - tell the user the purpose of the test. </li></ul><ul><li>3. Test </li></ul><ul><li>4. Debrief </li></ul>
  16. 16. Stages of a Test <ul><li>Intro - tell the user the purpose of the test. </li></ul><ul><li>A reminder that the test is confidential and should not be discussed with others </li></ul><ul><li>A statement that participation is the test is voluntary and the user may stop at any time </li></ul><ul><li>At FIT a human test subject document needs to be signed by the tester and returned to the university. </li></ul>
  17. 17. Performance Measurement <ul><li>Has the usability goals been met? (see page 194) </li></ul><ul><li>Test a set of tasks </li></ul><ul><li>The data collected: time to perform and error rate </li></ul><ul><li>Goals are abstract, so break them down. </li></ul><ul><li>Make sure you know the start and finish of a measured test </li></ul>
  18. 18. Thinking aloud <ul><li>“ verbalizing their thoughts” </li></ul><ul><li>Shows how users interpret each interface item </li></ul><ul><li>Users performed 9 % better using thinking aloud. </li></ul><ul><li>Constructive interaction (co-discovery learning) and coaching methods. </li></ul>
  19. 19. Usability Labs <ul><li>Is there really a need? </li></ul><ul><li>Why videotape? Impact analysis – you look after taping for a known problem. </li></ul><ul><li>Convince the manager and developers </li></ul><ul><li>Usability kiosks </li></ul>