Your SlideShare is downloading. ×
Usability testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Usability testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Usability Testing Chapter 6
  • 2. Reliability
    • Can you repeat the test?
  • 3. Reliability
    • Huge difference between test subjects
      • Therefore, only 2 subjects is not a good reliability check
    • Significance levels (e.g., p = .20)
    • Confidence levels
      • When should an usability problem be fixed?
    • Error rates have the highest variability
      • Therefore, more test subjects are needed
  • 4. Validity
    • Does this test reflect what usability issues you want to test?
    • Reliability measured with stats but validity requires methodological understanding
    • Typical problems
      • Wrong users or wrong tasks
      • Confounding effects – text based to GUI system
  • 5. Test Plans
    • What’s the goal of the test?
    • What is it you are trying to prove or show?
    • Pilot tests – show the problem areas of the test. E.g., the instructions
  • 6. Test Plans
    • The goal of the test?
    • Where and were will the test take place?
    • How long is each test session expected to take?
    • What computer support will be needed for the test?
    • What software needs to be ready for the test?
    • Who will serve as experimenters for the test?
    • How many test users are needed?
  • 7. Getting Test Users
    • Test subjects (users) should be representative of who will use the system.
    • Sales (“demoability”) – is it easy to show
    • Asking managers to choose subjects
      • They will pick the best or the worse
  • 8.
    • Formative evaluation
      • Help improve the interface as part of an iterative design process
    • Summative
      • Assessing the overall quality of an interface
  • 9. Novice and expert users
    • Most tests should test novice users
    • If possible, test for expert users. But test separately from novice users.
    • What are the effects of training or not training before a test?
  • 10. Between-Subject test
    • Simplest and most valid
    • Different tests subject using different systems
    • Possible problems:
      • Individual differences
      • Assignment to test groups (volunteer early or late)
  • 11. Within-Subject test
    • All users are tested on all systems
    • Problem: no longer a set of novice users when testing a secondary system
  • 12. Choosing experimenters
    • It’s better to get an usability person to conduct the test but you, the developer, can also run a test
    • In fact, it’s good to know the system well
    • From Usability Engineering (Nielsen)
      • “ It is possible for computer scientistws to learn user test methods and apply them with good results.”
    • However, designers try to explain away problems.
  • 13. Ethical aspect of testing human beings
    • Users should feel as comfortable as possible
    • Tell the test subjects:
      • No information will be revealed
      • Explain the testing environment. E.g., computer, etc.
    • The tester should not allow observers because they tend to influence results.
    • The tester should not interfere with the user. Let the user find the solution themselves
    • Emphasize that it is the system that is being tested, not the user
  • 14. Test tasks
    • Based on a task analysis. What is it the user needs to do.
    • Tasks should be small enough to be completed in time that the experiment takes.
    • Test tasks should be in writing. The user may refer to the steps.
  • 15. Stages of a Test
    • 1. Prep – is everything in working order?
    • 2. Intro - tell the user the purpose of the test.
    • 3. Test
    • 4. Debrief
  • 16. Stages of a Test
    • Intro - tell the user the purpose of the test.
    • A reminder that the test is confidential and should not be discussed with others
    • A statement that participation is the test is voluntary and the user may stop at any time
    • At FIT a human test subject document needs to be signed by the tester and returned to the university.
  • 17. Performance Measurement
    • Has the usability goals been met? (see page 194)
    • Test a set of tasks
    • The data collected: time to perform and error rate
    • Goals are abstract, so break them down.
    • Make sure you know the start and finish of a measured test
  • 18. Thinking aloud
    • “ verbalizing their thoughts”
    • Shows how users interpret each interface item
    • Users performed 9 % better using thinking aloud.
    • Constructive interaction (co-discovery learning) and coaching methods.
  • 19. Usability Labs
    • Is there really a need?
    • Why videotape? Impact analysis – you look after taping for a known problem.
    • Convince the manager and developers
    • Usability kiosks