Your SlideShare is downloading. ×
Usability testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Usability testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Usability Testing Chapter 6
  • 2. Reliability
    • Can you repeat the test?
  • 3. Reliability
    • Huge difference between test subjects
      • Therefore, only 2 subjects is not a good reliability check
    • Significance levels (e.g., p = .20)
    • Confidence levels
      • When should an usability problem be fixed?
    • Error rates have the highest variability
      • Therefore, more test subjects are needed
  • 4. Validity
    • Does this test reflect what usability issues you want to test?
    • Reliability measured with stats but validity requires methodological understanding
    • Typical problems
      • Wrong users or wrong tasks
      • Confounding effects – text based to GUI system
  • 5. Test Plans
    • What’s the goal of the test?
    • What is it you are trying to prove or show?
    • Pilot tests – show the problem areas of the test. E.g., the instructions
  • 6. Test Plans
    • The goal of the test?
    • Where and were will the test take place?
    • How long is each test session expected to take?
    • What computer support will be needed for the test?
    • What software needs to be ready for the test?
    • Who will serve as experimenters for the test?
    • How many test users are needed?
  • 7. Getting Test Users
    • Test subjects (users) should be representative of who will use the system.
    • Sales (“demoability”) – is it easy to show
    • Asking managers to choose subjects
      • They will pick the best or the worse
  • 8.
    • Formative evaluation
      • Help improve the interface as part of an iterative design process
    • Summative
      • Assessing the overall quality of an interface
  • 9. Novice and expert users
    • Most tests should test novice users
    • If possible, test for expert users. But test separately from novice users.
    • What are the effects of training or not training before a test?
  • 10. Between-Subject test
    • Simplest and most valid
    • Different tests subject using different systems
    • Possible problems:
      • Individual differences
      • Assignment to test groups (volunteer early or late)
  • 11. Within-Subject test
    • All users are tested on all systems
    • Problem: no longer a set of novice users when testing a secondary system
  • 12. Choosing experimenters
    • It’s better to get an usability person to conduct the test but you, the developer, can also run a test
    • In fact, it’s good to know the system well
    • From Usability Engineering (Nielsen)
      • “ It is possible for computer scientistws to learn user test methods and apply them with good results.”
    • However, designers try to explain away problems.
  • 13. Ethical aspect of testing human beings
    • Users should feel as comfortable as possible
    • Tell the test subjects:
      • No information will be revealed
      • Explain the testing environment. E.g., computer, etc.
    • The tester should not allow observers because they tend to influence results.
    • The tester should not interfere with the user. Let the user find the solution themselves
    • Emphasize that it is the system that is being tested, not the user
  • 14. Test tasks
    • Based on a task analysis. What is it the user needs to do.
    • Tasks should be small enough to be completed in time that the experiment takes.
    • Test tasks should be in writing. The user may refer to the steps.
  • 15. Stages of a Test
    • 1. Prep – is everything in working order?
    • 2. Intro - tell the user the purpose of the test.
    • 3. Test
    • 4. Debrief
  • 16. Stages of a Test
    • Intro - tell the user the purpose of the test.
    • A reminder that the test is confidential and should not be discussed with others
    • A statement that participation is the test is voluntary and the user may stop at any time
    • At FIT a human test subject document needs to be signed by the tester and returned to the university.
  • 17. Performance Measurement
    • Has the usability goals been met? (see page 194)
    • Test a set of tasks
    • The data collected: time to perform and error rate
    • Goals are abstract, so break them down.
    • Make sure you know the start and finish of a measured test
  • 18. Thinking aloud
    • “ verbalizing their thoughts”
    • Shows how users interpret each interface item
    • Users performed 9 % better using thinking aloud.
    • Constructive interaction (co-discovery learning) and coaching methods.
  • 19. Usability Labs
    • Is there really a need?
    • Why videotape? Impact analysis – you look after taping for a known problem.
    • Convince the manager and developers
    • Usability kiosks