This document summarizes a presentation on evaluating the usability of forms. It discusses how the presenters adapted usability evaluation methods developed for computers to evaluate non-computer products like tax forms. The key points are: the presenters used a methodology called MUSiC to measure the usability of tax forms; a video clip showed two different form layouts and had participants complete a calculation task; and usability evaluation involves testing a small number of real users completing tasks with the product and analyzing the results to identify any usability problems.

1. Usability Evaluation of Forms
Caroline Jarrett and Christine Yates
Presentation for the Information Design Association, London, 1998

2. Introductions
 Caroline Jarrett
 Forms specialist, usability consultant
 Christine Yates
 Head of usability, Inland Revenue Forms Unit

3. Background
 Needed a usable tax return
 Familiar with MUSiC methodology
 Measurement of Usability in Context
 National Physical Laboratory
 Developed for measuring computer usability
 Adapted methods for non-computer
 Used on Inland Revenue and other forms

4. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)

5. Why evaluate?
 Quantitative techniques
 Traditional qualitative
 Interaction with the product

6. Video clip
 Calculation of tax
 Two different layouts
 Participant gave appropriate permissions

7. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)

8. Product definition
 The form
 …. and the guidance
 …. in an envelope
 …. with a return envelope
 …. and a helpline
 …. the policy for dealing with it
 …. the source of the information

9. Why does it matter who the users are?
 You must complete your hardanger by 19th May
1999. Place your order for:
 22 count
 perle
 chart
 You must complete your annual return by 19th
May 1999. Place your order for:
 P14 (OCR)
 P11D
 Booklet 480

10. Who are the users?
 Recipient
 Advisor
 Friend or relative
 Customer assistance

11. Who are the users?

12. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)

13. The tasks reflect the goals
 Find the questions
 Understand the questions
 Put answers in the boxes
 Send it back
 Comply with deadlines

14. Structure of evaluation
 A participant
 Attempting a task
 With the product

15. Types of evaluation
 Measurement
 Diagnostic
 Concept

16. Participants
 Real users
 Proxy users
 Experts

17. Number of participants
 Statistical sample
 One is enough

18. Number of participants
 Statistical sample
 One is enough
 5 for a typical user evaluation
 Double for comparative evaluation
 Avoid evaluator exhaustion

19. Recruitment
 Who do you know?
 Lists from the client
 Market research recruitment

20. Running an evaluation
 Preparing an example
 Run through is essential
 Pre- and post- interviews
 Observers

21. Role of the interviewer
 Think aloud
 More natural
 More surprises
 More convincing
 Can provide measures
 With an interviewer
 Better control
 Keep them talking
 More detail
 Less nerve-wracking

22. Observation
 Self-reporting
 Interviewer
 Video
 Remote link

23. Rights of participants
 MRS code of conduct
 ESOMAR video code

24. Analysis and reporting
 Log of events
 List of problems
 Full methodological report
 Video report
 Personal observation

25. Types of problems
 Layout
 Question and answer
 Task structure

26. What is usability?
The extent to which a product can be
used by specified users
to achieve specified goals
with effectiveness, efficiency and satisfaction
in a specified context of use
(ISO 9241:11 1998)