This document discusses various methods for evaluating usability, including guidelines, expert opinion, and user testing. It emphasizes that real user testing where users actually use a product or prototype is the best approach, but also the most difficult. The document provides guidance on conducting user tests, such as using representative users, paper or electronic prototypes, asking both quantitative and qualitative questions, and capturing results through observation, metrics, questionnaires or interviews. It stresses the importance of neutral, planned questioning and analyzing competitors to improve one's own product.

1. Usability
Evaluation
Lon Barfield (26 slides)

2. Evaluation of anything
... that is designed for external consumption,
needs to be evaluated by external agents.
Can be a front end, film, pepsi challenge…
or a piece of writing.
Is the work fit for purpose?
Should be a no-brainer, but...

3. Reasons not to evaluate
It is a hassle
You lay your work open to external opinion
Alterations will mean more work
Some people think it is better just to assume it’s
great and stick your head in the sand...

4. Ways of evaluating
Rules/guidelines/patterns (‘heuristics’)
Can be difficult to apply
Use them in initial design
Expert opinion
Where do you get an unbiased expert from?
Real users
Best but hard work

5. Real users - usability testing
Some users
A thing for them to use
They have to actually use it
A test; questions, and results

6. Some users
One subject
Two subjects
Group
Representative users

7. A thing to use
Finished system
Competitors system
Prototype/simulation

8. Prototypes/simulations
Story boards
Paper based
Screen based (Photoshop, Flash)
Single track/walk through
Wizard of Oz

9. Paper based
Photocopoier, spreadsheet, tipex, scissors,
bluetac
By hand or printouts from layout program
Early and fast = incredible savings in effort
‘Paper psychology’

10. Good can be small and fast
Big and slow can be bad. “Our site needs
redesigning its rubbish, get 10 people into the
usability lab and conduct in depth analyses of
whats wrong with it…”
Here’s an idea; hand sketch it, small group,
‘where would you click for more information?’.

11. They have to use it!
Not about just asking a flatmate ‘is it user
friendly?’ They will say ‘Yeah, what’s on TV?’.
‘Do you want to stay my friend, and does this
look like I’ve put some work into it?’

12. Use it… Task oriented
Don’t ask:
Is it easy to use? Is it quick to use ?
Do ask:
How long does it take the user get an average
ticket if they already know how to use the
machine? When new users use it for the first
time can they all get the ticket they require?

13. Quantitive
Observations or questions
‘How many seconds to click it?’
‘Does the site have events information?’
Can do manipulations with the data
Vey focussed and might be measuring the
wrong thing

14. Qualitative
Interviews, chatty or open questions
More difficult
Data is less maleable
Interesting

15. Questions
Questions have to be:
Planned
Useful
Neutral
What to ask, how to ask it

16. What to ask - Life on Mars
‘Well, what do you think; is it easy and fast to
use?’
Mars probe. ‘Well, what do you think; is there
life on Mars?’
Break the big quesitons down into the detailed
quesitons that we can actually get answers to.

17. Life on Mars...
The National Research Council panel
nicknamed the "weird life" committee. The
group worries that scientists may be too Earthcentric when looking for extraterrestrial life. The
problem for scientists is that "you only find what
you're looking for," said Penn State University
geosciences professor Katherine Freeman

18. How to ask it?
Can be very tricky
The Elderberry Experiments 60’s (more later)

19. Neutral questioning
Societal and personal pressures
Your best friend likes your design - its the
Bradley effect (Tom Bradley 1982 mayor)
‘Do you mind if I smoke?’ Vs ‘Would you prefer
it if I didn’t smoke’
Scots referendum. “Do you agree that…”

20. Capturing results - which method
Method has to be appropriate
Observation doing a task
Metrics, talk through
Questions in a form
Questions in an interview

21. Capturing results
Video them and ask them ‘did you see the help
button on the page?’.... ‘Yes’
Simple closed questions use fill in forms with
data capture.

22. Open/closed questions
What do you think of the help function?
Did you use the help function?
I found the help function
a) interesting
b) boring…
Be very careful with multiple choices

23. Audio and video
Transcription/analysis is VERY time consuming
Audio for interviews
Video for visual oriented investigations

24. Interviews
Already a large amount of good practice
Clear
Asset management
Processing the recordings
Presenting results

25. … and competitor analysis
… is analysing your competitors to see how
you stack up against them
… is looking at what others are doing
it’s not about copy and pasting bits here and
there to create your own product. Not the
‘perfect solution’.

26. Conclusions
Appropriate
Considered
Fast and small and many
Slides on slideshare.net/lonbarfield