Expert Evaluations &
Design / Usability Heuristics
Will look at:
• Need for alternatives to user evaluation
• Methods of evaluating without end users
(using expert evaluators)
• Some heuristics / guidelines offered by
End User Evaluations
• End-user evaluations can be expensive
– The methods are very time consuming
– Users may not be willing
– To get truly ‘fresh’ eyes, so called “kleenex”
testing requires different players each time
• Concerns about leaks
– Few external play testers at early stages
– Friends & family play testers may be too kind
• As an alternative to some user testing,
expert evaluators / testers can be used
• Falconer details 10 inspection methods,
we will look at two:
– Cognitive Walkthrough
– Heuristic Evaluation
• In this approach experts imitate users
– Relatively quick and cheap
– Expert needs to be skilled in the system or to
be familiar with the prototype / design.
• Two phase process
– gathering inputs
– doing the walkthrough
Cog Walk Phase 1
• Two phase process.
• Phase 1: Gathering Inputs
A description of users (e.g. level of experience)
A description of system (or an operational system)
A description of the task to be carried out
A list of the actions required to complete the task
– step by step!
Cog Walk: Phase 2
• Phase 2: Walkthrough
– take each of the actions in the list
– and ask the following questions:
• Is the goal clear at this stage?
• Is the appropriate action obvious?
• Is it clear that the appropriate action leads to the
• What problems (or potential problems) are there in
performing the action?
• Essential that the expert tries to think like
the end user and not like themselves.
• For each step of the process, you ask
those questions and you write a ‘success
story’ - is it likely that the user will
successfully achieve this step. If not, why
Consider Dwarf Fortress
• A scenario:
– Task is to make beds for your dwarves
– what are the actions?
open designate menu,
choose ‘chop down trees’
construct carpenter’s workshop
open workshop menu, ‘add’ beds
look at dwarves - make sure there exists a dwarf with ‘wood
chopping’ and another with ‘carpentry’
• (even these actions are not low level enough - I’ve
abbreviated for ease of presentation)
• Involves assessing how closely an
interface or system conforms to a
predefined set of guidelines or heuristics.
– Nielsen’s usability heuristics
– Schneiderman’s eight golden rules
– Norman’s seven principles
Nielsen’s Usability Heuristics
– keep users informed about what is happening
Speak the user’s language
– dialogs should be expressed clearly using terms
familiar to the user
User control and freedom
– clearly marked exits and undo/redo
Consistency and standards
– even better than having good error messages
Nielsen’s Usability Heuristics
Minimise memory load
don’t have irrelevant or rarely needed information
Good error messages
accelerators (unseen by novices) speed up
interactions for experts
Aesthetic and minimalist design
recognition rather than recall
should indicate the problem and explain how to
Help and documentation
should be concise and easy to search
Norman’s 7 Principles
1: Use both knowledge in the world and
knowledge in the head.
2: Simplify the structure of tasks.
3: Make things visible.
4: Get the mappings right.
5: Exploit the power of constraints.
6: Design for error.
7: When all else fails, standardise.
(8 Golden Rules)
Strive for consistency
Enable frequent users to use shortcuts
Offer informative feedback
Design dialogues to yield closure
Offer error prevention & simple error handling
Permit easy reversal of actions
Support internal locus of control
Reduce short-term memory load
(Faulkner Chapter 7)
How Many Evaluators?
Different people find different problems.