It's been 30 years since Nielsen and Molich provided the first practical and widely applied framework to evaluate the usability of digital products. How well has this venerable set of the 10 Commandments for UX stood the test of time as the technology and user expectations changed? We'll discuss the original purpose of the heuristics, how to apply them to a structured UX evaluation, if they've stood the test of time, and whether they will apply as technology continues to evolve. We’ll also explore whether new heuristics should be developed to account for other, recent aspects of the modern interaction and technology landscape. To illustrate this we will introduce a proposed new set of heuristics for a specific and important part of our digital interactions – social media.

1. 30 years of usability heuristics
Chris Rourke – CEO
Kirsty Simpson – UX Consultant
November 2021

2. About us
UCD Gathering - November 2021 @crourke @uservision 2
Chris Rourke
@crourke @uservision
chris@uservision.co.uk
Kirsty Simpson
kirsty@uservision.co.uk

3. My goals for our session
I’d like to …
• Tell you a bit about why heuristics were created
• How they are applied
• Learn about your experience with them or similar methods
• Consider if they are relevant with today’s technology
• Present some new heuristics related to Social Media
4
Does that sound OK?

4. Agenda
UCD Gathering - November 2021 @crourke @uservision 5
„ Origin of the Heuristic Evaluation method
„ How to perform HE
„ Applying HE in projects
„ Social Media Heuristics

5. Agenda
UCD Gathering - November 2021 @crourke @uservision 6
„ Origin of the Heuristic Evaluation method
„ How to perform HE
„ Applying HE in projects
„ Social Media Heuristics

6. Origin of the Heuristic Evaluation method
When ‘discount’ was a dirty word in the usability world
• 1989– Nielsen: ‘Usability Engineering at a discount’ - Discount
was radical and unwelcome
• 1990 - Nielsen & Molich ‘Heuristic evaluation of user
interfaces’ – Proposes method & 10 heuristics
• 1994 - Usability Engineering book by Jakob Nielsen
• Very different than now where ‘Lean UX’ and integrating with
Agile are expected
• Adoption grew, especially after websites appeared from 1995
• Technology has massively changed (mobiles, VR/AR, voice and
more)
• So are the principles still valid?
7

7. Heuristic Evaluation
A usability evaluation method in which one or more evaluators compare an interactive
system to a list of heuristics and identify where the interactive system does not follow
those heuristics.
Heuristic - A generally recognised rule of thumb that helps to achieve usability.
The purpose of heuristics is to increase the chance that an evaluator will detect usability problems
during the usability evaluation of an interactive system.
Good heuristics should be:
„ Generally recognised - a reliable set of heuristics that stands the test of time
„ Comprehensible – for those less familiar with UX / usability
„ Useful
„ Manageable - not too many heuristics. Usually about 10 heuristics are used
8

8. Why use the method?
We want to be efficient in identifying and
fixing usability problems
The move from Waterfall to Agile fuelled desire
for more rapid yet still reliable and accurate
evaluation of products & software during
development.
Big quantitative lab studies were not that
Relying on a single expert was also risky
9
Credit: J Neilsen
Even the most successful individual evaluators missed some issues
that less successful evaluators found

9. Agenda
UCD Gathering - November 2021 @crourke @uservision 10
„ Origin of the Heuristic Evaluation method
„ How to perform HE
„ Applying HE in projects
„ Social Media Heuristics

10. Poll Question 1
Have you or your team ever performed a Heuristic Evaluation on a product / system
you are creating?
• Yes
• No
• Not sure / maybe
11

11. How to perform Heuristic Evaluation
The process is simple and flexible – adapt it to your team and process
„ Identify potential evaluators (3-5, not intimately familiar with the system)
„ Brief them on the process and the heuristics and introduce the system to evaluate
„ They evaluate independently (important) ideally a couple times. Record results.
„ Include issues related to heuristics and other issues they spot
„ Reconvene to debrief. Walkthrough, list issues. Avoid solution development for now.
12

12. 13
The heuristics

13. Visibility of system status
14
The system should always keep users informed about what is going on, through appropriate feedback
within reasonable time.
HEURISTIC 1

14. Match between system and real world
15
The interactive system should speak the users’ language, with words, phrases and
concepts familiar to the user, rather than system-oriented terms. Follow real-world
conventions, making information appear in a natural and logical order.
HEURISTIC 2
Jargon!!

15. User control and freedom
16
Users often choose system functions by mistake and will need a clearly marked “emergency exit” to
leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
HEURISTIC 3

16. Consistency and standards
17
Users should not have to wonder whether different words, situations, or actions mean the same thing.
Follow platform conventions.
HEURISTIC 4
Montage of interaction states from Google Material Design styleguide https://material.io/

17. Error prevention
18
Even better than good error messages is a careful design which prevents a problem from occurring in
the first place. Either eliminate error-prone conditions or check for them and present users with a
confirmation option before they commit to the action.
HEURISTIC 5

18. Recognition rather than recall
19
Minimise the user’s memory load by making objects, actions, and options visible. The user should not
have to remember information from one part of the dialogue to another. Instructions for use of the
interactive system should be visible or easily retrievable whenever appropriate.
HEURISTIC 6

19. Flexibility and efficiency of use
20
Accelerators – unseen by the novice user – may often speed up the interaction for the expert user
such that the system can cater to both inexperienced and experienced users. Allow users to tailor
frequent actions.
HEURISTIC 7

20. Aesthetic and minimalist design
21
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of
information in a dialogue competes with the relevant units of information and diminishes their relative
visibility.
HEURISTIC 8

21. Help users recognize, diagnose, and recover from errors
22
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and
constructively suggest a solution.
HEURISTIC 9

22. Help and documentation
23
Even though it is better if the system can be used without documentation, it may be necessary to
provide help and documentation. Any such information should be easy to search, focused on the
user’s task, list concrete steps to be carried out, and not be too large.
HEURISTIC 10
Contextual Help Interactive help

23. Frequent mistakes in heuristic evaluations
24
„ Evaluation is based on gut feeling rather than heuristics. Let
the heuristics drive the heuristic evaluation.
„ Evaluators do not fully understand the heuristics. Interpreting
them correctly requires some experience.
„ Many UX professionals say they do heuristic evaluations but
they actually do usability inspection, because they report
usability findings that could not possibly be found using the
heuristics.

24. Agenda
UCD Gathering - November 2021 @crourke @uservision 25
„ Origin of the Heuristic Evaluation method
„ How to perform HE
„ Applying HE in projects
„ Social Media Heuristics

25. Pros and Cons of Heuristic Evaluation
Pros of Heuristic Evaluation
„ Fast and cost-effective - Can be completed quickly, often one or two days. Can
be integrated into short turnaround review cycles (Agile). No need to recruit
and schedule test participants.
„ Heuristics can help the evaluators focus their attention on certain issues.
„ Heuristic evaluation avoids the ethical and practical challenges in evaluation
methods involving real users.
„ The team (rather than individual) aspect of the evaluation can increase
acceptance and uptake of findings. Avoid opinion wars.
„ Using a set of heuristics can help identify usability problems with individual
elements and how they impact the overall user experience.
26

26. Pros and Cons of Heuristic Evaluation
Cons of Heuristic Evaluation
„ Its fast but not instant - still need time to train evaluators and have independent parallel
evaluations, then the debrief.
„ Potentially Overlooks problems - Heuristic evaluation overlooks problems that are not
covered by the applied set of heuristics.
„ Limited number of heuristics - Usability is much too complex to be expressed in just 10
or even 50 heuristics.
„ False alarms - Problems identified by evaluators can often be false alarms. Some studies
have shown some of the problems identified by HE are not real problems for the users.
27

27. Applying HE in projects
Heuristic evaluations work well in today’s Lean methods of evolving from
problem statement to product
29
Credit: Interaction-design.org

28. Applying HE in projects
30
Heuristic evaluations work well in today’s Lean methods of evolving from
problem statement to product
Credit: Interaction-design.org

29. Creating your own Heuristics
You can increase the use and usefulness of your heuristic evaluations by creating your own set
„ The original heuristics are applicable even with modern designs and new technology.
„ Establishing your own design heuristics to evaluate your products, systems, websites can help
evangelise on usability / UX.
„ Indeed, this may better suit the design being reviewed. Nielsen and Molich’s heuristics can inform
and inspire a company’s own specific heuristics. Build these based on your own market research,
other design guidelines and requirements.
31

30. Poll Question 2
Have you or your team created your own set of heuristics / review points for evaluating
a product / system you are creating?
• Yes
• No
• Not sure / maybe
32

31. Agenda
UCD Gathering - November 2021 @crourke @uservision 33
„ Origin of the Heuristic Evaluation method
„ How to perform HE
„ Applying HE in projects
„ Social Media Heuristics

32. 1. User Regret (Nir Eyal)
Don’t do to others that they would not want done to them
Large companies who have shareholders to answer to, often value company profit, over
users’ psychological needs.
Often the case with social media providers, design decisions are taken to exploit human
vulnerabilities and keep users hooked and engaged for longer rather than with the users’
best interests at heart.
Tips: How can we consider whether users might regret using your product in the future?
„ Nir Eyal’s ‘Regret test’
„ Prioritisation matrices (e.g. ‘desirability’ vs. ‘negative impact’ or ‘addictiveness’)
34
impawards.com

33. 35
Example of a Priorisation matrix
nielsonnormangroup.com

34. 2. Harm Prevention (Carnegie Trust UK)
Social media providers should have a duty of care to prevent harm
“In the real world, if you visit your public library, swimming pool or workplace, you
have an expectation of safety. That’s because these places are protected by laws to
keep people safe, including a ‘Duty of Care’”
It’s only a matter of time before a ‘Duty of Care’ extends to social media products:
„ A UK Online Safety Bill is already in the works
„ This digital ‘Duty of Care’ approach is being adopted Internationally
„ Covid-19 has increased the urgency in preventing online harm
Tips: What can social media companies and UX Designers do?
„ Create and apply a digital duty of care policy
„ Measure your design against its ability to meet that policy
36
Photo by Malaya Sadler on Unsplash

35. 3. Help users recognise and recover from addictive behaviours
(Nir Eyal)
Software should make steps to protect users who are displaying addictive
behaviours or who are vulnerable
“When the user is addicted — no longer in control and wanting to stop but can’t — companies should intervene.
Standing idly by while reaping the rewards of users abusing their product is no longer acceptable — it is
exploitation” - Nir Eyal
Tips
„ The answer in protecting addicted users doesn’t lie in simply replacing addictive features
„ Nir Eyal believes social media platforms owe it to their customer base to:
„ Introduce Use and Abuse policies that protect vulnerable users
„ Introduce a product feature that prompts heavy users with an offer to help
37

36. 38
nirandfar.com
Example of prompt feature with an offer to help

37. 4. Help users access supportive resources where necessary
(Dr Sophie Mort, Psychologist)
Help users reframe negative thinking around social comparison
„ Social platforms exploit our innate human drive to evaluate ourselves
„ They act as a springboard for us to compare ourselves and to ‘benchmark’
Tips: How can we improve the mental health experience of the platform?
„ We can’t stop users from naturally comparing themselves to others
„ We can help them to re-frame their negative thinking.
„ Provide information upfront as part of the onboarding process
„ Provide a product feature that prompts heavy users with an offer to help
39
Medium.com

38. 5. Negative Brand Perceptions (N/A)
Empathise with users to combat negative brand perception issues
“User experience includes all the users' emotions, beliefs, preferences, perceptions, physical
and psychological responses, behaviours and accomplishments that occur before, during,
and after use.” - ISO 9241-210
„ Brand perceptions are just as important as the experience of a product
„ Design can be an incredibly potent tool in overcoming negative perceptions of a brand
Tips to overcome negative brand perceptions
„ Carry out usability tests and interviews to empathise with users
„ Evaluate user feeling through customer journey mapping and using Likert scale
„ Use these tools as a benchmark for future improvement
40
clipart-library.com

39. Questions
Please let us know your thoughts! Such as
• Questions on how to run Heuristic Evaluations
• Thoughts on the heuristics themselves
• Have they aged well?
• Creating your own Heuristics
• Social Media Heuristics
41

40. Thank you!
Chris Rourke
Kirsty Simpson
UCD Gathering - November 2021 @crourke @uservision 42
chris@uservision.co.uk @crourke
kirsty@uservision.co.uk