Baobab Health 2015 Usability Inspections

Usability Inspections:
Heuristic Evaluation
Harry Hochheiser
Biomedical Informatics
harryh@pitt.edu
Harry Hochheiser, harryh@pitt.edu Baobab Health, February 2015

Goal of Usability Assessments
• Can users complete tasks?
• Appropriateness of mental models
• Comparative efficiency
• Subjective satisfaction
• How do we assess?

No bright lines
Design
Functional
systemPaper
Prototype
Release
Usability Inspections
Usability Studies
Empirical User Studies, Case Studies,
Longitudinal Studies, Acceptance
Tests
Low
cost,
low
validity Higher
cost,
validity

Usability Inspections
• “clean-room” static examination of usability
• Methodically scrutinize interfaces in search of potential
problems
• Pros:
– Inexpensive – no users, relatively easy
– Identify major issues at a relatively early stage
• Cons:
– May miss problems: generally find < 50%
– All results are hypothetical – don’t know which problems might
really lead to errors

Common Questions across inspection
types
●
Who inspects?
●
Background
●
How many?
●
Which tasks?
●
How to interpret findings?
●
Problems are not “real” problems experience by
users
●
How important are they?

Broad classes of inspections
Heuristic Evaluations: How well does an interface conform
to guidelines for interface design?
Walkthrough: Analytic examination of interface and
interaction requirements, usually informed by some model
of the user
Many variants...

Who inspects?
Heuristic inspection
Usability experts
Domain experts
Combination? (Double experts)
Users should participate as users
when possible
3-5 experts? (Nielsen)
Or more...
Work alone, or in teams..
Walkthroughs
May require more cognitive
background
Domain expert feedback
helpful
Conducted by a team?

Tasks
Heuristic inspections
Set tasks
Open-ended exploration
Walkthroughs
Generally, specific tasks

Nielsen's Heuristics
Nielsen, 1994 - http://www.nngroup.com/articles/ten-usability-heuristics/
• Visibility of system status
• Match between system and real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnose, and recover from errors
• Help and documentation

Results 
Nielsen and Molich 1990 , Nielsen 1993
• Few false positives
• Find < 50% of errors
• “In general, we would expect aggregates of five
evaluators to find about two thirds of the usability
problems which is really quite good for an informal and
inexpensive technique like heuristic evaluation.”

Results 
Nielsen and Molich 1990

Results 
Nielsen and Molich 1990
Source of the “you only need 5 evaluators” rule…
Hotly contested…

Which Evaluators? 
Nielsen, 1992
• “Double Experts” - Domain and Usability expertise – considered
best
• Not always available – use some of each, or teams

Heuristic Evaluation Procedure
Evaluators work alone
(except for when they work in teams)
Optional observer can help explain confusing issues and to
record issues.
Go through interface several times
List specific tasks
Note discrepancies between interface and heuristic
Note any concerns - even those that don’t exactly fit heuristics
Individual evaluators meet to aggregate results
agree on interpretation and prioritization

• Visibility of system status
•“The system should always keep users informed about what is going on,
through appropriate feedback within reasonable time”
• Match between system and real world
“The system should speak the users’ language, with words, phrases, and
concepts familiar to the user, rather than system-oriented terms. Follow real-
world conventions, making information appear in a natural and logical order.”
• User control and freedom
“Users often choose system functions by mistake and will need a clearly marked
"emergency exit" to leave the unwanted state without having to go through an
extended dialogue. Support undo and redo.”

• Consistency and standards
“Users should not have to wonder whether different words, situations,
or actions mean the same thing. Follow platform conventions.”
• Error prevention
“Even better than good error messages is a careful design which prevents a
problem from occurring in the first place. Either eliminate error-prone
conditions or check for them and present users with a confirmation option
before they commit to the action.”
• Recognition rather than recall
“Minimize the user's memory load by making objects, actions, and options
visible. The user should not have to remember information from one part of
the dialogue to another. Instructions for use of the system should be visible or
easily retrievable whenever appropriate.”

• Flexibility and efficiency of use
“Accelerators -- unseen by the novice user -- may often speed up the
interaction for the expert user such that the system can cater to both
inexperienced and experienced users. Allow users to tailor frequent
actions.”
• Aesthetic and minimalist design
• “Dialogues should not contain information which is irrelevant or
rarely needed. Every extra unit of information in a dialogue competes
with the relevant units of information and diminishes their relative
visibility.”

• Help users recognize, diagnose, and recover from errors
“Error messages should be expressed in plain language (no codes), precisely
indicate the problem, and constructively suggest a solution.”
• Help and documentation
“Even though it is better if the system can be used without documentation, it
may be necessary to provide help and documentation. Any such information
should be easy to search, focused on the user's task, list concrete steps to be
carried out, and not be too large.”

How to Interpret?
Use severity judgments to prioritize fixes
Frequency of problem
Impact of problem
Persistence – will users be repeatedly bothered?
Multiple independent raters increase reliability
Bigger questions – does this design work at all?
As with usability studies, try to generalize
Don't solve lots of small problems if the design is inherently problematic

Severity Ratings
0. No problem
1. Cosmetic
2. Minor
3. Major
4. Catastrophe
Measure both impact and frequency?

Reporting heuristic violationshttp://ocw.mit.edu/courses/electrical-engineering-and-computer-science/
6-831-user-interface-design-and-implementation-spring-2011/lecture-notes/MIT6_831S11_lec23.pdf
• Goal - communicate problems
• Identify successes
• Be constructive
• Be specific

Heuristic Evaluation reportshttp://ocw.mit.edu/courses/electrical-engineering-and-computer-science/
6-831-user-interface-design-and-implementation-spring-2011/lecture-notes/MIT6_831S11_lec23.pdf
• Problem
• Heuristic
• Description
• Severity
• Recommendations
• Screenshot

Example: NIH Undiagnosed
Disease Program UDPICs LIMS tool

Example: NIH Undiagnosed
Disease Program UDPICs LIMS tool
Heuristic Visibility of system status; Consistency and standards; Recognition rather than
recall
Description The “gear” icon is often used to indicate a menu of systems settings and
preference. Although this is not a standard, it is a widely-used convention.
UDPICS uses this gear icon to access a menu that contains a variety of options
- some of which are settings and others providing additional system functionality
Figure 2). This may confuse users who select the menu looking only for settings
selections, or for those who look elsewhere for options found in this menu. The
use of the gear item also raises concerns for administrative users, who might be
confused by the gear icon on the one hand and the “settings and preferences”
menu on the left-hand navigation bar.  
Severity Minor
Suggested
Revisions
One possible solution would be to remove all non-settings items from this menu,
adding additional choices to the otherwise underutilized menu bar. Even if the
left-hand navigation bar is available only to administrators, placing all settings
menu in a single place and adjusting the available options based on user
privileges might be easier for users.

Making use of heuristic evaluation
results?
●
Have member of development team present during
inspection?
Can be done with usability studies and other inspections as well
They should observe, not analyze
●
Debriefing session to explore fixes
Consider redesigns where appropriate.

Other Inspection Methods
http://www.useit.com/papers/heuristic/inspection_summary.html
Heuristic estimation – compare interfaces
Feature Inspection – look for long sequences
of operations, unnatural sequences,
those requiring extensive skill...
Consistency inspection – compare designs for consistency
Standards Inspection

Which Heuristics?
• Multiple versions of Nielsen’s list
• Other versions
– Special cases?
• More heuristics – more focus
– ..but less study
• Too many heuristics – cumbersome, complicated

Web Usability Heuristics 
http://www.andybudd.com/archives/2007/01/
heuristics_for_modern_web_application_development/index.htm
• Design for user expectations
• Clarity
• Minimize Unnecessary Complexity and Cognitive Load
• Efficiency and Task Completion
• Provide Users with Context
• Consistency and Standards
• Prevent Errors
• Help users, notice, understand, and recover from errors
• Promote a pleasurable and positive web experience

Usability Evaluation of an Electronic Medication
Administration Record (eMAR) Application 
Guo, et al. 2011
• eMAR tasks
– Access to drug information, policies, and procedures related to
meds
– Context-specific patient clinical results
– Facilitate communication between nurses and pharmacists
– Verification of the five “rights” (patient, drug, dose, route, and
time)
• Heuristic Evaluation for usability
• Using Zhang’s heuristics and severity ratings

Heuristic evaluation of eMAR 
Guo, et al. 2011
• Tasks
– Login
– Order and modify medication
– Verify medication orders
– Access drug references
– Administer medication
– Edit eMAR
– Generate reports and review
• Procedure
– Train evaluators
– Conduct evaluation
– Calculate agreement
– Build consensus
– Review with staff – conduct tasks

Guo, et al. 2011
• Observations
– classified by heuristic,
– grouped into usability problems
– Organized by task

Guo, et al. 2011

Agreement?
• What if multiple reviewers disagree?
• Can calculate inter-rater reliability
• Cohen’s kappa
• Raw agreement
• Goal here is to find potential problems -
• being thorough is more important than agreeing

Baobab Health 2015 Usability Inspections

Recommended

Recommended

More Related Content

What's hot

What's hot (6)

Similar to Baobab Health 2015 Usability Inspections

Similar to Baobab Health 2015 Usability Inspections (20)

More from Harry Hochheiser

More from Harry Hochheiser (16)

Recently uploaded

Recently uploaded (20)

Baobab Health 2015 Usability Inspections