Presented by: Amanda Raven, M.S.I.E. Human Factors Safety Specialist, AHS Jason Laberge, M.Sc. Manager – Human Factors, AHS

1. Using Heuristic Evaluation During
Procurement to Improve Patient
Safety
Amanda Raven, M.S.I.E.
Human Factors Safety Specialist, AHS
Amanda.Raven@albertahealthservices.ca
Jason Laberge, M.Sc.
Manager – Human Factors, AHS
Jason.Laberge@albertahealthservices.ca

2. 2
Define Opportunity
• Difficult to use medical devices are a patient safety
threat
– Have been implicated in adverse events and close calls within
and outside of Alberta Health Services (AHS)
• This threat can be minimized through careful evaluation
during procurement, prior to purchase and
implementation
• Heuristic evaluation is one evaluation technique that can
be used for medical devices during procurement
– Allows AHS to “Try It Before You Buy It”

3. 3
Build Understanding
• Heuristic evaluation was used by the
AHS Human Factors team to evaluate
electronic ‘wayfinding’ kiosks that were
being purchased for the new South
Health Campus in Calgary
Definition:
Wayfinding = all aspects of
knowing where you are and
successfully navigating to your
destination

4. 4
Act to Improve
• Heuristic Evaluation can be
used to evaluate the design of
a device to determine:
– How easy is the device to use?
– What errors are likely to be
observed with the device?
• Criteria (heuristics) can be
developed and assessed on a
device specific basis
Sample Wayfinding Design
Heuristics:
• Wayfinding principles –
includes overview maps,
landmarks, paths, and
intuitive signage
• Anthropometrics and
accessibility – Meets
ergonomic and accessibility
requirements
• Minimize memory load –
Does not require users to
memorize a lot of
information
• Prevent Errors –
Designed to prevent errors
from happening in the first
place
• Reversible actions –
Allows users to recover
from errors or change their
minds

5. 5
Measurement
• During the wayfinding kiosk evaluation, each vendor
device was evaluated by 3 Human Factors Specialists
• Compliance scores are issued for each heuristic for
each device, resulting in an overall compliance score
100%
80%
60%
40%
20%
0%
Overall % Compliance
Heuristic Evaluation of Wayfinding Kiosk Vendors
4
3
2
1
0
Average Compliance Score
1 Vendor 1 2 Vendor 2 3 Vendor 3
• Lower compliance indicates
higher risk in terms of poor
design contributing to adverse
events
• AHS should purchase devices
with the highest compliance
scores

6. Normal Process With Heuristic Evaluation
6
Contribution to Patient Safety
Plan to Purchase
Create RFP
Select, purchase & deploy device
Identify common use errors
through close calls, adverse
events, patient harm
Implement reactive solutions (i.e.
training, policy) to reduce risk. With
serious issues, consider replacing
device ($$$).
Conduct Heuristic Evaluation to
identify usable and safe devices,
avoid devices with potential for major
use errors leading to adverse events
Select, purchase & deploy device
Use errors are identified prior to
purchase and deployment, proactively
reducing patient safety risk!

7. 7
Partnership
• Human Factors Specialists worked closely with the
Wayfinding working group at the South Health Campus
– Evaluation heuristics grounded in the defined wayfinding process
– Evaluations incorporated a
patient and volunteer
representative’s feedback
• Heuristic evaluations during
procurement support
engagement with users and
stakeholders for a medical
device
Image courtesy Joanne Ganton

8. 8
Sustain Results
• Heuristic evaluation in the procurement process leads
to sustainable change as it identifies usability concerns
with devices before they are brought into the health
system
– Avoids the negative impact that may have been observed if a
poorly designed device had been purchased
– Change is sustainable as usable devices remain in the health
system longer, rather than needing to constantly respond to
concerns with poorly designed devices.