MEDTECH 2013 Closing Plenary, Andy Shaudt, Director of Usability Services, National Center for Human Factors in Healthcare, MedStar Institute for Innovation, presents on Design and Development of Medical Devices through a Human Factors and Usability Lens on October 8, 2013

2. Design and Development of Medical Devices
through a Human Factors and Usability Lens
Andy Schaudt, M.S., M.B.A.
Director of Usability Services
National Center for Human Factors in Healthcare
MedStar Institute for Innovation
October 8, 2013

3. How on earth do we design
technologies/systems around drivers?
Do we give them bigger mirrors for putting on makeup?
Do we provide them a better steering wheel so they can drive with their knee?

4. Drivers are already at a
disadvantage because
they are human, let’s
not make it more
difficult/unsafe for
them…

6. Human Error in Healthcare
“More than 1 million patients are injured and approximately 180,000 die each year”
due to medical error – roughly equivalent to three commercial airliners crashed every
two days. (Andrews LB, Stocking C, Krizek T, et al. 1997)
8th leading cause of death in the United States (U.S.), ahead of motor vehicle
accidents, breast cancer and AIDS. (Leape 1994)
About half of adverse events were judged preventable with ordinary standards of care.
(Vincent, Neale, Woloshynowych, 2001)
$17.1 Billion annual cost (Van Den Bos, et al. 2011)
Might be 10x more than previous research has shown (Claussen, et al. 2011)

7. Human
Resources?
Engineering
People’s
Brains?
What is Human Factors Engineering?
“The scientific discipline concerning understanding of interactions among humans and
other elements of a system, and applying theory, principles, data, and other methods to
design in order to optimize human well-being and overall system performance.”
As defined by the Human Factors and Ergonomics Society.
Human
Factors
Psychology
Industrial
Engineering
Cognitive
Science
Anthropometry
Biomechanics

8. Human Factors and Usability
Benefits in Healthcare
• Can save lives
• Increase safety by reducing human error
• Improve system/device design
• Improve reputation
• Increases user acceptance
• Facilitates a stronger regulatory submission
• Believe it or not, it reduces overall costs…

9. Systems Approach
• Truly integrated systems result in higher
performance, higher reliability, improved safety
– Humans are one component in a larger system
– Focus on the interaction or interface between people and
the equipment and environment
– Fit the tools and environment to the person; not the
person to the tools and environment

10. Intended Users?
Healthcare Providers? Patients and Family Members?

11. Intended Environments?

12. Shift in Healthcare
• Expansion in ambulatory care
• Technology allowing for in-home care/monitoring
• Conversion in legacy systems to more mobile-based
technology
• Bottom line – users and environments are
expanding, and users’ roles are changing
• Incorporating human factors and usability early and
throughout device design is crucial

13. Primary Problem
• IOM Report in 2000
– Govt – 50% less errors in 5 years
• 13 years later…
– Essentially NO CHANGE
• Why?
– Focus still on individual performance
– Solutions inconsistent with safety
science
Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA. May 18 2005;293(19)
Wachter RM. The end of the beginning: Patient Safety Five Years After 'To Err Is Human'. Health Aff. 2004(11)
Wachter RM. Patient Safety At Ten: Unmistakable Progress, Troubling Gaps. Health Aff. 2010 (29:1)
Landrigan, Parry, et al. Temporal Trends in Rates of Patient Harm Resulting from Medical Care. NEJM 363(22):2010
Shekelle, Pronovost, et al. Advancing the science of patient safety. Ann Int Med 154(10):2011
Longo, Hewett, Ge, Schubert. The long road to patient safety: a status report on patient safety systems. JAMA, 294(22): 2005.

14. Error
• Human error is:
“A generic term to encompass all those occasions in
which a planned sequence of mental or physical activities
fails to achieve its intended outcome, and when these
failures cannot be attributed to some chance agency.” –
(Reason, 1990)
• Use error (most relevant for medical device manufacturers):
“act or omission of an act that has a different result than
intended by the manufacturer or expected by the
operator.” – (IEC, 2004)

15. Knowledge-Based
Rule-Based
Skill-Based
Improvisation in unfamiliar environments
No routines or rules available to help handle
Protocolized behavior
Process, Procedure
Automated Routines
Require little conscious attention
Figure adapted from: Embrey D. Understanding Human Behaviour and Error, Human Reliability Associates
Based on Rasmussen’s SRK Model of cognitive control, adapted to explain error by Reason (1990, 2008)

16. Human-machine System
Task
Lars-Ola Bligård (2007) [adapted from Sanders and McCormick, 1993; Danielsson, 2001]

17. Where Does the Breakdown Occur?
• Use errors occur because of a mismatch between the system
user, equipment, task and environment (FDA, 1999).
• Kaye and Crowley (2000) described six reasons why use-related
hazards occur:
1. Devices are used in ways that were not anticipated;
2. Devices are used in ways that were anticipated, but inadequately
controlled for;
3. Device use requires physical, perceptual, or cognitive abilities that
exceed those of the user;
4. Device use is inconsistent with user’s expectations or intuition about
device operation;
5. The use environment affects device operation and this effect is not
understood by the user; or
6. The user’s physical, perceptual, or cognitive capacities are exceeded
when using the device in a particular environment.

18. Defibrillator Example
Cardiac arrest
1. nurse with patient
2. charges unit…
3. clears patient…
4. presses “on” button
5. Machine powers down
– 2-3 minute delay in shock
Hoyer, Christensen, et al. Annals of Emergency Medicine 2008; 52(5): 512-514.
Fairbanks and Wears. Annals of Emergency Medicine 2008; 52(5): 519-521.

19. Error is Inevitable
• If error is inevitable… How to improve safety?
– Reduce the occurrence of human error
• With better design
• NOT training and policy
– Mitigate the effects of inevitable error
• With better design
– Better feedback
– Forcing functions

20. “Every system is designed to achieve exactly the results
it gets.”
- Don Berwick, former IHI president, former director of CMS

21. Another Challenge!
“The single greatest impediment to error prevention
in the medical industry is that we punish people for
making mistakes.”
--Lucian Leape, Testimony to congress

22. Medical Devices and Software
• More than 1/3 of medical device incidents involve
user error
• More than 1/2 of device recalls for design problems
involve the user interface
• Website/user interface design normally costs
upwards of 70% of total development costs
• More than 1/2 of device recalls for design problems
involve the user interface
~B.R. Phillips – Mayo Clinic

23. Let’s Recap…So Far
• You can’t eliminate human error
• Healthcare has a
“name, blame, shame, and
train” culture
• There are numerous
environments and users to
design a single system for… and
it’s changing
• Is the road ahead futile for
medical device designers?

24. Usability
• ISO 9241-11:1998
• The effectiveness, efficiency, and satisfaction with which
specified users achieve specified goals in particular
environments
• The components are explained as follows:
– Effectiveness: The accuracy and completeness with which
specified users can achieve specified goals in particular
environments.
– Efficiency: The resources expended in relation to the accuracy
and completeness of goals achieved.
– Satisfaction: The comfort and the acceptability of the work
system to its users and other people affected by its use.

25. Usability
engineering
process (IEC
60601-1-6)

26. Usability – Early and Often!
Pre-Market Post-Market
Design
Requirements
Development Deployment
Cost of Intervention in Product Lifecycle
# Design
Alternatives
$ of Design
Changes
Implement
usability early
and reduce #
of
interventions
and their
associated
costs!
Adapted from Bias &
Mayhew, 1994, p. 80

27. Tailor Your Usability Approach

28. Formative Recommendations
• Standards
– E.G. ANSI/AAMI HE75
– ISO/IEC 62366:2007
• Card Sorting
• Physical Ergonomics
• Storyboard
• Wireframe
• Task Analysis
– Hierarchical task analysis
– Cognitive task analysis
• Risk Analysis
– FMEA
– Task Analysis; Failure
Modes and their Effects;
rate
Severity, Occurrence, and
Detection; calculate Risk
Priority Numbers.
– When there isn’t a strong
“design control” and your
system relies on the
user, ask yourself why?
American National Standards Institute & Association for the Advancement of Medical Instrumentation (2009). HE75: Human Factors engineering
– Design of medical devices. Retrieved on July 25, 2013 from http://www.aami.org/publications/standards/he75.html

29. Formative Recommendations, cont..
• Heuristic Evaluation
– assess compliance of a
device or system with
regard to human factors
design principles and a
predetermined set of
design heuristics
• Heuristics
– Consistency & Standards
– Visibility of System State
– Match Between System &
World
– Minimalist
– Minimize Memory Load
– Informative Feedback
– Flexibility & Efficiency
– Good Error Messages
– Prevent Errors
– Clear Closure
– Others…
Heuristics - Nielsen, 1995

30. Formative Recommendations, cont..
• Formative User Testing
– Exploratory and is intended to find broad categories of use
challenges
– 4-5 users per group find ~80% of the problems (Virzi, 1992)
– Think aloud protocol
– Design to collect performance measures (e.g.
errors, violations, time-on-task) AND subjective feedback
(e.g. opinion ratings, design recommendations)
Virzi, R.. Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough? Human
Factors, 1992, 34(4), 457 468

31. Summative Recommendations
• Summative Evaluation
– Summative usability
evaluation is the final
validation intended to
demonstrate that all risks
have been identified and
adequately addressed and
that the impact of residual
risks is minimized or
acceptable. Usability
evaluation of a
complete, or near-
complete, design under
realistic conditions.
• Field User Events
• Post-market Usability
Evaluations
• Trade-off Studies

32. Regulatory Landscape
• Understand where your medical device fits here
• Medical Device?
– Class I General Controls
– Class II General Controls and Special Controls
– Class III General Controls and Premarket Approval
• Combination Product?
• Mobile Medical App?

33. Medical Device
• FDA Draft Human Factors Guidance - “The intent is to
improve the quality of the device user interface such that
errors that occur during use of the device are either
eliminated or reduced.”
– Identify anticipated use-related hazards (derived
analytically, see Section 6) and unanticipated use-related
hazards (derived through formative evaluations, see Section
7), and determine how hazardous use situations occur;
– Develop and apply strategies to mitigate or control use-related
hazards (see Section 8); and
– Demonstrate safe and effective device use through human
factors validation testing (see Section 10).
U.S. Department of Health and Human Services Food & Drug Administration Center for Drug Evaluation and Research
(2011). Draft Guidance for Applying Human Factors and Usability Engineering to Optimize Medical Device Design. Report
No. 1757. Retrieved on September 29, 2013 from:
http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM259760.pdf

34. Combination Product
• Examples:
drug/device, biologic/device, drug/biologic, or
drug/device/biologic
• No HF/usability guidance available (expected 2014)
• Human factors evaluation of training program
structure and materials is common
– Important to show the incorporation of user feedback in
the design evolution of the device/instrument AND the
training program

35. Mobile Medical App
• Final guidance available (not HF/Usability focused)
• Intended environment policies for/against?
• Multi-use device, not dedicated

36. Cost Arguments and Evidence for Human Factors Integration, Issue 1 / October 2006 / HFI DTC

37. In Summary
• Error in healthcare is substantial
• Medical device manufacturers can focus on reducing
“use errors” and mitigating their effects through human
factors/usability
• Incorporate user feedback/testing early and often
throughout the iterative design process
– Result in a safer product; benefits cost, time, and scope
• Keep a record of the user-focused design efforts and tell
your usability story in your regulatory submission

38. Andy Schaudt, M.S., M.B.A.
Director of Usability Services
National Center for Human Factors in Healthcare
MedStar Institute for Innovation
Andy.Schaudt@MedicalHFE.org
www.MedicalHumanFactors.net
202-244-9844