Polly Shelton, Human Sciences Consultant at PDD, presented this 20 minute talk at UK UPA (Usability Professional’s Association) and London Ergonomics Group (LEG) joint event on Medical Device Usability on the 26th January 2012.
Polly discusses the human centred design process as it applies to medical devices. She outlines the process, regulatory requirements in Europe and the US and talks through some case studies, finishing with some top tips for conducting and analysing usability tests of medical devices.
Usability Validation Testing of Medical Devices and SoftwareUXPA Boston
The U.S. FDA and international regulatory bodies require usability testing of medical devices, products, software, and systems as part of their overall validation. Manufacturers must demonstrate that all potential use-related hazards have been identified, prioritized, and mitigated. The method for demonstrating this is human factors/usability engineering (HF/UE) validation testing. However, the way we conduct these studies is in many ways different from the way we conduct studies of non-medical products and systems.
This topic is relevant to the Boston UX community given the convergence of consumer and medical devices, as well as the rise of wearable technologies and the apps that interact with them. This presentation will cover the key aspects of HF/UE validation (a.k.a. ‘summative’) testing and what the FDA expects in the final HF/UE summary report.
Importantly, this session will consist of half presentation and half Q&A, with the audience driving the discussion toward current issues, questions, and challenges that are relevant to them.
Speed your time to market by following the step-by-step instructions.
You'll learn the steps to success for product testing by a certification agency (like UL, TÜV, BSI, Intertek, etc.) to the IEC 60601-1 series of standards on medical electrical equipment and systems.
See the mistakes others have made that have slowed down their certification process (and how you can avoid them).
Take proactive steps so your product doesn’t need to get redesigned after testing starts.
Specifically, you will learn:
-The scope of IEC 60601-1 standards & if they apply to your product.
-What you need to know to classify your products to the IEC 60601-1 series.
-What is an isolation diagram and how does that help me with my design?
-Determine the applicable tests for your device.
-What are the marking and labeling requirements for the device?
-Know your critical components.
-What pre-tests to run and what’s not worth testing?
-Resources to help with this process and ways to reduce the paperwork off your backs.
Watch the presentation here: https://www.greenlight.guru/webinar/iec-60601-1
Usability Validation Testing of Medical Devices and SoftwareUXPA Boston
The U.S. FDA and international regulatory bodies require usability testing of medical devices, products, software, and systems as part of their overall validation. Manufacturers must demonstrate that all potential use-related hazards have been identified, prioritized, and mitigated. The method for demonstrating this is human factors/usability engineering (HF/UE) validation testing. However, the way we conduct these studies is in many ways different from the way we conduct studies of non-medical products and systems.
This topic is relevant to the Boston UX community given the convergence of consumer and medical devices, as well as the rise of wearable technologies and the apps that interact with them. This presentation will cover the key aspects of HF/UE validation (a.k.a. ‘summative’) testing and what the FDA expects in the final HF/UE summary report.
Importantly, this session will consist of half presentation and half Q&A, with the audience driving the discussion toward current issues, questions, and challenges that are relevant to them.
Speed your time to market by following the step-by-step instructions.
You'll learn the steps to success for product testing by a certification agency (like UL, TÜV, BSI, Intertek, etc.) to the IEC 60601-1 series of standards on medical electrical equipment and systems.
See the mistakes others have made that have slowed down their certification process (and how you can avoid them).
Take proactive steps so your product doesn’t need to get redesigned after testing starts.
Specifically, you will learn:
-The scope of IEC 60601-1 standards & if they apply to your product.
-What you need to know to classify your products to the IEC 60601-1 series.
-What is an isolation diagram and how does that help me with my design?
-Determine the applicable tests for your device.
-What are the marking and labeling requirements for the device?
-Know your critical components.
-What pre-tests to run and what’s not worth testing?
-Resources to help with this process and ways to reduce the paperwork off your backs.
Watch the presentation here: https://www.greenlight.guru/webinar/iec-60601-1
FDA Regulation of Promotion & Advertising -- Part 1: The BasicsMichael Swit
Presentation to the Compliance Online 2-day course on Ensuring Compliance with FDA Requirements for the Promotion & Advertising of Drugs and Medical Devices, March 22, 2018
“Regulatory writing department at Turacoz have the expertise to develop various regulatory documents such as Investigator Brochures (IBs), Protocols, Clinical Study Reports (CSRs), Common Technical Documents (CTDs) and pharmacovigilance documents such as Periodic Safety Update Reports (PSURs) and Risk Management Plans (RMPs). In these slides, we have presented an overview on Periodic safety update reports (PSURs) and also the guidelines such GVP modules and ICH E2c. We have also discussed the changes from old PSUR format to new Periodic Benefit-Risk Evaluation Report (PBRER) format.”
Risk management in the development of medical devices. This presentation was for a webinar where we discussed the basics of risk management, a general risk management lifecycle, the requirements of certain relevant standards (ISO 14971, IEC 62304, US FDA Title 21 CFR Part 11), and the practical method called HFMEA. The live demonstration shows you how risks can be managed and compliance achieved using the advanced risk management features of codeBeamer ALM, and also demonstrates the use of our (general) FMEA template.
New European Medical Device Regulations: Keeping Your Orthopaedic and Spine ...April Bright
For years, notified bodies and industry experts have warned orthopaedic device companies to prepare for the forthcoming EU Medical Device Regulation (MDR). Though the regulation is expected to be officially published in the first half of 2017, a recent study shows that not only are companies not prepared, but regulatory and quality affairs professionals say that they have not studied the regulation closely.
The regulation includes new requirements for orthopaedic (and especially spine) companies that must be met in a timely manner to keep products on the market. Among the MDR changes are more stringent clinical evidence, identification of a “qualified person,” implementation of UDI and rigorous postmarket oversight.
Dr. Li covers highlights of the regulation and the provided transition period toward fulfillment of the new expectations. He explains the impact of device up-classification and the need for additional clinical data toward passing this new regulation successfully and toward fulfilling the postmarket reporting requirements accordingly.
Medical Device Regulation (MDR) overview for Technion, May 25, 2021Levi Shapiro
On May 26, 2021, the EU introduced the most sweeping changes to the Medical Device legal framework since the mid 90's. Ulf Grundmann, Senior Partner, King & Spalding (Frankfurt), reviews some of these regulatory changes, from the perspective of medical device innovators. The presentation includes Scope and Definitions, Classification and Conformity Assessment, Placing a Device on the EU Market, UDI and EUDAMED, Supply Chain Obligations, PMS and Vigilance
An Inside Look at Changes to the New ISO 14971:2019 from a Member of the Stan...Greenlight Guru
The 3rd Edition of the medical device risk management standard, ISO 14971:2019, and its companion guidance document, ISO TR 24971:2019, will be published by year-end.
The new structure of the two documents will be presented so that the manufacturer can determine any changes to the risk management system and possibly the quality system that may be required.
These may include simple reference changes in procedures or revision to production and post-production processes that may be required.
Presenter Edwin Bills is an international member of the technical committee, ISO TC 210 JWG1, responsible for the revision of the third edition of ISO 14971 risk management standard.
TALK TAKEAWAYS:
• A detailed look at the new changes to ISO 14971:2019 and ISO TR 24971:2019.
• Reasons for the changes to the latest version
• How to prepare for the coming changes in the standard
This session took place live at the Greenlight Guru True Quality Virtual Summit, a three-day event for medical device professionals to learn to get their devices to market faster, stay ahead of regulatory changes, and use quality as their multiplier to grow their device business.
FDA Regulation of Promotion & Advertising -- Part 1: The BasicsMichael Swit
Presentation to the Compliance Online 2-day course on Ensuring Compliance with FDA Requirements for the Promotion & Advertising of Drugs and Medical Devices, March 22, 2018
“Regulatory writing department at Turacoz have the expertise to develop various regulatory documents such as Investigator Brochures (IBs), Protocols, Clinical Study Reports (CSRs), Common Technical Documents (CTDs) and pharmacovigilance documents such as Periodic Safety Update Reports (PSURs) and Risk Management Plans (RMPs). In these slides, we have presented an overview on Periodic safety update reports (PSURs) and also the guidelines such GVP modules and ICH E2c. We have also discussed the changes from old PSUR format to new Periodic Benefit-Risk Evaluation Report (PBRER) format.”
Risk management in the development of medical devices. This presentation was for a webinar where we discussed the basics of risk management, a general risk management lifecycle, the requirements of certain relevant standards (ISO 14971, IEC 62304, US FDA Title 21 CFR Part 11), and the practical method called HFMEA. The live demonstration shows you how risks can be managed and compliance achieved using the advanced risk management features of codeBeamer ALM, and also demonstrates the use of our (general) FMEA template.
New European Medical Device Regulations: Keeping Your Orthopaedic and Spine ...April Bright
For years, notified bodies and industry experts have warned orthopaedic device companies to prepare for the forthcoming EU Medical Device Regulation (MDR). Though the regulation is expected to be officially published in the first half of 2017, a recent study shows that not only are companies not prepared, but regulatory and quality affairs professionals say that they have not studied the regulation closely.
The regulation includes new requirements for orthopaedic (and especially spine) companies that must be met in a timely manner to keep products on the market. Among the MDR changes are more stringent clinical evidence, identification of a “qualified person,” implementation of UDI and rigorous postmarket oversight.
Dr. Li covers highlights of the regulation and the provided transition period toward fulfillment of the new expectations. He explains the impact of device up-classification and the need for additional clinical data toward passing this new regulation successfully and toward fulfilling the postmarket reporting requirements accordingly.
Medical Device Regulation (MDR) overview for Technion, May 25, 2021Levi Shapiro
On May 26, 2021, the EU introduced the most sweeping changes to the Medical Device legal framework since the mid 90's. Ulf Grundmann, Senior Partner, King & Spalding (Frankfurt), reviews some of these regulatory changes, from the perspective of medical device innovators. The presentation includes Scope and Definitions, Classification and Conformity Assessment, Placing a Device on the EU Market, UDI and EUDAMED, Supply Chain Obligations, PMS and Vigilance
An Inside Look at Changes to the New ISO 14971:2019 from a Member of the Stan...Greenlight Guru
The 3rd Edition of the medical device risk management standard, ISO 14971:2019, and its companion guidance document, ISO TR 24971:2019, will be published by year-end.
The new structure of the two documents will be presented so that the manufacturer can determine any changes to the risk management system and possibly the quality system that may be required.
These may include simple reference changes in procedures or revision to production and post-production processes that may be required.
Presenter Edwin Bills is an international member of the technical committee, ISO TC 210 JWG1, responsible for the revision of the third edition of ISO 14971 risk management standard.
TALK TAKEAWAYS:
• A detailed look at the new changes to ISO 14971:2019 and ISO TR 24971:2019.
• Reasons for the changes to the latest version
• How to prepare for the coming changes in the standard
This session took place live at the Greenlight Guru True Quality Virtual Summit, a three-day event for medical device professionals to learn to get their devices to market faster, stay ahead of regulatory changes, and use quality as their multiplier to grow their device business.
Epec Engineered Technologies is an experienced custom designer and manufacturer of high reliability user interface control panels. High reliability user interfaces are defined as being more dependable than standard membrane switches as they incorporate additional components into a full assembly such as displays, switches, interface electronics, and graphics with aesthetics to match demanding OEM applications.
7 Components to Medical Device Usability Testing SuccessMargee Moore
Despite the publication of various relevant guidance on medical device usability testing and standards for human factors testing confusion regarding best practices still exists. This presentation provides clear language and seven components for planning successful usability testing for medical device development.
User Interface Design for Medical Devices - The Relationship Between Usabilit...UXPA Boston
User Interface Design for Medical Devices - The Relationship Between Usability and Safety
Rich Newman's presentation from the UXPA Boston 2016 conference
In May 2022, the European In Vitro Diagnostics Regulation (IVDR) will apply in the world’s second-largest medical device market. The new Regulation will introduce major changes to how manufacturers obtain CE Marking and maintain access to the European market. Many companies have yet to prepare for compliance to these new requirements or organize their regulatory transition strategies. Oliver will present the ‘What will it take? Review IVDR readiness” to help you understand the scope of the new regulations.
This session took place live at the Greenlight Guru True Quality Virtual Summit, a three-day event for medical device professionals to learn to get their devices to market faster, stay ahead of regulatory changes, and use quality as their multiplier to grow their device business.
Glucose Strip, Urine Strip and Pregnancy Strip are three categories of IVD Strips that required to manufacturing in good quality and cost effective manner with accurate release timing and effective marketing strategy. To manufacture these strips, it should be well equipped with infrastructure, experienced team of research experts, proper regulatory compliances, dedicated lab facility and well developed supply chain network.
𝐂𝐨𝐧𝐭𝐚𝐜𝐭 𝐮𝐬
NIIR PROJECT CONSULTANCY SERVICES, DELHI
An ISO 9001:2015 Company
ENTREPRENEUR INDIA
106-E, Kamla Nagar, Opp. Mall ST,
New Delhi-110007, India.
Email: npcs.ei@gmail.com
info@entrepreneurindia.co
Tel: +91-11-23843955, 23845654, 23845886
Mobile: +91-9097075054, 8800733955
Website: https://www.entrepreneurindia.co
https://www.niir.org
As per FDAs UDI final ruling, GUDID will accept only DI information and it is the responsibility of the medical device manufacturer to maintain the PI information for any future regulatory and quality audits
This report analyzes the worldwide markets for Personal Protective Equipment in US$ Million Tby the following Product Segments: Traditional Workwear, Protective Clothing, Respiratory Protection Equipment, Eye & Face Protection Equipment, Hearing Protection Equipment, Head Protection Equipment, Fall Protection Equipment, and Others (Gloves and Footwear). The report provides separate comprehensive analytics for the US, Canada, Japan, Europe, Asia-Pacific, Middle East, and Latin America. Annual estimates and forecasts are provided for each region for the period 2007 through 2015. Also, a six-year historic analysis is provided for these markets. The report profiles 356 companies including many key and niche players such as 3M Company, Aearo Company, Alpha Pro Tech, Ltd., Ansell Healthcare Products LLC, Avon-ISI, BartelsRieger Atemschutztechnik GmbH & Co, Bekina NV, Bj
This one's an overview on the Bureau of Indian Standards which covers everything starting from its background to what it does and all the way to the certification procedures. But the main focus, however, is on the procedure part, especially the conformity assessment procedures in India and how BIS plays a role in the same.
Overview And Principles Of Standardization 27 May 2008anisazila
Similar to Medical Device Usability: Polly Shelton presents at UK UPA (Usability Professional’s Association) and London Ergonomics Group (LEG) joint. (20)
Unleash Your Inner Demon with the "Let's Summon Demons" T-Shirt. Calling all fans of dark humor and edgy fashion! The "Let's Summon Demons" t-shirt is a unique way to express yourself and turn heads.
https://dribbble.com/shots/24253051-Let-s-Summon-Demons-Shirt
Book Formatting: Quality Control Checks for DesignersConfidence Ago
This presentation was made to help designers who work in publishing houses or format books for printing ensure quality.
Quality control is vital to every industry. This is why every department in a company need create a method they use in ensuring quality. This, perhaps, will not only improve the quality of products and bring errors to the barest minimum, but take it to a near perfect finish.
It is beyond a moot point that a good book will somewhat be judged by its cover, but the content of the book remains king. No matter how beautiful the cover, if the quality of writing or presentation is off, that will be a reason for readers not to come back to the book or recommend it.
So, this presentation points designers to some important things that may be missed by an editor that they could eventually discover and call the attention of the editor.
You could be a professional graphic designer and still make mistakes. There is always the possibility of human error. On the other hand if you’re not a designer, the chances of making some common graphic design mistakes are even higher. Because you don’t know what you don’t know. That’s where this blog comes in. To make your job easier and help you create better designs, we have put together a list of common graphic design mistakes that you need to avoid.
Transforming Brand Perception and Boosting Profitabilityaaryangarg12
In today's digital era, the dynamics of brand perception, consumer behavior, and profitability have been profoundly reshaped by the synergy of branding, social media, and website design. This research paper investigates the transformative power of these elements in influencing how individuals perceive brands and products and how this transformation can be harnessed to drive sales and profitability for businesses.
Through an exploration of brand psychology and consumer behavior, this study sheds light on the intricate ways in which effective branding strategies, strategic social media engagement, and user-centric website design contribute to altering consumers' perceptions. We delve into the principles that underlie successful brand transformations, examining how visual identity, messaging, and storytelling can captivate and resonate with target audiences.
Methodologically, this research employs a comprehensive approach, combining qualitative and quantitative analyses. Real-world case studies illustrate the impact of branding, social media campaigns, and website redesigns on consumer perception, sales figures, and profitability. We assess the various metrics, including brand awareness, customer engagement, conversion rates, and revenue growth, to measure the effectiveness of these strategies.
The results underscore the pivotal role of cohesive branding, social media influence, and website usability in shaping positive brand perceptions, influencing consumer decisions, and ultimately bolstering sales and profitability. This paper provides actionable insights and strategic recommendations for businesses seeking to leverage branding, social media, and website design as potent tools to enhance their market position and financial success.
7 Alternatives to Bullet Points in PowerPointAlvis Oh
So you tried all the ways to beautify your bullet points on your pitch deck but it just got way uglier. These points are supposed to be memorable and leave a lasting impression on your audience. With these tips, you'll no longer have to spend so much time thinking how you should present your pointers.
White wonder, Work developed by Eva TschoppMansi Shah
White Wonder by Eva Tschopp
A tale about our culture around the use of fertilizers and pesticides visiting small farms around Ahmedabad in Matar and Shilaj.
Medical Device Usability: Polly Shelton presents at UK UPA (Usability Professional’s Association) and London Ergonomics Group (LEG) joint.
1. UKUPA and LEG, January 2012
Medical Device Usability
| Polly Shelton
| Human Factors Consultant
Page 1 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
2. pdd
Consumer
v
Industrial
Design
Medical
FMCG
Services
Page 2 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
3. History
Computer Aided
Design & Manufacture
Industrial Design
& Prototyping Engineering &
Analysis
1980 1990
Page 3 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
4. Innovation planning
Human factors Design Insight
2000 2010
Page 4 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
5. Medical devices group
Radiation Medical
does meter oxygen
Urine meter
Needle safe
catheter IV Pump
Wireless Inhaler dose
stethoscope counter
Heart-rate
monitor
Auto
injector
device Pen injector
Page 5 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
6. What is a medical device?
A product used to diagnose, treat or monitor a medical condition
Device classes (FDA)
• Class I – general controls
• Class II – general controls w/ special controls
• Class III – general controls & premarket approval
• & Combination products.
These products should all undergo a
usability engineering approach
Depending on the class, more or less HFE required
Class depends on complexity, use characteristics, risk
to patient & amount of regulatory control needed
Page 6 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
7. Human Factors and Usability
International/EU FDA (USA)
ISO/IEC 62366 ANSI/AAMI HE:75
Ensuring that products are developed for safe
and effective use and that potential use-errors
have been mitigated.
Page 7 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
8. History
International FDA (USA)
ISO/IEC 62366 ANSI/AAMI HE:75
Usability engineering process Medical device manufacturers
must demonstrate that all
International (EU & FDA) potential use-related hazards in
6 pages (+80page annex) their devices have been
identified, tested, and mitigated
Helps “design in” usability
and “design out” use error The device must be shown to be
Used with ISO 14971 usable by the target population
in the intended environment.
Page 8 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
9. History
International FDA (USA)
ISO/IEC 62366 ANSI/AAMI HE:75
A virtual encyclopaedia that Provides design principles
provides human factors and practical guidelines
engineering design guidance,
25 chapters, 465 pages
case studies, and checklists
Officially recognised by FDA
in October 2010
FDA draft guidance document
June 2011
Page 9 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
10. History
Human factors engineering guidelines and preferred
1988 AAMI HE: practices for the design of medical devices
Human factors engineering guidelines and preferred practices
1993 AAMI HE:48 for the design of medical devices
1999 Institute of Medicine Report : to Err is Human
2001 ANSI/AAMI HE:74 Human factors design process for medical devices FDA
Medical devices – Application of usability engineering March
2007 ISO/IEC 62366 to medical devices 2010, EU
Human factors design process for medical devices
2009 ANSI/AAMI HE:74 (reaffirmed)
FDA
November
2009 ANSI/AAMI HE:75 Human factors engineering – Design of medical devices
2010, FDA
Page 10 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
11. Process flowchart: IEC/ISO 62366
Page 11 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
12. Page 12 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
13. Usability & Risk management
How do they interact?
ISO 62366 has a complex flowchart
showing how.
Page 13 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
14. Usability engineering process
Insight Iteration Validation
Page 14 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
15. Expert
interviews
Task analysis
Iteration Validation
Insight
Desk research
Ethnographic Contextual
research enquiry
Page 15 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
16. Concept
Anthropometry &
refinement
biomechanics
Insight Validation
Iteration
Interaction design
Formative Use error risk
& prototyping
usability testing analysis
Page 16 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
17. Usability goals and Residual risk
acceptance criteria
Insight Iteration
Validation
Summative
usability testing
Regulatory
submission
Page 17 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
18. Usability engineering process
Expert
Formative usability
interviews
testing
Anthropometry &
biomechanics
Usability goals and
Ethnographic acceptance criteria Residual risk
research
Task analysis Concept
refinement
Insight Iteration Validation
Regulatory
Summative
submission
usability testing
Desk research
Contextual Use error
enquiry Interaction design risk analysis
& prototyping
Page 18 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
19. Ethnographic research
Observation, shadowing, journaling, contextual enquiry, journey mapping
• Gain a holistic view of your users, the
task and the context of use
• Identify opportunities and pain points
• Deep-dive into users needs and
experiences
• Create personas, scenarios,
journey/experience maps
Page 19 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
20. Desk research
User and task characteristics, human factors design guidelines
• Understand your stakeholders
and the problem
• Review similar products
• Gather anthropometric and
strength data
• Source best practice design
guidelines
Page 20 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
21. Development & refinement
Prototyping, cognitive walkthrough, heuristic review, formative usability tests
• Involve users early and often
• Test concepts
• Identify design problems early
• Understand mental models and user interactions
• Involve designers and clients where possible
• Inform the design
Page 21 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
22. Task analysis and user FMEA
Cognitive walkthrough, use scenarios, journey and task mapping
• Understand the task
• Identify physical, sensory and
cognitive challenges
• Identify and understand
potential risks and foreseeable
use error
• Control and mitigate use error
risk through design
Page 22 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
23. Validation & residual risk
Formal summative testing against quantitative usability goals
• Identify critical task scenarios
• Validate risk mitigation effectiveness
through testing with users
• Error analysis
• Root cause analysis
• Residual risk analysis and mitigation
• Follow-up testing
• Regulatory submission
Page 23 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
24. Medical device user testing
• Why do usability testing?
• Formative vs. summative
• Recruiting participants
• Legal and ethical considerations
• Task selection
• Setting usability goals
• Test environment
• Set-up and pilot
• Moderating sessions
• Providing assistance
• Providing training
• Testing instructions
• Data collection and error logging
• Error analysis and classification
Page 24 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
25. Why do usability testing?
Increase device sales,
Meet device
customer loyalty,
regulators’
reduce customer
expectations
support
Improve ease-of-use, Improve safety and
ease of learning and reduce medical
user experience device use errors
Page 25 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
26. For medical devices, poor usability
is literally a matter of life or death
Page 26 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
27. Some stats 100
medical
device alerts
“One third of medical device reports issued
each year involve use-error and human (MHRA,2010)
factors are inherent in virtually all and 403
device-related incidents.” FDA Manufacturer’s
10,280 Field Safety
Corrective
adverse Actions
incident Deaths and undertaken
reports in UK serious injuries
2010 (MHRA) involving
301 medical devices
An estimated15-
fatalities
up 29% 20% of all errors
last year
2,383 due to
mechanical
serious failure; 60-80%
injuries human error*
Page 27 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
*Institute of Medicine Report 2000, & Perrow 1984)
28. Formative vs. summative tests
What’s the difference?
Formative Summative
• Part of the iterative design process • Determines if a finished device is vulnerable to
• Prototypes of any fidelity potential harmful use errors
• More explorative, informing the design • Production equivalent devices
• Identifies an evolving device’s strengths • Tasks must include primary operating functions
and weaknesses and all safety-related tasks
• Extensive discussion of likes and dislikes • Actual or simulated use environments
• Preliminary usability objectives • Focus is on use-safety
• Small samples (5-8 users per group) • Effectiveness, efficiency, user satisfaction & ease
• User research and marketing questions of user learning also evaluated
can easily be incorporated • Usability goals and acceptance criteria
• Modest size report including design • Record use errors, close calls and root causes
recommendations • Can be large sample sizes (25+ per user group)
• 30-90 minute sessions • Training and final IFU may be included
• 90 minute + sessions
Page 28 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
29. Recruiting participants
• Reflect the actual user population
• Define inclusion and exclusion criteria within each user group
• Include users with physical, cognitive and sensory impairments
• Consider co-morbidities
• Use a medical specialist agency with BHBIA/EPhMRA
• Allow at least 4-6 weeks for recruitment
• Over-recruit by 10%
• Be prepared for difficult recruits and arranging in-home testing
where necessary
• Beware of “professional” participants
‐ ask for ID / registration numbers where possible
‐ patients to bring current medication/ a recent prescription
• Pilot your recruitment materials
• Test in different markets
Page 29 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
30. Legal and ethical considerations
• Ethics committee / IRB approval
• BHBIA and EPhMRA codes of conduct
• Data protection act, 1998
• ABPI code of conduct
• Adverse events and pharmacovigilance
• Country-specific requirements
• Video recording, streaming and transfer
• Incentives (BMA guidance):
‐ minimum level
‐ proportionate to time involved
‐ appropriate to respondent type and task nature
‐ air market value of the services provided
‐ vouchers and not cash for children
• Testing with children
Page 30 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
31. Task selection
• If possible, test every user task!
• If not, prioritize by:
‐ Safety Critical tasks, and
‐ Frequent tasks
• Address worst case scenarios where possible
• Link tasks to risk management
Page 31 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
32. Usability goals
• Derived from:
‐ Risk analysis
‐ Primary operating functions
‐ User interface requirements
‐ Worst case scenarios
• Define acceptance criteria by criticality and risk level
But….
• Don’t base the success of your test on the achievement of
usability goals against quantitative acceptance criteria!
• Always conduct:
‐ root cause analysis
‐ follow-up risk analysis
Page 32 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
33. Test environment
• Simulate the real context of use where possible
– Lighting levels
– Thermal environment
– Noise
– Furniture
– Other equipment
– Potential hazards
– Working space
– Multi-tasking and distractions
• Consider testing devices in-field
– In the home
– In clinical environments (e.g. ICU)
• For high risk products (e.g. infusion pumps)
consider validation with clinical evaluation.
Page 33 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
34. Set-up and pilot
• Test logistics, protocol and timings
• Equipment checklist
• Video recording
‐ Define work area
‐ If possible, use two cameras to capture detailed
interactions and scene-view
‐ Make sure the audio is ON!
‐ Remote control
• Live streaming
• Seating arrangements:
‐ Sit behind / next to participant
‐ Observer out of field of view
• Consider recruiting “real” pilot participant
• Pilot data collection materials and analysis
Page 34 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
35. Moderating sessions
• Protect participants’ rights
• Be professional and build trust
• Create rapport; but stay in charge
• Ask open questions
• Don’t be a robot! – remember the script is a guide
• Be unbiased
• Be consistent
• Let the participant speak
• Take care with “think-aloud”
• Give participants time
• Let participants struggle… but keep them safe
• Watch non-verbal cues
• Probe where appropriate to understand root causes
• Good seminar from UIE and Beth Loring
http://www.uie.com/events/virtual_seminars/good_moderating/
Page 35 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
36. Providing assistance
• An assisted task is a task failure
• Observing participants struggle, commit use errors or near
misses provides insight
• Only assist as a last resort
- safety risk
- repeated pattern of unproductive behaviour
- task abandonment
- after pre-established time-limit
- in order to continue
• Provide appropriate levels of assistance
• Pre-establish prompts for consistency
• All interventions should be reported
Page 36 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
37. Training
• Simulate the real-life training scenario
• Consider including a retention interval
• Simulate the worst case scenario where
appropriate
• Consider including some untrained users
Do not…
• Deliver better than normal training!
• Ask trainees to study for their test!
Page 37 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
38. Testing instructions
• Simulate the anticipated situation-of-use
• Ask users to behave naturally
- refer to IFU/ quick guide as they would at home
• Follow-up to address IFU in detail
- test all risk mitigations and warnings
• Evaluate and rate specific attributes e.g.
‐ Clarity of text
‐ Terminology
‐ Pictograms
‐ Structure & flow
‐ Text size and legibility
‐ Ease of use
‐ Ease of finding information
‐ Completeness
‐ Density of information
Page 38 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
39. Data collection and error logging
• Collect data concurrently
• If possible, have 2 people taking notes
• Debrief and align data post-session
• Review video where required Task log and Excel spread sheet
Data:
• Success rate
• Use errors, close calls, operational
difficulties and their root causes
• Assistance provided
• Verbatim comments
• Other observations and behaviours
• Responses to questions and ratings
Page 39 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
40. Error analysis & classification
• FDA are interested in understanding the root causes of errors
• Probe to understand observed errors
• ISO 62366 includes a taxonomy (adapted from James Reason)
• Close calls and operational difficulties should also be recorded
Page 40 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
41. Case Study: IV infusion set
• Pre-summative test
Changes
• 22 participants
recommend
• 3 hour sessions to design &
• 13 primary usability goals instructions
• 40 secondary usability goals
Training
needs
11 primary &
identified
14 secondary
usability
goals
10 15 Contextual
unmet major use additional insights
errors risks gathered
observed identified
to FMEA
Page 41 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
42. Case Study: easypod device Improved
ease-of-use,
ease-of-dosing,
comfort,
confidence
& safety
Ethnographic
fieldwork ROI: drug
identified sales rose
Full User need
Centred 7.5% in the
Design first year to
approach €58m
User
requirements
specification
Page 42 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
43. Further reading
Some good resources to get started
Page 43 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk
44. Get in contact
and keep updated via …
Page 44 COMMERCIAL IN CONFIDENCE | PDD Group Ltd 2011 ISO 9001 & ISD 134BS registered | www.pdd.co.uk