Early stage usability testing

PERSUADE BUDGET HOLDERS TO RELEASE
FUNDS FOR EARLY STAGE TESTING
Richard Featherstone
www.medical-device-usability.com 1

www.medical-device-usability.com
WHY IS IT DIFFICULT TO GET BUDGET FOR EARLY
STAGE TESTING?
‘It’s too risky - what if we find
something’s wrong?’
‘It’s too expensive – money is tight’
‘It’s too late - we’ve already
decided what we want’
‘It’s not essential – let’s just do the
validation’
‘It will delay submission’
22

‘IT’S TOO RISKY - WHAT IF WE
FIND SOMETHING’S WRONG?’
3www.medical-device-usability.com

A TRUE STORY
No formative evaluations with user types:
Two studies with employees
Performed by the product design company
• uFMEA written by engineers
• Design decisions made with no user
input
• “There are no critical tasks”
Submitted to regulators
Submitted to regulators
Formative evaluations with all user types:
Tested all elements of user interface
Independent of product designers
• Outputs fed into uFMEA
• Outputs fed into Design Reviews
• Outputs fed into Critical Task definitions
• Outputs fed into validation protocol
• $2bn+ turnover
• 9,000 employees
• Global, based in Europe
• Developing a product type on FDA
Watch List
• Aim – to enter a $bn market quickly
Company A
• $2bn+ turnover
• 9,000 employees
• Global, based in Europe
• Developing a product type on FDA
Watch List
• Aim – to enter a $bn market quickly
Company B

HF reviewer - “not approvable”
Clock stopped
Senior management replaced
All device development frozen
Delay of approx. 1 year
No access to a $bn market
HFE data accepted without
comment
Now in launch mode
Access to a $bn dollar market
A TRUE STORY
Rewrite uFMEA
Redefine the Critical tasks
Run a formative evaluation
Run the HF validation
Write a Usability SOP

‘IT’S TOO EXPENSIVE – MONEY IS
TIGHT’

WE ALL WANT THE LEAST EXPENSIVE OPTION

FORMATIVE COSTS IN PERSPECTIVE
Formative study RevalidationSummative study Delayed market entry
Assuming things go well… Assuming they don’t…
Redesign

COST OF DELAYED ENTRY TO MARKET
Lack of good quality early analysis can cause
• Delayed / extended reviewing time
• Questions / requests for clarification
Delay is expensive:
• Average daily sales of a top 10 drug = $15m
• Lost sales are removed from the END of life
because patent expiry doesn’t move

EARLY TESTING IS THE LEAST EXPENSIVE APPROACH
 Formative studies reduce risk in
the validation study
 Increase the likelihood of
successful review
 You only want to validate ONCE

FIXING A MISTAKE LATE IS VERY EXPENSIVE
 Formative studies reveal
problems early
 Give you opportunity to find
out why, and to fix it
 Fixing a problem late is
expensive and difficult
 If you do it early you have
maximum control

THERE IS STILL A PROBLEM WITH DEVICE DESIGN

‘IT’S TOO LATE - WE’VE ALREADY
DECIDED WHAT WE WANT’

LET’S REMIND OURSELVES ABOUT FORMATIVE
STUDIES
 Can be very quick to perform
 Uncover difficulties with the user interface
 Explore alternative designs
 A robust formative program informs the validation protocol
 Give weight to the uMFEA
 Help you to develop the IFU
 Make you sure that your critical tasks are right

‘IT WILL DELAY SUBMISSION’

PREVENT UNEXPECTED SURPRISES DURING
VALIDATION
 [FDA guidance on unexpected use errors in validation]
 If you have done your formative program well, there should be no
surprises in validation
16
…the application of HFE/UE during the development of…products
reduces the need for design modifications and costly updates after
market introduction and offers competitive advantages.

EARLY STAGE TESTING CAN SPEED UP APPROVAL
 Less likelihood of problems with
HF review
 Early stage testing reveals
unexpected use problems
 It should always be on the
CRITICAL PATH

‘IT’S NOT ESSENTIAL – LET’S JUST
DO THE VALIDATION’

FORMATIVE TESTING IS MORE IMPORTANT THAN
VALIDATION
Formative Summative
Allow you to make Small decisions
Changes are inexpensive
Force you to make BIGdecisions
Changes are EXPENSIVE
Flexibility to make changes Changes are constrained
Methods are flexible Methods are constrained
Respond quickly to new information New information could stop everything

TEST EARLY – AND GET RID OF THE BAD IDEAS

TEST EARLY – AND DESIGN OUT THE BIG PROBLEMS

TEST EARLY - AND STAY FLEXIBLE
www.medical-device-usability.com 22www.medical-device-usability.com

WHY SENIOR MANAGEMENT SHOULD PROVIDE FUNDS
FOR EARLY STAGE TESTING
 Early testing allows you to make small decisions early rather than big
decisions late
 It is the most cost effective / lowest risk option
 It increases your likelihood of a rapid approval

THANK YOU
Richard Featherstone
richard@medical-device-usability.com

Early stage usability testing

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Early stage usability testing

Similar to Early stage usability testing (20)

Recently uploaded

Recently uploaded (20)

Early stage usability testing