Getting Your Medical Device FDA Approved


Published on

Slides presented at "Getting Your Medical Device FDA Approved" event, presented by Mentor Graphics Embedded Software, discussing how to address the enhanced scrutiny from government agencies that can introduce significant delays with the commercial release of software-related medical devices.

Getting through the FDA review as quickly as possible requires a clear understanding of the development standards, documentation and testing that is now expected for Medical devices. During this session we discussed how FDA hot buttons affect your medical device submission will be discussed, including:

-Requirements for software development as outlined in IEC 62304
-Content considerations for premarket submissions
-Human factors engineering as a platform for enhanced user safety
-Provisions for data security and protection against unauthorized wireless access

We reviewed the design control requirements and product development approach that can shorten your medical device's path to market with a focus on safety, human factors engineering and security.

Published in: Technology, Business

Getting Your Medical Device FDA Approved

  1. 1. Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Qt is a registered trade mark of Digia Plc and/or its subsidiaries. All other trademarks mentioned in this document are trademarks of their respective owners. Seminar June 2014 Getting Your Medical Device FDA Approved
  2. 2. 2
  3. 3. Why and How Companies Fail  Software documentation inadequate; includes: *  not provided at all,  missing description,  missing traceability matrix,  missing list of anomalies, and/or  missing validation * “Analysis of Pre‐Market Review Times Under the 510(k) Program,” US Food and Drug  Administration,  CDRH, November 9, 2011 Affects 17% of  submissions! 3
  4. 4. Why and How Companies Fail Source: “Medical Device Recall Report: FY2003 to FY2012,” US Food and Drug Administration,  CDRH, March 21, 2014 “Failure to implement software design controls, and where  appropriate, testing procedures, as well as increasing complexity of  the medical device use environment (with increased connectivity and  interoperability) can lead to software anomalies often requiring a  correction or removal.” Software  Change Control Software  Design S/W Design  (Mfg.) Sum % of CDRH  Recalls 2008 13 141 2 156 18.3% 2009 9 111 1 121 15.4% 2010 4 73 3 80 8.9% 2011 11 182 10 203 15.8% 2012 12 169 5 186 15.5% Total 49 676 21 746 15.1% 4
  5. 5. Why and How Companies Fail “Failure to follow or otherwise address current guidance  document(s) or recognized standards ‐ FDA issues  guidance documents or recognizes a national or international  standard to help manufacturers determine what information  to include in a 510(k) submission generally and for certain  device types specifically. If a manufacturer fails to follow  current guidance (i.e. that which is up‐to‐date) for a  certain device type or a recognized standard, and offers  no explanation for its failure to do so, FDA would  consider that submission to be of poor quality and would  issue an AI Letter that quotes current guidance to obtain the  missing information.” * “Analysis of Pre‐Market Review Times Under the 510(k) Program,” US Food and Drug  Administration,  CDRH, November 9, 2011 41% of submissions affected! 5
  6. 6. The Primary Roadmap  Guidance for the Content  of Premarket Submissions  for Software Contained in  Medical Devices   DeviceRegulationandGuidance/  GuidanceDocuments/ucm089543.htm (Issued in 2005) 6
  7. 7. FDA Software Guidance  Software “level of concern”  Estimate (in the absence of mitigations) of the severity  of injury that a device failure or latent design flaw could  permit or inflict, either directly or indirectly, on a  patient or device operator:  Major: could result in death or serious injury   Moderate: could result in minor injury  Minor: unlikely to cause any injury  Documentation FDA recommends submitting  depends on the level of concern 7
  8. 8. Software‐related Documentation  Overview  Describe the design of your device  Document how your design was implemented  Demonstrate how the device produced by your design  implementation was tested  Show that you identified hazards appropriately and  managed risks effectively  Provide traceability to link together  design,  implementation, testing, and risk management 8
  9. 9. Software‐related Documentation  Device Hazard Analysis  Address all foreseeable hazards  (including intentional or inadvertent  misuse)  Identification of the hazardous event  Severity of the hazard  Cause(s) of the hazard  Method of control (e.g., alarm, hardware  design)  Corrective measures to eliminate, reduce, or  warn  Verification of correct control  implementation 9
  10. 10. Software‐related Documentation  Software Requirements Specification  Hardware Requirements  Programming Language Requirements  SW Performance and Functional Requirements  Architecture Design Chart  Software Design Specification  Traceability Analysis  SW Development Environment Description  V&V Documentation  Revision Level History  Unresolved Anomalies (Bug List) 10
  11. 11. Software Development Standard  IEC 62304:2006  Medical device software – Software life cycle  processes  SW development  SW maintenance  SW risk management   SW configuration  management  SW problem  resolution 11
  12. 12. Overview – IEC 62304:2006  Describes software development and maintenance  processes for medical devices but does not specify:  An organizational structure or responsibilities  The name, format or explicit content of documentation  The specific life‐cycle model to be employed  FDA Consensus Standard (September 2008)  IEC62304 conformance fulfills the “Software Development  Environment Description” section of a 510(k) submission  Normative standard in Europe for CE marking 12
  13. 13. Software Verification and Validation (V&V)  MINOR LOC:  System or device level testing, any integration testing  Pass/fail criteria  Summary of test results  MODERATE LOC:  V&V activities at the unit, integration, and system level  Summary list of V&V activities  Pass/fail criteria  Test results  MAJOR LOC:  V&V activities at the unit, integration, and system level  Unit, integration and system‐level test protocols  Pass/fail criteria  Test report, summary, test results 13
  14. 14. Software‐Related Documentation  General Principles of  Software Validation:  Final Guidance for  Industry and FDA Staff   MedicalDevices/  DeviceRegulationand Guidance/  GuidanceDocuments/  ucm085281.htm (Issued in 2002) 14
  15. 15. Software Design and Human Factors  Weave human factors engineering into entire design and  development process, including device design  requirements, analyses, and tests  Consider device safety and usability issues when  developing flowcharts, state diagrams, prototyping tools,  and test plans  Perform task and function analyses, risk analyses,  prototype tests and review, and full usability tests  Include participants from the user population(s) 15
  16. 16. Common User Interface (UI) Issues  UI complexity causes user confusion, delay in use,  or inability to use the device  UI makes it difficult for user to correct data entry  errors or modify device settings in a timely  fashion  UI falsely causes the user to believe a critical  situation exists when it does not, or vice‐versa  UI does not draw attention to dangerous  conditions of device operation or patient status  UI does not prevent known, likely data input  errors  16
  17. 17. Regulatory Basis for Human Factors  21 CFR 820.30, Design Controls (The need for human  factors is implied):  Design input – includes “needs of the user and  patient”  Design verification – performance criteria met  Design validation – “… devices conform to defined  user needs and intended uses and shall include  testing of production units under actual or  simulated use conditions. Design validation shall  include software validation and risk analysis….”  [incl. use‐related risks] 17
  18. 18. Human Factors Standards AAMI/ANSI HE75:2009 Human factors engineering – Design of medical devices 18 ISO/IEC 62366:2007 Medical devices – Application of  usability engineering to medical devices
  19. 19. FDA Human Factors Guidance  Applying HF&UE to  Optimize Medical  Device Design   MedicalDevices/  DeviceRegulationand Guidance/  GuidanceDocuments/  ucm259748.htm   NOTE: issued in 2011 –It  is not yet in effect but it  reflects FDA‐CDRH’s  current thinking and  approach to human  factors 19
  20. 20. 2011 Draft Human Factors Guidance FDA’s best thinking regarding:   Device Users, Use Environments and User Interfaces  Use‐Related Hazard Analytical Methods  Formative Evaluations  Mitigation and Control of Use‐Related Hazards  Design Verification Testing  Human Factors Validation 20 “Use error caused by designs that are either overly complex or contrary  to users' intuitive expectations for operation is one of the most  persistent and critical problems encountered by FDA.” ‐ General Principles of Software Validation, Final Guidance for Industry and FDA Staff  (2002)
  21. 21. Submission Problems with HF/UE  Not understanding that “usability goals” are not included  in FDA’s recognition of the standard  Simply state “in compliance” with 62366 and submit no HF  report  Not using US residents in validation studies (w/exceptions)  Devices modified to mitigate use error issues, then the  mitigation is not directly tested in validation testing  Formative evaluations either not done or pretending to be  validation testing  Overemphasis on complex protocol, technique, and test  conditions while the test focus, data, interpretations and  report are impossible to interpret.  21
  22. 22. A Good HF/UE Validation Report Includes:  Intended device users, uses, use environments,  and training  Device user interface (graphical & verbal  description)  Summary of known use problems  User task selection, characterization and  prioritization  Summary of formative evaluations  Validation testing 22
  23. 23. Validation Testing  Rationale for test type selected (i.e., simulated use or  clinical evaluation)   Number and type of test participants and rationale for  how they represent the intended user populations   Test goals, critical tasks and use scenarios studied   Technique for capturing unanticipated use errors   Definition of performance failures   Test results: Number of device uses, success and failure  occurrences   Subjective assessment by test participants of any critical  task failures and difficulties   Description and analysis of all task failures, implications  for additional risk mitigation 23
  24. 24. FDA’s Cybersecurity Concern  Network‐connected devices disabled by  malware  Targeting of patient data using wireless  technology  Uncontrolled passwords management  Failure to address security vulnerabilities via  update  Security vulnerabilities in OTS software ‐ Source:  FDA Safety Communication (June 13, 2013) 24
  25. 25. FDA Cybersecurity Guidance  Content of Premarket  Submissions for  Management of  Cybersecurity in  Medical Devices   MedicalDevices/  DeviceRegulationand Guidance/  GuidanceDocuments/  ucm356186.htm   NOTE: issued in 2013 –It is  not yet in effect but it  reflects FDA‐CDRH’s current  thinking and approach to  cybersecurity 25
  26. 26. 2013 Draft Cybersecurity Guidance General Principles:  Confidentiality – only authorized persons or  entities have access  Integrity – Accurate and complete data, not  improperly modified  Availability – Accessible and usable on a  timely basis in the expected manner 26
  27. 27. 2013 Draft Cybersecurity Guidance Documentation:  Hazard analysis addressing intentional and  unintentional cybersecurity risks  Traceability matrix linking controls to risks  Systematic plan for updates and patches  Demonstrate how the device will be  provided free of malware  IFU and specs for recommended anti‐virus  s/w and/or firewall 27
  28. 28. Conclusions:  Buy a copy of device‐relevant standards  Get free FDA guidance documents online  Read them  Gap assess your processes and current  records – then fix them  implement tools that drive consistent  outputs  Hire a HF/UE Engineer (or make a  consultant happy) 28
  29. 29. Thank You! For questions or assistance, contact me at or (972) 505‐1920 29
  30. 30. The Standard ‐ Illustrated 30
  31. 31. Medical Device Software under IEC 62304 George Romanski
  32. 32. © IEC 62304 • Medical Device Software – Software Lifecycle  Processes – Quality Management System* – RISK MANAGEMENT – Software Safety Classification – Development Process – Maintenance Process – Configuration Management* – Problem Reporting and Management* IEC‐62304 Medical Software * These processes are Universal between the standards
  33. 33. © Software Safety Assessment • For Avionics – ARP‐4761 (Safety Assessment)  • For Medical – ISO 14971 (Safety/Risk) Normative reference  • For Automotive  – Part of ISO 26262 • For Industrial – Part of IEC 61508 • For Trains – Part of EN 50128 IEC‐62304 Medical Software Different terms:  Design Assurance Level, Software Integrity Level, Class Same Principle: Consequence/Exposure, Severity ‐> Level for software Level chosen for System  then applied to Software ‐‐‐ How do you assess for RTOS?
  34. 34. © What level is Device Software?  • Software Faults do not follow “Gauss Normal”  Distribution (Bell Curve) – Given 10,000 lines of code – You test and find 10 software faults – What is the probability of finding another fault  with another test? • We Don’t know distribution of faults • Software must be assumed to be level C until  shown by Risk Assessment that a lower level is  applicable!  IEC‐62304 Medical Software
  35. 35. © Software Risks • Does it do what it should? • Does it do More than it should? • It does something wrong • Is an action late • Is an action too early • Are a sequence of actions in the wrong order? How can we be sure?  “enough” ALARP – As Low as Reasonably Practicable (RISK) IEC‐62304 Medical Software
  36. 36. © Software of Unknown Provenance (SOUP) RTOS Software (SOUP) Tasks Tasks Tasks Semaphores Semaphores ISRs Kalman Filter  Software (SOUP) “Wrapper” With ‘thin’ interface Medical Device Application Message  Queues Message  Queues ‘others’ Behavior managed by RTOS IEC‐62304 Medical Software
  37. 37. © Software of Unknown Provenance (SOUP) RTOS Software (SOUP) Tasks Tasks Tasks Semaphores Semaphores ISRs Kalman Filter  Software (SOUP) “Wrapper” With ‘thin’ interface Medical Device Application Message  Queues Message  Queues ‘others’ Behavior managed by RTOSDifferent SOUPs IEC‐62304 Medical Software
  38. 38. © Failure conditions potentially induced by RTOS • Data is incorrectly modified.  • Incorrect results are provided to the application. • Expected results are not provided or are provided past their  deadlines. • User code is not executed as expected (not run, incorrectly  run, and incorrectly sequenced). • Fault conditions are not detected. • Fault conditions are handled incorrectly. • False failure conditions are reported. • Incorrect or untimely response provided by the RTOS to  external or user generated events. IEC‐62304 Medical Software
  39. 39. © Failure conditions potentially induced by RTOS • Data is incorrectly modified.  • Incorrect results are provided to the application. • Expected results are not provided or are provided past their  deadlines. • User code is not executed as expected (not run, incorrectly  run, and incorrectly sequenced). • Fault conditions are not detected. • Fault conditions are handled incorrectly. • False failure conditions are reported. • Incorrect or untimely response provided by the RTOS to  external or user generated events. IEC‐62304 Medical Software Reduce risk by Using Certified  RTOS
  40. 40. © Managing Risk ‐ ISO 14971 Risk Management Plan Identify Risks Evaluate Control Verify Risk Management Processes Risk Management File IEC‐62304 Medical Software
  41. 41. © Medical Device – Based on Risk Assessment Develop  Hardware Develop  Software Integrate System Risk Analysis Risks Functional Requirements System IEC‐62304 Medical Software
  42. 42. © System Hazard – Software Class • System Hazard – No Injury – A – Non‐Serious injury – B – Death or Serious injury – C • If failure in Software leads to System Hazard • Software categorizes as – Class – A – Class – B – Class – C IEC‐62304 Medical Software ISO 14971 IEC 62304
  43. 43. © Risk Analysis • Failure Mode Analysis • Identify Potential Risks • Determine Risk Exposure – Continuous while operational – Frequent – Occasional • Can Software contribute to Hazards • Can Hazards be mitigated • Finding potential hazards in Software can be TOUGH! IEC‐62304 Medical Software
  44. 44. © Requirements Hierarchies – DO‐178B System Requirements High‐Level Requirements Low‐Level Requirements ARP 4754A Intended  System Behavior Intended  Software Behavior DO‐178C Requirements based on Software Architecture IEC‐62304 Medical Software
  45. 45. © Requirement/Design organization Risk Management Requirements Low‐Level Design ISO 14971 Intended  System Behavior Intended System/  Software Behavior IEC 62304 Software based on Requirements IEC‐62304 Medical Software
  46. 46. © Parameter Data Items  • Parameter Data Items can be developed and verified  separately if certain conditions are met • The high‐level requirements describe how the software uses  the parameter data items • The low‐level requirements define the structure, attributes  and allowable values of the parameter data items • Verification should show that every data element has the  correct value – or correct value is in equivalence class and  boundaries are verified e.g. Configuration Vectors IEC‐62304 Medical Software
  47. 47. © Our Experience • Develop verification evidence using a DO‐178C  software Lifecycle • All of the other Lifecycle processes will fall into  place.  • Some additional documents required – Safety Plan – Safety Manual – Staff Competency Assessment • Actual assessment required not just Resume IEC‐62304 Medical Software
  48. 48. © More Experience  • Interpretations and negotiations are prevalent  in Automotive, Medical and Rail industries. • TUV were strict on first Verocel certification – Plans, procedures and standards approved – Subsequent certifications were straightforward – “Manufacturing – reject tracking” needed to be  addressed (for software)?   IEC‐62304 Medical Software
  49. 49. © Finding and eliminating unintended behavior • Requirements describe intended behavior • Software developed against requirements (TRACED) • Tests written against requirements (ONLY) and  (TRACED) • Software coverage by tests measured • Any software not covered demonstrates “unintended  behavior”  • This is a risk that must be eliminated.    IEC‐62304 Medical Software
  50. 50. © Code Coverage Analysis • Requirements used to specify intended behavior • Write tests using Requirements ONLY – Normal Range – Robustness – Equivalence Class – Boundary Value • Run tests and measure how much code was executed • Assess is non‐executed code should not be there Coverage Analysis not required explicitly by IEC 62304, but Hard to mitigate “Unintended Functionality risk” without IEC‐62304 Medical Software
  51. 51. © Coding Standards • Standards Required – used to show goodness of  various properties • Code Layout • Code consistency • Readability • Avoidance of risky constructs • Etc.   IEC‐62304 Medical Software
  52. 52. © Code Review • Verifies Conformance to Standards • Verifies conformance to review criteria • Verifies code against intended behavior – Low‐level Design – Low‐level requirements IEC‐62304 Medical Software
  53. 53. © Code Analysis Tools (The silver bullet?) • Perform consistency checks • Perform checks against defined coding rules  (e.g. MISRA C) to find errors like: – Use of variable before initialization – Indexing out of bounds (simple cases) – Potential deadlock – Unreachable code (sometimes) – Arithmetic overflow (sometimes) Good quality step, reduction of potential faults, BUT! IEC‐62304 Medical Software
  54. 54. © Code Analysis Tools for Certification • Checks are incomplete • Hard to assess what checks must be  completed manually • No analysis against intended behavior – Low‐level design – Low‐level requirements Only partial credit may be taken! IEC‐62304 Medical Software
  55. 55. © Configuration Management Processes  • Identification  – Versions – Baseline • Change Control  – Documented process – Change Control Board • Configuration Accounting – History tracking – Access Controls IEC‐62304 Medical Software
  56. 56. © Segregation of Software Components RTOS APP_One Class C APP_Two Class A Components can be segregated and  treated as Class C on same computer. Fault Containment Memory Partitioning Time Partitioning Required App‐Two cannot adversely affect Class C software IEC‐62304 Medical Software
  57. 57. © Audit Risk • Scenario 1 – Prepare Verification evidence on paper – Instruct Engineers to give YES/NO answers – All information available, but difficult to locate.  – Auditor cannot make good assessment – Applicant passes with substandard/incomplete audit.  Not the Verocel Way! IEC‐62304 Medical Software
  58. 58. © The Verocel Approach • Plans, QA records, CM‐data, and Certification  data:  – Hyperlinked data – easy to find and trace • All data open and put on DVD‐ROM – Auditors can trace their own copies • All data extracted from Traceability database,  and CM repository IEC‐62304 Medical Software
  59. 59. © • Develop Plans and  Standards • Develop Certification  evidence using P&S • QA Checks that  P&S are  followed • Review Plans and Standards • Sample Cert evidence • Check QA Audits Auditors approach  If a controlled process was used consistently  And Sample is good  Then we can trust the rest of the certification  evidence and the software AuditorsDevelopers/Certifiers IEC‐62304 Medical Software
  60. 60. Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Qt is a registered trade mark of Digia Plc and/or its subsidiaries. All other trademarks mentioned in this document are trademarks of their respective owners. Andrew Caples June 2014 Mentor Embedded Solutions
  61. 61. 2 2 Trends for Mixed Criticality  Needed to meet stringent non-functional requirements — Cost — Space — Weight — Heat generation — Power consumption  Issues — Partitioning for safety assurance — Sharing for efficient resource usage — Hard Tasks must be guaranteed — Soft Tasks given best possible service — Must ensure the behavior of low criticality components does not adversely impact the behavior of higher criticality components
  62. 62. 3 3 Memory Protected Memory Protected Memory Protected Memory Protected Nucleus Processes  Memory Protected Modules — Prevents sub-systems from bringing down the system — No Virtual Addressing  Multiple Types — Applications — Libraries — Hybrids  Integrated with Sourcery CodeBench — Build projects via wizards — Debug / load modules — Profile module execution File Systems Peripheral Bus Drives GUINetworking Power Aware Kernel StorageLCD Ethernet/ Wireless Devices Memory Protected Application 1 Task 1 Task 2 … Task n Library 1 Function 1 Function 2 … Function n Hybrid 1 Task 1 Function 1 … Task n Function n Application 2 Task 1 Task 2 … Task n
  63. 63. 4 4 Nucleus Process Model Root Kernel Image User Process n User Process 2 User Process 3 Hardware User Process 1 User Process 2
  64. 64. 5 5 Foreground / Background Mode Static Application Root Kernel Image User Process n User Process 2 User Process 3 Hardware User Process 1 User Process 2
  65. 65. 6 66 Mentor Embedded Offerings
  66. 66. 7 77 Mentor Embedded United States Canada United Kingdom Ireland Netherlands Germany Denmark Sweden Finland Poland Armenia Russia China Japan Korea Taiwan Australia Singapore India Pakistan Israel Egypt Hungary ItalyAustria Switzerland Spain France Brazil 400+ engineers 50+ engineers in lead OSS community roles 10,000+ accepted OSS changes Deployed in 3 Billion+ devices 20,000+ Sourcery CodeBench users “Since 1996, Mentor Graphics has been the only EDA vendor with a broad product line of embedded tools and software IP. Integrating our software and hardware expertise more readily enables our customers to deal with the challenges of today’s multicore and power management complexities.” Walden Rhines, CEO