Quality Systems and Good Machine Learning
Practices
By: Nikita Angane
FDA expects every medical device manufacturer to have a robust and compliant quality system.
FDA has been taking great strides in establishing regulations for the digital health industry that
also facilitates research and development while maintaining high quality products.
In our previous blog: ARTIFICIAL INTELLIGENCE & MACHINE LEARNING
TECHNOLOGIES, we talked about how FDA plans on regulating devices with Artificial
Intelligence(AI) and Machine Learning (ML) technologies. In this blog, we will dive deeper into
how the FDA expects to see the integration of Quality System and Good Machine Learning
Practices (GMLP).
FDA envisions applying a Total Product Lifecycle (TPLC) regulatory approach for companies
developing AI/ML-based SaMD. The TPLC approach will assess the culture of quality and
organizational excellence that will give FDA assurance of the quality of software products
developed by that company. This approach is the core of the Software Pre-Cert Program.Error!
Bookmark notdefined.
Please read our blogs for more information on the Software Pre-Cert Program:
FDA’S DIGITAL SOFTWARE PRE-CERTIFICATION PROGRAM
AN UPDATE ON FDA’S DIGITAL SOFTWARE PRE-CERTIFICATION PROGRAM
This approach is particularly important for AI/ML-based devices because of its ability to adapt
and improve using real-world performance data. Another important element of this TPLC
approach is the integration of the Quality System and the Good Machine Learning Practices.
This helps balance the benefits, risks, safety, and effectiveness of the device while allowing for
the necessary technological advancements.
Good Machine Learning Practices are the best practices which are aligned with good software
engineering practices and good quality system practices. Some elements of GMLP are:Error!
Bookmark notdefined.
 The relevance of the available data to the clinical problem
 Data is acquired in a consistent manner and aligns with the intended use and the
modification plans submitted to the FDA
Page 2 of 2
 The appropriate separation between training, tuning and test datasets
 The clarity in the output and the algorithm used.
As part of the validation activities for the AI/ML devices, the manufacturers must demonstrate a
valid clinical association, analytical validation and clinical validation as per the SaMD: Clinical
Evaluation Guidance. The data required in the premarket review and in the study design depends
on the risk posed by the device to the users and patients, its intended use.i
AI/ML driven devices learn from real world data and hence prompts unique regulatory
considerations. FDA’s proposed regulatory framework is open for public comments. You can
submit your suggestions or comments on FDA’s Proposed Regulatory Framework for
Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical
Device (SaMD) - Discussion Paper and Request for Feedback using the following link
https://www.regulations.gov/comment?D=FDA-2019-N-1185-0001i
For any other questions that you may have about the proposed regulatory framework, please
contact us at 248-987-4497 or info@emmainternational.com.
iFDA (Jan 2019) Proposed Regulatory Framework for Modificationsto Artificial Intelligence/Machine Learning
(AI/ML)-Based Software as a Medical Device(SaMD) - Discussion Paper and Request for Feedback retrieved on 04-
10-2019 from
https://www.fda.gov/MedicalDevices/DigitalHealth/SoftwareasaMedicalDevice/ucm634612.htm
ValidClinical Association
•Is there a validclincial
associationbetweenthe
SaMD output andyour
SaMD targetedclinical
condition?
Analytical Validation
•Does yourSaMD correctly
process data to generate
accurate, reliable and
precise output data?
Clinical Validation
•Does use of your SaMD's
accurate, reliable and
precise output data
acheieve yourintended
purpose inyour target
populationinthe context
of clinical care?
Data Source: FDA (Jan 2019) Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/M L)-Based Software
as a Medical Device (SaMD) - Discussion Paper and Request for Feedback retrieved on 04-10-2019 from
https://www.fda.gov/MedicalDevices /DigitalHealth/SoftwareasaMedicalD evice/ucm634612.htm

Quality Systems and Good Machine Learning Practices

  • 1.
    Quality Systems andGood Machine Learning Practices By: Nikita Angane FDA expects every medical device manufacturer to have a robust and compliant quality system. FDA has been taking great strides in establishing regulations for the digital health industry that also facilitates research and development while maintaining high quality products. In our previous blog: ARTIFICIAL INTELLIGENCE & MACHINE LEARNING TECHNOLOGIES, we talked about how FDA plans on regulating devices with Artificial Intelligence(AI) and Machine Learning (ML) technologies. In this blog, we will dive deeper into how the FDA expects to see the integration of Quality System and Good Machine Learning Practices (GMLP). FDA envisions applying a Total Product Lifecycle (TPLC) regulatory approach for companies developing AI/ML-based SaMD. The TPLC approach will assess the culture of quality and organizational excellence that will give FDA assurance of the quality of software products developed by that company. This approach is the core of the Software Pre-Cert Program.Error! Bookmark notdefined. Please read our blogs for more information on the Software Pre-Cert Program: FDA’S DIGITAL SOFTWARE PRE-CERTIFICATION PROGRAM AN UPDATE ON FDA’S DIGITAL SOFTWARE PRE-CERTIFICATION PROGRAM This approach is particularly important for AI/ML-based devices because of its ability to adapt and improve using real-world performance data. Another important element of this TPLC approach is the integration of the Quality System and the Good Machine Learning Practices. This helps balance the benefits, risks, safety, and effectiveness of the device while allowing for the necessary technological advancements. Good Machine Learning Practices are the best practices which are aligned with good software engineering practices and good quality system practices. Some elements of GMLP are:Error! Bookmark notdefined.  The relevance of the available data to the clinical problem  Data is acquired in a consistent manner and aligns with the intended use and the modification plans submitted to the FDA
  • 2.
    Page 2 of2  The appropriate separation between training, tuning and test datasets  The clarity in the output and the algorithm used. As part of the validation activities for the AI/ML devices, the manufacturers must demonstrate a valid clinical association, analytical validation and clinical validation as per the SaMD: Clinical Evaluation Guidance. The data required in the premarket review and in the study design depends on the risk posed by the device to the users and patients, its intended use.i AI/ML driven devices learn from real world data and hence prompts unique regulatory considerations. FDA’s proposed regulatory framework is open for public comments. You can submit your suggestions or comments on FDA’s Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback using the following link https://www.regulations.gov/comment?D=FDA-2019-N-1185-0001i For any other questions that you may have about the proposed regulatory framework, please contact us at 248-987-4497 or info@emmainternational.com. iFDA (Jan 2019) Proposed Regulatory Framework for Modificationsto Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device(SaMD) - Discussion Paper and Request for Feedback retrieved on 04- 10-2019 from https://www.fda.gov/MedicalDevices/DigitalHealth/SoftwareasaMedicalDevice/ucm634612.htm ValidClinical Association •Is there a validclincial associationbetweenthe SaMD output andyour SaMD targetedclinical condition? Analytical Validation •Does yourSaMD correctly process data to generate accurate, reliable and precise output data? Clinical Validation •Does use of your SaMD's accurate, reliable and precise output data acheieve yourintended purpose inyour target populationinthe context of clinical care? Data Source: FDA (Jan 2019) Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/M L)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback retrieved on 04-10-2019 from https://www.fda.gov/MedicalDevices /DigitalHealth/SoftwareasaMedicalD evice/ucm634612.htm