An overview of the “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)” https://www.fda.gov/media/122535/download Presented on May 19th, 2019 in Chicago at the BSN 2019 workshop on "Using mHealth Technology to Enable the Clinical Trial of the Future". Disclaimer: Views/thoughts expressed in these slides are mine and do not necessarily reflect those of my employer.

1. REGULATING AI & ML IN MEDICINE
Shyamal Patel, PhD
Digital Medicine & Translational Imaging
Pfizer, Inc.
An overview of the “Proposed Regulatory Framework for Modifications
to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a
Medical Device (SaMD)”
https://www.fda.gov/media/122535/download
5/19/19 1

2. AI, machine learning and deep learning
Artificial Intelligence
Machine Learning
Deep Learning
1950s Today1980s
ApplicationBreadth
Automated Driving
Speech Recognition
RoboticsObject Recognition
Bioinformatics
Recommender Systems
Spam Detection
Fraud Detection
Weather Forecasting
Algorithmic Trading
Sentiment Analysis
Medical Diagnosis
Health Monitoring
Computer Board Games
Machine Translation
Knowledge Representation
Perception
Reasoning
Interactive Programs
Expert Systems
25/19/19

3. AI/ML around us
Source: https://beebom.com/examples-of-artificial-intelligence/
35/19/19

4. Medicine is becoming increasingly data driven
Chaussabel, D., & Pulendran, B. (2015). A vision and a prescription for big data-enabled medicine. Nature
Publishing Group, 16(5), 435–439. http://doi.org/10.1038/ni.3151
45/19/19

5. AI/ML in medicine
Sources: SkinVision, PathAI, CheXNet
assist radiologists in
detection of pneumonia
from chest X-rays at a
level exceeding practicing
radiologists
assist pathologists in
making diagnoses and
identify patients that
benefit from novel
therapies
assist consumers in
performing self-checks of
skin health and provide
indications for risk of skin
cancer
55/19/19
Skin Health Pathology Reads Radiology Workflow

6. Classical programming VS AI/ML
• learn efficiently from large, heterogeneous datasets
• learn complex non-linear relationships
• adapt to the unique needs of each individual
• update continuously or when more data becomes available
Advantages of AI/ML
Classical Programming
Data + Rules = Answers
AI/ML
Data + Answers = Rules
65/19/19

7. When would an AI/ML-based solution require
premarket submission for an algorithm change?
Sensor data
Images
…
Diagnosis
Intervention
…
Input AI/ML algorithm Output
75/19/19

8. Risk categorization of Software as a Medical Device
(SaMD)
SaMD When software is intended to be used for one or more
medical purposes without being part of a hardware medical
device
AI/ML-based
SaMD
When algorithms are intended to diagnose, treat, cure,
mitigate or prevent diseases
85/19/19

9. Types of AI/ML-based SaMD modifications
• Changing the types of input signals
• Changing compatibility with different devices
Modifications related to inputs
• Retraining with new data
• Changes in AI/ML architecture or parameters
Modifications related to
performance
• Changing the significance of information
• Changing the intended patient population
Modifications related to
intended use
i
ii
iii
95/19/19
Sensor data
Images
…
Diagnosis
Intervention
…

10. Proposed Total Product Life-Cycle (TPLC) approach
105/19/19

11. 1. Quality systems and Good Machine Learning
Practices (GMLP)
• Acquire data in a consistent, clinically relevant and generalizable manner
• Maintain proper separation between training, tuning and testing datasets
• Maintain appropriate level of clarity of output and algorithm to end user
• Define performance metrics and success criteria in advance
• Select AI/ML technique after careful analytical consideration of factors like simplicity,
interpretability, accuracy, robustness, speed and scalability
115/19/19

12. 2. Modification plan submission during initial
premarket review
Scope of modifications
SaMD Pre-specifications
(SPS)
Manage & control risks
Algorithm Change Protocol
(ACP)
SPS
Inputs
Performance
Intended Use
ACP
Data
Management
Retraining
Performance
Evaluation
Update
Procedures
125/19/19

13. 3.Approach for modifications with an established
SPS andACP
135/19/19

14. 4. Real-world performance monitoring and
transparency
Monitoring Understand how product is being used in the real-world, identify
opportunities for improvement and respond proactively to safety or
usability concerns
Transparency Provide periodic updates to the stakeholders (e.g. FDA, collaborators,
clinicians, patients) on implementation of updates (as per SPS and
ACP) and related changes in performance metrics
Monitoring/repor
ting type and
frequency based
on
risk of the device
number and type of modifications
maturity of algorithms
145/19/19

15. Example 1: Intensive care unit
SPS Modify algorithm to ensure consistent
performance across sub-population
Reduce false alarm rates while
maintaining or increasing sensitivity
ACP Methods for database generation,
reference standard labeling, and
comparative analysis
Specification of performance
requirements and statistical analysis
plan
Detect onset of
physiological
instability
Risk category III: ‘drive clinical management’ in a ‘critical healthcare situation or condition’
155/19/19
ECG, Blood Pressure, Pulse
Oximetry

16. Example 1: Intensive care unit
Modification
Scenario
1A
Algorithm modified in accordance with ACP to achieve lower false-
alarm rate while maintaining sensitivity on an independent
validation dataset
Update algorithm and labeling in accordance with modified SaMD
performance and communicate to users
165/19/19
Detect onset of
physiological
instability
Risk category III: ‘drive clinical management’ in a ‘critical healthcare situation or condition’
ECG, Blood Pressure, Pulse
Oximetry

17. Example 1: Intensive care unit
Modification
Scenario
1B
Algorithm re-trained on additional data can now predict onset of
physiological instability 15 minute in advance while maintaining
same sensitivity and false-alarm rate
Update algorithm, labeling and intended use to indicate change in
alarm condition
175/19/19
Detect onset of
physiological
instability
Risk category III: ‘drive clinical management’ in a ‘critical healthcare situation or condition’
ECG, Blood Pressure, Pulse
Oximetry

18. Example 2: Skin lesion mobile medical app
SPS Improve sensitivity and specificity of
skin lesion characterization by using
real-world data
Extend usage with other smartphones
with similar image acquisition
capabilities and monitor performance
ACP Methods for database generation,
reference standard labeling, and
comparative analysis
Acceptance criteria for image
acquisition systems, design of
validation study and plan for real-world
performance
Physical
characteristics of skin
lesion
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
185/19/19
Smartphone camera image

19. Example 2: Skin lesion mobile medical app
Modification
Scenario
2A
Algorithm actively learning on real-world data achieved improved
sensitivity and specificity in analytical validation (per ACP)
Update algorithm and labeling in accordance with modified SaMD
performance and communicate to users
195/19/19
Physical
characteristics of skin
lesion
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
Smartphone camera image

20. Example 2: Skin lesion mobile medical app
Modification
Scenario
2B
Analytical validation (per ACP) demonstrated two additional
smartphones with similar image acquisition systems achieved
performance consistent with initial system
Update labeling to reflect new app compatibility and communicate to
users
205/19/19
Physical
characteristics of skin
lesion
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
Smartphone camera image

21. Example 2: Skin lesion mobile medical app
Modification
Scenario
2C
New algorithm analyses the physical characteristics of skin lesions
and provides recommendation to the user based on assessment
of malignancy
Distribute a new app that will now be patient-facing (instead of the
dermatologist)
215/19/19
Physical
characteristics of skin
lesion
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
Smartphone camera image

22. Example 3: X-ray feeding tube misplacement
SPS Improve accuracy of incorrect tube
placement by using real-world data
Extend notifications to nursing staff
based on achieving pre-specified
performance
ACP Methods for real-world data collection,
reference standard, performance, and
comparative analysis
Analytical validation of performance
improvement and clinical validation for
high confidence cases
Detect feeding tube
placement error
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
225/19/19
Chest X-ray

23. Example 3: X-ray feeding tube misplacement
Modification
Scenario
3A
Algorithm re-trained and re-validated on real-world data achieved
higher accuracy and clinical validation of high confidence
cases
Update algorithm with new version and add notifications to nursing
staff in parallel with radiologists for high confidence cases
235/19/19
Detect feeding tube
placement error
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
Chest X-ray

24. Example 3: X-ray feeding tube misplacement
Modification
Scenario
3B
New algorithm trained and validated to identify pneumonia using
development and validation process similar to SPS and ACP with
some adaptations.
Market new algorithm for pneumonia detection from chest x-ray
245/19/19
Detect feeding tube
placement error
Risk category II: ‘drive clinical management’ in a ‘serious healthcare situation or condition’
Chest X-ray