Downloaded 26 times
Uploaded byICS
Machine Learning in Medical Devices Webinar
The document discusses machine learning in medical devices. It describes how the FDA regulates machine learning as a medical device and proposes a framework for modifications to AI/ML models. The framework allows manufacturers to make pre-approved changes if they follow a predetermined change control plan or submit changes through an algorithm change protocol, software pre-specifications, and total product lifecycle approach. The document also discusses developing medical devices with machine learning and important safety considerations.
More Related Content
Similar to Machine Learning in Medical Devices Webinar
![[Wroclaw #6] Medical device security](https://cdn.slidesharecdn.com/ss_thumbnails/medicaldevicesecurityowaspnopictures-170531065557-thumbnail.jpg?width=640&height=640&fit=bounds)
Machine Learning in Medical Devices Webinar
- 1. Machine Learning in MedicalDevices June 18, 2020 Milton Yarberry, Director of Medical Programs, ICS Stephen Olsen, Senior FAE Manager,BlackBerry QNX
- 2. 2 Agenda ● Machine Learning ○What is it? ○ Why is this special? ○ The FDA’s treatment ● Use case: Medical devices ○ How to make use of AI or ML in a medical device ● Medical Device Safety Considerations ● Development lifecycle with Machine Learning ● Safeguards for Medical Devices with Machine Learning Machine Learning in Medical Devices
- 3. Google’s DeepDream What doesa network trained to see dogs see in an image without dogs? Initially it was invented to help scientists and engineers to see what a deep neural network is seeing when it is looking in a given image. 3
- 4.
- 5. What is AI/ML? Without usingexplicit instructions Relying on patterns and inference instead AI FDA defines ML as “…techniques used to design and train software algorithms to learn from and act on data…” “…when intended to treat, diagnose, cure, mitigate, or prevent disease or other conditions, are medical devices…” 1 5
- 6. Why ML isdifferent. What’s hard? Wikipedia “without using explicit instructions, relying on patterns and inference instead.” Recognition, categorization but not procedural Not ... word processors email clients multimedia players video games Apocryphal, tanks or clouds? Verification, local minima Overtraining (remembering the answers) Prove “Ground truth” (vs. gold standard) Different from conventional algorithms Why ML now? ML applications Challenges Compute power Since 1984, Processor power Up ~1,000,000X Data storage Up ~5,000,000X 6
- 7. ML products withalgorithms that continually adapt based on new data require a new pre-market submission (510K). The Problem 7
- 8. April 2, 2019 FDAproposes regulatory framework for modifications to an AI/ML https://www.fda.gov/media/122535/download New from the FDA 8
- 9. Summary, FDA allows trustedmanufacturers to make pre-approved changes if they follow a predetermined change control plan FDA’s Proposed Framework 9
- 10. Summary, FDA allows trustedmanufacturers to make pre-approved changes if they follow a predetermined change control plan or, FDA allows changes to an ML-enabled SaMD with an SPS using ACP in a TPLC. FDA’s Proposed Framework 10
- 11. FDA’s Proposed Framework SPS ACP TPLC 11 SaMDSoftware as a Medical Device (phone, laptop, tablet) SaMD Pre-Specifications (this is what’s going to change) Algorithmic Change Protocol (how the change is controlled) Total Product Life Cycle (an ML lifecycle that includes data)
- 12. Software as aMedical Device ● Software that runs on a generic computing platform (computer, mobile phone, tablet computer) ● "Software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device.“(IMDRF) ● No specially made hardware, doesn’t control hardware motors, sensors, switches SaMD Examples: Viewer for MRI images, ECG app, radiological image analysis (Computer Aided Detection) SaMD 12
- 13. SPS – SaMDPre-Specification ● Anticipated modifications to performance or inputs or intended use ● A region of potential changes, around specifications and labeling ACP – Algorithm Change Protocol ● Plan to control risks while making changes ● Step-by-step process ○ Data management ○ Re-training ○ Performance evaluation ○ Update procedures SPS & ACP 13
- 14. A regulatory framework 4principles: 1. Quality systems and good machine learning practice (corporate reputation) 2. Initial premarket assurance of safety and effectiveness (premarket review) 3. Approach for modifications after initial review with an established SPS and ACP (change plan) 4. Transparency and real-time monitoring of AI/ML-based SaMD (PAY ATTENTION) TPLC – Total Product Life Cycle 14
- 15. TPLC on anAI workflow 15
- 16. FDA’s Software Precertification(Pre-Cert) Program https://www.fda.gov/medical-devices/digital-health/digital-health-software-precertification-pre-cert-program 16
- 17. Old-way vs. New-way(Pre-Cert) Software Precertification Program: Working Model – Version 1.0 – January 2019 17
- 18. 18 Agenda ● Machine Learning ○What is it? ○ Why is this special? ○ The FDA’s treatment ● Use case: Medical devices ○ How to make use of AI or ML in a medical device ● Medical Device Safety Considerations ● Development lifecycle with Machine Learning ● Safeguards for Medical Devices with Machine Learning Machine Learning in Medical Devices
- 19. Software Foundation —Safe ● Safe Real Time OS (QNX) ○ Certificate IEC 62304/61508 ○ Safety Manual ○ Fault Tolerant ○ APIs (POSIX) ○ Networking ○ Well Tested/Deployed 19
- 20. ● Security Features ○User permissions ○ Application permissions ○ FIPS 140-2 ○ OpenSSL ○ Secure File System ○ Lifecycle Management ○ Virus Detection/Eradication Software Foundation — Security 20
- 21. Machine Learning inCloud ● Collect test set data on Edge ● Propagate data securely to cloud (On/Off Premises) ● Train the model ● Distribute model securely to Edge devices Test Data Machine Learning 21
- 22. Security: Multilayered withLifecycle Management 22
- 23. Research Security Lifecycle Support Product Planning Sourcing Test Production Contract Award Field TrialsMaintenance Development Integration Refresh Security Lifecycle Support Field Updates Penetration testing Forensic services Incident response Security best practices training Regulatory compliance support Security tooling & testing Secure boot Access control Virtualization Secure reference implementations Runtime separation & isolation Secure over-the-air software Secure manufacturing Managed PKI FIPS certified crypto Complete Security Lifecycle Support 23
- 24. QNX Runtime Security— Layered and completely integrated Applications Temporal & Spatial Control Runtime Integrity Application Sandboxing Authorization Tamper Resistance HW Validation Medical Device Manufacturer and 3rd party apps (embedded & connected) Control & Restrict CPU usage, Resource Access, Protect against defects and rogue execution Monitor system behavior, Intrusion detection logging & reporting Whitelisting, Pathspace control, Resource Access, Abilities, Trusted Code Execution Mandatory Access Control (MAC) Secure boot, Signed Execution, Image Verification, Integrity Measurement model Unique Certificate through Certicom Secure Manufacturing 24
- 25. SafeGuards ● Training wheelsso you cannot fall. ● Validating the output of the model. Ensuring that it does not break major rules. 25
- 26. Summary ● Machine Learning ●Use case: Medical devices ○ How to make use of AI or ML in a medical device ● Medical Device Safety Considerations ● Development lifecycle with Machine Learning ● Safeguards for Medical Devices with Machine Learning 26