The presentation aims at raising the regulatory awareness of human reproduction healthcare professionals, introduce tools and techniques that can help them on regulatory journey and avoid pitfalls.

1. Copyright © 2023 Vlad Stirbu
Certifying artificial intelligence
Vlad Știrbu
ESHRE 2023
Copenhagen
This work is licensed under a Creative Commons Attribution 4.0 International License.

2. Copyright © 2023 Vlad Stirbu
Who am I?
Current
● University of Jyväskylä, PostDoc Researcher
● University of Helsinki, PostDoc Researcher
● CompliancePal, Founder and lead developer, Expert
witness
● RegOps Days, Organizer
Past
● Nokia Technologies, Principal Software Engineer
● Nokia Research Center, Researcher

3. Copyright © 2023 Vlad Stirbu
No conflict of interest

4. Copyright © 2023 Vlad Stirbu
Learning objectives
● Regulatory landscape awareness
● Tools and techniques
● Avoid regulatory pitfalls

5. Copyright © 2023 Vlad Stirbu
AI regulations in media vs reality
● General product safety (2001)
● Medical Device Directive (2007)
● Medical Device Regulation (2017)
● AI Act (2023)

6. Copyright © 2023 Vlad Stirbu
Definitions
Artificial intelligence: the theory and development of computer systems that are
able to perform tasks that normally require human intelligence, such as visual
perception, speech recognition, learning, decision-making, and natural language
processing (IEEE Position Statement)
Medical software: medical device functionality that is implemented in software

7. Copyright © 2023 Vlad Stirbu
Design Control
Safety
Effectiveness
Adapted from FDA 1997

8. Copyright © 2023 Vlad Stirbu
AI Development Phases
Deployment
Software engineers
● Integrate model into the
larger software system
● Deploy and operate
✓ Application
Data preparation
Data engineers
● Data sources
● Label data
✓ Training data
Training
Data scientists
● Develop algorithms
● Perform experiments
with model candidates
✓ Models

9. Copyright © 2023 Vlad Stirbu
Technical debt in machine learning systems
Sculley et al. 2015

10. Copyright © 2023 Vlad Stirbu
AI development in medical products
Deployment
Capability to detect data and
concept drifts
Continuous quality assurance
Model redeployment (see FDA
2023)
Automation: DevOps, RegOps
(Stirbu et al 2022)
Data preparation
Data assessments
Data management
Data Cards (Pushkarna et al. 2022)
Training
Monitoring metrics
Assessment of actual medical
benefits
Model Cards (Mitchell et al 2019)
Regulatory lock

11. Copyright © 2023 Vlad Stirbu
Cybersecurity, Privacy, Information Security
● Connected systems
● Regulations
○ MDR/FDA Guidelines
○ GDPR
○ HIPAA
● Standards
○ ISO 27001

12. Copyright © 2023 Vlad Stirbu
Regulatory approache differences
USA
● Risk-benefit centered
● Recognized Third Parties
UE
● Conformance centered
● Notified Bodies

13. Copyright © 2023 Vlad Stirbu
Design Control Revisited
Adapted from FDA 1997
AI alignment
Traceability
Explainability
Interpretability

14. Copyright © 2023 Vlad Stirbu
Conclusions
Establish QMS provisions covering AI development
Documented proof of AI alignment

15. Copyright © 2023 Vlad Stirbu
[1] IEEE Position Statement Artificial Intelligence,
https://globalpolicy.ieee.org/wp-content/uploads/2019/06/IEEE18029.pdf
[2] FDA - Center for Devices and Radiological Health: Design Control Guidance for Medical Device Manufacturers,
1997
[3] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F.,
Dennison, D.: Hidden technical debt in machine learning systems. In: Proceedings of the 28th International
Conference on Neural Information Processing Systems. NIPS’15, 2015
[4] FDA - Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial
Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions, 2023
[5] Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. 2022. Data Cards: Purposeful and Transparent
Dataset Documentation for Responsible AI. In 2022 ACM Conference on Fairness, Accountability, and
Transparency (FAccT '22)
[6] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena
Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of
the Conference on Fairness, Accountability, and Transparency (FAT* '19)
[7] Stirbu, V., Granlund, T. & Mikkonen, T. Continuous design control for machine learning in certified medical
systems. Software Qual J (2022).
Bibliography