This document discusses the need for algorithmvigilance, which is the systematic monitoring of algorithms used in healthcare to identify unintended impacts and ensure equitable outcomes. As the use of AI and machine learning grows in healthcare decision-making, there is evidence of data and algorithmic biases that can differentially impact populations. The author argues that just as pharmacovigilance monitors drug effects, algorithmvigilance is increasingly important to evaluate algorithms, understand adverse effects, and prevent inequities. An algorithm-driven healthcare learning cycle is proposed that continuously monitors effects, identifies needed changes, and improves algorithms based on real-world use.

1. Algorithmovigilance:
Considerations for Systematic
Monitoring and Continuous
Learning for Equitable AI-Driven
Healthcare
Peter J. Embi, MD, MS, FACP, FACMI, FIAHSI
President & CEO, Regenstrief Institute
Betley Professor of Medicine & Associate Dean,
IU School of Medicine
Associate Director, Indiana CTSI
VP for Learning Health Systems, IU Health
Incoming Chair, Biomedical Informatics,
Vanderbilt University Medical Center
Session: Translating Digital Equity
CTSA Program Annual Meeting
December 3, 2021

2. Algorithm-driven Healthcare
• Growth in Algorithm-driven healthcare via decision support
• Growing evidence of:
• Data and algorithmic bias
• Differential impacts across populations
• Unintended impacts
• Health equity concerns
• Need for a “learning health system” approach to:
• Monitor whether algorithm-driven healthcare has the intended effects
• Mechanisms to identify and respond to unintended effects
• An analogy from traditional clinical trials…

3. Clinical Trial Phases
Phase I:
• Testing for
Safety
• 20-80
participants
Phase II:
• Testing for
Efficacy
• 100-300
participants
Phase III:
• Testing for
Effectiveness
• 300-3000
FDA Review/
Approval
• Approved
for market
use
Phase IV: Post-
marketing
• Post-
marketing
testing
• Thousands+

4. • Science relating to the collection, detection,
assessment, monitoring, and prevention of
adverse effects with pharmaceutical products.
Pharmacovigilance:
• Individual Case Safety Reports (standards)
• Coding of adverse events
• Seriousness determination
• Expedited reporting
• Spontaneous reporting
• Aggregate reporting
Adverse Effect Reporting
Pharmacovigilance
Processes for discovering adverse effects
Systematic surveillance approaches growing
Increasingly leveraging EHRs and related data for
monitoring of safety and effectiveness

5. ”Algorithmovigilance”
“The scientific methods and
activities relating to the evaluation,
monitoring, understanding, and
prevention of adverse effects of
algorithms in health care.”
Akin to pharmacovigilance for
monitoring drug effects
Increasingly important as AI/ML-
derived algorithms are used

6. Need to
monitor
healthcare
Algorithms
Biases in Data:
Known and Unknown
Caution about
generalizability
Unexpected results
can be expected
Must monitor to
promote trust
Need new systems
and approaches
Essential for safe,
equitable, effective care

7. Algorithm-driven healthcare learning cycle
Monitor to
determine
effects
Change
based
upon
findings
Deploy
and use in
healthcare
settings

8. Thanks!
@embimd
pembi@regenstrief.org
peter.embi@vumc.org