The document discusses the capabilities and limitations of machine learning (ML) in medicine, highlighting its potential to dramatically improve prognosis, diagnostic accuracy, and patient management by utilizing vast amounts of data. However, it also points out challenges such as unclear diagnostic standards, reliance on structured data, and limitations in predictive power without sufficient historical context. Solutions proposed include collecting diverse data types and reframing complex health questions to better inform predictions and interventions.

1. Machine Learning and
Prediction in Medicine
Chad You

3. Agenda
Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5953825/
• Capabilities of ML in Medicine
• Limitations of ML
• Solutions

4. Capabilities of
ML in Medicine
• Improve prognosis dramatically
• Data could be drawn directly
from EHRs or claims databases,
allowing models to use
thousands of rich predictor
variables.
• Precisely identify metastatic
cancer patients with high
mortality rates.
• Prognostic algorithms will come
into use in recent years.

5. Capabilities of
ML in Medicine
• Displace much of the work of
radiologists and anatomical
pathologists
• Algorithms can replace a second
radiologist reading
mammograms and will soon
exceed human accuracy.
• Algorithms need no sleep, and
their vigilance is the same at
2am as at 9am.

6. Capabilities of
ML in Medicine
• Improve diagnostic accuracy
• Algorithms will soon generate
differential diagnoses, suggest
high-value tests, and reduce
overuse of testing.

8. Limitations of
ML
• Diagnosis
• Gold standard for diagnosis is unclear
in many conditions, which makes it
harder to train algorithms.
• High-value EHR data are often stored
in unstructured formats that are
inaccessible to algorithms without
layers of preprocessing.
• Models need to be built and
validated individually for each
diagnosis.

9. Limitations of
ML
• Prediction
• No ML can squeeze out information
that is not present.
• Clinical data alone have relatively
limited predictive power for
hospital readmissions.

10. Limitations of
ML
• Prediction
• Accumulating multiple years of
historical data is worse than simply
using the most recent year of data.
• Half-life of clinical data: 4 months

11. Limitations of
ML
• Prediction
• To assess the usefulness of
prediction models, we must evaluate
the accuracy in predicting future
events, not on their ability to
recapitulate historical trends.

12. Limitation Example

13. Solutions
• Get varieties of data
• social-demographics
• personal genomics
• mobile-sensor readouts
• patient’s credit history
• Web-browsing logs

14. Solutions
• Reframing complex phenomena in
terms of limited multiple-choice
questions
• Will you have a heart attack
within 10 years?
• Are you more or less likely than
average to end up back in the
hospital within 30 days?

15. Solutions
• Predicting events early enough for a
relevant intervention to influence care
decisions and outcomes.