SlideShare a Scribd company logo
Machine learning versus traditional
statistical modeling and medical
doctors
Maarten van Smeden
Leiden University Medical Center
IBS ROeS - Lausanne
September 10, 2019
IBS ROeS 2019, Lausanne MaartenvSmeden
Left out artificial intelligence?
In medical research, “artificial intelligence” usually
just means “machine learning” or “algorithm”
IBS ROeS 2019, Lausanne MaartenvSmeden
Tech company business model
IBS ROeS 2019, Lausanne MaartenvSmeden
Tech company business model
https://bit.ly/2HSp8X5; https://bit.ly/2Z0Pfop; https://bit.ly/2KIcpHG; https://bit.ly/33IJhr9
IBS ROeS 2019, Lausanne MaartenvSmeden
Other success stories
https://go.nature.com/2VG2hS7; https://bbc.in/2Z1drXQ
IBS ROeS 2019, Lausanne MaartenvSmeden
IBM Watson winning Jeopardy
https://bbc.in/2TMvV8I
IBS ROeS 2019, Lausanne MaartenvSmeden
IBM Watson for oncology
https://bit.ly/2LxiWGj
IBS ROeS 2019, Lausanne MaartenvSmedenForsting, J Nuc Med, 2017, DOI: 10.2967/jnumed.117.190397
IBS ROeS 2019, Lausanne MaartenvSmeden
Machine learning everywhere (selection of last month)
https://bit.ly/2ka0HLq; https://go.nature.com/33TQgO6; https://bit.ly/2kp6X23; https://bit.ly/2lZuKWt; https://bit.ly/2lI298g
What are these
Machine Learning methods?
IBS ROeS 2019, Lausanne MaartenvSmeden
“Everything is an ML method”
https://bit.ly/2lEVn33
IBS ROeS 2019, Lausanne MaartenvSmeden
“ML methods come from computer science”
https://bit.ly/2zhbwPv; https://stanford.io/2TVp1xK; https://stanford.io/2ZfED0k
Leo Breiman Jerome H Friedman Trevor Hastie
CART, random forest Gradient boosting Elements of statistical learning
Education Physics/Math Physics Statistics
Job title Professor of Statistics Professor of Statistics Professor of Statistics
IBS ROeS 2019, Lausanne MaartenvSmeden
“ML methods for prediction, statistics for explaining”
Damen, BMJ, 2016, DOI:10.1136/bmj.i2416
363 developed models how many?
Decision trees 0
Random forests 0
Support vector machines 0
Nearest neighbor algorithms 0
Neural networks 1
IBS ROeS 2019, Lausanne MaartenvSmeden
“ML methods for prediction, statistics for explaining”
1See further: Kreiff and Diaz Ordaz; https://bit.ly/2m1eYdK
ML and causal inference, small selection1
• Superlearner (e.g. van der Laan)
• High dimensional propensity scores (e.g. Schneeweiss)
• The book of why (Pearl)
Wednesday 10:40-12:10 Keynote Session 3
Els Goetghebeur: Plea for a marriage of
machine learning and causal inference
IBS ROeS 2019, Lausanne MaartenvSmeden
Two cultures
Breiman, Stat Sci, 2001, DOI: 10.1214/ss/1009213726
IBS ROeS 2019, Lausanne MaartenvSmedenRobert Tibshirani: https://stanford.io/2zqEGfr
Machine learning: large grant = $1,000,000
Statistics: large grant = $50,000
IBS ROeS 2019, Lausanne MaartenvSmeden
Statistics Machine learning
Covariates Features
Outcome variable Target
Model Network, graphs
Parameters Weights
Model for discrete var. Classifier
Model for continuous var. Regression
Log-likelihood Loss
Multinomial regression Softmax
Measurement error Noise
Subject/observation Sample/instance
Dummy coding One-hot encoding
Measurement invariance Concept drift
Statistics Machine learning
Prediction Supervised learning
Latent variable modeling Unsupervised learning
Fitting Learning
Prediction error Error
Sensitivity Recall
Positive predictive value Precision
Contingency table Confusion matrix
Measurement error model Noise-aware ML
Structural equation model Gaussian Bayesian network
Gold standard Ground truth
Derivation–validation Training–test
Experiment A/B test
Adapted from Daniel Obserski: https://bit.ly/2YN12Xf and Robert Tibshirani: https://stanford.io/2zqEGfr
Language
IBS ROeS 2019, Lausanne MaartenvSmeden
ML refers to a culture, not to methods
Distinguishing between statistics and machine learning
• Substantial overlap methods used by both cultures
• Substantial overlap analysis goals
• Attempts to separate the two frequently result in disagreement
Pragmatic approach:
I’ll use “ML” to refer to models roughly outside of the traditional regression
types of analysis: decision trees (and descendants), SVMs, neural networks,
boosting etc.
Examples where “ML” has
done well
IBS ROeS 2019, Lausanne MaartenvSmeden
IBS ROeS 2019, Lausanne MaartenvSmeden
Example: retinal disease
Gulshan et al, JAMA, 2016, 10.1001/jama.2016.17216; Picture retinopathy: https://bit.ly/2kB3X2w
Diabetic retinopathy
Deep learning (= Neural network)
• 128,000 images
• Transfer learning (preinitialization)
• Sensitivity and specificity > .90
• Estimated from training data
IBS ROeS 2019, Lausanne MaartenvSmeden
Example: lymph node metastases
Bejnordi et al, JAMA, 2018, doi: 10.1001/jama.2017.14585. See our letter to the editor for a critical discussion: https://bit.ly/2kcYS0e
Deep learning competition
• 390 teams signed up, 23 submitted
• Only 270 images for training
• Test AUC range: 0.56 to 0.99
IBS ROeS 2019, Lausanne MaartenvSmeden
Deep learning on images
Many similar studies and challenges in radiology, pathology,
dermatology, opthalmology, gastroenterology, cardiology, ….
Topol, Nature Medicine, 2019, DOI: 10.1038/s41591-018-0300-7
IBS ROeS 2019, Lausanne MaartenvSmeden
Other sources of “medical” data
• Large scale gene expression data
• e.g. diagnosis of acute myeloid leukemia
https://bit.ly/2k8Ao8e
• Prognostication by text mining electronic health records
• e.g. predicting life expectancy
https://bit.ly/2k8Ao8e
• Analyzing social media posts
• e.g. pharmacovigilance, adverse events monitoring via Twitter posts
https://bit.ly/2m0KKrg
Examples where “ML” has
done poorly
IBS ROeS 2019, Lausanne MaartenvSmeden
Skin cancer and rulers
Esteva et al., Nature, 2016, DOI: 10.1038/nature21056; https://bit.ly/2lE0vV0
IBS ROeS 2019, Lausanne MaartenvSmeden
Predicting mortality – the conclusion
PlosOne, 2018, DOI: 10.1371/journal.pone.0202344
IBS ROeS 2019, Lausanne MaartenvSmeden
Predicting mortality – the results
PlosOne, 2018, DOI: 10.1371/journal.pone.0202344
IBS ROeS 2019, Lausanne MaartenvSmeden
Predicting mortality – the media
PlosOne, 2018, DOI: 10.1371/journal.pone.0202344; https://bit.ly/2Q6H41R; https://bit.ly/2m3RLrn
IBS ROeS 2019, Lausanne MaartenvSmeden
HYPE!
IBS ROeS 2019, Lausanne MaartenvSmeden
Systematic review clinical prediction models
Christodoulou et al. Journal of Clinical Epidemiology, 2019, doi: 10.1016/j.jclinepi.2019.02.004
“ML” versus traditional
statistics and medical
doctors
IBS ROeS 2019, Lausanne MaartenvSmeden
Comparison “ML” vs statistical models
• Machine learning methods versus statistical models is a false
dichotomy
• Advanced “ML” shows promise, especially in areas that are
not the traditional “tabular data” (e.g. images)
• Tabular data settings where “ML” can be compared with
traditional regression model techniques show little added value
in medical applications
IBS ROeS 2019, Lausanne MaartenvSmeden
Classification versus risk prediction
Most ML “classifiers” don’t come naturally with risk prediction, i.e.
a probability estimate of predicted outcome for individuals
• Possibly much large sample size needed to obtain reliable
(calibrated) risk predictions1 than reliable classifications
• Models can be trained to be optimized for a certain predictive
performance (e.g. AUC, sensitivity, calibration)
• Which performance to use to compare models are optimized
for different types of performance?
• What about the patient outcomes?
Van Smeden et al., Stat Meth Med Res, 2019
IBS ROeS 2019, Lausanne MaartenvSmeden
Where do we stand on “ML” vs doctors?
Domain: radiology and pathology
• Article hits: 12,000
• After screening: 22
• Out-of-sample comparison “ML” vs doctors: 2
Faes et al., Lancet preprint, 2019, https://ssrn.com/abstract=3384923
IBS ROeS 2019, Lausanne MaartenvSmeden
Fair “ML” vs doctor comparisons
Three basic principles
• Doctors should work under realistic time constraints and have
access to all regular diagnostic information, including relevant
additional diagnostic testing, unless there are compelling
reasons not to do so
• The output generated by algorithms and physicians should be
evaluated on the same scale
• Performance over-optimism should be avoided
Van Smeden et al., JAMA, 2018, doi:
IBS ROeS 2019, Lausanne MaartenvSmeden
Fair “ML” vs doctor comparisons
Several barriers for diagnosis/prognosis
• Absence of a gold standard for most diseases1
• Errors/unclear category are to be expected
• Errors are transferred to algorithm
• Risk overestimating the performance
• Which performance measures should we be looking at?
• AUC, sens/spec, predictive values, F1?
• What about patient outcomes?
1See: Reitsma, Journal of Clinical Epidemiology, 2009, doi: 10.1016/j.jclinepi.2009.02.005
IBS ROeS 2019, Lausanne MaartenvSmeden
My plea
To big data (and use it) and back to trials
• There is a need to evaluate and compare the performance of
well developed statistical learning models on patient outcomes
(e.g. survival, response to treatment, PROs, etc.)
• The analogue of test-treatment trials in diagnostic research:
algorithm-treatment trials
IBS ROeS 2019, Lausanne MaartenvSmeden
IBS ROeS 2019, Lausanne MaartenvSmeden

More Related Content

What's hot

Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
Maarten van Smeden
 
A gentle introduction to AI for medicine
A gentle introduction to AI for medicineA gentle introduction to AI for medicine
A gentle introduction to AI for medicine
Maarten van Smeden
 
Performance Metrics for Machine Learning Algorithms
Performance Metrics for Machine Learning AlgorithmsPerformance Metrics for Machine Learning Algorithms
Performance Metrics for Machine Learning Algorithms
Kush Kulshrestha
 
Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...
Maarten van Smeden
 
Data Science - Part III - EDA & Model Selection
Data Science - Part III - EDA & Model SelectionData Science - Part III - EDA & Model Selection
Data Science - Part III - EDA & Model Selection
Derek Kane
 
Algorithm based medicine
Algorithm based medicineAlgorithm based medicine
Algorithm based medicine
Maarten van Smeden
 
Big Data in Medicine
Big Data in MedicineBig Data in Medicine
Big Data in Medicine
Nasir Arafat
 
HEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHM
HEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHMHEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHM
HEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHM
amiteshg
 
The Basics of Statistics for Data Science By Statisticians
The Basics of Statistics for Data Science By StatisticiansThe Basics of Statistics for Data Science By Statisticians
The Basics of Statistics for Data Science By Statisticians
Stat Analytica
 
Big Data Analytics for Healthcare
Big Data Analytics for HealthcareBig Data Analytics for Healthcare
Big Data Analytics for Healthcare
Chandan Reddy
 
Smart Data Slides: Machine Learning - Case Studies
Smart Data Slides: Machine Learning - Case StudiesSmart Data Slides: Machine Learning - Case Studies
Smart Data Slides: Machine Learning - Case Studies
DATAVERSITY
 
Ethical Issues in Machine Learning Algorithms. (Part 1)
Ethical Issues in Machine Learning Algorithms. (Part 1)Ethical Issues in Machine Learning Algorithms. (Part 1)
Ethical Issues in Machine Learning Algorithms. (Part 1)
Vladimir Kanchev
 
Machine Learning in Healthcare Diagnostics
Machine Learning in Healthcare DiagnosticsMachine Learning in Healthcare Diagnostics
Machine Learning in Healthcare Diagnostics
Larry Smarr
 
Developing and validating statistical models for clinical prediction and prog...
Developing and validating statistical models for clinical prediction and prog...Developing and validating statistical models for clinical prediction and prog...
Developing and validating statistical models for clinical prediction and prog...
Evangelos Kritsotakis
 
Big Data, Artificial Intelligence & Healthcare
Big Data, Artificial Intelligence & HealthcareBig Data, Artificial Intelligence & Healthcare
Big Data, Artificial Intelligence & Healthcare
Iris Thiele Isip-Tan
 
Data analytics
Data analyticsData analytics
Data analytics
davidfergarcia
 
Introduction To Survival Analysis
Introduction To Survival AnalysisIntroduction To Survival Analysis
Introduction To Survival Analysis
federicorotolo
 
Data Exploration.pptx
Data Exploration.pptxData Exploration.pptx
Data Exploration.pptx
PerumalPitchandi
 
Data Mining : Healthcare Application
Data Mining : Healthcare ApplicationData Mining : Healthcare Application
Data Mining : Healthcare Application
osman ansari
 
Kaplan meier survival curves and the log-rank test
Kaplan meier survival curves and the log-rank testKaplan meier survival curves and the log-rank test
Kaplan meier survival curves and the log-rank test
zhe1
 

What's hot (20)

Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
 
A gentle introduction to AI for medicine
A gentle introduction to AI for medicineA gentle introduction to AI for medicine
A gentle introduction to AI for medicine
 
Performance Metrics for Machine Learning Algorithms
Performance Metrics for Machine Learning AlgorithmsPerformance Metrics for Machine Learning Algorithms
Performance Metrics for Machine Learning Algorithms
 
Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...
 
Data Science - Part III - EDA & Model Selection
Data Science - Part III - EDA & Model SelectionData Science - Part III - EDA & Model Selection
Data Science - Part III - EDA & Model Selection
 
Algorithm based medicine
Algorithm based medicineAlgorithm based medicine
Algorithm based medicine
 
Big Data in Medicine
Big Data in MedicineBig Data in Medicine
Big Data in Medicine
 
HEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHM
HEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHMHEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHM
HEART DISEASE PREDICTION USING NAIVE BAYES ALGORITHM
 
The Basics of Statistics for Data Science By Statisticians
The Basics of Statistics for Data Science By StatisticiansThe Basics of Statistics for Data Science By Statisticians
The Basics of Statistics for Data Science By Statisticians
 
Big Data Analytics for Healthcare
Big Data Analytics for HealthcareBig Data Analytics for Healthcare
Big Data Analytics for Healthcare
 
Smart Data Slides: Machine Learning - Case Studies
Smart Data Slides: Machine Learning - Case StudiesSmart Data Slides: Machine Learning - Case Studies
Smart Data Slides: Machine Learning - Case Studies
 
Ethical Issues in Machine Learning Algorithms. (Part 1)
Ethical Issues in Machine Learning Algorithms. (Part 1)Ethical Issues in Machine Learning Algorithms. (Part 1)
Ethical Issues in Machine Learning Algorithms. (Part 1)
 
Machine Learning in Healthcare Diagnostics
Machine Learning in Healthcare DiagnosticsMachine Learning in Healthcare Diagnostics
Machine Learning in Healthcare Diagnostics
 
Developing and validating statistical models for clinical prediction and prog...
Developing and validating statistical models for clinical prediction and prog...Developing and validating statistical models for clinical prediction and prog...
Developing and validating statistical models for clinical prediction and prog...
 
Big Data, Artificial Intelligence & Healthcare
Big Data, Artificial Intelligence & HealthcareBig Data, Artificial Intelligence & Healthcare
Big Data, Artificial Intelligence & Healthcare
 
Data analytics
Data analyticsData analytics
Data analytics
 
Introduction To Survival Analysis
Introduction To Survival AnalysisIntroduction To Survival Analysis
Introduction To Survival Analysis
 
Data Exploration.pptx
Data Exploration.pptxData Exploration.pptx
Data Exploration.pptx
 
Data Mining : Healthcare Application
Data Mining : Healthcare ApplicationData Mining : Healthcare Application
Data Mining : Healthcare Application
 
Kaplan meier survival curves and the log-rank test
Kaplan meier survival curves and the log-rank testKaplan meier survival curves and the log-rank test
Kaplan meier survival curves and the log-rank test
 

Similar to Machine learning versus traditional statistical modeling and medical doctors

Using Healthcare Data for Research @ The Hyve - Campus Party 2016
Using Healthcare Data for Research @ The Hyve - Campus Party 2016Using Healthcare Data for Research @ The Hyve - Campus Party 2016
Using Healthcare Data for Research @ The Hyve - Campus Party 2016
Kees van Bochove
 
Big data in research: possibilities and pitfalls
Big data in research: possibilities and pitfallsBig data in research: possibilities and pitfalls
Big data in research: possibilities and pitfalls
Joppe Nijman
 
Deep learning for biomedicine
Deep learning for biomedicineDeep learning for biomedicine
Deep learning for biomedicine
Deakin University
 
2016 ACS Semantic Approaches for Biochemical Knowledge Discovery
2016 ACS Semantic Approaches for Biochemical Knowledge Discovery2016 ACS Semantic Approaches for Biochemical Knowledge Discovery
2016 ACS Semantic Approaches for Biochemical Knowledge Discovery
Michel Dumontier
 
Bias in covid 19 models
Bias in covid 19 modelsBias in covid 19 models
Bias in covid 19 models
Laure Wynants
 
Final APEC ERW 25 Aug 2022.pdf
Final APEC ERW 25 Aug 2022.pdfFinal APEC ERW 25 Aug 2022.pdf
Final APEC ERW 25 Aug 2022.pdf
pantapong
 
IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...
IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...
IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...
IRJET Journal
 
IVD Market Size and Growth Trend
IVD Market Size and Growth TrendIVD Market Size and Growth Trend
IVD Market Size and Growth Trend
Bruce Carlson
 
How to compare typing techniques: do’s and Don’t’s
How to compare typing techniques:do’s and Don’t’sHow to compare typing techniques:do’s and Don’t’s
How to compare typing techniques: do’s and Don’t’s
João André Carriço
 
Possibilities and pitfalls of AI in PICU
Possibilities and pitfalls of AI in PICUPossibilities and pitfalls of AI in PICU
Possibilities and pitfalls of AI in PICU
Joppe Nijman
 
The big data challenge in healthcare and how can business intelligence best d...
The big data challenge in healthcare and how can business intelligence best d...The big data challenge in healthcare and how can business intelligence best d...
The big data challenge in healthcare and how can business intelligence best d...
HealthXn
 
InSTEDD: TED Prize Follow Up
InSTEDD: TED Prize Follow UpInSTEDD: TED Prize Follow Up
InSTEDD: TED Prize Follow Up
InSTEDD
 
Big data and machine learning: opportunità per la medicina di precisione e i ...
Big data and machine learning: opportunità per la medicina di precisione e i ...Big data and machine learning: opportunità per la medicina di precisione e i ...
Big data and machine learning: opportunità per la medicina di precisione e i ...
Fondazione Giannino Bassetti
 
The absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problemThe absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problem
Maarten van Smeden
 
Journal for Clinical Studies: Close Cooperation Between Data Management and B...
Journal for Clinical Studies: Close Cooperation Between Data Management and B...Journal for Clinical Studies: Close Cooperation Between Data Management and B...
Journal for Clinical Studies: Close Cooperation Between Data Management and B...
KCR
 
ML, biomedical data & trust
ML, biomedical data & trustML, biomedical data & trust
ML, biomedical data & trust
Paul Agapow
 
Artificial Intelligence in Medicine.pdf
Artificial Intelligence in Medicine.pdfArtificial Intelligence in Medicine.pdf
Artificial Intelligence in Medicine.pdf
zeeshan811731
 
IRJET - Prediction and Analysis of Multiple Diseases using Machine Learni...
IRJET -  	  Prediction and Analysis of Multiple Diseases using Machine Learni...IRJET -  	  Prediction and Analysis of Multiple Diseases using Machine Learni...
IRJET - Prediction and Analysis of Multiple Diseases using Machine Learni...
IRJET Journal
 
IRJET- Cancer Disease Prediction using Machine Learning over Big Data
IRJET- Cancer Disease Prediction using Machine Learning over Big DataIRJET- Cancer Disease Prediction using Machine Learning over Big Data
IRJET- Cancer Disease Prediction using Machine Learning over Big Data
IRJET Journal
 
PREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHES
PREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHESPREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHES
PREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHES
IRJET Journal
 

Similar to Machine learning versus traditional statistical modeling and medical doctors (20)

Using Healthcare Data for Research @ The Hyve - Campus Party 2016
Using Healthcare Data for Research @ The Hyve - Campus Party 2016Using Healthcare Data for Research @ The Hyve - Campus Party 2016
Using Healthcare Data for Research @ The Hyve - Campus Party 2016
 
Big data in research: possibilities and pitfalls
Big data in research: possibilities and pitfallsBig data in research: possibilities and pitfalls
Big data in research: possibilities and pitfalls
 
Deep learning for biomedicine
Deep learning for biomedicineDeep learning for biomedicine
Deep learning for biomedicine
 
2016 ACS Semantic Approaches for Biochemical Knowledge Discovery
2016 ACS Semantic Approaches for Biochemical Knowledge Discovery2016 ACS Semantic Approaches for Biochemical Knowledge Discovery
2016 ACS Semantic Approaches for Biochemical Knowledge Discovery
 
Bias in covid 19 models
Bias in covid 19 modelsBias in covid 19 models
Bias in covid 19 models
 
Final APEC ERW 25 Aug 2022.pdf
Final APEC ERW 25 Aug 2022.pdfFinal APEC ERW 25 Aug 2022.pdf
Final APEC ERW 25 Aug 2022.pdf
 
IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...
IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...
IRJET- Develop Futuristic Prediction Regarding Details of Health System for H...
 
IVD Market Size and Growth Trend
IVD Market Size and Growth TrendIVD Market Size and Growth Trend
IVD Market Size and Growth Trend
 
How to compare typing techniques: do’s and Don’t’s
How to compare typing techniques:do’s and Don’t’sHow to compare typing techniques:do’s and Don’t’s
How to compare typing techniques: do’s and Don’t’s
 
Possibilities and pitfalls of AI in PICU
Possibilities and pitfalls of AI in PICUPossibilities and pitfalls of AI in PICU
Possibilities and pitfalls of AI in PICU
 
The big data challenge in healthcare and how can business intelligence best d...
The big data challenge in healthcare and how can business intelligence best d...The big data challenge in healthcare and how can business intelligence best d...
The big data challenge in healthcare and how can business intelligence best d...
 
InSTEDD: TED Prize Follow Up
InSTEDD: TED Prize Follow UpInSTEDD: TED Prize Follow Up
InSTEDD: TED Prize Follow Up
 
Big data and machine learning: opportunità per la medicina di precisione e i ...
Big data and machine learning: opportunità per la medicina di precisione e i ...Big data and machine learning: opportunità per la medicina di precisione e i ...
Big data and machine learning: opportunità per la medicina di precisione e i ...
 
The absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problemThe absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problem
 
Journal for Clinical Studies: Close Cooperation Between Data Management and B...
Journal for Clinical Studies: Close Cooperation Between Data Management and B...Journal for Clinical Studies: Close Cooperation Between Data Management and B...
Journal for Clinical Studies: Close Cooperation Between Data Management and B...
 
ML, biomedical data & trust
ML, biomedical data & trustML, biomedical data & trust
ML, biomedical data & trust
 
Artificial Intelligence in Medicine.pdf
Artificial Intelligence in Medicine.pdfArtificial Intelligence in Medicine.pdf
Artificial Intelligence in Medicine.pdf
 
IRJET - Prediction and Analysis of Multiple Diseases using Machine Learni...
IRJET -  	  Prediction and Analysis of Multiple Diseases using Machine Learni...IRJET -  	  Prediction and Analysis of Multiple Diseases using Machine Learni...
IRJET - Prediction and Analysis of Multiple Diseases using Machine Learni...
 
IRJET- Cancer Disease Prediction using Machine Learning over Big Data
IRJET- Cancer Disease Prediction using Machine Learning over Big DataIRJET- Cancer Disease Prediction using Machine Learning over Big Data
IRJET- Cancer Disease Prediction using Machine Learning over Big Data
 
PREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHES
PREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHESPREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHES
PREDICTION OF COVID-19 USING MACHINE LEARNING APPROACHES
 

More from Maarten van Smeden

Uncertainty in AI
Uncertainty in AIUncertainty in AI
Uncertainty in AI
Maarten van Smeden
 
UMC Utrecht AI Methods Lab
UMC Utrecht AI Methods LabUMC Utrecht AI Methods Lab
UMC Utrecht AI Methods Lab
Maarten van Smeden
 
Rage against the machine learning 2023
Rage against the machine learning 2023Rage against the machine learning 2023
Rage against the machine learning 2023
Maarten van Smeden
 
Associate professor lecture
Associate professor lectureAssociate professor lecture
Associate professor lecture
Maarten van Smeden
 
Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...
Maarten van Smeden
 
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Maarten van Smeden
 
Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...
Maarten van Smeden
 
Predictimands
PredictimandsPredictimands
Predictimands
Maarten van Smeden
 
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient healthPrognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Maarten van Smeden
 
Clinical prediction models for covid-19: alarming results from a living syste...
Clinical prediction models for covid-19: alarming results from a living syste...Clinical prediction models for covid-19: alarming results from a living syste...
Clinical prediction models for covid-19: alarming results from a living syste...
Maarten van Smeden
 
Five questions about artificial intelligence
Five questions about artificial intelligenceFive questions about artificial intelligence
Five questions about artificial intelligence
Maarten van Smeden
 
Development and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutionsDevelopment and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutions
Maarten van Smeden
 
Prediction models for diagnosis and prognosis related to COVID-19
Prediction models for diagnosis and prognosis related to COVID-19Prediction models for diagnosis and prognosis related to COVID-19
Prediction models for diagnosis and prognosis related to COVID-19
Maarten van Smeden
 
Correcting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confoundingCorrecting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confounding
Maarten van Smeden
 
Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead
Maarten van Smeden
 
Living systematic reviews: now and in the future
Living systematic reviews: now and in the futureLiving systematic reviews: now and in the future
Living systematic reviews: now and in the future
Maarten van Smeden
 
Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19
Maarten van Smeden
 
The statistics of the coronavirus
The statistics of the coronavirusThe statistics of the coronavirus
The statistics of the coronavirus
Maarten van Smeden
 
COVID-19 related prediction models for diagnosis and prognosis - a living sys...
COVID-19 related prediction models for diagnosis and prognosis - a living sys...COVID-19 related prediction models for diagnosis and prognosis - a living sys...
COVID-19 related prediction models for diagnosis and prognosis - a living sys...
Maarten van Smeden
 
The basics of prediction modeling
The basics of prediction modeling The basics of prediction modeling
The basics of prediction modeling
Maarten van Smeden
 

More from Maarten van Smeden (20)

Uncertainty in AI
Uncertainty in AIUncertainty in AI
Uncertainty in AI
 
UMC Utrecht AI Methods Lab
UMC Utrecht AI Methods LabUMC Utrecht AI Methods Lab
UMC Utrecht AI Methods Lab
 
Rage against the machine learning 2023
Rage against the machine learning 2023Rage against the machine learning 2023
Rage against the machine learning 2023
 
Associate professor lecture
Associate professor lectureAssociate professor lecture
Associate professor lecture
 
Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...
 
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
 
Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...
 
Predictimands
PredictimandsPredictimands
Predictimands
 
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient healthPrognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient health
 
Clinical prediction models for covid-19: alarming results from a living syste...
Clinical prediction models for covid-19: alarming results from a living syste...Clinical prediction models for covid-19: alarming results from a living syste...
Clinical prediction models for covid-19: alarming results from a living syste...
 
Five questions about artificial intelligence
Five questions about artificial intelligenceFive questions about artificial intelligence
Five questions about artificial intelligence
 
Development and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutionsDevelopment and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutions
 
Prediction models for diagnosis and prognosis related to COVID-19
Prediction models for diagnosis and prognosis related to COVID-19Prediction models for diagnosis and prognosis related to COVID-19
Prediction models for diagnosis and prognosis related to COVID-19
 
Correcting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confoundingCorrecting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confounding
 
Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead
 
Living systematic reviews: now and in the future
Living systematic reviews: now and in the futureLiving systematic reviews: now and in the future
Living systematic reviews: now and in the future
 
Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19
 
The statistics of the coronavirus
The statistics of the coronavirusThe statistics of the coronavirus
The statistics of the coronavirus
 
COVID-19 related prediction models for diagnosis and prognosis - a living sys...
COVID-19 related prediction models for diagnosis and prognosis - a living sys...COVID-19 related prediction models for diagnosis and prognosis - a living sys...
COVID-19 related prediction models for diagnosis and prognosis - a living sys...
 
The basics of prediction modeling
The basics of prediction modeling The basics of prediction modeling
The basics of prediction modeling
 

Recently uploaded

A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715
A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715
A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715
Sérgio Sacani
 
Post RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) MetabolismPost RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) Metabolism
Areesha Ahmad
 
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
Christian Birchler
 
Complementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotailComplementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotail
Sérgio Sacani
 
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with PhytoremediationPhytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Gurjant Singh
 
No black holes from light einstein general relativity
No black holes from light einstein general relativityNo black holes from light einstein general relativity
No black holes from light einstein general relativity
Sérgio Sacani
 
Bioconversion of sago waste and oil cakes into biobutanol using Environmental...
Bioconversion of sago waste and oil cakes into biobutanol using Environmental...Bioconversion of sago waste and oil cakes into biobutanol using Environmental...
Bioconversion of sago waste and oil cakes into biobutanol using Environmental...
Dr NEETHU ASOKAN
 
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
US Environmental Protection Agency (EPA), Center for Computational Toxicology and Exposure
 
Concept of Balanced Diet & Nutrients.pdf
Concept of Balanced Diet & Nutrients.pdfConcept of Balanced Diet & Nutrients.pdf
Concept of Balanced Diet & Nutrients.pdf
SELF-EXPLANATORY
 
End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...
End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...
End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...
Faculty of Applied Chemistry and Materials Science
 
Ancient Theory, Abiogenesis , Biogenesis
Ancient Theory, Abiogenesis , BiogenesisAncient Theory, Abiogenesis , Biogenesis
Ancient Theory, Abiogenesis , Biogenesis
SoniaBajaj10
 
Potential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptxPotential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptx
J. Bovas Joel BFSc
 
Pancreas_functional anatomy_enzymes.pptx
Pancreas_functional anatomy_enzymes.pptxPancreas_functional anatomy_enzymes.pptx
Pancreas_functional anatomy_enzymes.pptx
muralinath2
 
Analytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina BujorAnalytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina Bujor
Faculty of Applied Chemistry and Materials Science
 
Burn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptxBurn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptx
sohil4260
 
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
Sérgio Sacani
 
Protein: Structure and Function (The Agricultural Magazine)
Protein: Structure and Function (The Agricultural Magazine)Protein: Structure and Function (The Agricultural Magazine)
Protein: Structure and Function (The Agricultural Magazine)
Dr. Lenin Kumar Bompalli
 
17. 20240529_Ingrid Olesen_MariGreen summer school.pdf
17. 20240529_Ingrid Olesen_MariGreen summer school.pdf17. 20240529_Ingrid Olesen_MariGreen summer school.pdf
17. 20240529_Ingrid Olesen_MariGreen summer school.pdf
marigreenproject
 
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen BergstedtFish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Faculty of Applied Chemistry and Materials Science
 
MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)
MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)
MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)
Steffi Friedrichs
 

Recently uploaded (20)

A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715
A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715
A NICER VIEW OF THE NEAREST AND BRIGHTEST MILLISECOND PULSAR: PSR J0437−4715
 
Post RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) MetabolismPost RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) Metabolism
 
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
 
Complementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotailComplementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotail
 
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with PhytoremediationPhytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with Phytoremediation
 
No black holes from light einstein general relativity
No black holes from light einstein general relativityNo black holes from light einstein general relativity
No black holes from light einstein general relativity
 
Bioconversion of sago waste and oil cakes into biobutanol using Environmental...
Bioconversion of sago waste and oil cakes into biobutanol using Environmental...Bioconversion of sago waste and oil cakes into biobutanol using Environmental...
Bioconversion of sago waste and oil cakes into biobutanol using Environmental...
 
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
 
Concept of Balanced Diet & Nutrients.pdf
Concept of Balanced Diet & Nutrients.pdfConcept of Balanced Diet & Nutrients.pdf
Concept of Balanced Diet & Nutrients.pdf
 
End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...
End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...
End of pipe treatment: Unlocking the potential of RAS waste - Carlos Octavio ...
 
Ancient Theory, Abiogenesis , Biogenesis
Ancient Theory, Abiogenesis , BiogenesisAncient Theory, Abiogenesis , Biogenesis
Ancient Theory, Abiogenesis , Biogenesis
 
Potential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptxPotential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptx
 
Pancreas_functional anatomy_enzymes.pptx
Pancreas_functional anatomy_enzymes.pptxPancreas_functional anatomy_enzymes.pptx
Pancreas_functional anatomy_enzymes.pptx
 
Analytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina BujorAnalytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina Bujor
 
Burn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptxBurn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptx
 
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
SOFIA/HAWC+ FAR-INFRARED POLARIMETRIC LARGE-AREA CMZ EXPLORATION (FIREPLACE) ...
 
Protein: Structure and Function (The Agricultural Magazine)
Protein: Structure and Function (The Agricultural Magazine)Protein: Structure and Function (The Agricultural Magazine)
Protein: Structure and Function (The Agricultural Magazine)
 
17. 20240529_Ingrid Olesen_MariGreen summer school.pdf
17. 20240529_Ingrid Olesen_MariGreen summer school.pdf17. 20240529_Ingrid Olesen_MariGreen summer school.pdf
17. 20240529_Ingrid Olesen_MariGreen summer school.pdf
 
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen BergstedtFish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
 
MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)
MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)
MACRAMÉ ChIPs @Behoerdenklausur 2024 (Berlin)
 

Machine learning versus traditional statistical modeling and medical doctors

  • 1. Machine learning versus traditional statistical modeling and medical doctors Maarten van Smeden Leiden University Medical Center IBS ROeS - Lausanne September 10, 2019
  • 2. IBS ROeS 2019, Lausanne MaartenvSmeden Left out artificial intelligence? In medical research, “artificial intelligence” usually just means “machine learning” or “algorithm”
  • 3. IBS ROeS 2019, Lausanne MaartenvSmeden Tech company business model
  • 4. IBS ROeS 2019, Lausanne MaartenvSmeden Tech company business model https://bit.ly/2HSp8X5; https://bit.ly/2Z0Pfop; https://bit.ly/2KIcpHG; https://bit.ly/33IJhr9
  • 5. IBS ROeS 2019, Lausanne MaartenvSmeden Other success stories https://go.nature.com/2VG2hS7; https://bbc.in/2Z1drXQ
  • 6. IBS ROeS 2019, Lausanne MaartenvSmeden IBM Watson winning Jeopardy https://bbc.in/2TMvV8I
  • 7. IBS ROeS 2019, Lausanne MaartenvSmeden IBM Watson for oncology https://bit.ly/2LxiWGj
  • 8. IBS ROeS 2019, Lausanne MaartenvSmedenForsting, J Nuc Med, 2017, DOI: 10.2967/jnumed.117.190397
  • 9. IBS ROeS 2019, Lausanne MaartenvSmeden Machine learning everywhere (selection of last month) https://bit.ly/2ka0HLq; https://go.nature.com/33TQgO6; https://bit.ly/2kp6X23; https://bit.ly/2lZuKWt; https://bit.ly/2lI298g
  • 10. What are these Machine Learning methods?
  • 11. IBS ROeS 2019, Lausanne MaartenvSmeden “Everything is an ML method” https://bit.ly/2lEVn33
  • 12. IBS ROeS 2019, Lausanne MaartenvSmeden “ML methods come from computer science” https://bit.ly/2zhbwPv; https://stanford.io/2TVp1xK; https://stanford.io/2ZfED0k Leo Breiman Jerome H Friedman Trevor Hastie CART, random forest Gradient boosting Elements of statistical learning Education Physics/Math Physics Statistics Job title Professor of Statistics Professor of Statistics Professor of Statistics
  • 13. IBS ROeS 2019, Lausanne MaartenvSmeden “ML methods for prediction, statistics for explaining” Damen, BMJ, 2016, DOI:10.1136/bmj.i2416 363 developed models how many? Decision trees 0 Random forests 0 Support vector machines 0 Nearest neighbor algorithms 0 Neural networks 1
  • 14. IBS ROeS 2019, Lausanne MaartenvSmeden “ML methods for prediction, statistics for explaining” 1See further: Kreiff and Diaz Ordaz; https://bit.ly/2m1eYdK ML and causal inference, small selection1 • Superlearner (e.g. van der Laan) • High dimensional propensity scores (e.g. Schneeweiss) • The book of why (Pearl) Wednesday 10:40-12:10 Keynote Session 3 Els Goetghebeur: Plea for a marriage of machine learning and causal inference
  • 15. IBS ROeS 2019, Lausanne MaartenvSmeden Two cultures Breiman, Stat Sci, 2001, DOI: 10.1214/ss/1009213726
  • 16. IBS ROeS 2019, Lausanne MaartenvSmedenRobert Tibshirani: https://stanford.io/2zqEGfr Machine learning: large grant = $1,000,000 Statistics: large grant = $50,000
  • 17. IBS ROeS 2019, Lausanne MaartenvSmeden Statistics Machine learning Covariates Features Outcome variable Target Model Network, graphs Parameters Weights Model for discrete var. Classifier Model for continuous var. Regression Log-likelihood Loss Multinomial regression Softmax Measurement error Noise Subject/observation Sample/instance Dummy coding One-hot encoding Measurement invariance Concept drift Statistics Machine learning Prediction Supervised learning Latent variable modeling Unsupervised learning Fitting Learning Prediction error Error Sensitivity Recall Positive predictive value Precision Contingency table Confusion matrix Measurement error model Noise-aware ML Structural equation model Gaussian Bayesian network Gold standard Ground truth Derivation–validation Training–test Experiment A/B test Adapted from Daniel Obserski: https://bit.ly/2YN12Xf and Robert Tibshirani: https://stanford.io/2zqEGfr Language
  • 18. IBS ROeS 2019, Lausanne MaartenvSmeden ML refers to a culture, not to methods Distinguishing between statistics and machine learning • Substantial overlap methods used by both cultures • Substantial overlap analysis goals • Attempts to separate the two frequently result in disagreement Pragmatic approach: I’ll use “ML” to refer to models roughly outside of the traditional regression types of analysis: decision trees (and descendants), SVMs, neural networks, boosting etc.
  • 19. Examples where “ML” has done well
  • 20. IBS ROeS 2019, Lausanne MaartenvSmeden
  • 21. IBS ROeS 2019, Lausanne MaartenvSmeden Example: retinal disease Gulshan et al, JAMA, 2016, 10.1001/jama.2016.17216; Picture retinopathy: https://bit.ly/2kB3X2w Diabetic retinopathy Deep learning (= Neural network) • 128,000 images • Transfer learning (preinitialization) • Sensitivity and specificity > .90 • Estimated from training data
  • 22. IBS ROeS 2019, Lausanne MaartenvSmeden Example: lymph node metastases Bejnordi et al, JAMA, 2018, doi: 10.1001/jama.2017.14585. See our letter to the editor for a critical discussion: https://bit.ly/2kcYS0e Deep learning competition • 390 teams signed up, 23 submitted • Only 270 images for training • Test AUC range: 0.56 to 0.99
  • 23. IBS ROeS 2019, Lausanne MaartenvSmeden Deep learning on images Many similar studies and challenges in radiology, pathology, dermatology, opthalmology, gastroenterology, cardiology, …. Topol, Nature Medicine, 2019, DOI: 10.1038/s41591-018-0300-7
  • 24. IBS ROeS 2019, Lausanne MaartenvSmeden Other sources of “medical” data • Large scale gene expression data • e.g. diagnosis of acute myeloid leukemia https://bit.ly/2k8Ao8e • Prognostication by text mining electronic health records • e.g. predicting life expectancy https://bit.ly/2k8Ao8e • Analyzing social media posts • e.g. pharmacovigilance, adverse events monitoring via Twitter posts https://bit.ly/2m0KKrg
  • 25. Examples where “ML” has done poorly
  • 26. IBS ROeS 2019, Lausanne MaartenvSmeden Skin cancer and rulers Esteva et al., Nature, 2016, DOI: 10.1038/nature21056; https://bit.ly/2lE0vV0
  • 27. IBS ROeS 2019, Lausanne MaartenvSmeden Predicting mortality – the conclusion PlosOne, 2018, DOI: 10.1371/journal.pone.0202344
  • 28. IBS ROeS 2019, Lausanne MaartenvSmeden Predicting mortality – the results PlosOne, 2018, DOI: 10.1371/journal.pone.0202344
  • 29. IBS ROeS 2019, Lausanne MaartenvSmeden Predicting mortality – the media PlosOne, 2018, DOI: 10.1371/journal.pone.0202344; https://bit.ly/2Q6H41R; https://bit.ly/2m3RLrn
  • 30. IBS ROeS 2019, Lausanne MaartenvSmeden HYPE!
  • 31. IBS ROeS 2019, Lausanne MaartenvSmeden Systematic review clinical prediction models Christodoulou et al. Journal of Clinical Epidemiology, 2019, doi: 10.1016/j.jclinepi.2019.02.004
  • 33. IBS ROeS 2019, Lausanne MaartenvSmeden Comparison “ML” vs statistical models • Machine learning methods versus statistical models is a false dichotomy • Advanced “ML” shows promise, especially in areas that are not the traditional “tabular data” (e.g. images) • Tabular data settings where “ML” can be compared with traditional regression model techniques show little added value in medical applications
  • 34. IBS ROeS 2019, Lausanne MaartenvSmeden Classification versus risk prediction Most ML “classifiers” don’t come naturally with risk prediction, i.e. a probability estimate of predicted outcome for individuals • Possibly much large sample size needed to obtain reliable (calibrated) risk predictions1 than reliable classifications • Models can be trained to be optimized for a certain predictive performance (e.g. AUC, sensitivity, calibration) • Which performance to use to compare models are optimized for different types of performance? • What about the patient outcomes? Van Smeden et al., Stat Meth Med Res, 2019
  • 35. IBS ROeS 2019, Lausanne MaartenvSmeden Where do we stand on “ML” vs doctors? Domain: radiology and pathology • Article hits: 12,000 • After screening: 22 • Out-of-sample comparison “ML” vs doctors: 2 Faes et al., Lancet preprint, 2019, https://ssrn.com/abstract=3384923
  • 36. IBS ROeS 2019, Lausanne MaartenvSmeden Fair “ML” vs doctor comparisons Three basic principles • Doctors should work under realistic time constraints and have access to all regular diagnostic information, including relevant additional diagnostic testing, unless there are compelling reasons not to do so • The output generated by algorithms and physicians should be evaluated on the same scale • Performance over-optimism should be avoided Van Smeden et al., JAMA, 2018, doi:
  • 37. IBS ROeS 2019, Lausanne MaartenvSmeden Fair “ML” vs doctor comparisons Several barriers for diagnosis/prognosis • Absence of a gold standard for most diseases1 • Errors/unclear category are to be expected • Errors are transferred to algorithm • Risk overestimating the performance • Which performance measures should we be looking at? • AUC, sens/spec, predictive values, F1? • What about patient outcomes? 1See: Reitsma, Journal of Clinical Epidemiology, 2009, doi: 10.1016/j.jclinepi.2009.02.005
  • 38. IBS ROeS 2019, Lausanne MaartenvSmeden My plea To big data (and use it) and back to trials • There is a need to evaluate and compare the performance of well developed statistical learning models on patient outcomes (e.g. survival, response to treatment, PROs, etc.) • The analogue of test-treatment trials in diagnostic research: algorithm-treatment trials
  • 39. IBS ROeS 2019, Lausanne MaartenvSmeden
  • 40. IBS ROeS 2019, Lausanne MaartenvSmeden