SlideShare a Scribd company logo
Maarten van Smeden, PhD
Statistician / assistant prof epidemiologic methods
RSS North Eastern Local Group
27 September 2021
Prediction models for diagnosis
and prognosis related to COVID19
SLIDES AVAILABLE ON SLIDESHARE (MaartenvanSmeden)
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Source: https://scitechdaily.com/mits-new-neural-network-liquid-machine-learning-system-adapts-to-changing-conditions/
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Source: https://www.gehealthcare.com/article/no-matter-how-you-slice-it-this-ai-tech-is-changing-mr-neuro-imaging
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Source: XKCD
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Machine learning is everywhere
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
COVID
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
“As of today, we have deployed the system in 16 hospitals, and it is
performing over 1,300 screenings per day”
MedRxiv pre-print only, 23 March 2020,
doi.org/10.1101/2020.03.19.20039354
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
FDA APPROVED
FDA APPROVED
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Diagnostic and prognostic (prediction) models
• To support clinical decision-making for individual patients
• Combining and giving appropriate weights to several inputs
e.g. signs and symptoms, lab test results, demographics, CT image
characteristics, …
• Risk prediction
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
https://isaric4c.net/risk/
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
If done right, prediction models improve care:
increased diagnostic/prognostic speed,
uniformity and accuracy at reduced costs
Help allocate and prioritize scarce health care
resources
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
But…
Enfield et al. Chest, 2011, doi: 10.1378/chest.1118087
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Source: https://openai.com/blog/multimodal-neurons/
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
claiming that a classifier trained on
zillions of human-labelled images
containing cats and no cats, is
recognizing cats is just stupid – a
human can see a handful of cats,
including cartoons of pink panthers,
and lions and tigers and panthers, and
then can not only recognize many
other types of cats, but even if they
lose their sight, might have a pretty
good go at telling whether they are
holding their moggy or their doggy
https://bit.ly/326ghK8
Jon Crowcroft
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Adversarial example
https://bit.ly/2N4mQFo; https://bit.ly/2W7X9rF
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Skin cancer and rulers
Esteva et al., Nature, 2016, DOI: 10.1038/nature21056; https://bit.ly/2lE0vV0
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Kiani et al. NPJ dig med, 2020, doi: 10.1038/s41746-020-0232-8
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Source: https://www.shutterstock.com/nl/image-photo/risk-benefit-balance-managing-reward-on-1739271863
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
Doi: 10.1136/bmj.m1328
1916 titles screened
15 studies included
describing 19 models
(first submitted version)
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
Doi: 10.1136/bmj.m1328
1st submission
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
Doi: 10.1136/bmj.m1328
2690 titles screened
27 studies included
describing 31 models
(published version)
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
Doi: 10.1136/bmj.m1328
Received peer review
feedback
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
Doi: 10.1136/bmj.m1328
Replied to peer
review + update
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Timeline
Doi: 10.1136/bmj.m1328
Paper accepted
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
• Published on 7 April 2020
• 18 days between idea and article acceptance (sprint)
• Invited by BMJ as the first ever living review (marathon)
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Latest version (Update 3)
DOI: 10.1136/bmj.m1328
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Aims
What is available?
• Up to date overview of COVID related prediction models
• Critical appraisal
Which models can we support for use in clinical practice?
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Facts and figures
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Risk of bias assesment
Participants
Predictors
Outcome
Analysis
Signalling
questions in
4 domains:
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Prediction model related to prognosis and diagnosis
3 main types models
1. Patient X is infected / COVID-19 pneumonia
diagnostic
2. Patient X will die from COVID-19 / need respiratory support
prognostic
3. Currently healthy person X will get severe COVID-19
general population
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Data extraction
• 43 researchers, duplicate reviews
• Extraction form based on CHARMS checklist & PROBAST
• Assessment of each prediction model separately
if more than one was developed/validated
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Results
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Median (IQR)
Sample size
344
(134 to 748)
Number of events
70
(37 to 160)
Results
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
114 out of 236 models (48%) were available in a format for use in clinical practice.
Results
COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Logistic regression
34%
Neural network / deep
learning
32%
Other (Cox PH, SVM,
random forest,..)
34%
Results
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Commonly included predictors
Prognosis
• Age
• Sex
• Comorbidities
• Temperature
• Heart rate
• Respiratory rate
• Oxygen saturation
• Blood pressure
• Image features
• lymphocyte count
• C reactive protein
Diagnosis
• Flu-like signs and symptoms
• Cough
• Sputum
• Contact with covid-19 confirmed case
• Neutrophil count
• Electrolytes
• Leukocytes
• Liver enzymes
• Red cell distribution width
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Reported performance – AUC range
• General population models: 0.71 to 0.99
• Diagnostic models: 0.65 to 0.99
• Diagnostic severity models: 0.80 to 0.99
• Diagnostic imaging models: 0.70 to 0.99
• Prognosis models: 0.54 to 0.99
prediction horizon varied from 1 to 37 days
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Participants
• Inappropriate or unclear in/exclusion or study design
Predictors
• Scored “unknown” in imaging studies
Outcome
• Subjective or proxy outcomes
Analysis
• Small sample size
• Overfitting and optimism
• Inappropriate or incomplete evaluation of performance
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Calibration rarely assessed
Xie, Hungerford et al
Conclusion update 3 living review
“…models are all at high or unclear risk of bias”
We do “not recommend any of the current prediction
models to be used in practice, but one diagnostic and
one prognostic model originated from higher quality
studies and should be (independently) validated in other
datasets”
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Impact
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Review limitations
• Other models are available (e.g., in-house developed models, proprietary
algorithms) without a scientific report - outside scope of the review
• Indications that quality is not better
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
How can things improve?
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
How can things improve?
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
How can things improve?
Image courtesy Laure Wynants
Usable
models
Representative
data
Multi-
disciplinary
Share
models
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Realizing the pipeline of implementation failure
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Low hanging fruit: better reporting urgently needed
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Future steps
• Update 4 living review already ongoing (~90,000 new COVID articles)
• Will be pre-printed on our website: https://www.covprecise.org/
• COVID precise consortium: IPD-MA of external validations of COVID-19
related prognostic models (soon to be submitted)
• WHO commissioned project on validation and update for LMIC countries
• Comparison of preprint and peer reviewed prediction models
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Positive exceptions
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Beyond a single setting and single validation
• https://www.covprecise.org/
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
https://www.covprecise.org/
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
https://www.prognosisresearch.com/
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Team science
Coordination team
Laure Wynants (Maastricht)
Maarten van Smeden (Utrecht)
Carl Moons
Ben Van Calster (Leuven)
Reviewers
Thomas Debray (Utrecht)
Valentijn de Jong
Ewoud Schuit
Hans Reitsma
Toshi Takada
Lotty Hooft
Anneke Damen
Constanza Navarro
Florien van Royen
Pauline Heus
Luc Smits (Maastricht)
Sander van Kuijk
Bas van Bussel
Iwan van der Horst
Ewout Steyerberg (Leiden)
Anna Lohmann
Kim Luijken
Georg Heinze (Vienna)
Maria Haller
Michael Kammer
Christine Wallisch
Nina Kreuzberger (Cologne)
Nicole Skoetz
Darren Dahly (Cork)
Michael Harhay (Philadelphia)
Robert Wolff (York)
Ioana Tzoulaki (London)
Gary Collins (Oxford)
Jie Ma
Paula Dhiman
Richard Riley (Keele)
Kym Snell
Matthew Sperrin (Manchester)
Jamie Sergeant
Glen Martin
Jack Wilkinson
Chunhu Shi
Jan Verbakel (Leuven)
David McLernon (Aberdeen)
Information specialist
René Spijker (Utrecht)
Advisors
Marc Bonten (Utrecht)
Maarten De Vos (Leuven)
Liesbet Henckaerts
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
Slides:
https://www.slideshare.net/MaartenvanSmeden
Thanks to
• Laure Wynants (Maastricht)
• Georg Heinze and team (Vienna)
• Ewoud Schuit (Utrecht)
• Gary Collins (Oxford)
• Maarten De Vos (Leuven)
for some of the materials for these slides
Email: M.vanSmeden@umcutrecht.nl
Twitter: @MaartenvSmeden
http://www.mvansmeden.net/
M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden
COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis

More Related Content

What's hot

Five questions about artificial intelligence
Five questions about artificial intelligenceFive questions about artificial intelligence
Five questions about artificial intelligence
Maarten van Smeden
 
Development and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutionsDevelopment and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutions
Maarten van Smeden
 
Introduction to prediction modelling - Berlin 2018 - Part I
Introduction to prediction modelling - Berlin 2018 - Part IIntroduction to prediction modelling - Berlin 2018 - Part I
Introduction to prediction modelling - Berlin 2018 - Part I
Maarten van Smeden
 
The absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problemThe absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problem
Maarten van Smeden
 
Machine learning versus traditional statistical modeling and medical doctors
Machine learning versus traditional statistical modeling and medical doctorsMachine learning versus traditional statistical modeling and medical doctors
Machine learning versus traditional statistical modeling and medical doctors
Maarten van Smeden
 
Thoughts on Machine Learning and Artificial Intelligence
Thoughts on Machine Learning and Artificial IntelligenceThoughts on Machine Learning and Artificial Intelligence
Thoughts on Machine Learning and Artificial Intelligence
Maarten van Smeden
 
The basics of prediction modeling
The basics of prediction modeling The basics of prediction modeling
The basics of prediction modeling
Maarten van Smeden
 
Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...
Maarten van Smeden
 
Dichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatisticianDichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatistician
Laure Wynants
 
Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
Maarten van Smeden
 
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Ewout Steyerberg
 
How to establish and evaluate clinical prediction models - Statswork
How to establish and evaluate clinical prediction models - StatsworkHow to establish and evaluate clinical prediction models - Statswork
How to establish and evaluate clinical prediction models - Statswork
Stats Statswork
 
Make clinical prediction models great again
Make clinical prediction models great againMake clinical prediction models great again
Make clinical prediction models great again
BenVanCalster
 
Bias in covid 19 models
Bias in covid 19 modelsBias in covid 19 models
Bias in covid 19 models
Laure Wynants
 
Machine learning in medicine: calm down
Machine learning in medicine: calm downMachine learning in medicine: calm down
Machine learning in medicine: calm down
BenVanCalster
 
Artificial intelligence to fight against covid19
Artificial intelligence to fight against covid19Artificial intelligence to fight against covid19
Artificial intelligence to fight against covid19
saritamathania
 
SRGE COVID-19 Publications 2020
SRGE COVID-19 Publications 2020SRGE COVID-19 Publications 2020
SRGE COVID-19 Publications 2020
Aboul Ella Hassanien
 
Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...
Laure Wynants
 

What's hot (18)

Five questions about artificial intelligence
Five questions about artificial intelligenceFive questions about artificial intelligence
Five questions about artificial intelligence
 
Development and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutionsDevelopment and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutions
 
Introduction to prediction modelling - Berlin 2018 - Part I
Introduction to prediction modelling - Berlin 2018 - Part IIntroduction to prediction modelling - Berlin 2018 - Part I
Introduction to prediction modelling - Berlin 2018 - Part I
 
The absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problemThe absence of a gold standard: a measurement error problem
The absence of a gold standard: a measurement error problem
 
Machine learning versus traditional statistical modeling and medical doctors
Machine learning versus traditional statistical modeling and medical doctorsMachine learning versus traditional statistical modeling and medical doctors
Machine learning versus traditional statistical modeling and medical doctors
 
Thoughts on Machine Learning and Artificial Intelligence
Thoughts on Machine Learning and Artificial IntelligenceThoughts on Machine Learning and Artificial Intelligence
Thoughts on Machine Learning and Artificial Intelligence
 
The basics of prediction modeling
The basics of prediction modeling The basics of prediction modeling
The basics of prediction modeling
 
Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...
 
Dichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatisticianDichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatistician
 
Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
 
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
 
How to establish and evaluate clinical prediction models - Statswork
How to establish and evaluate clinical prediction models - StatsworkHow to establish and evaluate clinical prediction models - Statswork
How to establish and evaluate clinical prediction models - Statswork
 
Make clinical prediction models great again
Make clinical prediction models great againMake clinical prediction models great again
Make clinical prediction models great again
 
Bias in covid 19 models
Bias in covid 19 modelsBias in covid 19 models
Bias in covid 19 models
 
Machine learning in medicine: calm down
Machine learning in medicine: calm downMachine learning in medicine: calm down
Machine learning in medicine: calm down
 
Artificial intelligence to fight against covid19
Artificial intelligence to fight against covid19Artificial intelligence to fight against covid19
Artificial intelligence to fight against covid19
 
SRGE COVID-19 Publications 2020
SRGE COVID-19 Publications 2020SRGE COVID-19 Publications 2020
SRGE COVID-19 Publications 2020
 
Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...Prediction research in a pandemic: 3 lessons from a living systematic review ...
Prediction research in a pandemic: 3 lessons from a living systematic review ...
 

Similar to Clinical prediction models for covid-19: alarming results from a living systematic review

7) z. psiis14733099 (sikkema, reina)
7) z. psiis14733099 (sikkema, reina)7) z. psiis14733099 (sikkema, reina)
7) z. psiis14733099 (sikkema, reina)
MastifahSunarso
 
Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...
Maarten van Smeden
 
Corona warriors and their challenges
Corona warriors and their challengesCorona warriors and their challenges
Corona warriors and their challenges
shatrughanPareek
 
Pharma herald bulletin
Pharma herald bulletinPharma herald bulletin
Pharma herald bulletin
ArvindGupta251
 
Estimates of the severity of coronavirus disease 2019 - a model-based analysis
Estimates of the severity of coronavirus disease 2019 - a model-based analysisEstimates of the severity of coronavirus disease 2019 - a model-based analysis
Estimates of the severity of coronavirus disease 2019 - a model-based analysis
Guy Boulianne
 
Covid reference01 book Dr. Freddy Flores Malpartida
Covid reference01 book Dr. Freddy Flores MalpartidaCovid reference01 book Dr. Freddy Flores Malpartida
Covid reference01 book Dr. Freddy Flores Malpartida
Freddy Flores Malpartida
 
Covid reference01 book
Covid reference01 bookCovid reference01 book
Covid reference01 book
Freddy Flores Malpartida
 
Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...
Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...
Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...
Jake Chen
 
Information services continuity during the COVID-19 pandemics: Lessons learne...
Information services continuity during the COVID-19 pandemics: Lessons learne...Information services continuity during the COVID-19 pandemics: Lessons learne...
Information services continuity during the COVID-19 pandemics: Lessons learne...
Patrice Chalon
 
Blood safety issues 2018
Blood safety issues 2018Blood safety issues 2018
Blood safety issues 2018
Albert Farrugia
 
COVID-19 trivia and facts for educators and nonprofits
COVID-19 trivia and facts for educators and nonprofitsCOVID-19 trivia and facts for educators and nonprofits
COVID-19 trivia and facts for educators and nonprofits
Cynthia Calongne
 
Impact of COVID-19 caronavirus on poverty in Pakistan: a case study of Sindh
Impact of COVID-19 caronavirus on poverty in Pakistan: a case study of SindhImpact of COVID-19 caronavirus on poverty in Pakistan: a case study of Sindh
Impact of COVID-19 caronavirus on poverty in Pakistan: a case study of Sindh
SubmissionResearchpa
 
Module 1a n cov introduction v2
Module 1a  n cov introduction v2Module 1a  n cov introduction v2
Module 1a n cov introduction v2
OlgaPaterson1
 
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient healthPrognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Maarten van Smeden
 
Professor Aboul ella COVID-19 related publications
Professor Aboul ella COVID-19 related publications Professor Aboul ella COVID-19 related publications
Professor Aboul ella COVID-19 related publications
Aboul Ella Hassanien
 
4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...
4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...
4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...
JafarAbbas18
 
Hellenic Innovation Forum, Γιώργος Κλέτσας
Hellenic Innovation Forum, Γιώργος ΚλέτσαςHellenic Innovation Forum, Γιώργος Κλέτσας
Hellenic Innovation Forum, Γιώργος Κλέτσας
Starttech Ventures
 
To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...
To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...
To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...
YogeshIJTSRD
 
Algorithm based medicine
Algorithm based medicineAlgorithm based medicine
Algorithm based medicine
Maarten van Smeden
 
HA101: Demystifying SARS-CoV-2 Testing for COVID-19-First Edition
HA101: Demystifying SARS-CoV-2 Testing for COVID-19-First EditionHA101: Demystifying SARS-CoV-2 Testing for COVID-19-First Edition
HA101: Demystifying SARS-CoV-2 Testing for COVID-19-First Edition
Health Advances
 

Similar to Clinical prediction models for covid-19: alarming results from a living systematic review (20)

7) z. psiis14733099 (sikkema, reina)
7) z. psiis14733099 (sikkema, reina)7) z. psiis14733099 (sikkema, reina)
7) z. psiis14733099 (sikkema, reina)
 
Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...Guideline for high-quality diagnostic and prognostic applications of AI in he...
Guideline for high-quality diagnostic and prognostic applications of AI in he...
 
Corona warriors and their challenges
Corona warriors and their challengesCorona warriors and their challenges
Corona warriors and their challenges
 
Pharma herald bulletin
Pharma herald bulletinPharma herald bulletin
Pharma herald bulletin
 
Estimates of the severity of coronavirus disease 2019 - a model-based analysis
Estimates of the severity of coronavirus disease 2019 - a model-based analysisEstimates of the severity of coronavirus disease 2019 - a model-based analysis
Estimates of the severity of coronavirus disease 2019 - a model-based analysis
 
Covid reference01 book Dr. Freddy Flores Malpartida
Covid reference01 book Dr. Freddy Flores MalpartidaCovid reference01 book Dr. Freddy Flores Malpartida
Covid reference01 book Dr. Freddy Flores Malpartida
 
Covid reference01 book
Covid reference01 bookCovid reference01 book
Covid reference01 book
 
Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...
Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...
Lessons from COVID-19: How Are Data Science and AI Changing Future Biomedical...
 
Information services continuity during the COVID-19 pandemics: Lessons learne...
Information services continuity during the COVID-19 pandemics: Lessons learne...Information services continuity during the COVID-19 pandemics: Lessons learne...
Information services continuity during the COVID-19 pandemics: Lessons learne...
 
Blood safety issues 2018
Blood safety issues 2018Blood safety issues 2018
Blood safety issues 2018
 
COVID-19 trivia and facts for educators and nonprofits
COVID-19 trivia and facts for educators and nonprofitsCOVID-19 trivia and facts for educators and nonprofits
COVID-19 trivia and facts for educators and nonprofits
 
Impact of COVID-19 caronavirus on poverty in Pakistan: a case study of Sindh
Impact of COVID-19 caronavirus on poverty in Pakistan: a case study of SindhImpact of COVID-19 caronavirus on poverty in Pakistan: a case study of Sindh
Impact of COVID-19 caronavirus on poverty in Pakistan: a case study of Sindh
 
Module 1a n cov introduction v2
Module 1a  n cov introduction v2Module 1a  n cov introduction v2
Module 1a n cov introduction v2
 
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient healthPrognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient health
 
Professor Aboul ella COVID-19 related publications
Professor Aboul ella COVID-19 related publications Professor Aboul ella COVID-19 related publications
Professor Aboul ella COVID-19 related publications
 
4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...
4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...
4-Journal of Medical Virology - 2021 - Zhang - Using different machine learni...
 
Hellenic Innovation Forum, Γιώργος Κλέτσας
Hellenic Innovation Forum, Γιώργος ΚλέτσαςHellenic Innovation Forum, Γιώργος Κλέτσας
Hellenic Innovation Forum, Γιώργος Κλέτσας
 
To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...
To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...
To Assess the Severity and Mortality among Covid 19 Patients after Having Vac...
 
Algorithm based medicine
Algorithm based medicineAlgorithm based medicine
Algorithm based medicine
 
HA101: Demystifying SARS-CoV-2 Testing for COVID-19-First Edition
HA101: Demystifying SARS-CoV-2 Testing for COVID-19-First EditionHA101: Demystifying SARS-CoV-2 Testing for COVID-19-First Edition
HA101: Demystifying SARS-CoV-2 Testing for COVID-19-First Edition
 

More from Maarten van Smeden

Uncertainty in AI
Uncertainty in AIUncertainty in AI
Uncertainty in AI
Maarten van Smeden
 
UMC Utrecht AI Methods Lab
UMC Utrecht AI Methods LabUMC Utrecht AI Methods Lab
UMC Utrecht AI Methods Lab
Maarten van Smeden
 
Rage against the machine learning 2023
Rage against the machine learning 2023Rage against the machine learning 2023
Rage against the machine learning 2023
Maarten van Smeden
 
A gentle introduction to AI for medicine
A gentle introduction to AI for medicineA gentle introduction to AI for medicine
A gentle introduction to AI for medicine
Maarten van Smeden
 
Associate professor lecture
Associate professor lectureAssociate professor lecture
Associate professor lecture
Maarten van Smeden
 
Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...
Maarten van Smeden
 
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Maarten van Smeden
 
Predictimands
PredictimandsPredictimands
Predictimands
Maarten van Smeden
 
Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead
Maarten van Smeden
 
Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19
Maarten van Smeden
 
Anatomy of a successful science thread
Anatomy of a successful science threadAnatomy of a successful science thread
Anatomy of a successful science thread
Maarten van Smeden
 
Is it causal, is it prediction or is it neither?
Is it causal, is it prediction or is it neither?Is it causal, is it prediction or is it neither?
Is it causal, is it prediction or is it neither?
Maarten van Smeden
 
Regression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questionsRegression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questions
Maarten van Smeden
 

More from Maarten van Smeden (13)

Uncertainty in AI
Uncertainty in AIUncertainty in AI
Uncertainty in AI
 
UMC Utrecht AI Methods Lab
UMC Utrecht AI Methods LabUMC Utrecht AI Methods Lab
UMC Utrecht AI Methods Lab
 
Rage against the machine learning 2023
Rage against the machine learning 2023Rage against the machine learning 2023
Rage against the machine learning 2023
 
A gentle introduction to AI for medicine
A gentle introduction to AI for medicineA gentle introduction to AI for medicine
A gentle introduction to AI for medicine
 
Associate professor lecture
Associate professor lectureAssociate professor lecture
Associate professor lecture
 
Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...Improving epidemiological research: avoiding the statistical paradoxes and fa...
Improving epidemiological research: avoiding the statistical paradoxes and fa...
 
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
Shrinkage in medical prediction: the poor man’s solution for an inadequate sa...
 
Predictimands
PredictimandsPredictimands
Predictimands
 
Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead Why the EPV≥10 sample size rule is rubbish and what to use instead
Why the EPV≥10 sample size rule is rubbish and what to use instead
 
Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19Voorspelmodellen en COVID-19
Voorspelmodellen en COVID-19
 
Anatomy of a successful science thread
Anatomy of a successful science threadAnatomy of a successful science thread
Anatomy of a successful science thread
 
Is it causal, is it prediction or is it neither?
Is it causal, is it prediction or is it neither?Is it causal, is it prediction or is it neither?
Is it causal, is it prediction or is it neither?
 
Regression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questionsRegression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questions
 

Recently uploaded

20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
Sharon Liu
 
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptxlargeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
muralinath2
 
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
khadija07kubra
 
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptxSCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
WALTONMARBRUCAL
 
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
Sérgio Sacani
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Hossein Fani
 
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptx
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptxTOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptx
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptx
imansiipandeyy
 
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDEANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
RanjithaSL
 
degree Certificate of Aston University
degree Certificate of Aston Universitydegree Certificate of Aston University
degree Certificate of Aston University
ebgyz
 
Gametogenesis: Male gametes Formation Process / Spermatogenesis .pdf
Gametogenesis: Male gametes Formation Process / Spermatogenesis .pdfGametogenesis: Male gametes Formation Process / Spermatogenesis .pdf
Gametogenesis: Male gametes Formation Process / Spermatogenesis .pdf
SELF-EXPLANATORY
 
1,1 and 1,2 Migratory insertion reactions.pptx
1,1 and 1,2 Migratory insertion reactions.pptx1,1 and 1,2 Migratory insertion reactions.pptx
1,1 and 1,2 Migratory insertion reactions.pptx
nharnahquophi8080
 
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptxBIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
alishyt102010
 
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopyA mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
Sérgio Sacani
 
SCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptx
SCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptxSCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptx
SCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptx
JoanaBanasen1
 
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptxBragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Lisandro Cunci
 
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpectiveGasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Recupera
 
MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...
MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...
MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...
Steffi Friedrichs
 
LOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratoryLOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratory
JCKH
 
SDG (sustainable development goal) of government
SDG (sustainable development goal) of governmentSDG (sustainable development goal) of government
SDG (sustainable development goal) of government
Patna ,Bihar
 
Liver & Gall Bladder 23098463278654387654328765439875.pptx
Liver & Gall Bladder 23098463278654387654328765439875.pptxLiver & Gall Bladder 23098463278654387654328765439875.pptx
Liver & Gall Bladder 23098463278654387654328765439875.pptx
muralinath2
 

Recently uploaded (20)

20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
 
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptxlargeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
largeintestinepathologiesconditions-240627071428-3c936a47 (2).pptx
 
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
2. Osmotic pressure, osmotic potential, turgor pressure, wall pressure, water...
 
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptxSCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
SCIENCEgfvhvhvkjkbbjjbbjvhvhvhvjkvjvjvjj.pptx
 
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
The cryptoterrestrial hypothesis: A case for scientific openness to a conceal...
 
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, ...
 
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptx
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptxTOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptx
TOPIC: INTRODUCTION TO FORENSIC SCIENCE.pptx
 
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDEANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
ANTIGENS_.pptx ( Ranjitha SL) PRESENTATION SLIDE
 
degree Certificate of Aston University
degree Certificate of Aston Universitydegree Certificate of Aston University
degree Certificate of Aston University
 
Gametogenesis: Male gametes Formation Process / Spermatogenesis .pdf
Gametogenesis: Male gametes Formation Process / Spermatogenesis .pdfGametogenesis: Male gametes Formation Process / Spermatogenesis .pdf
Gametogenesis: Male gametes Formation Process / Spermatogenesis .pdf
 
1,1 and 1,2 Migratory insertion reactions.pptx
1,1 and 1,2 Migratory insertion reactions.pptx1,1 and 1,2 Migratory insertion reactions.pptx
1,1 and 1,2 Migratory insertion reactions.pptx
 
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptxBIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
BIOPHYSICS Interactions of molecules in 3-D space-determining binding and.pptx
 
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopyA mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
A mature quasar at cosmic dawn revealed by JWST rest-frame infrared spectroscopy
 
SCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptx
SCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptxSCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptx
SCIENTIFIC INVESTIGATIONS – THE IMPORTANCE OF FAIR TESTING.pptx
 
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptxBragg Brentano Alignment for D4 with LynxEye Rev3.pptx
Bragg Brentano Alignment for D4 with LynxEye Rev3.pptx
 
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpectiveGasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
Gasification and Pyrolyssis of plastic Waste under a Circular Economy perpective
 
MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...
MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...
MACRAMÉ-ChiPs: Patchwork Project Family & Sibling Projects (24th Meeting of t...
 
LOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratoryLOB LOD LOQ for method validation in laboratory
LOB LOD LOQ for method validation in laboratory
 
SDG (sustainable development goal) of government
SDG (sustainable development goal) of governmentSDG (sustainable development goal) of government
SDG (sustainable development goal) of government
 
Liver & Gall Bladder 23098463278654387654328765439875.pptx
Liver & Gall Bladder 23098463278654387654328765439875.pptxLiver & Gall Bladder 23098463278654387654328765439875.pptx
Liver & Gall Bladder 23098463278654387654328765439875.pptx
 

Clinical prediction models for covid-19: alarming results from a living systematic review

  • 1. Maarten van Smeden, PhD Statistician / assistant prof epidemiologic methods RSS North Eastern Local Group 27 September 2021 Prediction models for diagnosis and prognosis related to COVID19 SLIDES AVAILABLE ON SLIDESHARE (MaartenvanSmeden)
  • 2. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Source: https://scitechdaily.com/mits-new-neural-network-liquid-machine-learning-system-adapts-to-changing-conditions/
  • 3. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Source: https://www.gehealthcare.com/article/no-matter-how-you-slice-it-this-ai-tech-is-changing-mr-neuro-imaging
  • 4. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Source: XKCD
  • 5. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Machine learning is everywhere
  • 6. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis COVID
  • 7. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis “As of today, we have deployed the system in 16 hospitals, and it is performing over 1,300 screenings per day” MedRxiv pre-print only, 23 March 2020, doi.org/10.1101/2020.03.19.20039354
  • 8. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis FDA APPROVED FDA APPROVED
  • 9. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Diagnostic and prognostic (prediction) models • To support clinical decision-making for individual patients • Combining and giving appropriate weights to several inputs e.g. signs and symptoms, lab test results, demographics, CT image characteristics, … • Risk prediction
  • 10. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 11. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 12. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis https://isaric4c.net/risk/
  • 13. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 14. If done right, prediction models improve care: increased diagnostic/prognostic speed, uniformity and accuracy at reduced costs Help allocate and prioritize scarce health care resources
  • 15. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis But… Enfield et al. Chest, 2011, doi: 10.1378/chest.1118087
  • 16. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Source: https://openai.com/blog/multimodal-neurons/
  • 17. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis claiming that a classifier trained on zillions of human-labelled images containing cats and no cats, is recognizing cats is just stupid – a human can see a handful of cats, including cartoons of pink panthers, and lions and tigers and panthers, and then can not only recognize many other types of cats, but even if they lose their sight, might have a pretty good go at telling whether they are holding their moggy or their doggy https://bit.ly/326ghK8 Jon Crowcroft
  • 18. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Adversarial example https://bit.ly/2N4mQFo; https://bit.ly/2W7X9rF
  • 19. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Skin cancer and rulers Esteva et al., Nature, 2016, DOI: 10.1038/nature21056; https://bit.ly/2lE0vV0
  • 20. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Kiani et al. NPJ dig med, 2020, doi: 10.1038/s41746-020-0232-8
  • 21. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Source: https://www.shutterstock.com/nl/image-photo/risk-benefit-balance-managing-reward-on-1739271863
  • 22. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 23. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline
  • 24. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline Doi: 10.1136/bmj.m1328 1916 titles screened 15 studies included describing 19 models (first submitted version)
  • 25. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline Doi: 10.1136/bmj.m1328 1st submission
  • 26. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline Doi: 10.1136/bmj.m1328 2690 titles screened 27 studies included describing 31 models (published version)
  • 27. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline Doi: 10.1136/bmj.m1328 Received peer review feedback
  • 28. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline Doi: 10.1136/bmj.m1328 Replied to peer review + update
  • 29. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Timeline Doi: 10.1136/bmj.m1328 Paper accepted
  • 30. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis • Published on 7 April 2020 • 18 days between idea and article acceptance (sprint) • Invited by BMJ as the first ever living review (marathon)
  • 31. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Latest version (Update 3) DOI: 10.1136/bmj.m1328
  • 32. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Aims What is available? • Up to date overview of COVID related prediction models • Critical appraisal Which models can we support for use in clinical practice?
  • 33. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Facts and figures
  • 34. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Risk of bias assesment Participants Predictors Outcome Analysis Signalling questions in 4 domains:
  • 35. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Prediction model related to prognosis and diagnosis 3 main types models 1. Patient X is infected / COVID-19 pneumonia diagnostic 2. Patient X will die from COVID-19 / need respiratory support prognostic 3. Currently healthy person X will get severe COVID-19 general population
  • 36. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Data extraction • 43 researchers, duplicate reviews • Extraction form based on CHARMS checklist & PROBAST • Assessment of each prediction model separately if more than one was developed/validated
  • 37. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Results
  • 38. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Median (IQR) Sample size 344 (134 to 748) Number of events 70 (37 to 160) Results
  • 39. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis 114 out of 236 models (48%) were available in a format for use in clinical practice. Results
  • 40. COVID Precise M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Logistic regression 34% Neural network / deep learning 32% Other (Cox PH, SVM, random forest,..) 34% Results
  • 41. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Commonly included predictors Prognosis • Age • Sex • Comorbidities • Temperature • Heart rate • Respiratory rate • Oxygen saturation • Blood pressure • Image features • lymphocyte count • C reactive protein Diagnosis • Flu-like signs and symptoms • Cough • Sputum • Contact with covid-19 confirmed case • Neutrophil count • Electrolytes • Leukocytes • Liver enzymes • Red cell distribution width
  • 42. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Reported performance – AUC range • General population models: 0.71 to 0.99 • Diagnostic models: 0.65 to 0.99 • Diagnostic severity models: 0.80 to 0.99 • Diagnostic imaging models: 0.70 to 0.99 • Prognosis models: 0.54 to 0.99 prediction horizon varied from 1 to 37 days
  • 43. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 44. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 45. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Participants • Inappropriate or unclear in/exclusion or study design Predictors • Scored “unknown” in imaging studies Outcome • Subjective or proxy outcomes Analysis • Small sample size • Overfitting and optimism • Inappropriate or incomplete evaluation of performance
  • 46. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Calibration rarely assessed Xie, Hungerford et al
  • 47. Conclusion update 3 living review “…models are all at high or unclear risk of bias” We do “not recommend any of the current prediction models to be used in practice, but one diagnostic and one prognostic model originated from higher quality studies and should be (independently) validated in other datasets”
  • 48. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 49. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 50. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 51. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 52. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Impact
  • 53. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Review limitations • Other models are available (e.g., in-house developed models, proprietary algorithms) without a scientific report - outside scope of the review • Indications that quality is not better
  • 54. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis How can things improve?
  • 55. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis How can things improve?
  • 56. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis How can things improve? Image courtesy Laure Wynants Usable models Representative data Multi- disciplinary Share models
  • 57. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Realizing the pipeline of implementation failure
  • 58. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Low hanging fruit: better reporting urgently needed
  • 59. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis
  • 60. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Future steps • Update 4 living review already ongoing (~90,000 new COVID articles) • Will be pre-printed on our website: https://www.covprecise.org/ • COVID precise consortium: IPD-MA of external validations of COVID-19 related prognostic models (soon to be submitted) • WHO commissioned project on validation and update for LMIC countries • Comparison of preprint and peer reviewed prediction models
  • 61. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Positive exceptions
  • 62. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Beyond a single setting and single validation • https://www.covprecise.org/
  • 63. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis https://www.covprecise.org/
  • 64. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis https://www.prognosisresearch.com/
  • 65. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Team science Coordination team Laure Wynants (Maastricht) Maarten van Smeden (Utrecht) Carl Moons Ben Van Calster (Leuven) Reviewers Thomas Debray (Utrecht) Valentijn de Jong Ewoud Schuit Hans Reitsma Toshi Takada Lotty Hooft Anneke Damen Constanza Navarro Florien van Royen Pauline Heus Luc Smits (Maastricht) Sander van Kuijk Bas van Bussel Iwan van der Horst Ewout Steyerberg (Leiden) Anna Lohmann Kim Luijken Georg Heinze (Vienna) Maria Haller Michael Kammer Christine Wallisch Nina Kreuzberger (Cologne) Nicole Skoetz Darren Dahly (Cork) Michael Harhay (Philadelphia) Robert Wolff (York) Ioana Tzoulaki (London) Gary Collins (Oxford) Jie Ma Paula Dhiman Richard Riley (Keele) Kym Snell Matthew Sperrin (Manchester) Jamie Sergeant Glen Martin Jack Wilkinson Chunhu Shi Jan Verbakel (Leuven) David McLernon (Aberdeen) Information specialist René Spijker (Utrecht) Advisors Marc Bonten (Utrecht) Maarten De Vos (Leuven) Liesbet Henckaerts
  • 66. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis Slides: https://www.slideshare.net/MaartenvanSmeden Thanks to • Laure Wynants (Maastricht) • Georg Heinze and team (Vienna) • Ewoud Schuit (Utrecht) • Gary Collins (Oxford) • Maarten De Vos (Leuven) for some of the materials for these slides Email: M.vanSmeden@umcutrecht.nl Twitter: @MaartenvSmeden http://www.mvansmeden.net/
  • 67. M.vanSmeden@umcutrecht.nl | Twitter: @MaartenvSmeden COVID PRECISE: prediction of COVID-19 related prognosis and diagnosis