Cara Menggugurkan Kandungan Dengan Cepat Selesai Dalam 24 Jam Secara Alami Bu...
Prediction research in a pandemic: 3 lessons from a living systematic review of diagnostic and prognostic models
1. Prediction research in a Pandemic
3 lessons from a living systematic review
of diagnostic and prognostic models
Laure Wynants PhD
Maastricht University, Department of Epidemiology
KU Leuven, Department of Development and Regeneration, EPI-Centre
laure.wynants@maastrichtuniversity.nl
@laure_wynants
AI in tackling Covid-19 crisis: opportunities and challenges
Digital Transition and Single Market Observatory Webinar
2. “As of today, we have deployed the system in 16 hospitals, and it is
performing over 1,300 screenings per day”
MedRxiv pre-print, 23 March 2020, doi.org/10.1101/2020.03.19.20039354
8. Improve care and reduce costs.
Help allocate scarce resources.
A good model could…
9. Poor models can make things worse
Inaccurate predictions -> harmful decisions
(Van Calster & Vickers, Med Dec Mak, 2015)
ICU scores during H1N1 pandemic (Enfield, Chest, 2011)
10. Aims of the living review
What is available?
o Up to date overview & critical appraisal
o Learn from other countries
14. Results
114 out of 236 models (48%) were available in a format for use in clinical practice.
15. No-one has an overview
• >500 newer models in scientific journals
• Models not in scientific journals
(proprietary or in-house developed algorithms)
17. Performance
• General population models: 0.71 to ≥0.99
• Diagnostic models: 0.65 to ≥ 0.99
• Diagnostic severity models: 0.80 to ≥ 0.99
• Diagnostic imaging models: 0.70 to ≥ 0.99
• Prognosis models: 0.54 to ≥0.99
(prediction horizon varies from 1 to 37 days, if reported)
18. Risk of biased predictive performance (PROBAST)
226 high
6 unclear
4 low
19. Participants
Inappropriate in/exclusion or study
design
Predictors
Particularly problematic in imaging studies
Outcome
Subjective or proxy outcomes
Analysis
Limited sample size
Overfitting and optimism
Risk of bias – common causes
24. 4 low risk of bias: QCOVID models
Development (910 GPs, 6 083 102 patients)
Outcome 1: Death with COVID-19
Outcome 2: Hospitalization with COVID-19
Predictors
Excellent performance in validation study in 80% of English adults
doi: 10.1136/bmj.m3731
25. Lesson 3: too much competition threatens patient safety
27. Conclusion
• Despite reports of impressive predictive performance, much of the
growing body of literature on prediction research for covid-19 is of low
quality
• Do not trust a good reported performance alone – underlying data &
analysis & validation matter!
28. Team science
Coordination team
Laure Wynants (Maastricht)
Maarten van Smeden (Utrecht)
Carl Moons
Ben Van Calster (Leuven)
Reviewers
Thomas Debray (Utrecht)
Valentijn de Jong
Ewoud Schuit
Hans Reitsma
Toshi Takada
Lotty Hooft
Anneke Damen
Constanza Navarro
Florien van Royen
Pauline Heus
Luc Smits (Maastricht)
Sander van Kuijk
Bas Van Bussel
Iwan van der Horst
Ewout Steyerberg (Leiden)
Anna Lohmann
Kim Luijken
Georg Heinze (Vienna)
Maria Haller
Christine Wallisch
Nina Kreuzberger (Cologne)
Nicole Skoetz
Darren Dahly (Cork)
Michael Harhay (Philadelphia)
Robert Wolff (York)
Ioana Tzoulaki (London)
Gary Collins (Oxford)
Jie Ma
Paula Dhiman
Richard Riley (Keele)
Kym Snell
Matthew Sperrin (Manchester)
Jamie Sergeant
Glen Martin
Jack Wilkinson
Chunhu Shi
Jan Verbakel (Leuven)
Information specialist
René Spijker (Utrecht)
Advisors
Maarten De Vos (Leuven)
Liesbet Henckaerts
Marc Bonten (Utrecht)
Acknowledgements
Contributions by
Cochrane Prognosis Methods Group
Covid-19 patient and laypeople panel
Financial support by Internal Funds KU Leuven, KOOR, and the
COVID-19 Fund
