SlideShare a Scribd company logo
Dichotomania
and other challenges for the collaborating biostatistician
A perspective on principles, responsibilities and potential solutions
Laure Wynants PhD
laure.wynants@maastrichtuniversity.nl
@laure_wynants
Dichotomania
and other challenges for the collaborating biostatistician
A perspective on principles, responsibilities and potential solutions
Laure Wynants PhD
laure.wynants@maastrichtuniversity.nl
@laure_wynants
or teaching
2018 conference invitation
“Statistics made very easy for clinicians”
2018 conference invitation
“Statistics made very easy for clinicians”
“Free coffee and croissants while statisticians explain the P-
value crisis”
Goodman S. A dirty dozen: twelve p-value misconceptions. Semin Hematol. 2008 Jul;45(3):135-40.
6
Estimated probability of causal attribution according to the null P-value, modeled using fractional polynomials with a cutpoint at P = 0.05.
A Psychometric Experiment in Causal Inference to Estimate Evidential Weights Used by Epidemiologists
Holman, C. D’Arcy J.; Arnold-Reed, Diane E.; de Klerk, Nicholas; McComb, Christine; English, Dallas R., Epidemiology12(2):246-255, 2001.
https://xkcd.com/
Dichotomania and other challenges for the collaborating biostatistician
A conversation between a researcher and a statistician
• R: “We need some statistical testing for these plots.”
• S: “Why? These are not your main research questions
in this paper.”
• R: “I am not fishing for significant findings. I am aware
of the dangers. These are hypotheses we investigated
in earlier work. If the tests are significant, we know this
is confirmed in our new data.”
A conversation between a researcher and a statistician
• R: “We need some statistical testing for these plots.”
• S: “Why? These are not your main research questions
in this paper.”
• R: “I am not fishing for significant findings. I am aware
of the dangers. These are hypotheses we investigated
in earlier work. If the tests are significant, we know this
is confirmed in our new data.”
• R: “If it is not significant, we will discuss further. We
just didn’t have enough power then.”
Reporting and publication bias
“Trim and fill” funnel plot of Ki-67 expression for overall
survival in ovarian cancer patients (Qiu et al. Arch
Gynecol Obstet 2019)
Missing studies Published studies
Science as a disorderly mass of stray
observations, inconclusive results and
fledgling explanations.
And yet, as soon as their hypotheses were
turned into peer-reviewed papers,
researchers claimed that such facts had
always spoken for themselves.
How (not) to report results
Replication crisis
Ioannidis JAMA 2005: all original research published in 3 major journals in 1990-2003 and cited >1000 times:
49 studies, 45 claimed that the intervention was effective.
32% could not be replicated:
16% was contradicted (no effect found)
16% estimated effects were too strong (to the point that subsequent studies cast doubt on effect being clinically
important)
44% were replicated
11% no subsequent larger/better designed replication studies
Problems were worst for small RCTS and non-randomized studies.
Begly & Ellis Nature 2012: replication of landmark preclinical cancer studies: only 11% could be reproduced.
Journal impact factor Number of articles Mean number of citations of
non-reproduced articles
Mean number of citations of
reproduced articles
>20 21 248 (range 3–800) 231 (range 82–519)
5–19 32 169 (range 6–1,909) 13 (range 3–24)
• SARS-Cov-2 “viral loads in the very young do not differ significantly from
those of adults. Based on these results, we have to caution against an
unlimited re-opening of schools and kindergartens in the present situation”
• Ill-defined research question, comparison between all age groups (45
comparisons), test as if non-ordered categories.
• Reanalyzes with more appropriate techniques finds opposite conclusion.
• https://medium.com/@d_spiegel/is-sars-cov-2-viral-load-lower-in-young-children-than-adults-
8b4116d28353
A mistake in the operating room can
threaten the life of one patient;
A mistake in the statistical analysis or
interpretation can lead to hundreds of early
deaths.
Andrew Vickers, Biostatistician, Memorial Sloan Kettering Cancer Center
Some reactions to previous presentations
MDs
- surprise that meta-analysis can be biased
- “Our statisticians did not tell us this”
Altman BMJ 1994,
republished almost unchanged 20 years later…
“Put simply, much poor research arises
because researchers feel compelled for
career reasons to carry out research that
they are ill equipped to perform, and
nobody stops them.”
Statisticians
- “Oh no not this again”
- “We know this already”
- “P-values are not the problem”
- “It’s not us, it’s them”
Dichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatistician
An ethical statistician…
identifies and mitigates any preferences on the part of the investigators or data providers that might
predetermine or influence the analyses/results
only support studies that have pre-defined objectives and that are capable of producing useful results
strives to explain any expected adverse consequences of failure to follow through on an agreed-upon
sampling or analytic plan
shall indicate the risks and possible consequences if their professional judgement is overruled
Views or opinions based on general knowledge or belief should be clearly distinguished from views or
opinions derived from the statistical analyses being reported.
Taken from RSS code and ASA ethical guidelines
An ethical statistician…
recognizes […] research practices and standards can differ across disciplines, and statisticians do not have
obligations to standards of other professions that conflict with these guidelines
shall take personal responsibility for work bearing their name
avoids compromising scientific validity for expediency
should always be aware of their overriding responsibility to the public good […] A Fellow’s obligations to
employers, clients and the profession can never override this; and Fellows should seek to avoid situations
and not enter into undertakings which compromise this responsibility
Taken from RSS code and ASA ethical guidelines
An ethical statistician…
conveys the findings in ways that are both honest and meaningful to the user/reader
shall seek to conform to recognised good practice including quality standards which are in their judgement
relevant, and shall encourage others to do likewise
shall seek to advance knowledge and understanding of statistical science and advocate its use. This
advocacy of statistical science should extend to employers, clients, colleagues and the general public
Taken from RSS code and ASA ethical guidelines
#AITA?
@numbersman77
What can we do better?
ATOM
- Accept uncertainty (no more ***, interpret confidence intervals)
- Be Thoughtful (research question, design, clinically relevant effect size, registered reports)
- Be Open (conflicts of interest, registration, share data, code, analysis protocols, publish all results)
- Be Modest (exploratory, retrospective, secondary analyses (no harking); interpret studies in broader context)
Ronald L. Wasserstein, Allen L. Schirm & Nicole A. Lazar (2019) Moving to a World Beyond “p < 0.05”, The American Statistician, 73:sup1, 1-19
What can we do better?
Distinguish between different applications
E.g. Cox 2020
- Two-decision situation (health screening: return tomorrow vs next year; control error)
- Subject-matter hypothesis (difference in trt, H0: there is no difference; p-value as measure of uncertainty)
- Dividing hypothesis (at which level does CI only contain positive/negative effects?)
- Tests of model adequacy(normality assumption, informal role, judgement required)
What can we do better?
• “How to” teaching vs understanding
Simulated data (Bishop 2020)
• Be explicit about how principles extend to observational
research
• Software
• Conceptual clarity in educational material (Greenland 2019)
“Significance level”: α or p-value?
“P-value”: observed value p or random variable P?
It won’t be easy
• Misconception fatigue in teaching
It won’t be easy
Wang et al Annals of Internal Medicine 2018
It won’t be easy
No statistician can do this alone
• A responsibility for each of us
• A role for professional organizations
• A necessity to put this on the agenda of ISCB
• Thanks to John Carlin and Jonathan Sterne - even if
there are no free croissants
Dichotomania and other challenges for the collaborating biostatistician
Dichotomania and other challenges for the collaborating biostatistician
Testimation bias
Steyerberg et al JCE 1999
Replicability without p-values?
• Hanson (1958) – anthropology, sociology, psychology
- Statements with “confirmation criteria” (test): >70% confirmed
- Statements without confirmation criteria: <46% confirmed
• Basic Applied Social Psychology banned p-values
- Overinterpret / overstate descriptive results
Lakens et al 2020

More Related Content

What's hot

Predictimands
PredictimandsPredictimands
Predictimands
Maarten van Smeden
 
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Ewout Steyerberg
 
Correcting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confoundingCorrecting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confounding
Maarten van Smeden
 
Five questions about artificial intelligence
Five questions about artificial intelligenceFive questions about artificial intelligence
Five questions about artificial intelligence
Maarten van Smeden
 
ML and AI: a blessing and curse for statisticians and medical doctors
ML and AI: a blessing and curse forstatisticians and medical doctorsML and AI: a blessing and curse forstatisticians and medical doctors
ML and AI: a blessing and curse for statisticians and medical doctors
Maarten van Smeden
 
Introduction to prediction modelling - Berlin 2018 - Part II
Introduction to prediction modelling - Berlin 2018 - Part IIIntroduction to prediction modelling - Berlin 2018 - Part II
Introduction to prediction modelling - Berlin 2018 - Part II
Maarten van Smeden
 
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient healthPrognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Maarten van Smeden
 
Algorithm based medicine: old statistics wine in new machine learning bottles?
Algorithm based medicine: old statistics wine in new machine learning bottles?Algorithm based medicine: old statistics wine in new machine learning bottles?
Algorithm based medicine: old statistics wine in new machine learning bottles?
Maarten van Smeden
 
Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...
Maarten van Smeden
 
Uncertainty in AI
Uncertainty in AIUncertainty in AI
Uncertainty in AI
Maarten van Smeden
 
Development and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutionsDevelopment and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutions
Maarten van Smeden
 
Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
Maarten van Smeden
 
Minimally important differences v2
Minimally important differences v2Minimally important differences v2
Minimally important differences v2
Stephen Senn
 
Has modelling killed randomisation inference frankfurt
Has modelling killed randomisation inference frankfurtHas modelling killed randomisation inference frankfurt
Has modelling killed randomisation inference frankfurt
Stephen Senn
 
The basics of prediction modeling
The basics of prediction modeling The basics of prediction modeling
The basics of prediction modeling
Maarten van Smeden
 
Clinical trials are about comparability not generalisability V2.pptx
Clinical trials are about comparability not generalisability V2.pptxClinical trials are about comparability not generalisability V2.pptx
Clinical trials are about comparability not generalisability V2.pptx
StephenSenn2
 
The Seven Habits of Highly Effective Statisticians
The Seven Habits of Highly Effective StatisticiansThe Seven Habits of Highly Effective Statisticians
The Seven Habits of Highly Effective Statisticians
Stephen Senn
 
On p-values
On p-valuesOn p-values
On p-values
Maarten van Smeden
 
PRISMA
PRISMAPRISMA
Metaanalysis copy
Metaanalysis    copyMetaanalysis    copy
Metaanalysis copy
Amandeep Kaur
 

What's hot (20)

Predictimands
PredictimandsPredictimands
Predictimands
 
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019
 
Correcting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confoundingCorrecting for missing data, measurement error and confounding
Correcting for missing data, measurement error and confounding
 
Five questions about artificial intelligence
Five questions about artificial intelligenceFive questions about artificial intelligence
Five questions about artificial intelligence
 
ML and AI: a blessing and curse for statisticians and medical doctors
ML and AI: a blessing and curse forstatisticians and medical doctorsML and AI: a blessing and curse forstatisticians and medical doctors
ML and AI: a blessing and curse for statisticians and medical doctors
 
Introduction to prediction modelling - Berlin 2018 - Part II
Introduction to prediction modelling - Berlin 2018 - Part IIIntroduction to prediction modelling - Berlin 2018 - Part II
Introduction to prediction modelling - Berlin 2018 - Part II
 
Prognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient healthPrognosis-based medicine: merits and pitfalls of forecasting patient health
Prognosis-based medicine: merits and pitfalls of forecasting patient health
 
Algorithm based medicine: old statistics wine in new machine learning bottles?
Algorithm based medicine: old statistics wine in new machine learning bottles?Algorithm based medicine: old statistics wine in new machine learning bottles?
Algorithm based medicine: old statistics wine in new machine learning bottles?
 
Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...Sample size for binary logistic prediction models: Beyond events per variable...
Sample size for binary logistic prediction models: Beyond events per variable...
 
Uncertainty in AI
Uncertainty in AIUncertainty in AI
Uncertainty in AI
 
Development and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutionsDevelopment and evaluation of prediction models: pitfalls and solutions
Development and evaluation of prediction models: pitfalls and solutions
 
Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
 
Minimally important differences v2
Minimally important differences v2Minimally important differences v2
Minimally important differences v2
 
Has modelling killed randomisation inference frankfurt
Has modelling killed randomisation inference frankfurtHas modelling killed randomisation inference frankfurt
Has modelling killed randomisation inference frankfurt
 
The basics of prediction modeling
The basics of prediction modeling The basics of prediction modeling
The basics of prediction modeling
 
Clinical trials are about comparability not generalisability V2.pptx
Clinical trials are about comparability not generalisability V2.pptxClinical trials are about comparability not generalisability V2.pptx
Clinical trials are about comparability not generalisability V2.pptx
 
The Seven Habits of Highly Effective Statisticians
The Seven Habits of Highly Effective StatisticiansThe Seven Habits of Highly Effective Statisticians
The Seven Habits of Highly Effective Statisticians
 
On p-values
On p-valuesOn p-values
On p-values
 
PRISMA
PRISMAPRISMA
PRISMA
 
Metaanalysis copy
Metaanalysis    copyMetaanalysis    copy
Metaanalysis copy
 

Similar to Dichotomania and other challenges for the collaborating biostatistician

Big Data: Learning from MIMIC- Celi
Big Data: Learning from MIMIC- CeliBig Data: Learning from MIMIC- Celi
Big Data: Learning from MIMIC- Celi
intensivecaresociety
 
1_Intro to Research.pdf
1_Intro to Research.pdf1_Intro to Research.pdf
1_Intro to Research.pdf
MharCastro
 
Running head OVERVIEW .docx
Running head OVERVIEW                                            .docxRunning head OVERVIEW                                            .docx
Running head OVERVIEW .docx
glendar3
 
Running head OVERVIEW .docx
Running head OVERVIEW                                            .docxRunning head OVERVIEW                                            .docx
Running head OVERVIEW .docx
todd581
 
02 young vpi lecture 2014
02 young vpi lecture 201402 young vpi lecture 2014
02 young vpi lecture 2014
jemille6
 
Mind the Gap Health Systems Research and the Search for Answers
Mind the Gap Health Systems Research and the Search for AnswersMind the Gap Health Systems Research and the Search for Answers
Mind the Gap Health Systems Research and the Search for Answers
Heather Gilmartin
 
Primary-Care-Research-2017.ppt
Primary-Care-Research-2017.pptPrimary-Care-Research-2017.ppt
Primary-Care-Research-2017.ppt
ANANDBABUGOPATHOTI1
 
Care research
Care researchCare research
Care research
ssuserd770cd
 
Primary-Care-Research-2017.ppt
Primary-Care-Research-2017.pptPrimary-Care-Research-2017.ppt
Primary-Care-Research-2017.ppt
Murali Krishna
 
Surviving statistics lecture 1
Surviving statistics lecture 1Surviving statistics lecture 1
Surviving statistics lecture 1
MikeBlyth
 
Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...
Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...
Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...
John Hoey
 
20050325 Design of clinical trails in radiology
20050325 Design of clinical trails in radiology20050325 Design of clinical trails in radiology
20050325 Design of clinical trails in radiology
Internet Medical Journal
 
Primary-Care-Research-2017.pptx
Primary-Care-Research-2017.pptxPrimary-Care-Research-2017.pptx
Primary-Care-Research-2017.pptx
MohamedAbdi347025
 
Sample Of Research Essay
Sample Of Research EssaySample Of Research Essay
Sample Of Research Essay
Paper Writing Service Cheap
 
P-values the gold measure of statistical validity are not as reliable as many...
P-values the gold measure of statistical validity are not as reliable as many...P-values the gold measure of statistical validity are not as reliable as many...
P-values the gold measure of statistical validity are not as reliable as many...
David Pratap
 
MethodUnderstandingStatisticalSignificanceMatthew .docx
MethodUnderstandingStatisticalSignificanceMatthew .docxMethodUnderstandingStatisticalSignificanceMatthew .docx
MethodUnderstandingStatisticalSignificanceMatthew .docx
ARIV4
 
Factors That Impacted Effective Diabetes Management Within...
Factors That Impacted Effective Diabetes Management Within...Factors That Impacted Effective Diabetes Management Within...
Factors That Impacted Effective Diabetes Management Within...
Susan Tullis
 
MAKING SENSE OFSTATISTICSWhat statistics tell you an.docx
MAKING SENSE OFSTATISTICSWhat statistics tell you an.docxMAKING SENSE OFSTATISTICSWhat statistics tell you an.docx
MAKING SENSE OFSTATISTICSWhat statistics tell you an.docx
smile790243
 
What is research
What is researchWhat is research
What is research
Jahanzeb Jahan
 
تحليل البيانات وتفسير المعطيات
تحليل البيانات وتفسير المعطياتتحليل البيانات وتفسير المعطيات
تحليل البيانات وتفسير المعطيات
مركز البحوث الأقسام العلمية
 

Similar to Dichotomania and other challenges for the collaborating biostatistician (20)

Big Data: Learning from MIMIC- Celi
Big Data: Learning from MIMIC- CeliBig Data: Learning from MIMIC- Celi
Big Data: Learning from MIMIC- Celi
 
1_Intro to Research.pdf
1_Intro to Research.pdf1_Intro to Research.pdf
1_Intro to Research.pdf
 
Running head OVERVIEW .docx
Running head OVERVIEW                                            .docxRunning head OVERVIEW                                            .docx
Running head OVERVIEW .docx
 
Running head OVERVIEW .docx
Running head OVERVIEW                                            .docxRunning head OVERVIEW                                            .docx
Running head OVERVIEW .docx
 
02 young vpi lecture 2014
02 young vpi lecture 201402 young vpi lecture 2014
02 young vpi lecture 2014
 
Mind the Gap Health Systems Research and the Search for Answers
Mind the Gap Health Systems Research and the Search for AnswersMind the Gap Health Systems Research and the Search for Answers
Mind the Gap Health Systems Research and the Search for Answers
 
Primary-Care-Research-2017.ppt
Primary-Care-Research-2017.pptPrimary-Care-Research-2017.ppt
Primary-Care-Research-2017.ppt
 
Care research
Care researchCare research
Care research
 
Primary-Care-Research-2017.ppt
Primary-Care-Research-2017.pptPrimary-Care-Research-2017.ppt
Primary-Care-Research-2017.ppt
 
Surviving statistics lecture 1
Surviving statistics lecture 1Surviving statistics lecture 1
Surviving statistics lecture 1
 
Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...
Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...
Published Research, Flawed, Misleading, Nefarious - Use of Reporting Guidelin...
 
20050325 Design of clinical trails in radiology
20050325 Design of clinical trails in radiology20050325 Design of clinical trails in radiology
20050325 Design of clinical trails in radiology
 
Primary-Care-Research-2017.pptx
Primary-Care-Research-2017.pptxPrimary-Care-Research-2017.pptx
Primary-Care-Research-2017.pptx
 
Sample Of Research Essay
Sample Of Research EssaySample Of Research Essay
Sample Of Research Essay
 
P-values the gold measure of statistical validity are not as reliable as many...
P-values the gold measure of statistical validity are not as reliable as many...P-values the gold measure of statistical validity are not as reliable as many...
P-values the gold measure of statistical validity are not as reliable as many...
 
MethodUnderstandingStatisticalSignificanceMatthew .docx
MethodUnderstandingStatisticalSignificanceMatthew .docxMethodUnderstandingStatisticalSignificanceMatthew .docx
MethodUnderstandingStatisticalSignificanceMatthew .docx
 
Factors That Impacted Effective Diabetes Management Within...
Factors That Impacted Effective Diabetes Management Within...Factors That Impacted Effective Diabetes Management Within...
Factors That Impacted Effective Diabetes Management Within...
 
MAKING SENSE OFSTATISTICSWhat statistics tell you an.docx
MAKING SENSE OFSTATISTICSWhat statistics tell you an.docxMAKING SENSE OFSTATISTICSWhat statistics tell you an.docx
MAKING SENSE OFSTATISTICSWhat statistics tell you an.docx
 
What is research
What is researchWhat is research
What is research
 
تحليل البيانات وتفسير المعطيات
تحليل البيانات وتفسير المعطياتتحليل البيانات وتفسير المعطيات
تحليل البيانات وتفسير المعطيات
 

Recently uploaded

20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
Sharon Liu
 
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen BergstedtFish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Faculty of Applied Chemistry and Materials Science
 
Complementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotailComplementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotail
Sérgio Sacani
 
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Dr. sreeremya S
 
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
US Environmental Protection Agency (EPA), Center for Computational Toxicology and Exposure
 
Potential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptxPotential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptx
J. Bovas Joel BFSc
 
Burn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptxBurn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptx
sohil4260
 
AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...
AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...
AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...
Faculty of Applied Chemistry and Materials Science
 
Introduction to Space (Our Solar System)
Introduction to Space (Our Solar System)Introduction to Space (Our Solar System)
Introduction to Space (Our Solar System)
vanshgarg8002
 
Potential of Marine renewable and Non renewable energy.pptx
Potential of Marine renewable and Non renewable energy.pptxPotential of Marine renewable and Non renewable energy.pptx
Potential of Marine renewable and Non renewable energy.pptx
J. Bovas Joel BFSc
 
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPYReview Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
niranjangiri009
 
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptxAN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
kalpnayadav03021986
 
Post RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) MetabolismPost RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) Metabolism
Areesha Ahmad
 
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
Sérgio Sacani
 
Adjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyerAdjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyer
pablovgd
 
Structure of Sperm / Spermatozoon .pdf
Structure of  Sperm / Spermatozoon  .pdfStructure of  Sperm / Spermatozoon  .pdf
Structure of Sperm / Spermatozoon .pdf
SELF-EXPLANATORY
 
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with PhytoremediationPhytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Gurjant Singh
 
Analytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina BujorAnalytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina Bujor
Faculty of Applied Chemistry and Materials Science
 
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physicsTHE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
Dr. sreeremya S
 
MCQ in Electrostatics. for class XII pptx
MCQ in Electrostatics. for class XII  pptxMCQ in Electrostatics. for class XII  pptx
MCQ in Electrostatics. for class XII pptx
ArunachalamM22
 

Recently uploaded (20)

20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
20240710 ACMJ Diagrams Set 3.docx . Apache, Csharp, Mysql, Javascript stack a...
 
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen BergstedtFish in the Loop: Exploring RAS - Julie Hansen Bergstedt
Fish in the Loop: Exploring RAS - Julie Hansen Bergstedt
 
Complementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotailComplementary interstellar detections from the heliotail
Complementary interstellar detections from the heliotail
 
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
Direct instructions, towards hundred fold yield,layering,budding,grafting,pla...
 
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
Accessing Data to Support Pesticide Residue and Emerging Contaminant Analysis...
 
Potential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptxPotential of Marine Renewable and Non renewable energy.pptx
Potential of Marine Renewable and Non renewable energy.pptx
 
Burn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptxBurn child health Nursing 3rd year presentation..pptx
Burn child health Nursing 3rd year presentation..pptx
 
AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...
AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...
AlgaeBrew project - Unlocking the potential of microalgae for the valorisatio...
 
Introduction to Space (Our Solar System)
Introduction to Space (Our Solar System)Introduction to Space (Our Solar System)
Introduction to Space (Our Solar System)
 
Potential of Marine renewable and Non renewable energy.pptx
Potential of Marine renewable and Non renewable energy.pptxPotential of Marine renewable and Non renewable energy.pptx
Potential of Marine renewable and Non renewable energy.pptx
 
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPYReview Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
Review Article:- A REVIEW ON RADIOISOTOPES IN CANCER THERAPY
 
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptxAN EMPIRE ACROSS THE THREE CONTINENTS.pptx
AN EMPIRE ACROSS THE THREE CONTINENTS.pptx
 
Post RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) MetabolismPost RN - Biochemistry (Unit 7) Metabolism
Post RN - Biochemistry (Unit 7) Metabolism
 
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
A Strong He II λ1640 Emitter with an Extremely Blue UV Spectral Slope at z=8....
 
Adjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyerAdjusted NuGOweek 2024 Ghent programme flyer
Adjusted NuGOweek 2024 Ghent programme flyer
 
Structure of Sperm / Spermatozoon .pdf
Structure of  Sperm / Spermatozoon  .pdfStructure of  Sperm / Spermatozoon  .pdf
Structure of Sperm / Spermatozoon .pdf
 
Phytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with PhytoremediationPhytoremediation: Harnessing Nature's Power with Phytoremediation
Phytoremediation: Harnessing Nature's Power with Phytoremediation
 
Analytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina BujorAnalytical methods for blue residues characterization - Oana Crina Bujor
Analytical methods for blue residues characterization - Oana Crina Bujor
 
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physicsTHE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
THE ESSENCE OF CHANGE CHAPTER ,energy,conversion,life is easy,laws of physics
 
MCQ in Electrostatics. for class XII pptx
MCQ in Electrostatics. for class XII  pptxMCQ in Electrostatics. for class XII  pptx
MCQ in Electrostatics. for class XII pptx
 

Dichotomania and other challenges for the collaborating biostatistician

  • 1. Dichotomania and other challenges for the collaborating biostatistician A perspective on principles, responsibilities and potential solutions Laure Wynants PhD laure.wynants@maastrichtuniversity.nl @laure_wynants
  • 2. Dichotomania and other challenges for the collaborating biostatistician A perspective on principles, responsibilities and potential solutions Laure Wynants PhD laure.wynants@maastrichtuniversity.nl @laure_wynants or teaching
  • 3. 2018 conference invitation “Statistics made very easy for clinicians”
  • 4. 2018 conference invitation “Statistics made very easy for clinicians” “Free coffee and croissants while statisticians explain the P- value crisis”
  • 5. Goodman S. A dirty dozen: twelve p-value misconceptions. Semin Hematol. 2008 Jul;45(3):135-40.
  • 6. 6 Estimated probability of causal attribution according to the null P-value, modeled using fractional polynomials with a cutpoint at P = 0.05. A Psychometric Experiment in Causal Inference to Estimate Evidential Weights Used by Epidemiologists Holman, C. D’Arcy J.; Arnold-Reed, Diane E.; de Klerk, Nicholas; McComb, Christine; English, Dallas R., Epidemiology12(2):246-255, 2001. https://xkcd.com/
  • 8. A conversation between a researcher and a statistician • R: “We need some statistical testing for these plots.” • S: “Why? These are not your main research questions in this paper.” • R: “I am not fishing for significant findings. I am aware of the dangers. These are hypotheses we investigated in earlier work. If the tests are significant, we know this is confirmed in our new data.”
  • 9. A conversation between a researcher and a statistician • R: “We need some statistical testing for these plots.” • S: “Why? These are not your main research questions in this paper.” • R: “I am not fishing for significant findings. I am aware of the dangers. These are hypotheses we investigated in earlier work. If the tests are significant, we know this is confirmed in our new data.” • R: “If it is not significant, we will discuss further. We just didn’t have enough power then.”
  • 10. Reporting and publication bias “Trim and fill” funnel plot of Ki-67 expression for overall survival in ovarian cancer patients (Qiu et al. Arch Gynecol Obstet 2019) Missing studies Published studies
  • 11. Science as a disorderly mass of stray observations, inconclusive results and fledgling explanations. And yet, as soon as their hypotheses were turned into peer-reviewed papers, researchers claimed that such facts had always spoken for themselves.
  • 12. How (not) to report results
  • 13. Replication crisis Ioannidis JAMA 2005: all original research published in 3 major journals in 1990-2003 and cited >1000 times: 49 studies, 45 claimed that the intervention was effective. 32% could not be replicated: 16% was contradicted (no effect found) 16% estimated effects were too strong (to the point that subsequent studies cast doubt on effect being clinically important) 44% were replicated 11% no subsequent larger/better designed replication studies Problems were worst for small RCTS and non-randomized studies. Begly & Ellis Nature 2012: replication of landmark preclinical cancer studies: only 11% could be reproduced. Journal impact factor Number of articles Mean number of citations of non-reproduced articles Mean number of citations of reproduced articles >20 21 248 (range 3–800) 231 (range 82–519) 5–19 32 169 (range 6–1,909) 13 (range 3–24)
  • 14. • SARS-Cov-2 “viral loads in the very young do not differ significantly from those of adults. Based on these results, we have to caution against an unlimited re-opening of schools and kindergartens in the present situation” • Ill-defined research question, comparison between all age groups (45 comparisons), test as if non-ordered categories. • Reanalyzes with more appropriate techniques finds opposite conclusion. • https://medium.com/@d_spiegel/is-sars-cov-2-viral-load-lower-in-young-children-than-adults- 8b4116d28353
  • 15. A mistake in the operating room can threaten the life of one patient; A mistake in the statistical analysis or interpretation can lead to hundreds of early deaths. Andrew Vickers, Biostatistician, Memorial Sloan Kettering Cancer Center
  • 16. Some reactions to previous presentations MDs - surprise that meta-analysis can be biased - “Our statisticians did not tell us this”
  • 17. Altman BMJ 1994, republished almost unchanged 20 years later… “Put simply, much poor research arises because researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them.”
  • 18. Statisticians - “Oh no not this again” - “We know this already” - “P-values are not the problem” - “It’s not us, it’s them”
  • 23. An ethical statistician… identifies and mitigates any preferences on the part of the investigators or data providers that might predetermine or influence the analyses/results only support studies that have pre-defined objectives and that are capable of producing useful results strives to explain any expected adverse consequences of failure to follow through on an agreed-upon sampling or analytic plan shall indicate the risks and possible consequences if their professional judgement is overruled Views or opinions based on general knowledge or belief should be clearly distinguished from views or opinions derived from the statistical analyses being reported. Taken from RSS code and ASA ethical guidelines
  • 24. An ethical statistician… recognizes […] research practices and standards can differ across disciplines, and statisticians do not have obligations to standards of other professions that conflict with these guidelines shall take personal responsibility for work bearing their name avoids compromising scientific validity for expediency should always be aware of their overriding responsibility to the public good […] A Fellow’s obligations to employers, clients and the profession can never override this; and Fellows should seek to avoid situations and not enter into undertakings which compromise this responsibility Taken from RSS code and ASA ethical guidelines
  • 25. An ethical statistician… conveys the findings in ways that are both honest and meaningful to the user/reader shall seek to conform to recognised good practice including quality standards which are in their judgement relevant, and shall encourage others to do likewise shall seek to advance knowledge and understanding of statistical science and advocate its use. This advocacy of statistical science should extend to employers, clients, colleagues and the general public Taken from RSS code and ASA ethical guidelines
  • 27. What can we do better? ATOM - Accept uncertainty (no more ***, interpret confidence intervals) - Be Thoughtful (research question, design, clinically relevant effect size, registered reports) - Be Open (conflicts of interest, registration, share data, code, analysis protocols, publish all results) - Be Modest (exploratory, retrospective, secondary analyses (no harking); interpret studies in broader context) Ronald L. Wasserstein, Allen L. Schirm & Nicole A. Lazar (2019) Moving to a World Beyond “p < 0.05”, The American Statistician, 73:sup1, 1-19
  • 28. What can we do better? Distinguish between different applications E.g. Cox 2020 - Two-decision situation (health screening: return tomorrow vs next year; control error) - Subject-matter hypothesis (difference in trt, H0: there is no difference; p-value as measure of uncertainty) - Dividing hypothesis (at which level does CI only contain positive/negative effects?) - Tests of model adequacy(normality assumption, informal role, judgement required)
  • 29. What can we do better? • “How to” teaching vs understanding Simulated data (Bishop 2020) • Be explicit about how principles extend to observational research • Software • Conceptual clarity in educational material (Greenland 2019) “Significance level”: α or p-value? “P-value”: observed value p or random variable P?
  • 30. It won’t be easy • Misconception fatigue in teaching
  • 31. It won’t be easy Wang et al Annals of Internal Medicine 2018
  • 33. No statistician can do this alone • A responsibility for each of us • A role for professional organizations • A necessity to put this on the agenda of ISCB • Thanks to John Carlin and Jonathan Sterne - even if there are no free croissants
  • 37. Replicability without p-values? • Hanson (1958) – anthropology, sociology, psychology - Statements with “confirmation criteria” (test): >70% confirmed - Statements without confirmation criteria: <46% confirmed • Basic Applied Social Psychology banned p-values - Overinterpret / overstate descriptive results Lakens et al 2020