•Download as PPTX, PDF•

10 likes•5,558 views

How to combine results from randomised clinical trials on the additive scale with real world data to provide predictions on the clinically relevant scale for individual patients

Report

Share

Report

Share

Regression shrinkage: better answers to causal questions

The document discusses a presentation on regression shrinkage and its implications for causal inference in epidemiological research. The presentation argues that alternative statistical models to logistic regression, such as Firth's correction, are generally "better" as they reduce bias. Firth's correction shrinks estimated coefficients towards less extreme values, reducing finite sample bias compared to maximum likelihood estimation. Simulations show that Firth's correction reduces bias in estimated odds ratios from around 25% to approximately 3%.

Improving predictions: Lasso, Ridge and Stein's paradox

Slides of masterclass "Improving predictions: Lasso, Ridge and Stein's paradox" at the (Dutch) National Institute for Public Health and the Environment (RIVM)

The replication crisis: are P-values the problem and are Bayes factors the so...

Today’s posterior is tomorrow’s prior. Dennis Lindley1 (P2)
It has been claimed that science is undergoing a replication crisis and that when looking for culprits, the cult of significance is the chief suspect. It has also been claimed that Bayes factors might provide a solution.
In my opinion, these claims are misleading and part of the problem is our understanding of the purpose and nature of replication, which has only recently been subject to formal analysis2. What we are or should be interested in is truth. Replication is a coherence not a correspondence requirement3 and one that has a strong dependence on the size of the replication study4.
Consideration of Bayes factors raises a puzzling question. Should the Bayes factor for a replication study be calculated as if it were the initial study? If the answer is yes, the approach is not fully Bayesian and furthermore the Bayes factors will be subject to exactly the same replication ‘paradox’ as P-values. If the answer is no, then in what sense can an initially found Bayes factor be replicated and what are the implications for how we should view replication of P-values?
A further issue is that little attention has been paid to false negatives and, by extension to true negative values. Yet, as is well known from the theory of diagnostic tests, it is meaningless to consider the performance of a test in terms of false positives alone.
I shall argue that we are in danger of confusing evidence with the conclusions we draw and that any reforms of scientific practice should concentrate on producing evidence that is reliable as it can be qua evidence. There are many basic scientific practices in need of reform. Pseudoreplication5, for example, and the routine destruction of information through dichotomisation6 are far more serious problems than many matters of inferential framing that seem to have excited statisticians.
References
1. Lindley DV. Bayesian statistics: A review. SIAM; 1972.
2. Devezer B, Navarro DJ, Vandekerckhove J, Ozge Buzbas E. The case for formal methodology in scientific reform. R Soc Open Sci. Mar 31 2021;8(3):200805. doi:10.1098/rsos.200805
3. Walker RCS. Theories of Truth. In: Hale B, Wright C, Miller A, eds. A Companion to the Philosophy of Language. John Wiley & Sons,; 2017:532-553:chap 21.
4. Senn SJ. A comment on replication, p-values and evidence by S.N.Goodman, Statistics in Medicine 1992; 11:875-879. Letter. Statistics in Medicine. 2002;21(16):2437-44.
5. Hurlbert SH. Pseudoreplication and the design of ecological field experiments. Ecological monographs. 1984;54(2):187-211.
6. Senn SJ. Being Efficient About Efficacy Estimation. Research. Statistics in Biopharmaceutical Research. 2013;5(3):204-210. doi:10.1080/19466315.2012.754726

Why I hate minimisation

Minimisation is an approach to allocating patients to treatment in clinical trials that forces a greater degree of balance than does randomisation. Here I explain why I dislike it.

Clinical trials are about comparability not generalisability V2.pptx

Lecture delivered at the September 2022 EFSPI meeting in Basle in which I argued that the patients in a clinical trial should not be viewed as being a representative sample of some target population.

Choosing Regression Models

This document provides an introduction to choosing regression models. It discusses basic considerations like determining the purpose of the model, choosing appropriate predictors, and whether predictors or the outcome need transformation. Temporal sequence and prior knowledge are important factors in choosing predictors. The type of data, case ascertainment, and results of model fitting also influence predictor choice. Transforming predictors or the outcome can improve the model fit in some cases. The key is using statistical tools together with experience and understanding, not as a substitute for scientific insight.

Is it causal, is it prediction or is it neither?

This document discusses the differences between explanatory models, predictive models, and descriptive models. Explanatory models aim to understand causal relationships by examining regression coefficients and testing theories. Predictive models focus on predicting future observations without considering causality, and addressing overfitting. Descriptive models simply capture data structures. While the goals differ, the document notes problems like generalizability and model misspecification are common challenges. It provides examples of epidemiological and medical prediction models and emphasizes the need for external validation of predictive performance.

ML and AI: a blessing and curse forstatisticians and medical doctors

Freiburg, Germany, Institut für Medizinische Biometrie und Statistik, Biometrischen Kolloquium. Organized by Prof Willi Sauerbrei.

Regression shrinkage: better answers to causal questions

The document discusses a presentation on regression shrinkage and its implications for causal inference in epidemiological research. The presentation argues that alternative statistical models to logistic regression, such as Firth's correction, are generally "better" as they reduce bias. Firth's correction shrinks estimated coefficients towards less extreme values, reducing finite sample bias compared to maximum likelihood estimation. Simulations show that Firth's correction reduces bias in estimated odds ratios from around 25% to approximately 3%.

Improving predictions: Lasso, Ridge and Stein's paradox

Slides of masterclass "Improving predictions: Lasso, Ridge and Stein's paradox" at the (Dutch) National Institute for Public Health and the Environment (RIVM)

The replication crisis: are P-values the problem and are Bayes factors the so...

Today’s posterior is tomorrow’s prior. Dennis Lindley1 (P2)
It has been claimed that science is undergoing a replication crisis and that when looking for culprits, the cult of significance is the chief suspect. It has also been claimed that Bayes factors might provide a solution.
In my opinion, these claims are misleading and part of the problem is our understanding of the purpose and nature of replication, which has only recently been subject to formal analysis2. What we are or should be interested in is truth. Replication is a coherence not a correspondence requirement3 and one that has a strong dependence on the size of the replication study4.
Consideration of Bayes factors raises a puzzling question. Should the Bayes factor for a replication study be calculated as if it were the initial study? If the answer is yes, the approach is not fully Bayesian and furthermore the Bayes factors will be subject to exactly the same replication ‘paradox’ as P-values. If the answer is no, then in what sense can an initially found Bayes factor be replicated and what are the implications for how we should view replication of P-values?
A further issue is that little attention has been paid to false negatives and, by extension to true negative values. Yet, as is well known from the theory of diagnostic tests, it is meaningless to consider the performance of a test in terms of false positives alone.
I shall argue that we are in danger of confusing evidence with the conclusions we draw and that any reforms of scientific practice should concentrate on producing evidence that is reliable as it can be qua evidence. There are many basic scientific practices in need of reform. Pseudoreplication5, for example, and the routine destruction of information through dichotomisation6 are far more serious problems than many matters of inferential framing that seem to have excited statisticians.
References
1. Lindley DV. Bayesian statistics: A review. SIAM; 1972.
2. Devezer B, Navarro DJ, Vandekerckhove J, Ozge Buzbas E. The case for formal methodology in scientific reform. R Soc Open Sci. Mar 31 2021;8(3):200805. doi:10.1098/rsos.200805
3. Walker RCS. Theories of Truth. In: Hale B, Wright C, Miller A, eds. A Companion to the Philosophy of Language. John Wiley & Sons,; 2017:532-553:chap 21.
4. Senn SJ. A comment on replication, p-values and evidence by S.N.Goodman, Statistics in Medicine 1992; 11:875-879. Letter. Statistics in Medicine. 2002;21(16):2437-44.
5. Hurlbert SH. Pseudoreplication and the design of ecological field experiments. Ecological monographs. 1984;54(2):187-211.
6. Senn SJ. Being Efficient About Efficacy Estimation. Research. Statistics in Biopharmaceutical Research. 2013;5(3):204-210. doi:10.1080/19466315.2012.754726

Why I hate minimisation

Minimisation is an approach to allocating patients to treatment in clinical trials that forces a greater degree of balance than does randomisation. Here I explain why I dislike it.

Clinical trials are about comparability not generalisability V2.pptx

Lecture delivered at the September 2022 EFSPI meeting in Basle in which I argued that the patients in a clinical trial should not be viewed as being a representative sample of some target population.

Choosing Regression Models

This document provides an introduction to choosing regression models. It discusses basic considerations like determining the purpose of the model, choosing appropriate predictors, and whether predictors or the outcome need transformation. Temporal sequence and prior knowledge are important factors in choosing predictors. The type of data, case ascertainment, and results of model fitting also influence predictor choice. Transforming predictors or the outcome can improve the model fit in some cases. The key is using statistical tools together with experience and understanding, not as a substitute for scientific insight.

Is it causal, is it prediction or is it neither?

This document discusses the differences between explanatory models, predictive models, and descriptive models. Explanatory models aim to understand causal relationships by examining regression coefficients and testing theories. Predictive models focus on predicting future observations without considering causality, and addressing overfitting. Descriptive models simply capture data structures. While the goals differ, the document notes problems like generalizability and model misspecification are common challenges. It provides examples of epidemiological and medical prediction models and emphasizes the need for external validation of predictive performance.

ML and AI: a blessing and curse forstatisticians and medical doctors

Freiburg, Germany, Institut für Medizinische Biometrie und Statistik, Biometrischen Kolloquium. Organized by Prof Willi Sauerbrei.

Correcting for missing data, measurement error and confounding

Presentation for the epidemiologic methods group at Julius Center for Health Sciences and Primary Care, UMC Utrecht on Nov 30 2020

Prognosis-based medicine: merits and pitfalls of forecasting patient health

Presentation at the Forecasting for Social Good Workshop, International Symposium on Forecasting, in Oxford (UK) on Sunday July 10 2022

Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019

Title"Clinical prediction models in the age of artificial intelligence and big data", presented at the Basel Biometrics Society seminar Nov 1, 2019, Basel, by Ewout Steyerberg, with substantial inout from Maarten van Smeden and Ben van Calster

P-values in crisis

A comment in Nature, signed by over 800 researchers, called for a rise up against statistical significance. This was followed by a special issue of The American Statistician aimed at halting the use of the term "statistically significant", and new guidelines for statistical reporting in the New England Journal of Medicine. These slides discuss the broader context of the "p-value crisis" and alternatives for communicating the conclusions after statistical analyses.
Target audience: Medical researchers; Scientists involved in conducting or interpreting analyses and communicating the results of scientific research, as well as readers of scientific publications.
Learning objectives:
To understand the context of the reproducibility crisis in medical research.
To learn about problems with p-values and alternatives to report findings.
To understand how (not) to interpret significant and insignificant findings.
To learn how to communicate research findings in a modest, thoughtful, and transparent way.

Personalised medicine a sceptical view

Some grounds for believing that the current enthusiasm about personalised medicine is exaggerated, founded on poor statistics and represents a disappointing loss of ambition.

QUANTIFYING THE IMPACT OF DIFFERENT APPROACHES FOR HANDLING CONTINUOUS PREDIC...

Continuous predictors are often dichotomized or categorized in prognostic models, despite recommendations against this practice. This study investigated the impact of different approaches to handling continuous predictors on model performance and validation. The researchers found that dichotomizing continuous predictors, either at the median or an "optimal" cut-point, led to substantially worse model discrimination, calibration, and clinical utility compared to analyzing predictors linearly or with fractional polynomials. The negative impact of dichotomizing was more pronounced at smaller sample sizes. Maintaining continuous predictors yielded better prognostic performance and validation than dichotomizing.

Uncertainty in AI

Presentation about uncertainties in AI for the SIDM the future of diagnosis congres in Utrecht, July 4 2023

Introduction to prediction modelling - Berlin 2018 - Part I

This document describes an introduction to prediction modeling workshop given by Maarten van Smeden. It discusses using linear regression to predict systolic blood pressure at discharge for heart failure patients using variables like age, gender, medications and blood pressure at admission. Descriptive analyses of the simulated data of 7,000 patients are shown. The goal is to develop a model to predict individual patient outcomes using combinations of predictor variables.

What is the point of point estimates

The importance of measurements of uncertainty is explained and illustrated with the help of a famous experiment in nutrition from 1930 and a complex cross-over trial from the 1990s,

The Seven Habits of Highly Effective Statisticians

This document provides advice on habits that make statisticians effective. It discusses the importance of understanding causation, control, comparison and counterfactuals when thinking about effectiveness. It warns against proposing habits as causes without proper evaluation. Seven key habits are identified: read, listen, understand, think, do, calculate, and communicate. The document illustrates these habits through examples of invalid inversion, regression to the mean, and statistical mistakes. It emphasizes understanding concepts fundamentally rather than just mathematically and finding simple ways to communicate ideas.

Development and evaluation of prediction models: pitfalls and solutions

Slides for the statistics in practice session for the Biometrisches Kolloqium (organized by the Deutsche Region der Internationalen Biometrischen Gesellschaft), 18 March 2021

Dichotomania and other challenges for the collaborating biostatistician

Conference presentation at ISCB 41 in the session
"Biostatistical inference in practice: moving beyond false
dichotomies"
A comment in Nature, signed by over 800 researchers, called for the scientific community to “retire statistical significance”. The responses included a call to halt the use of the term „statistically significant”, and changes in journal’s author guidelines. The leading discourse among statisticians is that inadequate statistical training of clinical researchers and publishing practices are to blame for the misuse of statistical testing. In this presentation, we search our collective conscience by reviewing ethical guidelines for statisticians in light of the p-value crisis, examine what this implies for us when conducting analyses in collaborative work and teaching, and whether the ATOM (accept uncertainty; be thoughtful, open and modest) principles can guide us.

Introduction to prediction modelling - Berlin 2018 - Part II

This document summarizes the key steps in building a risk prediction model:
1. Conduct research design and data collection, typically using a prospective cohort study.
2. Choose statistical model, outcome, and candidate predictors based on clinical knowledge.
3. Perform initial data analysis including descriptive statistics and assessing predictors.
4. Specify and estimate the prediction model, addressing issues like handling continuous predictors and missing data.
5. Evaluate the model's performance using measures like discrimination and calibration and perform internal validation to account for overoptimism.
6. Present the final model following reporting guidelines like TRIPOD.

The basics of prediction modeling

Here are the steps to solve this exercise:
1) Given:
Prev = 30%
Se = 99%
Sp = 95%
2) Calculate other metrics:
PPV = 75%
NPV = 99.7%
3) Re-calculate NPV assuming Prev of 10%:
NPV = 99.95%
4) Re-calculate NPV assuming Prev of 80%:
NPV = 91.2%
So in summary, the NPV decreases as the prevalence increases, since with a higher prevalence there is a higher chance that a negative test result represents a false negative.

Algorithm based medicine: old statistics wine in new machine learning bottles?

The document summarizes a seminar presentation given by Maarten van Smeden on algorithm based medicine and machine learning. Some key points made in the presentation include: the terminology of artificial intelligence often refers to machine learning or algorithms in medical research; examples are given of areas where machine learning has performed well, such as detecting diabetic retinopathy and lymph node metastases; examples are also provided of where machine learning has done poorly, such as predicting recidivism and mortality; and the sources of prediction error from machine learning models are discussed.

Predictimands

Presentation at Workshop on missing values and estimands in diagnostic accuracy studies, Hamburg (Germany)

Clinical prediction models

Introduction to clinical prediction models for the clinical trials and research methodology meeting by the Turkish society of cardiology in Istanbul

Why the EPV≥10 sample size rule is rubbish and what to use instead

This document discusses issues with the commonly used EPV≥10 sample size rule for prognostic/diagnostic prediction modeling. It argues that the rule has no strong rationale and that sample size is still important even when using more sophisticated methods. It presents evidence that logistic regression coefficients are subject to finite sample bias and introduces Firth's correction as a method to reduce this bias. While this method improves matters, the document cautions that sample size planning still requires consideration of multiple factors specific to the model and validation rather than relying on a single rule-of-thumb.

Statistics and ML 21Oct22 sel.pptx

Presentation on similarities and differences between statistical and machine learning research fields for the @UM_MiCHAMP Big Data & AI in Health Seminar Series; October 21, 2022

NNTs, responder analysis & overlap measures

Unfortunately, some have interpreted Numbers Needed to Treat as indicating the proportion of patients on whom the treatment has had a causal effect. This interpretation is very rarely, if ever, necessarily correct. It is certainly inappropriate if based on a responder dichotomy. I shall illustrate the problem using simple causal models.
One also sometimes encounters the claim that the extent to which two distributions of outcomes overlap from a clinical trial indicates how many patients benefit. This is also false and can be traced to a similar causal confusion.

The challenge of small data

1) The document discusses the challenge of studying rare diseases due to the small amount of available data. It proposes that N-of-1 trials, where individual patients are repeatedly randomized to treatment or control, could help address this issue.
2) It provides examples of how careful experimental design and statistical analysis are important even with small data sets. Factors like randomization, blocking, and replication can increase efficiency and validity.
3) Analyzing an N-of-1 trial for a rare disease, the document explores objectives like determining if one treatment is better, estimating average effects, and predicting effects for future patients. It discusses randomization and sampling philosophies and mixed effects models.

In search of the lost loss function

Sample size determination in clinical trials is considered from various ethical and practical perspectives. It is concluded that cost is a missing dimension and that the value of information is key.

Correcting for missing data, measurement error and confounding

Presentation for the epidemiologic methods group at Julius Center for Health Sciences and Primary Care, UMC Utrecht on Nov 30 2020

Prognosis-based medicine: merits and pitfalls of forecasting patient health

Presentation at the Forecasting for Social Good Workshop, International Symposium on Forecasting, in Oxford (UK) on Sunday July 10 2022

Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019

Title"Clinical prediction models in the age of artificial intelligence and big data", presented at the Basel Biometrics Society seminar Nov 1, 2019, Basel, by Ewout Steyerberg, with substantial inout from Maarten van Smeden and Ben van Calster

P-values in crisis

A comment in Nature, signed by over 800 researchers, called for a rise up against statistical significance. This was followed by a special issue of The American Statistician aimed at halting the use of the term "statistically significant", and new guidelines for statistical reporting in the New England Journal of Medicine. These slides discuss the broader context of the "p-value crisis" and alternatives for communicating the conclusions after statistical analyses.
Target audience: Medical researchers; Scientists involved in conducting or interpreting analyses and communicating the results of scientific research, as well as readers of scientific publications.
Learning objectives:
To understand the context of the reproducibility crisis in medical research.
To learn about problems with p-values and alternatives to report findings.
To understand how (not) to interpret significant and insignificant findings.
To learn how to communicate research findings in a modest, thoughtful, and transparent way.

Personalised medicine a sceptical view

Some grounds for believing that the current enthusiasm about personalised medicine is exaggerated, founded on poor statistics and represents a disappointing loss of ambition.

QUANTIFYING THE IMPACT OF DIFFERENT APPROACHES FOR HANDLING CONTINUOUS PREDIC...

Continuous predictors are often dichotomized or categorized in prognostic models, despite recommendations against this practice. This study investigated the impact of different approaches to handling continuous predictors on model performance and validation. The researchers found that dichotomizing continuous predictors, either at the median or an "optimal" cut-point, led to substantially worse model discrimination, calibration, and clinical utility compared to analyzing predictors linearly or with fractional polynomials. The negative impact of dichotomizing was more pronounced at smaller sample sizes. Maintaining continuous predictors yielded better prognostic performance and validation than dichotomizing.

Uncertainty in AI

Presentation about uncertainties in AI for the SIDM the future of diagnosis congres in Utrecht, July 4 2023

Introduction to prediction modelling - Berlin 2018 - Part I

This document describes an introduction to prediction modeling workshop given by Maarten van Smeden. It discusses using linear regression to predict systolic blood pressure at discharge for heart failure patients using variables like age, gender, medications and blood pressure at admission. Descriptive analyses of the simulated data of 7,000 patients are shown. The goal is to develop a model to predict individual patient outcomes using combinations of predictor variables.

What is the point of point estimates

The importance of measurements of uncertainty is explained and illustrated with the help of a famous experiment in nutrition from 1930 and a complex cross-over trial from the 1990s,

The Seven Habits of Highly Effective Statisticians

This document provides advice on habits that make statisticians effective. It discusses the importance of understanding causation, control, comparison and counterfactuals when thinking about effectiveness. It warns against proposing habits as causes without proper evaluation. Seven key habits are identified: read, listen, understand, think, do, calculate, and communicate. The document illustrates these habits through examples of invalid inversion, regression to the mean, and statistical mistakes. It emphasizes understanding concepts fundamentally rather than just mathematically and finding simple ways to communicate ideas.

Development and evaluation of prediction models: pitfalls and solutions

Slides for the statistics in practice session for the Biometrisches Kolloqium (organized by the Deutsche Region der Internationalen Biometrischen Gesellschaft), 18 March 2021

Dichotomania and other challenges for the collaborating biostatistician

Conference presentation at ISCB 41 in the session
"Biostatistical inference in practice: moving beyond false
dichotomies"
A comment in Nature, signed by over 800 researchers, called for the scientific community to “retire statistical significance”. The responses included a call to halt the use of the term „statistically significant”, and changes in journal’s author guidelines. The leading discourse among statisticians is that inadequate statistical training of clinical researchers and publishing practices are to blame for the misuse of statistical testing. In this presentation, we search our collective conscience by reviewing ethical guidelines for statisticians in light of the p-value crisis, examine what this implies for us when conducting analyses in collaborative work and teaching, and whether the ATOM (accept uncertainty; be thoughtful, open and modest) principles can guide us.

Introduction to prediction modelling - Berlin 2018 - Part II

This document summarizes the key steps in building a risk prediction model:
1. Conduct research design and data collection, typically using a prospective cohort study.
2. Choose statistical model, outcome, and candidate predictors based on clinical knowledge.
3. Perform initial data analysis including descriptive statistics and assessing predictors.
4. Specify and estimate the prediction model, addressing issues like handling continuous predictors and missing data.
5. Evaluate the model's performance using measures like discrimination and calibration and perform internal validation to account for overoptimism.
6. Present the final model following reporting guidelines like TRIPOD.

The basics of prediction modeling

Here are the steps to solve this exercise:
1) Given:
Prev = 30%
Se = 99%
Sp = 95%
2) Calculate other metrics:
PPV = 75%
NPV = 99.7%
3) Re-calculate NPV assuming Prev of 10%:
NPV = 99.95%
4) Re-calculate NPV assuming Prev of 80%:
NPV = 91.2%
So in summary, the NPV decreases as the prevalence increases, since with a higher prevalence there is a higher chance that a negative test result represents a false negative.

Algorithm based medicine: old statistics wine in new machine learning bottles?

The document summarizes a seminar presentation given by Maarten van Smeden on algorithm based medicine and machine learning. Some key points made in the presentation include: the terminology of artificial intelligence often refers to machine learning or algorithms in medical research; examples are given of areas where machine learning has performed well, such as detecting diabetic retinopathy and lymph node metastases; examples are also provided of where machine learning has done poorly, such as predicting recidivism and mortality; and the sources of prediction error from machine learning models are discussed.

Predictimands

Presentation at Workshop on missing values and estimands in diagnostic accuracy studies, Hamburg (Germany)

Clinical prediction models

Introduction to clinical prediction models for the clinical trials and research methodology meeting by the Turkish society of cardiology in Istanbul

Why the EPV≥10 sample size rule is rubbish and what to use instead

This document discusses issues with the commonly used EPV≥10 sample size rule for prognostic/diagnostic prediction modeling. It argues that the rule has no strong rationale and that sample size is still important even when using more sophisticated methods. It presents evidence that logistic regression coefficients are subject to finite sample bias and introduces Firth's correction as a method to reduce this bias. While this method improves matters, the document cautions that sample size planning still requires consideration of multiple factors specific to the model and validation rather than relying on a single rule-of-thumb.

Statistics and ML 21Oct22 sel.pptx

Presentation on similarities and differences between statistical and machine learning research fields for the @UM_MiCHAMP Big Data & AI in Health Seminar Series; October 21, 2022

NNTs, responder analysis & overlap measures

Unfortunately, some have interpreted Numbers Needed to Treat as indicating the proportion of patients on whom the treatment has had a causal effect. This interpretation is very rarely, if ever, necessarily correct. It is certainly inappropriate if based on a responder dichotomy. I shall illustrate the problem using simple causal models.
One also sometimes encounters the claim that the extent to which two distributions of outcomes overlap from a clinical trial indicates how many patients benefit. This is also false and can be traced to a similar causal confusion.

Correcting for missing data, measurement error and confounding

Correcting for missing data, measurement error and confounding

Prognosis-based medicine: merits and pitfalls of forecasting patient health

Prognosis-based medicine: merits and pitfalls of forecasting patient health

Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019

Prediction, Big Data, and AI: Steyerberg, Basel Nov 1, 2019

P-values in crisis

P-values in crisis

Personalised medicine a sceptical view

Personalised medicine a sceptical view

QUANTIFYING THE IMPACT OF DIFFERENT APPROACHES FOR HANDLING CONTINUOUS PREDIC...

QUANTIFYING THE IMPACT OF DIFFERENT APPROACHES FOR HANDLING CONTINUOUS PREDIC...

Uncertainty in AI

Uncertainty in AI

Introduction to prediction modelling - Berlin 2018 - Part I

Introduction to prediction modelling - Berlin 2018 - Part I

What is the point of point estimates

What is the point of point estimates

The Seven Habits of Highly Effective Statisticians

The Seven Habits of Highly Effective Statisticians

Development and evaluation of prediction models: pitfalls and solutions

Development and evaluation of prediction models: pitfalls and solutions

Dichotomania and other challenges for the collaborating biostatistician

Dichotomania and other challenges for the collaborating biostatistician

Introduction to prediction modelling - Berlin 2018 - Part II

Introduction to prediction modelling - Berlin 2018 - Part II

The basics of prediction modeling

The basics of prediction modeling

Algorithm based medicine: old statistics wine in new machine learning bottles?

Algorithm based medicine: old statistics wine in new machine learning bottles?

Predictimands

Predictimands

Clinical prediction models

Clinical prediction models

Why the EPV≥10 sample size rule is rubbish and what to use instead

Why the EPV≥10 sample size rule is rubbish and what to use instead

Statistics and ML 21Oct22 sel.pptx

Statistics and ML 21Oct22 sel.pptx

NNTs, responder analysis & overlap measures

NNTs, responder analysis & overlap measures

The challenge of small data

1) The document discusses the challenge of studying rare diseases due to the small amount of available data. It proposes that N-of-1 trials, where individual patients are repeatedly randomized to treatment or control, could help address this issue.
2) It provides examples of how careful experimental design and statistical analysis are important even with small data sets. Factors like randomization, blocking, and replication can increase efficiency and validity.
3) Analyzing an N-of-1 trial for a rare disease, the document explores objectives like determining if one treatment is better, estimating average effects, and predicting effects for future patients. It discusses randomization and sampling philosophies and mixed effects models.

In search of the lost loss function

Sample size determination in clinical trials is considered from various ethical and practical perspectives. It is concluded that cost is a missing dimension and that the value of information is key.

What is your question

There are many questions one might ask of a clinical trial, ranging from what was the effect in the patients studied to what might the effect be in future patients via what was the effect in individual patients? The extent to which the answer to these questions is similar depends on various assumptions made and in some cases the design used may not permit any meaningful answer to be given at all.
A related issue is confusion between randomisation, random sampling, linear model and true multivariate based modelling. These distinctions don’t matter much for some purposes and under some circumstances but for others they do.
A yet further issue is that causal analysis in epidemiology, which has brought valuable insights in many cases, has tended to stress point estimates and ignore standard errors. This has potentially misleading consequences.
An understanding of components of variation is key. Unfortunately, the development of two particular topics in recent years, evidence synthesis by the evidence based medicine movement and personalised medicine by bench scientists has either paid scant attention to components of variation or to the questions being asked or both resulting in confusion about many issues.
For instance, it is often claimed that numbers needed to treat indicate the proportion of patients for whom treatments work, that inclusion criteria determine the generalisability of results and that heterogeneity means that a random effects meta-analysis is required. None of these is true. The scope for personalised medicine has very plausibly been exaggerated and an important cause of variation in the healthcare system, physicians, is often overlooked.
I shall argue that thinking about questions is important.

What is your question

1) Clinical trials can aim to answer causal or predictive questions, but the questions and assumptions need to be clear.
2) For causal questions, randomization-based analyses that account for study design are important. Linear models can be valid if they respect randomization.
3) Claims of personalized medicine are often overstated due to misinterpretations of variation in trial data. Repeated trials on individuals using designs like n-of-1 trials could provide more insight into response heterogeneity.

Clinical trials: quo vadis in the age of covid?

A discussion of the role of clinical trials in the age of COVID. My contribution to the phastar 2020 life sciences summit https://phastar.com/phastar-life-science-summit

To infinity and beyond v2

The statistical revolution of the 20th century was largely concerned with developing methods for analysing small datasets. Student’s paper of 1908 was the first in the English literature to address the problem of second order uncertainty (uncertainty about the measures of uncertainty) seriously and was hailed by Fisher as heralding a new age of statistics. Much of what Fisher did was concerned with problems of what might be called ‘small data’, not only as regards efficient analysis but also as regards efficient design and in addition paying close attention to what was necessary to measure uncertainty validly.
I shall consider the history of some of these developments, in particular those that are associated with what might be called the Rothamsted School, starting with Fisher and having its apotheosis in John Nelder’s theory of General Balance and see what lessons they hold for the supposed ‘big data’ revolution of the 21st century.

Minimally important differences

Lecture given to the workshop on minimally important differences at the 2nd EuroQol Academy Meeting 8 March 2017 in Noordwijk, NL

Minimally important differences v2

When estimating sample sizes for clinical trials there are several different views that might be taken as to what definition and meaning should be given to the sought-for treatment effect. However, if the concept of a ‘minimally important difference’ (MID) does have relevance to interpreting clinical trials (which can be disputed) then its value cannot be the same as the ‘clinically relevant difference’ (CRD) that would be used for planning them.
A doubly pernicious use of the MID is as a means of classifying patients as responders and non-responders. Not only does such an analysis lead to an increase in the necessary sample size but it misleads trialists into making causal distinctions that the data cannot support and has been responsible for exaggerating the scope for personalised medicine.
In this talk these statistical points will be explained using a minimum of technical detail.

Numbers needed to mislead

1) The document discusses how clinical trial responses are often misinterpreted through the use of dichotomies and responder analysis.
2) An example simulation is provided showing how dichotomizing a continuous outcome measure and conducting responder analysis on a single clinical trial can misleadingly suggest some patients respond to treatment while others do not.
3) In reality, the simulation shows that all patients may experience the same proportional benefit from treatment, but dichotomizing the data obscures this and encourages unfounded conclusions about personalized medicine.

Clinical trials are about comparability not generalisability V2.pptx

It is a fundamental but common mistake to regard clinical trials as being a form of representative inference. The key issue is comparability. Experiments do not involve typical material. In clinical trials; it is concurrent control that is key and randomisation is a device for calculating standard errors appropriately that should reflect the design.
Generalisation beyond the clinical trial always involves theory.

Does clinical research help me take care of my patient?

A presentation by Derek Angus at the 2017 meeting of the Scandinavian Society of Anaestesiology and Intensive Care Medicine.
All available content from SSAI2017: https://scanfoam.org/ssai2017/
Delivered in collaboration between scanFOAM, SSAI & SFAI.

Clinical prediction models:development, validation and beyond

This document appears to be a slide deck on the topic of clinical prediction models. It discusses:
- The differences between explanatory, predictive, and descriptive models.
- Challenges with predictive models like overfitting and the need for shrinkage methods.
- Sample size criteria like events per variable (EPV) and challenges validating models with low EPV.
- Methods for validating predictive performance like apparent, internal, and external validation and quantifying optimism.
- Additional validation strategies like bootstrapping and the importance of assessing calibration.

8 screening.pptxscreening.pptxscreening.

This document discusses screening tests and their evaluation. It defines screening as applying a test to asymptomatic individuals to identify those at high risk of disease. Key criteria for diseases suitable for screening include being a major health problem, having a recognizable pre-symptomatic stage, and having effective early treatment. Important features of screening tests are that they are reliable, sensitive, specific, acceptable and inexpensive. Sensitivity measures the test's ability to correctly identify those with disease, while specificity measures its ability to correctly identify those without disease. Predictive values indicate the likelihood that individuals with positive or negative test results truly have or do not have the disease.

To infinity and beyond

The document discusses lessons from small experiments and the Rothamsted School approach to experimental design and analysis. It provides three key lessons:
1) Variances matter - if you cannot estimate variances precisely, you do not know how to interpret your results or make inferences. The Rothamsted approach matches the analysis to the experimental design to properly account for variances.
2) Experimental designs should eliminate sources of variation that can be controlled, like blocking by centers. This allows the analysis to focus on remaining uncontrolled variations.
3) Lord's paradox arises because some analyses, like comparing change scores, do not adjust for important baseline covariates, while other analyses do adjust and find significant effects. Proper analysis depends on

Understanding randomisation

Randomisation balances both known and unknown factors that influence outcomes between treatment groups on average. Conditioning the analysis on available baseline covariates further reduces uncertainty by accounting for imbalances. While there are many potential covariates, their combined influence is bounded; randomisation thus remains important for accounting for unknown factors. Correct analysis requires matching the randomisation design to the analysis to make valid statistical inferences.

Screening in biomedical sciences

This presentation has been prepared to highlight the most important points about screening.
It builds on previous -even little-knowledge about screening in biomedical sciences.

Testing of hypothesis and Goodness of fit

Testing of Hypothesis and Goodness of Fit
This document discusses hypothesis testing and goodness of fit. It defines hypothesis testing as a procedure to determine if sample data agrees with a hypothesized population characteristic. The key steps are stating the null and alternative hypotheses, selecting a significance level, determining the test distribution, defining rejection regions, performing the statistical test, and drawing a conclusion. Common hypothesis tests discussed include the Student's t-test and chi-square test of goodness of fit.

Depersonalising medicine

1) The document discusses the overhyping of pharmacogenetics and personalized medicine, arguing that much of the claimed benefits are based on weak evidence.
2) It notes that clinical trials often fail to properly distinguish between genetic and non-genetic sources of variability in treatment response.
3) The author argues that without better understanding of variability through improved trial design and analysis, it is impossible to know if and how much variation is truly genetic in nature and able to be addressed through personalized treatment strategies.

Sampling and Sample Size

The document discusses key concepts related to sampling and sample size, including:
- The difference between a population and a sample, with a sample being a subset of the population.
- Factors that influence sample representativeness, such as sampling procedure, sample size, and participation rate.
- The importance of defining the target population, sampling frame, sampling method, and determining an appropriate sample size.
- The two main types of sampling techniques - probability sampling and non-probability sampling. Probability sampling allows results to be generalized while non-probability sampling does not.
- Formulas for calculating sample sizes needed for estimating population means, comparing two independent samples, and estimating proportions.
- Examples

Surveillance and screening-cp.pptx

Surveillance involves the systematic ongoing collection, analysis, and interpretation of health data to monitor disease frequency and spread. Surveillance data can be used to estimate disease burden, understand disease natural history, detect epidemics, monitor control measures, and inform public health planning. Effective surveillance requires establishing objectives, case definitions, data sources, and dissemination mechanisms, as well as evaluating the system. Screening tests asymptomatic populations to detect disease early. Key principles of screening include choosing appropriate diseases, tests, treatments, and considering costs. Test performance is measured by sensitivity, specificity, and predictive values, which depend on disease prevalence and help interpret individual test results.

The challenge of small data

The challenge of small data

In search of the lost loss function

In search of the lost loss function

What is your question

What is your question

What is your question

What is your question

Clinical trials: quo vadis in the age of covid?

Clinical trials: quo vadis in the age of covid?

To infinity and beyond v2

To infinity and beyond v2

Minimally important differences

Minimally important differences

Minimally important differences v2

Minimally important differences v2

Numbers needed to mislead

Numbers needed to mislead

Clinical trials are about comparability not generalisability V2.pptx

Clinical trials are about comparability not generalisability V2.pptx

Does clinical research help me take care of my patient?

Does clinical research help me take care of my patient?

Clinical prediction models:development, validation and beyond

Clinical prediction models:development, validation and beyond

8 screening.pptxscreening.pptxscreening.

8 screening.pptxscreening.pptxscreening.

To infinity and beyond

To infinity and beyond

Understanding randomisation

Understanding randomisation

Screening in biomedical sciences

Screening in biomedical sciences

Testing of hypothesis and Goodness of fit

Testing of hypothesis and Goodness of fit

Depersonalising medicine

Depersonalising medicine

Sampling and Sample Size

Sampling and Sample Size

Surveillance and screening-cp.pptx

Surveillance and screening-cp.pptx

Has modelling killed randomisation inference frankfurt

Lecture originally given in Frankfurt in 2006 discussing difference between design based and model based approaches to analysis of experiments

Vaccine trials in the age of COVID-19

The response to the COVID-19 crisis by various vaccine developers has been extraordinary, both in terms of speed of response and the delivered efficacy of the vaccines. It has also raised some fascinating issues of design, analysis and interpretation. I shall consider some of these issues, taking as my example, five vaccines: Pfizer/BioNTech, AstraZeneca/Oxford, Moderna, Novavax, and J&J Janssen but concentrating mainly on the first two. Among matters covered will be concurrent control, efficient design, issues of measurement raised by two-shot vaccines and implications for roll-out, and the surprising effectiveness of simple analyses. Differences between the five development programmes as they affect statistics will be covered but some essential similarities will also be discussed.

Approximate ANCOVA

Talk given at ISCB 2016 Birmingham
For indications and treatments where their use is possible, n-of-1 trials represent a promising means of investigating potential treatments for rare diseases. Each patient permits repeated comparison of the treatments being investigated and this both increases the number of observations and reduces their variability compared to conventional parallel group trials.
However, depending on whether the framework for analysis used is randomisation-based or model- based produces puzzling difference in inferences. This can easily be shown by starting on the one hand with the randomisation philosophy associated with the Rothamsted school of inference and building up the analysis through the block + treatment structure approach associated with John Nelder’s theory of general balance (as implemented in GenStat®) or starting on the other hand with a plausible variance component approach through a mixed model. However, it can be shown that these differences are related not so much to modelling approach per se but to the questions one attempts to answer: ranging from testing whether there was a difference between treatments in the patients studied, to predicting the true difference for a future patient, via making inferences about the effect in the average patient.
This in turn yields interesting insight into the long-run debate over the use of fixed or random effect meta-analysis.
Some practical issues of analysis will also be covered in R and SAS®, in which languages some functions and macros to facilitate analysis have been written. It is concluded that n-of-1 hold great promise in investigating chronic rare diseases but that careful consideration of matters of purpose, design and analysis is necessary to make best use of them.
Acknowledgement
This work is partly supported by the European Union’s 7th Framework Programme for research, technological development and demonstration under grant agreement no. 602552. “IDEAL”

A century of t tests

The story of Student's t-test including the history of the trial at Kalamazoo that provided the data that WS Gosset used to illustrate his test.

Is ignorance bliss

It is argued that when it comes to nuisance parameters an assumption of ignorance is harmful. On the other hand this raises problems as to how far one should go in searching for further data when combining evidence.

What should we expect from reproducibiliry

Is there really a reproducibility crisis and if so are P-values to blame? Choose any statistic you like and carry out two identical independent studies and report this statistic for each. In advance of collecting any data, you ought to expect that it is just as likely that statistic 1 will be smaller than statistic 2 as vice versa. Once you have seen statistic 1, things are not so simple but if they are not so simple, it is that you have other information in some form. However, it is at least instructive that you need to be careful in jumping to conclusions about what to expect from reproducibility. Furthermore, the forecasts of good Bayesians ought to obey a Martingale property. On average you should be in the future where you are now but, of course, your inferential random walk may lead to some peregrination before it homes in on “the truth”. But you certainly can’t generally expect that a probability will get smaller as you continue. P-values, like other statistics are a position not a movement. Although often claimed, there is no such things as a trend towards significance.
Using these and other philosophical considerations I shall try and establish what it is we want from reproducibility. I shall conclude that we statisticians should probably be paying more attention to checking that standard errors are being calculated appropriately and rather less to inferential framework.

De Finetti meets Popper

Views of the role of hypothesis falsification in statistical testing do not divide as cleanly between frequentist and Bayesian views as is commonly supposed. This can be shown by considering the two major variants of the Bayesian approach to statistical inference and the two major variants of the frequentist one.
A good case can be made that the Bayesian, de Finetti, just like Popper, was a falsificationist. A thumbnail view, which is not just a caricature, of de Finetti’s theory of learning, is that your subjective probabilities are modified through experience by noticing which of your predictions are wrong, striking out the sequences that involved them and renormalising.
On the other hand, in the formal frequentist Neyman-Pearson approach to hypothesis testing, you can, if you wish, shift conventional null and alternative hypotheses, making the latter the strawman and by ‘disproving’ it, assert the former.
The frequentist, Fisher, however, at least in his approach to testing of hypotheses, seems to have taken a strong view that the null hypothesis was quite different from any other and there was a strong asymmetry on inferences that followed from the application of significance tests.
Finally, to complete a quartet, the Bayesian geophysicist Jeffreys, inspired by Broad, specifically developed his approach to significance testing in order to be able to ‘prove’ scientific laws.
By considering the controversial case of equivalence testing in clinical trials, where the object is to prove that ‘treatments’ do not differ from each other, I shall show that there are fundamental differences between ‘proving’ and falsifying a hypothesis and that this distinction does not disappear by adopting a Bayesian philosophy. I conclude that falsificationism is important for Bayesians also, although it is an open question as to whether it is enough for frequentists.

In Search of Lost Infinities: What is the “n” in big data?

In designing complex experiments, agricultural scientists, with the help of their statistician collaborators, soon came to realise that variation at different levels had very different consequences for estimating different treatment effects, depending on how the treatments were mapped onto the underlying block structure. This was a key feature of the Rothamsted approach to design and analysis and a strong thread running through the work of Fisher, Yates and Nelder, being expressed in topics such as split-pot designs, recovering inter-block information and fractional factorials. The null block-structure of an experiment is key to this philosophy of design and analysis. However modern techniques for analysing experiments stress models rather than symmetries and this modelling approach requires much greater care in analysis, with the consequence that you can easily make mistakes and often will.
In this talk I shall underline the obvious, but often unintentionally overlooked, fact that understanding variation at the various levels at which it occurs is crucial to analysis. I shall take three examples, an application of John Nelder’s theory of general balance to Lord’s Paradox, the use of historical data in drug development and a hybrid randomised non-randomised clinical trial, the TARGET study, to show that the data that many, including those promoting a so-called causal revolution, assume to be ‘big’ may actually be rather ‘small’. The consequence is that there is a danger that the size of standard errors will be underestimated or even that the appropriate regression coefficients for adjusting for confounding may not be identified correctly.
I conclude that an old but powerful experimental design approach holds important lessons for observational data about limitations in interpretation that mere numbers cannot overcome. Small may be beautiful, after all.

Seventy years of RCTs

This year marks the 70th anniversary of the Medical Research Council randomised clinical trial (RCT) of streptomycin in tuberculosis led by Bradford Hill. This is widely regarded as a landmark in clinical research. Despite its widespread use in drug regulation and in clinical research more widely and its high standing with the evidence based medicine movement, the RCT continues to attracts criticism. I show that many of these criticisms are traceable to failure to understand two key concepts in statistics: probabilistic inference and design efficiency. To these methodological misunderstandings can be added the practical one of failing to appreciate that entry into clinical trials is not simultaneous but sequential.
I conclude that although randomisation should not be used as an excuse for ignoring prognostic variables, it is valuable and that many standard criticisms of RCTs are invalid.

The Rothamsted school meets Lord's paradox

Lords ‘paradox’ is a notoriously difficult puzzle that is guaranteed to provoke discussion, dissent and disagreement. Two statisticians analyse some observational data and come to radically different conclusions, each of which has acquired defenders over the years since Lord first proposed his puzzle in 1967. It features in the recent Book of Why by Pearl and McKenzie, who use it to demonstrate the power of Pearl’s causal calculus, obtaining a solution they claim is unambiguously right. They also claim that statisticians have failed to get to grips with causal questions for well over a century, in fact ever since Karl Pearson developed Galton’s idea of correlation and warned the scientific world that correlation is not causation.
However, only two years before Lord published his paradox John Nelder outlined a powerful causal calculus for analyzing designed experiments based on a careful distinction between block and treatment structure. This represents an important advance in formalizing the approach to analysing complex experiments that started with Fisher 100 years ago, when he proposed splitting variability using the square of the standard deviation, which he called the variance, continued with Yates and has been developed since the 1960s by Rosemary Bailey, amongst others. This tradition might be referred to as The Rothamsted School. It is fully implemented in Genstat® but, as far as I am aware, not in any other package.
With the help of Genstat®, I demonstrate how the Rothamsted School would approach Lord’s paradox and come to a solution that is not the same as the one reached by Pearl and McKenzie, although given certain strong but untestable assumptions it would reduce to it. I conclude that the statistical tradition may have more to offer in this respect than has been supposed.

The revenge of RA Fisher

Presidents' invited lecture ISCB Vigo 2017
Discusses various issues to do with how randomised clinical trials should be analysed. See also https://errorstatistics.com/2017/07/01/s-senn-fishing-for-fakes-with-fisher-guest-post/

The story of MTA/02

History of how and why a complex cross-over trial was designed to prove the equivalence of two formulations of a beta-agonist and what the eventual results were. Presented at the Newton Institute 28 July 2008. Warning: following the important paper by Kenward & Roger Biostatistics, 2010, I no longer think the random effects analysis is appropriate, although, in fact the results are pretty much the same as for the fixed effects analysis.

Confounding, politics, frustration and knavish tricks

2008 Bradford Hill Lecture. An explanation of some problems with the propensity score and why its supposed superiority to ANCOVA is doubtful

And thereby hangs a tail

The document summarizes key differences between the work of William Sealy Gosset ("Student") and Ronald A. Fisher on the development of statistical hypothesis testing using P-values. While Student is often credited with developing the t-test, he did not establish it in its modern form or interpret it using significance tests. Fisher reformulated the t-statistic and stressed its alternative interpretation for significance testing, though he did not originate the concept of P-values. There remains ongoing debate around the Bayesian and frequentist interpretations of P-values and their role in null hypothesis significance testing.

The revenge of RA Fisher

1) The document discusses a study by Carlisle that analyzed distributions of means from randomized controlled trials. It found an excess of trials that appeared too well balanced, potentially due to how trials are actually randomized and analyzed.
2) Three explanations are provided for Carlisle's findings: covariates in trials may not be independent; trials are often randomized using blocking or other techniques that balance covariates but the analysis does not account for this; and the published trials analyzed are a selected subset due to publication bias.
3) The key points are that how trials are randomized and analyzed can impact balance in unexpected ways, covariates are often correlated not independent, and publication bias further selects the trials seen, limiting what can be concluded from

P value wars

The document summarizes recent criticisms of the use of p-values and significance testing in research. It discusses how influential researchers like Ioannidis and replicability studies have questioned whether most published findings are true. It also summarizes the American Statistical Association's statement on p-values, which advises that scientific conclusions should not be based solely on p-value thresholds. The document then provides background on the history of p-values, noting that the debate is not just between frequentist and Bayesian approaches, but between different Bayesian views. It concludes by advising that p<0.05 is a weak standard, replication is important, and researchers should follow the ASA's recommendations to avoid overreliance on p-values alone.

Thinking statistically v3

This document provides an overview of four statistical "paradoxes": 1) Covariate measurement error in randomized clinical trials, 2) Meta-analysis of sequential trials, 3) Dawid's selection paradox, and 4) Publication bias in the medical literature. For each paradox, brief explanations are given of the statistical issues involved and how they can be understood. Key points include that meta-analysis of sequential trials can account for early stopping by weighting trials by information amount, Dawid's selection paradox shows that Bayesian analysis is unaffected by selection of optimal outcomes, and publication bias may appear as a difference in acceptance rates between positive and negative results but could also be explained by differences in paper quality and targeting of journals. Graphs and heuristics

Seven myths of randomisation

I describe some common misunderstandings regarding randomisation
(Version given at Sheffield University)

Has modelling killed randomisation inference frankfurt

Has modelling killed randomisation inference frankfurt

Vaccine trials in the age of COVID-19

Vaccine trials in the age of COVID-19

Approximate ANCOVA

Approximate ANCOVA

A century of t tests

A century of t tests

Is ignorance bliss

Is ignorance bliss

What should we expect from reproducibiliry

What should we expect from reproducibiliry

De Finetti meets Popper

De Finetti meets Popper

In Search of Lost Infinities: What is the “n” in big data?

In Search of Lost Infinities: What is the “n” in big data?

Seventy years of RCTs

Seventy years of RCTs

The Rothamsted school meets Lord's paradox

The Rothamsted school meets Lord's paradox

The revenge of RA Fisher

The revenge of RA Fisher

The story of MTA/02

The story of MTA/02

Confounding, politics, frustration and knavish tricks

Confounding, politics, frustration and knavish tricks

And thereby hangs a tail

And thereby hangs a tail

The revenge of RA Fisher

The revenge of RA Fisher

P value wars

P value wars

Thinking statistically v3

Thinking statistically v3

Seven myths of randomisation

Seven myths of randomisation

Get Covid Testing at Fit to Fly PCR Test

A Fit-to-Fly PCR Test is a crucial service for travelers needing to meet the entry requirements of various countries or airlines. This test involves a polymerase chain reaction (PCR) test for COVID-19, which is considered the gold standard for detecting active infections. At our travel clinic in Leeds, we offer fast and reliable Fit to Fly PCR testing, providing you with an official certificate verifying your negative COVID-19 status. Our process is designed for convenience and accuracy, with quick turnaround times to ensure you receive your results and certificate in time for your departure. Trust our professional and experienced medical team to help you travel safely and compliantly, giving you peace of mind for your journey.

R3 Stem Cell Therapy: A New Hope for Women with Ovarian Failure

Discover the groundbreaking advancements in stem cell therapy by R3 Stem Cell, offering new hope for women with ovarian failure. This innovative treatment aims to restore ovarian function, improve fertility, and enhance overall well-being, revolutionizing reproductive health for women worldwide.

Michigan HealthTech Market Map 2024 with Policy Makers, Academic Innovation C...

Michigan HealthTech Market Map 2024. Includes 7 categories: Policy Makers, Academic Innovation Centers, Digital Health Providers, Healthcare Providers, Payers / Insurance, Device Companies, Life Science Companies, Innovation Accelerators. Developed by the Michigan-Israel Business Accelerator

COLOUR CODING IN THE PERIOPERATIVE NURSING PRACTICE.

COLOUR CODING IN THE PERIOPERATIVE ENVIRONMENT HAS COME TO STAY ,SOME SENCE OF HUMOUR WILL BE APPRECIATED AT THE RIGHT TIME BY THE PATIENT AND OTHER SURGICAL TEAM MEMBERS.

一比一原版(EUR毕业证)鹿特丹伊拉斯姆斯大学毕业证如何办理

EUR毕业证offer【微信95270640】《鹿特丹伊拉斯姆斯大学毕业证书原版制作EUR成绩单》【Q微信95270640】《仿制EUR毕业证成绩单鹿特丹伊拉斯姆斯大学学位证书pdf电子图》，A.为什么留学生需要操作留信认证?
留信认证全称全国留学生信息服务网认证,隶属于北京中科院。①留信认证门槛条件更低,费用更美丽,并且包过,完单周期短,效率高②留信认证虽然不能去国企,但是一般的公司都没有问题,因为国内很多公司连基本的留学生学历认证都不了解。这对于留学生来说,这就比自己光拿一个证书更有说服力,因为留学学历可以在留信网站上进行查询!
B.为什么我们提供的毕业证成绩单具有使用价值？
查询留服认证是国内鉴别留学生海外学历的唯一途径，但认证只是个体行为，不是所有留学生都操作，所以没有办理认证的留学生的学历在国内也是查询不到的，他们也仅仅只有一张文凭。所以这时候我们提供的和学校颁发的一模一样的毕业证成绩单，就有了使用价值。
实体公司，专业可靠，办理加拿大毕业证|办美国成绩单|做德国文凭学历认证|办新西兰学位证，办澳洲文凭认证，办留信网认证（网上可查，实体公司，专业可靠）铸就十年品质！信誉！实体公司！
办理国外鹿特丹伊拉斯姆斯大学毕业证书 #成绩单改成绩 #教育部学历学位认证 #毕业证认证 #留服认证 #使馆认证（留学回国人员证明） #（证）等
真实教育部认证教育部存档中国教育部留学服务中心认证（即教育部留服认证）网站100%可查.
真实使馆认证（即留学人员回国证明）使馆存档可通过大使馆查询确认.
留信网认证国家专业人才认证中心颁发入库证书留信网永久存档可查.
鹿特丹伊拉斯姆斯大学鹿特丹伊拉斯姆斯大学毕业证假文凭毕业证 #成绩单等全套材料从防伪到印刷从水印到钢印烫金跟学校原版100%相同.
国际留学归国服务中心：实体公司注册经营行业标杆精益求精！
国外毕业证学位证成绩单办理流程：
1客户提供办理鹿特丹伊拉斯姆斯大学鹿特丹伊拉斯姆斯大学毕业证假文凭信息：姓名生日专业学位毕业时间等（如信息不确定可以咨询顾问：我们有专业老师帮你查询）；
2开始安排制作鹿特丹伊拉斯姆斯大学毕业证成绩单电子图；
3鹿特丹伊拉斯姆斯大学毕业证成绩单电子版做好以后发送给您确认；
4鹿特丹伊拉斯姆斯大学毕业证成绩单电子版您确认信息无误之后安排制作成品；
5鹿特丹伊拉斯姆斯大学成品做好拍照或者视频给您确认；
6快递给客户（国内顺丰国外DHLUPS等快递邮寄鹿特丹伊拉斯姆斯大学鹿特丹伊拉斯姆斯大学毕业证假文凭）。地跟着父亲昼夜兼程第二天凌晨就辗转到了父亲的城哇父亲的城真的好大好美哟走出广州火车东站的那一刻山娃感受到了一种从未有过的激动和震撼太美了可爱的广州父亲的城山娃惊喜得几乎叫出声来山娃觉得父亲太伟大了居然能单匹马地跑到这么远这么大这么美的地方赚大钱高楼大厦鳞次栉比大街小巷人潮涌动山娃一路张望一路惊叹他发现城里的桥居然层层叠叠扭来扭去桥下没水却有着水一般的车水马龙山娃惊诧于城里的公交车那么大那么美不的

National Rural Health Mission(NRHM).pptx

This document describes the NRHM Scheme, its goals, objectives, components, and new initiatives.

Hypertension and it's role of physiotherapy in it.

This particular slides consist of- what is hypertension,what are it's causes and it's effect on body, risk factors, symptoms,complications, diagnosis and role of physiotherapy in it.
This slide is very helpful for physiotherapy students and also for other medical and healthcare students.
Here is summary of hypertension -
Hypertension, also known as high blood pressure, is a serious medical condition that occurs when blood pressure in the body's arteries is consistently too high. Blood pressure is the force of blood pushing against the walls of blood vessels as the heart pumps it. Hypertension can increase the risk of heart disease, brain disease, kidney disease, and premature death.

Top massage center in ajman chandrima Spa

We are one of the top Massage Spa Ajman Our highly skilled, experienced, and certified massage therapists from different corners of the world are committed to serving you with a soothing and relaxing experience. Luxuriate yourself at our spas in Sharjah and Ajman, which are indeed enriched with an ambiance of relaxation and tranquility. We could confidently claim that we are one of the most affordable Spa Ajman and Sharjah as well, where you can book the massage session of your choice for just 99 AED at any time as we are open 24 hours a day, 7 days a week.
Visit : https://massagespaajman.com/
Call : 052 987 1315

nurs fpx 4050 assessment 4 final care coordination plan.pdf

nurs fpx 4050 assessment 4 final care coordination plan.pdf

Sectional dentures for microstomia patients.pptx

Microstomia, characterized by an abnormally small oral aperture, presents significant challenges in prosthodontic treatment, including limited access for examination, difficulties in impression making, and challenges with prosthesis insertion and removal. To manage these issues, customized impression techniques using sectional trays and elastomeric materials are employed. Prostheses may be designed in segments or with flexible materials to facilitate handling. Minimally invasive procedures and the use of digital technologies can enhance patient comfort. Education and training for patients on prosthesis care and maintenance are crucial for compliance. Regular follow-up and a multidisciplinary approach, involving collaboration with other specialists, ensure comprehensive care and improved quality of life for microstomia patients.

The Importance of Black Women Understanding the Chemicals in Their Personal C...

Certain chemicals, such as phthalates and parabens, can disrupt the body's hormones and have significant effects on health. According to data, hormone-related health issues such as uterine fibroids, infertility, early puberty and more aggressive forms of breast and endometrial cancers disproportionately affect Black women. Our guest speaker, Jasmine A. McDonald, PhD, an Assistant Professor in the Department of Epidemiology at Columbia University in New York City, discusses the scientific reasons why Black women should pay attention to specific chemicals in their personal care products, like hair care, and ways to minimize their exposure.

Pediatric Emergency Care for Children | Apollo Hospital

Pediatric Emergency Care for Children | Apollo HospitalApollo 24/7 Adult & Paediatric Emergency Services

At Apollo Hospital, Lucknow, U.P., we provide specialized care for children experiencing dehydration and other symptoms. We also offer NICU & PICU Ambulance Facility Services. Consult our expert today for the best pediatric emergency care.
For More Details:
Map: https://cutt.ly/BwCeflYo
Name: Apollo Hospital
Address: Singar Nagar, LDA Colony, Lucknow, Uttar Pradesh 226012
Phone: 08429021957
Opening Hours: 24X7Surface guided deep inspiration breath hold (SG-DIBH) in ultra-hypofractionat...

Cynthia Aristei
Professor
University of Perugia and Perugia General Hospital
Italy

Professional Secrecy: Forensic Medicine Lecture

Professional Secrecy: Forensic Medicine Lecture , Medical jurisprudence

Bath patient Fundamental of Nursing.pptx

This is presentation about how to bath a patient.
It is a fundamental of nursing.

VEDANTA AIR AMBULANCE SERVICES IN REWA AT A COST-EFFECTIVE PRICE.pdf

Air Ambulance Services In Rewa works in close coordination with ground-based emergency services, including local Emergency Medical Services, fire departments, and law enforcement agencies.
More@: https://tinyurl.com/2shrryhx
More@: https://tinyurl.com/5n8h3wp8

HUMAN BRAIN.pptx.PRIYA BHOJWANI@GAMIL.COM

BY PRIYA BHOJWANI

Sexual Disorders.gender identity disorderspptx

Gender identity disorder, paraphilias , sexual dysfunction

English Drug and Alcohol Commissioners June 2024.pptx

Presentation made by Mat Southwell to the Harm Reduction Working Group of the English Drug and Alcohol Commissioners. Discuss stimulants, OAMT, NSP coverage and community-led approach to DCRs. Focussing on active drug user perspectives and interests

Get Covid Testing at Fit to Fly PCR Test

Get Covid Testing at Fit to Fly PCR Test

R3 Stem Cell Therapy: A New Hope for Women with Ovarian Failure

R3 Stem Cell Therapy: A New Hope for Women with Ovarian Failure

Michigan HealthTech Market Map 2024 with Policy Makers, Academic Innovation C...

Michigan HealthTech Market Map 2024 with Policy Makers, Academic Innovation C...

COLOUR CODING IN THE PERIOPERATIVE NURSING PRACTICE.

COLOUR CODING IN THE PERIOPERATIVE NURSING PRACTICE.

一比一原版(EUR毕业证)鹿特丹伊拉斯姆斯大学毕业证如何办理

一比一原版(EUR毕业证)鹿特丹伊拉斯姆斯大学毕业证如何办理

DELIRIUM BY DR JAGMOHAN PRAJAPATI.......

DELIRIUM BY DR JAGMOHAN PRAJAPATI.......

National Rural Health Mission(NRHM).pptx

National Rural Health Mission(NRHM).pptx

Hypertension and it's role of physiotherapy in it.

Hypertension and it's role of physiotherapy in it.

Top massage center in ajman chandrima Spa

Top massage center in ajman chandrima Spa

nurs fpx 4050 assessment 4 final care coordination plan.pdf

nurs fpx 4050 assessment 4 final care coordination plan.pdf

Sectional dentures for microstomia patients.pptx

Sectional dentures for microstomia patients.pptx

The Importance of Black Women Understanding the Chemicals in Their Personal C...

The Importance of Black Women Understanding the Chemicals in Their Personal C...

Pediatric Emergency Care for Children | Apollo Hospital

Pediatric Emergency Care for Children | Apollo Hospital

Surface guided deep inspiration breath hold (SG-DIBH) in ultra-hypofractionat...

Surface guided deep inspiration breath hold (SG-DIBH) in ultra-hypofractionat...

Professional Secrecy: Forensic Medicine Lecture

Professional Secrecy: Forensic Medicine Lecture

Bath patient Fundamental of Nursing.pptx

Bath patient Fundamental of Nursing.pptx

VEDANTA AIR AMBULANCE SERVICES IN REWA AT A COST-EFFECTIVE PRICE.pdf

VEDANTA AIR AMBULANCE SERVICES IN REWA AT A COST-EFFECTIVE PRICE.pdf

HUMAN BRAIN.pptx.PRIYA BHOJWANI@GAMIL.COM

HUMAN BRAIN.pptx.PRIYA BHOJWANI@GAMIL.COM

Sexual Disorders.gender identity disorderspptx

Sexual Disorders.gender identity disorderspptx

English Drug and Alcohol Commissioners June 2024.pptx

English Drug and Alcohol Commissioners June 2024.pptx

- 1. Clinical Trials are not Enough Stephen Senn (c) Stephen Senn 1
- 2. Acknowledgements (c) Stephen Senn 2 Acknowledgements This work is partly supported by the European Union’s 7th Framework Programme for research, technological development and demonstration under grant agreement no. 602552. “IDEAL”
- 3. Basic Thesis • Clinical trials are experiments • At every stage of drug development, even phase III, they are unrepresentative of ‘real world’ clinical practice • The key to using their results to inform ‘real world’ decisions is not to make the trial more representative • The key is to use appropriate scales for analysis and transfer the results into practical real-world decision making • This will require – A different attitude – More modelling – Extensive use of auxiliary real world data (c) Stephen Senn 3
- 4. An Example • Dan Moerman’s analysis more than 30 years ago of Smith-Kline-French’s development of Tagamet® – Tagamet® (cimetidine) was the best selling drug of its day • 31 trials in 1692 patients – in 17 countries – Duodenal, Gastric, Mixed – 13 significant according to Moerman • Used Chi-square test with Yates’s correction (c) Stephen Senn 4
- 5. (c) Stephen Senn 5 on the risk- difference scale and there is significant evidence of heterogeneity
- 6. What the EBM movement used to conclude from this sort of thing • The treatment effect varies according to the type of patient • We want RCTS that automatically deliver the right decision • We can’t rely on the results from RCTs which involve artificially selected patients • We need large simple trials in representative populations • They need to be reported using numbers needed to treat (NNTs) (c) Stephen Senn 6
- 7. Problems • At best you can hope to recommend treatments that are beneficial on average • But in practice you can never guarantee that trials are representative of practice anyway • You lose the opportunity to study issues more deeply • NNTs are a terrible scale for doing analysis (c) Stephen Senn 7
- 8. (c) Stephen Senn 8 Here the analysis is on the log-odds ratio scale and heterogeneity is much reduced and not even ‘significant’.
- 9. 9 “If you need statistics to prove it I don’t believe it” You can’t prove it with statistics but everybody believes it Thanks to Pat Ballew’s blog site (c) Stephen Senn 9 That is to say, we see non-random variation too easily. This example does NOT give strong evidence treatment by trial interaction
- 10. Lessons • If not carefully, studied random variation can be underestimated • Differences from trial to trial in true effect may be less than one thinks • Finding a good scale is important • BUT The additive scale is not necessarily the relevant one (c) Stephen Senn 10
- 11. What not to do • The solution is not to attempt to make trials more representative • The solution is to measure appropriately and translate appropriately • This requires the following – Good scales – Good analysis – Good modelling – Good supplementary real world data (c) Stephen Senn 11
- 12. (c) Stephen Senn 12 Chasing sub- groups leads nowhere
- 13. Solution? (c) Stephen Senn 14 “a possible resolution is to use the additive measure at the point of analysis and transform to the relevant scale at the point of implementation. This transformation at the point of medical decision-making will require auxiliary information on the level of background risk of the patient.” Senn, Statistics in Medicine, 2004
- 14. How we already use modelling, data and additive scales • Interspecies scaling • Bioequivalence – log relative bioavailability is additive but difference in absolute bioavailability is not • Dose proportionality • Use of additive scales in phase III – Log hazard – Log-odds ratio (c) Stephen Senn 15
- 15. (c) Stephen Senn 17Controlled Clinical Trials, 1989
- 16. Implications of the Lubsen-Tijssen Model • We need to study treatment benefit on disaggregated (of harm) additive scale • We will need real world data on harms • We will need real world data on background risk • We will need models • We will need cooperation between – Medics and statisticians working on clinical trials – Statisticians, epidemiologists, health economists, medics and others working in real world data (c) Stephen Senn 18
- 17. Example of Atrial Fibrillation • Such patients are at higher risk of stroke • Meta-analysis (reproduced in Hart et al 2007)concluded that warfarin has a beneficial protective effect • But there is a risk of intracranial bleeding • Who should get warfarin? (c) Stephen Senn 19
- 18. (c) Stephen Senn 20
- 19. (c) Stephen Senn 21
- 20. So you have atrial fibrillation • Should you take warfarin? • What else do you need to know? – The difference in risk taking warfarin or not – The rate of side effects – The consequences of side-effects • These cannot be answered (alone) by analysis of RCTs with pre-specified efficacy measures on the additive scale • The RCTs has to be translated and supplemented by real world data (c) Stephen Senn 22
- 21. (c) Stephen Senn 23
- 22. (c) Stephen Senn 24 Estimate based on 6 v 3 cases only
- 23. The reimburser’s perspective • What benefit and harm to the population will accrue from recommending warfarin prophylaxis? • How much will it cost? • How can its use be optimised? • Who should get it? (c) Stephen Senn 25
- 24. (c) Stephen Senn 26
- 25. Reimburser’s needs Requirements • A means of separating patients by risk • A means of establishing risk distribution in the population of patients above any threshold chosen • A means of determining expected benefits and costs Solutions • These figures cannot be delivered by clinical trials alone but will require – Cohort studies/case control studies – Health surveys – Economic modelling (c) Stephen Senn 27
- 26. We need to model background risk • Sort of data set we could use is that provided by the UK Clinical Practice Research Data Link CPRD • Could use this to model – A) Predictors of risk of stroke – B) Distribution of risk levels in the population • Former relevant to individuals to make decisions • Latter is relevant to reimbursers (c) Stephen Senn 28
- 27. Is there a Trust Problem? • Yes • Clinical trials provide a “template of trust” whereby regulators can mandate sponsors to provide the proof • Modelling + real world data cannot provide these guarantees • But this is no excuse – Whether or not you model, others will – You need to know as much as possible about your own drugs and where and when to use them (c) Stephen Senn 29
- 28. Finally I leave you with this though (c) Stephen Senn 30 Any damn fool can analyse a clinical trial and frequently does But doing it properly involves skilful analysis, understanding what the results mean requires intelligence insight and experience, and applying the results intelligently needs more of the same plus modelling and real world data And to whom do we look to provide these skills? Statisticians!