Blocked backdoor path. F meets the criteria for traditional confounder, but it is not a counfounder and this is not a confounded study
Causal null: whether having low education increases risk for type II diabetes. We measured mother’s diabetes status, but do not have measures of family income when the individual was growing up or if the mother had any genes that would increase the risk for diabetes. Under the assumptions in the DAG, should we adjust for mother’s diabetes status? Assumptions that if poor during childhood, then poor as adult and poor associated with diabetes and low education. Mother’s diabetes status will be statistically associated with education. They share a common prior cause. Meets criteria for statistical association Conditioning on mother’s diabetes unblocks the blocked backdoor path and induces a spurious statistical association between low education and diabetes. Does not meet criteria for graphical confounder. Basketball player tall or fast.
Argue a high level of inductive reasoning here
1. The unity of all science consistsalone in the method, not its material.Pearson K. The grammar of science. London, Black, 1892.
2. Statistics is the study of uncertainty.Savage LJ. The foundations of statistics. New York, Wiley,1954.
3. The aim of statistics reviewingAccurate and transparent description of the uncertainty inpresented findings.“Statisticians are experts in handling uncertainty”.Lindley DV. The philosophy of statistics. The Statistician 2000;49:293-337.
4. Medical research methodology Ethics concerns RandomizationExperi- Controlled conditionsmental Internal validity Short follow up Sample size restrictionsStudydesignObser- Few ethics concernsvational Bias adjustments External validity Long follow up Few sample size restrictions
5. Statistical aspects – internal validity Internal validity by design (blocking ofExperi- known risk factors and randomization ofmental other) Potential for confounding: noneStudydesign Internal validity by statistical analysisObser- (confounding adjustment for known andvational measured risk factors) Potential for confounding: massive
6. Confounder (or case-mix) adjustmentHow much of the variation in endpoints can be explained by knownfactors, and how much has unknown causes?Variation with unknown origin95%-99% Arthroplasty revision85%-95% EQ-5D, SF3670%-80% Coronary heart disease
7. Risk factors, confounding, and theillusion of statistical control...it is essential to remember that “statistical control” is nothing morethan a highly fallible process filled with judgment calls that often gounnoticed in practice.Christenfeld NJS, Sloan RP, Carrol D, Greenland S.Psychosomatic Medicine 2004;66:868–875
8. Linear regression analysisSimple model Multiple model (or multivariable, but not multivariate)
9. Stepwise regression
10. StatisticsWe calculated odds ratios by logistic regressionanalysis, to estimate the relationship betweenfailure of the osteotomy and possible preoperativerisk factors. We performed multivariate, stepwise(backward) logistic regression and enteredvariables with a p-value of ≤ 0.05 into the model.
11. Unified theory of biasBias can be reduced to or explained by 3 structures1. Reverse causationOutcome precedes exposure measurement or outcome can haveeffect on exposure. Measurement error or Information bias.2. Common causeConfounding by association, confounding by indication.3. Conditioning on common effectsCollider, selection bias, time varying confounding.
12. Covariate selectionAdequate Background KnowledgeConfounder identification must be based on understanding of thecausal structure linking the variables being studied (treatment anddisease).Condition on the minimal set of variables necessary to removeconfounding.Inadequate Background KnowledgeRemove known instrumental variables, colliders, intermediates(variables with post treatment measurement.
13. ConfoundingUnder-adjustmentoccurs when a confounder is not adjusted for.Over-adjustmentcan occur from adjusting instrumental variables, intermediatevariables, colliders, variables caused by outcome.
14. ConfounderCommon cause, i.e., confounderConfounder L distort the effect oftreatment A on disease YAlways adjust for confounders, unlesssmall data set and confounder hasstrong association with treatment andweek association with outcome
16. Intermediate variableAdjusting for intermediate variable I in afixed covariate model will remove the effectof treatment A on disease/outcome YIn a fixed covariate model we do not want toinclude variables influenced by A or Y
18. ColliderAdjusting for a collider can produce biasConditioning on common effect F withoutadjustment of U1 or U2 will induce anassociation between U1 and U2, which willconfound the association between A and Y
19. Collider example
20. Variables associated withtreatment or disease onlyInclusion of variables associated with treatment only can cause biasand imprecisionVariables associated with disease but not treatment (risk factors)can be included in models. They are expected to decrease varianceof treatment effect without increasing biasIncluding variables associated with disease reduces the chance ofmissing important confounders
21. Reality is complicated
24. Any claim coming from an observational study ismost likely to be wrong12 randomised trials have tested 52 observational claims (about theeffects of vitamine B6, B12, C, D, E, beta carotene, hormone replace-ment therapy, folic acid and selenium).“They all confirmed no claims in the direction of the observationalclaim. We repeat that figure: 0 out of 52. To put it in another way,100% of the observational claims failed to replicate. In fact, fiveclaims (9.6%) are statistically significant in the opposite direction tothe observational claim.”Young S, Karr A. Deming, data and observational studies.Significance, September 2011.
25. Medical research methodology Hypothesis generation Pre-specified hypotheses Exploration ConfirmationAcademic analysis freedom Legislation, regulatory guidelines Uncertainty tolerance Uncertainty intolerance Aetiology Study scope Treatment
26. Medical research methodologyExperi- Laboratory Randomized clinicalmental experiments trialsStudydesignObser-vational Epidemiological Patient register studies studies Aetiology Study scope Treatment
27. Statistical aspects - precision Bonferroni correction Protected type-1 error rateExperi- within endpoints for specified endpointsmental Few type-2 error Sample size based on the considerations type-2 error rateStudydesign Multiplicity issues Specified type-1 and -2 errorObser- not addressed uncertaintyvational (confidence intervals) Sample size not based on type-2 error rate No multiplicity consideration for safety endpoints Aetiology Study scope Treatment
28. Drug development Discovery Phase 1 (Phase 0) Phase 2Experi- Phase 3mental Phase 4Studydesign PMS (Phase 5)Obser-vational Aetiology Study scope Treatment
29. Device development Biomechanics in vitro, etc. RandomizedExperi- performancemental trialsStudydesign Safety follow-upObser- in registriesvational Aetiology Study orientation Treatment
30. It is impossible to do clinical research so badly thatit cannot be published“There seems to be no study too fragmented, no hypothesis tootrivial, no literature citation too biased or too egotistical, no designtoo warped, no methodology too bungled, no presentation ofresults too inaccurate, no argument too circular, no conclusionstoo trifling or too unjustified, and no grammar and syntax toooffensive for a paper to end up in print.”Drummond Rennie 1986 (editor of NEJM and JAMA)
31. Arthroplasty registry analysesCrucial issues- Fulfillment of methodological assumptions (Gaussian distr, homogeneity of variance, proportionality, linearity, etc.)- Confounding adjustment (risk factors, causality, linearity, etc.)- Clinical significance and estimation uncertainty (95%CI).Should be avoided- P-value culture- Bonferroni correction- Post-hoc power- Predictions
32. Thank you for your attention!
33. Indicators for statistical review- Randomized trials- Patient registry (safety) studies- Analyses of knees, hips, elbows... (bilateral observations)- Pseudo-replicates (esp. in laboratory experiments)- “No difference” manuscripts- Stepwise regression- ???