All modern medical science publications 60%Source: Pubmed
Randomized clinical trial of streptomycin and tubercolosis (1948) Bradford Hill & MRCSource: Pubmed
Cohort study of smoking and lung cancer (1954) Bradford Hill & Doll Case-control study of smoking and lung cancer (1950) Bradford Hill & Doll Randomized clinical trial of streptomycin and tubercolosis (1948) Bradford Hill & MRCSource: Pubmed
Cohort study of smoking and lung cancer (1954) Bradford Hill & Doll Case-control study of smoking and lung cancer (1950) Bradford Hill & Doll Evidence based medicine Systematic reviews and Randomized clinical meta analyses trial of streptomycin (The Cochrane and tubercolosis (1948) collaboration 1993) Bradford Hill & MRCSource: Pubmed
Evidence levels1. Strong evidence from at least one systematic review of multiple well-designed randomized controlled trials.2. Strong evidence from at least one properly designed randomized controlled trial of appropriate size.3. Evidence from well-designed trials such as pseudo-randomized or non-randomized trials, cohort studies, time series or matched case-controlled studies.4. Evidence from well-designed non-experimental studies from more than one center or research group or from case reports.5. Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees.
Any claim coming from an observationalstudy is most likely to be wrong12 randomised trials have tested 52 observational claims (about theeffects of vitamine B6, B12, C, D, E, beta carotene, hormonereplacement therapy, folic acid and selenium).“They all confirmed no claims in the direction of the observationalclaim. We repeat that figure: 0 out of 52. To put it in another way,100% of the observational claims failed to replicate. In fact, five claims(9.6%) are statistically significant in the opposite direction to theobservational claim.”Stanley Young and Allan Karr, Significance, September 2011
GuidelinesSystematic reviews and meta analyses benefit from astandardized, transparent and accurate reporting of studies.STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA,GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK,CONSORT...
Internal validity Internal validity by design (blocking ofExperi- known risk factors and randomization ofmental unknown) Potential for confounding: noneStudydesign Internal validity by statistical analysisObser- (confounding adjustment for known andvational measured risk factors) Potential for confounding: massive
Confounder (or case-mix) adjustmentHow much of the variation in endpoints can be explained by knownfactors, and how much has unknown causes?Unexplained variation (1-r2)95%-99% Arthroplasty revision85%-95% EQ-5D, SF3650%-70% Coronary heart disease risk
Analysis strategies and publication guidelinesNARAThe Nordic Arthroplasty Register Association (NARA) study groupdecided in September 2009 at a meeting in Lund, Sweden, todevelop guidelines for statistical analysis of arthroplasty qualityregister data.The guidelines were published In April, 2011.Acta Orthopaedica 2011;82:253-267.
The NARA GuidelinesA collaborative effort by1. Independent observations (Pulkkinen & Mäkelä )2. Competing risks (Mehnert & Pedersen)3. Proportional hazard rates (Espehaug & Furnes)4. Rankable revision risk estimates (Ranstam & Kärrholm)The NARA study groupLI Havelin, LB Engesæter AM Fenstad (NO)S Overgaard, A Odgaard (DA)A Eskelinen, V Remes, P Virolainen (FI)G Garellick, M Sundberg, O Robertsson (SE)
The NARA Guidelineshave been developed to– define a NARA consensus view on statistical analysis– describe foreseeable problems and recommend solutions– improve the comparability of reports– facilitate reading, writing and reviewing of reports
The NARA Guidelinesare not intended to– stifle creativity– promote uniformity
NARA GuidelinesStructure1. Review of underlying assumptions2. Consequences of departure from these assumptions3. Verifying that the assumptions are fulfilled4. Possible methodological alternatives5. Practical recommendations
Independent observations Pseudoreplication Two rats are sampled from a population with a mean (μ) of 50 and a standard deviation (σ) of 10, and ten measure- ments of an arbitrary outcome variable are made on each rat. - Biological variability. - Measurement errors.
Independent observationsRipatti S and Palmgren J. Estimation of multivariate frailty models using penalizedpartial likelihood. Biometrics 2000, 56:1016-1022.Schwarzer G, Schumacher M, Maurer TB and Ochsner PE. Statistical analysis offailure times in total joint replacement. J Clin Epidemiol 2001, 54:997-1003.Visuri T, Turula KB, Pulkkinen P and Nevalainen J. Survivorship of hip prosthesis inprimary arthrosis: influence of bilaterality and interoperative time in 45,000 hipprostheses from the Finnish endoprosthesis register. Acta Orthop Scand 2002,73:287-290.Robertsson O and Ranstam J. No bias of ignored bilaterality when analysing therevision risk of knee prostheses: Analysis of a population based sample of 44,590patients with 55,298 knee prostheses from the national Swedish Knee ArthroplastyRegister. BMC Musculoskeletal Disorders 2003, 4:1.Lie SA, Engesaeter LB, Havelin LI, Gjessing HK and Vollset SE. Dependency issuesin survival analyses of 55,782 primary hip replacements from 47,355 patients. StatMed. 2004 Oct 30;23(20):3227-40.
Independent observationsRecommendationsThe inclusion of bilateral observations in analysis of knee- and hipprosthesis survival does not seem to affect the reliability of theresults, but this need not be the case with other types ofprostheses.The number of bilateral observations should always be presented.Sensitivity analyses can be useful when the results robustnessagainst departures from the assumption of independence.
Competing risksKaplan-Meier analysisThe statistical analysis of arthroplasty failure is primarily about thelength of time from primary operation to revision.Not all patients are revised during follow up. The length of followup usually differ, and some patients are withdrawn before end offollow up; these observations are “censored”.With Kaplan-Meier analysis censored observations are included inthe analysis, until their censoring.
Competing risksKaplan-Meier assumptionThe time at which a patient gets a revision is assumed to beindependent of the censoring mechanism. Other events than the onestudied are competing risk events if they alter the risk of beingrevised. Re-revision Revision Primary Death operation Death
Competing risksAlternative method: Cumulative incidenceThe probability that a particular event, such as revision or acompeting risk event, has occurred before a given time.The cumulative incidence function for an event of interest can becalculated by appropriately accounting for the presence of competingrisk events.Censored observations can be included in the analysis.
Competing risksWith competing risk events Kaplan-Meier estimates will overestimatethe real failure risk.
Competing risksRecommendationsWith competing risks the Kaplan-Meier failure function over-estimatesthe revision risk.An alternative method can be to calculate the cumulative incidence ofrevisions. However, from the patients perspective this may be lessrelevant.The presence of competing risks should always be presented andboth the number and types of censored observations should bedescribed.
Competing risks- do not condition on the future;- do not regard individuals at risk after they have died; and- stick to this world.
Proportional hazard ratesAdjustment for case-mix effectsRisk estimates can be adjusted for the confounding effect of animbalance of known and measured risk factors using statisticalmodeling.This is usually achieved using a Cox model.
Proportional hazard ratesCox modelThe Cox model is a regression model for revision times (or morespecifically, hazard rates).The purpose of the model is to explore the simultaneous effectsof different factors on the revision risk.
Proportional hazard ratesThe Cox model is based on theassumption of proportionalhazards (conditional revisionrisks). It is also known as the“proportional hazards model”.The assumption of proportionalhazards is not always fulfilled.
Proportional hazard ratesSchoenfeld residualThe covariate value for the implant that failed minus its expectedvalue.
Proportional hazard ratesConsequencesWhen the effect of one or more of the prognostic factors in a Coxregression model changes over time, the average hazard ratio forsuch a prognostic factor is under- or overestimated.Weighted estimation in Cox regression (Schempers method) is aparsimonious alternative without additional parameters.
Proportional hazard ratesRecommendationsNon-proportional hazards may be an interesting finding in itself.In register studies with large sample sizes, analyses can usuallybe performed by partitioning follow up time, by stratification, or byincluding time dependent covariates.If the average relative risk is of interest, Schempers method canbe an alternative.It should always be evaluated whether the assumption onproportional hazard is fulfilled or not. Testing the Schoenfeldresiduals may be a solution.
Rankable revision risk estimatesRanking is a problematic method for comparisons. If ranking isperformed, the uncertainty in the ranks should be clearly indicated,preferably with confidence intervals.Consequences of misclassification (registration errors) should beevaluated and case-mix effects considered as far as possible.
FinallyRevisions and updatesThe guidelines should be open for revision and updating.They have been developed as a consensus and should evolve asa consensus.Experience and feedback is essential.Forward your suggestions to the NARA study group!
GuidelinesGuidelines are particularly prevalent in clinical trials.CONSORTICH E9 - Statistical Principles for Clinical TrialsEMA Points to Consider, on multiplicity, baseline covariates,superiority and non-inferiority, etc. and similar documents from theFDAEtc.