IPCW attempts to control for time-dependent confounders as well as baseline covariatesSNM also tries to adjust for time-dependent confounders as well as baseline covariatesSNM uses the counterfactual survival model to estimate counterfactual survival for each value of psi. It then also models the hazard of the treatment process in order to identify the ‘true’ value of psi. The hazard of the treatment process is modelled as a function of baseline and time-dependent covariates, and counterfactual survival for each value of psi. It is assumed that when all covariates are included the treatment process is not related to counterfactual survival – ie it is random. The value of psi that gives counterfactual survival times that allows this assumption to be borne out is the ‘true’ value of psi.So assume that you can fully explain the treatment decision and the survival process using covariates. Assume that counterfactual survival times are the same for all individuals, controlling for covariates. Once have accounted for all time-dependent confounding the association between exposure and survival can be attributed soley to the treatment effect.No unobserved confounders, model (Cox) for hazard of treatment must be correct, and SNM for counterfactual survival must be correct.Treatment group and disease progression are perfect predictors of receiving treatment and thus would be dropped from a logistic regression modelling the treatment process. Without these (particularly disease progression) our models would be biased as it is an important prognostic covariate. Hence need to think of another method of application 2-stage approach. Including the whole dataset would not work as exp group can’t crossover, and crossover can’t happen before disease progression.Particularly prone to bias with high xo% given way we’ve implemented it in the control group after disease progression – high xo leaves little control ‘controls’.
Say why we need a sim study. With real data we don’t know the truth and so can’t assess how well the different methods do. With a sim study we do know the truth, and we can assess the methods against this.
Economic evaluation. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study.
Methods for adjusting survivalestimates in the presence oftreatment crossover: a simulationstudyNicholas Latimer, University of SheffieldCollaborators: Paul Lambert, Keith Abrams, Michael Crowther, AllanWailoo and James Morden
2ContentsWhat is the treatment crossover problemPotential solutionsSimulation studyResultsConclusions
3Treatment switching – the problem In RCTs often patients are allowed to switch from the control treatment to the new intervention after a certain timepoint (eg disease progression) OS estimates will be confounded OS is a key input into the QALY calculation Cost effectiveness results will be inaccurate an ITT analysis is likely to underestimate the treatment benefit Inconsistent and inappropriate treatment recommendations could be made
4Treatment switching – the problem Control Treatment True OS difference PFS PPS Intervention PFS PPS Control Intervention RCT OS difference PFS PPS Survival timeSwitching is likely to result in an underestimate of the treatment effect
What is usually done to adjust? No clear consensus Numerous „naive‟ approaches have been taken in NICE appraisals, eg: Very prone to Take no action at all selection bias Exclude or censor all patients who crossover – crossover isn‟t random Occasionally more complex statistical methods have been used, eg: Rank Preserving Structural Failure Time Models (RPSFTM) Inverse Probability of Censoring Weights (IPCW) And others are available from the literature, eg: Structural Nested Models (SNM)
6What are the consequences? TA 215, Pazopanib for RCC [51% of control switched] ITT: OS HR (vs IFN) = 1.26 ICER = Dominated Censor patients: HR = 0.80 ICER = £71,648 Exclude patients: HR = 0.48 ICER = £26,293 IPCW: HR = 0.80 ICER = £72,274 RPSFTM: HR = 0.63 ICER = £38,925
Potential solutionsRPSFTM (and IPE algorithm) Randomisation-based approach, estimates counterfactual survival timesKey assumption: common treatment effectIPCW Observational-based approach, censors xo patients, weights remaining patientsKey assumptions: “no unmeasured confounders”; must model OS and crossoverSNM Observational version of RPSFTMKey assumptions: “no unmeasured confounders”; must model OS and crossover
Simulation study (1) None of these methods are perfect But we need to know which are likely to produce least bias in different scenariosSimulation study Simulate survival data for two treatment groups, applying crossover that is linked to patient characteristics/prognosis In some scenarios simulate a treatment effect that changes over time In some scenarios simulate a treatment effect that remains constant over time Test different %s of crossover, and different treatment effect sizesHow does the bias and coverage associated with each method compare?
9 Simulation study (2)Methods assessed Naive methods ITT Exclude crossover patients Censor crossover patients Treatment as a time-dependent covariate Complex methods RPSFTM IPE algorithm IPCW SNM
10 Results: common treatment effect RPSFTM / IPE worked very well IPCW and SNM performed ok when crossover % was lower 50.00 ITT IPCW RPSFTM IPE SNM 40.00 AUC mean bias (%) 30.00 20.00 10.00 0.0019 20 27 28 31 32 39 40 43 44 51 -10.00 IPCW and SNM performed poorly when crossover % was very high Naive methods performed poorly (generally led to higher bias than ITT)
11 Results: effect 15% in xo patients RPSFTM / IPE produced higher bias than previous scenarios IPCW and SNM performed similarly to RPSFTM / IPE providing crossover < 90% ITT IPCW RPSFTM IPE SNM 45.00 AUC mean bias (%) 35.00 25.00 15.00 5.00 -5.0017 18 25 26 29 30 37 38 41 42 49 -15.00 IPCW and SNM performed poorly when crossover % was very high Bias not always lower than that associated with the ITT analysis
12 Results: effect 25% in xo patients RPSFTM / IPE produced even more bias IPCW and SNM produce less bias than RPSFTM / IPE providing crossover < 90% 50.00 ITT IPCW RPSFTM IPE SNM AUC mean bias (%) 40.00 30.00 20.00 10.00 0.0023 24 33 34 35 36 45 46 47 48 57 -10.00 -20.00 No „good‟ options when crossover % is very high Often ITT analysis likely to result in least bias (esp. when trt effect low)
13Conclusions Treatment crossover is an important issue that has come to the fore in HE arena Current methods for dealing with treatment crossover are imperfect Our study offers evidence on bias in different scenarios (subject to limitations) RPSFTM / IPE produce low bias when treatment effect is common But are very sensitive to this IPCW / SNM are not affected by changes in treatment effect between groups, but in (relatively) small trial datasets observational methods are volatile Especially when crossover % is very high (leaving low n in control group) Very important to assess trial data, crossover mechanism, treatment effect to determine which method likely to be most appropriate Don’t just pick one!!
A topical analogy... 28 England Spain Vs (control group) (intervention group) Kick-off...
A topical analogy... 29Halftime: England 0 – 3 Spain Spain are statistically significantlybetter than England For ethical reasons, theSpanish team are cloned and theEnglish team are sent home. Sothe second half is Spain VsSpain
A topical analogy... 30Full time: England 2 – 5 Spain Spain still win, but not as comfortably asthey would haveSwitching is likely to result in an underestimate ofthe true supremacy of the intervention
36Conclusions Treatment crossover is an important issue that has come to the fore in HE arena Current methods for dealing with treatment crossover are imperfect Our study offers evidence on bias in different scenarios (subject to limitations) 1. RPSFTM / IPE produce low bias when treatment effect is common 2. When treatment effect ≈ 15% lower in crossover patients RPSFTM / IPE and IPCW / SNM methods produce similar levels of bias (5-10%) [provided suitable data available for obs methods and <90% crossover] 3. When treatment effect ≈ 25% lower IPCW/SNM produce less bias than RPSFTM/IPE [provided suitable data available for obs methods and <90% crossover] [and significant bias likely to remain ITT analysis may offer least bias] 4.Very important to assess trial data, crossover mechanism, treatment effect to determine which method likely to be most appropriate Don’t just pick one!!