Methods for adjusting survivalestimates in the presence oftreatment crossover: a simulationstudyNicholas Latimer, Universi...
2ContentsWhat is the treatment crossover problemPotential solutionsSimulation studyResultsConclusions
3Treatment switching – the problem In RCTs often patients are allowed to switch from the control treatment to the new inte...
4Treatment switching – the problem Control Treatment                                              True OS difference      ...
What is usually done to adjust? No clear consensus Numerous „naive‟ approaches have been taken in NICE appraisals, eg:    ...
6What are the consequences? TA 215, Pazopanib for RCC [51% of control switched]    ITT:      OS HR (vs IFN) = 1.26  ICER ...
Potential solutionsRPSFTM (and IPE algorithm)   Randomisation-based approach, estimates counterfactual survival timesKey a...
Simulation study (1)   None of these methods are perfect   But we need to know which are likely to produce least bias in d...
9 Simulation study (2)Methods assessed  Naive methods     ITT     Exclude crossover patients     Censor crossover patients...
10     Results: common treatment effect                         RPSFTM / IPE worked very well                         IPCW...
11 Results: effect 15%                                                      in xo patients                         RPSFTM ...
12 Results: effect 25%                                                             in xo patients                         ...
13Conclusions  Treatment crossover is an important issue that has come to the fore in HE arena  Current methods for dealin...
14Back-up Slides28/09/2012 © The University of Sheffield
15                                Data Generation (1)   Used a two-stage Weibull model to generate underlying survival tim...
16 Data Generation (2)   The survival hazard function was based upon a Weibull (see Bender   et al 2005):                ...
17Data Generation (3)  We applied a reduced treatment effect to crossover patients – this was  equivalent to the baseline ...
18       Data Generation (4)              We then selected parameter values in order that „realistic‟ datasets            ...
19                                 Data Generation (5)We made several assumptions about the „crossover mechanism‟:    1. C...
20                                     ScenariosVariable                                         Value                    ...
21                                     ScenariosVariable                                         Value                    ...
22                                     ScenariosVariable                                         Value                    ...
23                                     ScenariosVariable                                         Value                    ...
24                                     ScenariosVariable                                         Value                    ...
25                                     ScenariosVariable                                         Value                    ...
26                                     ScenariosVariable                                         Value                    ...
27 Estimating AUC1. ‘Survivor function’ approach   Apply treatment effect to survivor function (or hazard function)   esti...
A topical analogy...                        28     England                    Spain                    Vs  (control group)...
A topical analogy...                      29Halftime: England 0 – 3 Spain Spain are statistically significantlybetter tha...
A topical analogy...                               30Full time: England 2 – 5 Spain Spain still win, but not as comfortab...
31Results (4)While the SNM and IPCW methods appeared to produce similar levels of bias, theSNM approach was more volatile:...
32 Results (5)Relationship between bias and treatment crossover %               140               120               100   ...
33 Results (5)Relationship between bias and treatment crossover %               140               120               100   ...
34 Results (5)Relationship between bias and treatment crossover %               140               120               100   ...
35 Results (5)Relationship between bias and treatment crossover %               140               120               100   ...
36Conclusions      Treatment crossover is an important issue that has come to the fore in HE arena      Current methods fo...
Upcoming SlideShare
Loading in...5
×

Economic evaluation. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study.

505

Published on

Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
505
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • IPCW attempts to control for time-dependent confounders as well as baseline covariatesSNM also tries to adjust for time-dependent confounders as well as baseline covariatesSNM uses the counterfactual survival model to estimate counterfactual survival for each value of psi. It then also models the hazard of the treatment process in order to identify the ‘true’ value of psi. The hazard of the treatment process is modelled as a function of baseline and time-dependent covariates, and counterfactual survival for each value of psi. It is assumed that when all covariates are included the treatment process is not related to counterfactual survival – ie it is random. The value of psi that gives counterfactual survival times that allows this assumption to be borne out is the ‘true’ value of psi.So assume that you can fully explain the treatment decision and the survival process using covariates. Assume that counterfactual survival times are the same for all individuals, controlling for covariates. Once have accounted for all time-dependent confounding the association between exposure and survival can be attributed soley to the treatment effect.No unobserved confounders, model (Cox) for hazard of treatment must be correct, and SNM for counterfactual survival must be correct.Treatment group and disease progression are perfect predictors of receiving treatment and thus would be dropped from a logistic regression modelling the treatment process. Without these (particularly disease progression) our models would be biased as it is an important prognostic covariate. Hence need to think of another method of application  2-stage approach. Including the whole dataset would not work as exp group can’t crossover, and crossover can’t happen before disease progression.Particularly prone to bias with high xo% given way we’ve implemented it in the control group after disease progression – high xo leaves little control ‘controls’.
  • Say why we need a sim study. With real data we don’t know the truth and so can’t assess how well the different methods do. With a sim study we do know the truth, and we can assess the methods against this.
  • Transcript of "Economic evaluation. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study."

    1. 1. Methods for adjusting survivalestimates in the presence oftreatment crossover: a simulationstudyNicholas Latimer, University of SheffieldCollaborators: Paul Lambert, Keith Abrams, Michael Crowther, AllanWailoo and James Morden
    2. 2. 2ContentsWhat is the treatment crossover problemPotential solutionsSimulation studyResultsConclusions
    3. 3. 3Treatment switching – the problem In RCTs often patients are allowed to switch from the control treatment to the new intervention after a certain timepoint (eg disease progression) OS estimates will be confounded OS is a key input into the QALY calculation Cost effectiveness results will be inaccurate  an ITT analysis is likely to underestimate the treatment benefit Inconsistent and inappropriate treatment recommendations could be made
    4. 4. 4Treatment switching – the problem Control Treatment True OS difference PFS PPS Intervention PFS PPS Control  Intervention RCT OS difference PFS PPS Survival timeSwitching is likely to result in an underestimate of the treatment effect
    5. 5. What is usually done to adjust? No clear consensus Numerous „naive‟ approaches have been taken in NICE appraisals, eg: Very prone to Take no action at all selection bias Exclude or censor all patients who crossover – crossover isn‟t random Occasionally more complex statistical methods have been used, eg: Rank Preserving Structural Failure Time Models (RPSFTM) Inverse Probability of Censoring Weights (IPCW) And others are available from the literature, eg: Structural Nested Models (SNM)
    6. 6. 6What are the consequences? TA 215, Pazopanib for RCC [51% of control switched] ITT: OS HR (vs IFN) = 1.26  ICER = Dominated Censor patients: HR = 0.80  ICER = £71,648 Exclude patients: HR = 0.48  ICER = £26,293 IPCW: HR = 0.80  ICER = £72,274 RPSFTM: HR = 0.63  ICER = £38,925
    7. 7. Potential solutionsRPSFTM (and IPE algorithm) Randomisation-based approach, estimates counterfactual survival timesKey assumption: common treatment effectIPCW Observational-based approach, censors xo patients, weights remaining patientsKey assumptions: “no unmeasured confounders”; must model OS and crossoverSNM Observational version of RPSFTMKey assumptions: “no unmeasured confounders”; must model OS and crossover
    8. 8. Simulation study (1) None of these methods are perfect But we need to know which are likely to produce least bias in different scenariosSimulation study Simulate survival data for two treatment groups, applying crossover that is linked to patient characteristics/prognosis In some scenarios simulate a treatment effect that changes over time In some scenarios simulate a treatment effect that remains constant over time Test different %s of crossover, and different treatment effect sizesHow does the bias and coverage associated with each method compare?
    9. 9. 9 Simulation study (2)Methods assessed Naive methods ITT Exclude crossover patients Censor crossover patients Treatment as a time-dependent covariate Complex methods RPSFTM IPE algorithm IPCW SNM
    10. 10. 10 Results: common treatment effect RPSFTM / IPE worked very well IPCW and SNM performed ok when crossover % was lower 50.00 ITT IPCW RPSFTM IPE SNM 40.00 AUC mean bias (%) 30.00 20.00 10.00 0.0019 20 27 28 31 32 39 40 43 44 51 -10.00 IPCW and SNM performed poorly when crossover % was very high Naive methods performed poorly (generally led to higher bias than ITT)
    11. 11. 11 Results: effect 15% in xo patients RPSFTM / IPE produced higher bias than previous scenarios IPCW and SNM performed similarly to RPSFTM / IPE providing crossover < 90% ITT IPCW RPSFTM IPE SNM 45.00 AUC mean bias (%) 35.00 25.00 15.00 5.00 -5.0017 18 25 26 29 30 37 38 41 42 49 -15.00 IPCW and SNM performed poorly when crossover % was very high Bias not always lower than that associated with the ITT analysis
    12. 12. 12 Results: effect 25% in xo patients RPSFTM / IPE produced even more bias IPCW and SNM produce less bias than RPSFTM / IPE providing crossover < 90% 50.00 ITT IPCW RPSFTM IPE SNM AUC mean bias (%) 40.00 30.00 20.00 10.00 0.0023 24 33 34 35 36 45 46 47 48 57 -10.00 -20.00 No „good‟ options when crossover % is very high Often ITT analysis likely to result in least bias (esp. when trt effect low)
    13. 13. 13Conclusions Treatment crossover is an important issue that has come to the fore in HE arena Current methods for dealing with treatment crossover are imperfect Our study offers evidence on bias in different scenarios (subject to limitations) RPSFTM / IPE produce low bias when treatment effect is common  But are very sensitive to this IPCW / SNM are not affected by changes in treatment effect between groups, but in (relatively) small trial datasets observational methods are volatile  Especially when crossover % is very high (leaving low n in control group) Very important to assess trial data, crossover mechanism, treatment effect to determine which method likely to be most appropriate Don’t just pick one!!
    14. 14. 14Back-up Slides28/09/2012 © The University of Sheffield
    15. 15. 15 Data Generation (1) Used a two-stage Weibull model to generate underlying survival times and a time-dependent covariate (called „CEA‟) Longitudinal model for CEA (for ith patient at time t):where is the random intercept is the slope for a patient in the control arm is the slope for a patient in the treatment arm (all is the change in the intercept for a patient with bad prognosis compared to a patient without bad prognosis Picked parameter values such that CEA increased over time, more slowly in the experimental group, and was higher in the badprog group 28/09/2012 © The University of Sheffield
    16. 16. 16 Data Generation (2) The survival hazard function was based upon a Weibull (see Bender et al 2005):  1 h(t )   t exp( X ) In our case, X = 1 * trt i + ( * log(t) ) * trt i +  2 badprogi +  (cea(t) )where  1 is the log hazard ratio (the baseline treatment effect)  is the time-dependent change in the treatment effect  2 is the impact of a bad prognosis baseline covariate on survival  is the coefficient of CEA, indicating its effect on survival We used this to generate our survival times So, CEA has an effect on survival over time, and there is a separate time-dependent treatment effect 28/09/2012 © The University of Sheffield
    17. 17. 17Data Generation (3) We applied a reduced treatment effect to crossover patients – this was equivalent to the baseline treatment effect multiplied by a factor to ensure that this effect was less than (or equal to) the average effect received in the experimental group28/09/2012 © The University of Sheffield
    18. 18. 18 Data Generation (4) We then selected parameter values in order that „realistic‟ datasets were created: 1.00 2.5 AF actual 0.80 AF predicted trtrand = 0 trtrand = 1 2 0.60 Acceleration Factor 1.5 0.40 0.20 1 0.00 0.5 0 200 400 600 800 1000 1200 analysis timeNumber at risk 0 trtrand = 0 251 154 86 52 25 9 0 0 200 400 600 800 1000 1200 trtrand = 1 249 186 126 71 42 21 0 Time (days) 28/09/2012 © The University of Sheffield
    19. 19. 19 Data Generation (5)We made several assumptions about the „crossover mechanism‟: 1. Crossover could only occur after disease progression (disease progression was approximately half of OS, calculated for each patient using a beta(5,5) distribution) 2. Crossover could only occur at 3 „consultations‟ following disease progression These were set at 21 day intervals Probability of crossover highest at initial consultation, then falls in second and third 3. Crossover probability depended on time-dependent covariates: CEA value at progression (high value reduced chance of crossover) Time to disease progression (high value increased chance of crossover) This was altered in scenarios to test a simpler mechanism where probability only depended on CEA Given all this, CEA was a time-dependent confounder 28/09/2012 © The University of Sheffield
    20. 20. 20 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  severityInitially assumed treatment effect Alter to test two levels of treatment effect Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions over timePrognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
    21. 21. 21 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  (2) severityInitially assumed treatment effect Alter to test two levels of treatment effect Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions over timePrognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
    22. 22. 22 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  (2) severityInitially assumed treatment effect Alter to test two levels of treatment effect  (4)Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions over timePrognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
    23. 23. 23 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  (2) severityInitially assumed treatment effect Alter to test two levels of treatment effect  (4)Time-dependence of treatment effect Treatment effect received is equal in all crossover  (8) patients, and equals baseline treatment effect multplied by a factor. However set α to zero in (12) some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions over timePrognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
    24. 24. 24 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  (2) severityInitially assumed treatment effect Alter to test two levels of treatment effect  (4)Time-dependence of treatment effect Treatment effect received is equal in all crossover  (8) patients, and equals baseline treatment effect multplied by a factor. However set α to zero in (12) some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions  (24)over timePrognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
    25. 25. 25 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  (2) severityInitially assumed treatment effect Alter to test two levels of treatment effect  (4)Time-dependence of treatment effect Treatment effect received is equal in all crossover  (8) patients, and equals baseline treatment effect multplied by a factor. However set α to zero in (12) some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions  (24)over timePrognosis of crossover patients Test three crossover mechanisms in which different  (72) groups become more likely to cross over This combined to 72 scenarios 28/09/2012 © The University of Sheffield
    26. 26. 26 ScenariosVariable Value AlternativeSample size 500 Number of prognosis groups (prog) 2 Probability of good prognosis 0.5 Probability of poor prognosis 0.5 Maximum follow-up time 3 years (1095 days) Multiplication of OS survival time due Log hazard ratio = 0.5 to bad prognosis groupSurvival time distribution Alter parameters to test two levels of disease  severityInitially assumed treatment effect Alter to test two levels of treatment effect Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenariosProbability of switching treatment Test two levels of treatment crossover proportions over timePrognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
    27. 27. 27 Estimating AUC1. ‘Survivor function’ approach Apply treatment effect to survivor function (or hazard function) estimated for experimental group  calculate AUC2. ‘Extrapolation’ approach Extrapolate counterfactual dataset to required time-point (only relevant for RPSFTM/IPE approaches)  calculate AUC3. ‘Shrinkage’ approach Use estimated acceleration factor to „shrink‟ survival times in crossover patients in order to obtain an adjusted dataset  calculate AUC (only relevent for AF-based approaches) 28/09/2012 © The University of Sheffield
    28. 28. A topical analogy... 28 England Spain Vs (control group) (intervention group) Kick-off...
    29. 29. A topical analogy... 29Halftime: England 0 – 3 Spain Spain are statistically significantlybetter than England For ethical reasons, theSpanish team are cloned and theEnglish team are sent home. Sothe second half is Spain VsSpain
    30. 30. A topical analogy... 30Full time: England 2 – 5 Spain Spain still win, but not as comfortably asthey would haveSwitching is likely to result in an underestimate ofthe true supremacy of the intervention
    31. 31. 31Results (4)While the SNM and IPCW methods appeared to produce similar levels of bias, theSNM approach was more volatile: Relatively often failed to converge (in up to 90% of sims in one scenario) Typically when disease severity was high and treatment effect was low Only produced lower bias than the ITT analysis in 38% of scenarios (compared to 60% for the IPCW method) Was more sensitive to increasing crossover %28/09/2012 © The University of Sheffield
    32. 32. 32 Results (5)Relationship between bias and treatment crossover % 140 120 100 80 Mean % bias 60 40 ITT 20 0 60% 70% 80% 90% 100% -20 -40 Crossover proportion (at-risk patients) 28/09/2012 © The University of Sheffield
    33. 33. 33 Results (5)Relationship between bias and treatment crossover % 140 120 100 80 Mean % bias 60 IPE RPSFTM 40 ITT 20 0 60% 70% 80% 90% 100% -20 -40 28/09/2012 © The University ofCrossover Sheffield proportion (at-risk patients)
    34. 34. 34 Results (5)Relationship between bias and treatment crossover % 140 120 100 80 Mean % bias IPCW 60 IPE 40 RPSFTM ITT 20 0 60% 70% 80% 90% 100% -20 -40 28/09/2012 © The University ofCrossover Sheffield proportion (at-risk patients)
    35. 35. 35 Results (5)Relationship between bias and treatment crossover % 140 120 100 80 IPCW Mean % bias 60 IPE RPSFTM 40 SNM 20 ITT 0 60% 70% 80% 90% 100% -20 -40 28/09/2012 © The University ofCrossover Sheffield proportion (at-risk patients)
    36. 36. 36Conclusions Treatment crossover is an important issue that has come to the fore in HE arena Current methods for dealing with treatment crossover are imperfect Our study offers evidence on bias in different scenarios (subject to limitations) 1. RPSFTM / IPE produce low bias when treatment effect is common 2. When treatment effect ≈ 15% lower in crossover patients RPSFTM / IPE and IPCW / SNM methods produce similar levels of bias (5-10%) [provided suitable data available for obs methods and <90% crossover] 3. When treatment effect ≈ 25% lower IPCW/SNM produce less bias than RPSFTM/IPE [provided suitable data available for obs methods and <90% crossover] [and significant bias likely to remain  ITT analysis may offer least bias] 4.Very important to assess trial data, crossover mechanism, treatment effect to determine which method likely to be most appropriate  Don’t just pick one!!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×