Uploaded byrgveroniki
PDF, PPTX1,138 views

2010 smg training_cardiff_day2_session1_salanti

This document outlines the concepts and methods of multiple-treatments meta-analysis (MTM). MTM allows for the simultaneous comparison of multiple interventions for a condition by combining both direct and indirect evidence from randomized controlled trials. Key advantages of MTM include the ability to rank treatments, comprehensively use all available data, and compare interventions not directly compared in trials. The document discusses MTM approaches using frequentist meta-regression and Bayesian statistics.

Embed presentation

Download as PDF, PPTX
Multiple-Treatments Meta-Analysis Dr Georgia Salanti & Sofia Dias With thanks to Julian Higgins, Tony Ades, Andrea Cipriani, Corrado Barbui University of Ioannina Greece
Outline – Part I • Concept • Simple indirect comparison • Advantages of the methods • MTM using frequentist meta-regression • Presentation of results • The notion of Inconsistency
Outline – Part II • Bayesian MTM model • Comparison of models
Evidence Based Medicine Levels of evidence For Therapy, Prevention, Aetiology and Harm Randomized Controlled trials (RCTs) Meta-analysis of RCTs Two interventions Centre for Evidence Based Medicine, University of Oxford Cohort studies, Case-control studies • Backbone: meta-analysis • Rigorous statistical models • Clinical practice guidelines – NICE, WHO, The Cochrane Collaboration, HuGENet
Fluoxetine: 28€ Venlafaxine:111€ Sertaline: 76 € “Although Mirtazapine is likely to have a faster onset of action than Sertraline and Paroxetine no significant differences were observed...” “Venlafaxine tends to have a favorable trend in response rates compared with duloxetine” “…statistically significant differences in terms of efficacy …. between Fluoxetine and Venlafaxine, but the clinical meaning of these differences is uncertain…” “…meta-analysis highlighted a trend in favour of Sertraline over other Fluoxetine” 12 new generation antidepressants 19 meta-analyses published in the last two years
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapineduloxetine escitalopram sertraline milnacipran bupropion paroxetine milnacipran duloxetine escitalopram fluvoxamine ? 12 new generation antidepressants 19 meta-analyses published in the last two years
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapineduloxetine escitalopram milnacipran bupropion paroxetine milnacipran escitalopram fluvoxamine paroxetine 0% sertraline 7% citalopram 0% escitalopram 26% fluoxetine 0% fluvoxamine 0% milnacipran 1% venlafaxine 11% reboxetine 0% bupropion 0% mirtazapine 54% duloxetine 0% sertraline duloxetine? Probability to be the best 12 new generation antidepressants 19 meta-analyses published in the last two years
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapineduloxetine escitalopram milnacipran bupropion paroxetine milnacipran escitalopram fluvoxamine paroxetine 0% sertraline 7% citalopram 0% escitalopram 26% fluoxetine 0% fluvoxamine 0% milnacipran 1% venlafaxine 11% reboxetine 0% bupropion 0% mirtazapine 54% duloxetine 0% sertraline duloxetine? Probability to be the best Current meta-analysis misses data! 12 new generation antidepressants 19 meta-analyses published in the last two years
A new methodological framework Two interventions Randomized Controlled trials (RCTs) Meta-analysis of RCTs Cohort studies, Case-control studies
Two interventions Multiple-treatments meta-analysis Many different intervention Randomized Controlled trials (RCTs) Meta-analysis of RCTs Cohort studies, Case-control studies A new methodological framework
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
sertraline citalopram fluoxetine Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
Indirect comparison A B C • We can obtain an indirect estimate for A vs B from RCTs comparing A vs C and B vs C: MDAB = MDAC – MDBC Var(MDAB) = Var(MDAC) + Var(MDBC)
Placebo Toothpaste Varnish Rinse Gel 69 4 1 6 31 13 3 1 No treat 9 4 4
Placebo Toothpaste Gel 69 13 ?
Simple exercise: prevented mean caries Comparison MD CIs Placebo vs Toothpaste -0.34 (-0.41, -0.28) Placebo vs Gel -0.19 (-0.30, -0.10) How to compare Gel to Toothpaste? Estimate indirect MD and a 95% CI Toothpaste Gel Placebo
Flash back to stats… Each estimate has uncertainty as conveyed by the variance, the standard error and the 95% CI Variance=SE2 95% CI (Low CI, High CI): x-1.96·SE to x+1.96·SE : SE=(High CI – Low CI)/3.92
Pen and paper (and calculator!) exercise! Indirect MDGvsT= MDPvsT – MDPvsG Indirect MDGvsT = -0.34 – (-0.19)= -0.15 Variance Indirect MDGvsT = Variance MDPvsT + Variance MDPvsG Variance MDPvsT = ((high CI –low CI)/3.92)2 Variance MDPvsT= ((-0.28– (-0.41))/3.92)2 =0.0011 Variance MDGvsT= ((-0.10– (-0.30))/3.92)2 =0.0026 Variance Indirect MDGvsT = 0.0011+0.0026=0.0037 SE Indirect MDGvsT = sqrt(0.0037)=0.061 95% CI for Indirect MDGvsT = (-0.15 – 1.96·0.061, -0.15 + 1.96·0.061) 95% CI for Indirect MDGvsT = (-0.27, -0.03)
Placebo Toothpaste Varnish Rinse Gel 69 4 1 3 6 31 13 3 1 No treat 9 4 4
Combining direct and indirect evidence • Inverse variance method • Each estimate is ‘weighted’ by the inverse of the variance • Then a common (pooled) result is obtained! IndirectDirect Indirect Indirect Direct Direct MDMD var 1 var 1 var 1 var 1 MDpooled   
You can do this with any measure... lnOR, lnRR, RD, mean difference, HR, Peto’s lnOR etc… Indirect MDGvsT = - 0.15 Variance Indirect MDGvsT = 0.0037 Direct MDGvsT = 0.04 Variance Direct MDGvsT = 0.011 Pooled MDGvsT= -0.14 037.0 1 011.0 1 15.0 0037.0 1 04.0 011.0 1 MDpooled    
sertraline citalopram fluoxetine Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons LORSF v1 LORCF v2 Indirect estimation LORSC = LORSF - LORCF Var(LORSC) = v1+ v2 LORSC Var(LORSC)
sertraline citalopram fluoxetine Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons LORSF v1 LORFC v2 LORSC Var(LORSC) Combine the direct estimate with the indirect estimate using IV methods Get a combined LOR! v4<v3 LORSF v4 Combined LORSC v3 Indirect estimation LORSC = LORSF - LORCF Var(LORSC) = v1+ v2
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Choose basic parameters
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram All other contrasts are functional!
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram All other contrasts are functional!
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram All other contrasts are functional!
Advantages of MTM – Ranking of many treatments for the same condition (see later) – Comprehensive use of all available data (indirect evidence) – Comparison of interventions which haven’t been directly compared in any experiment
Bevacizumab Fluorouracil and leucovorin Fluorouracil and leucovorin+bevacizumab Fluorouracil and leucovorin+irinotecan Fluorouracil and leucovorin+ irinotecan+bevacizumab Fluorouracil and leucovorin+irinotecan +oxaliplatinFluorouracil+leucovorin+oxaliplatin Fluorouracil and leucovorin + oxaliplatin + bevacizumab Irinotecan Irinotecan + oxaliplatin Oxaliplatin Colorectal Cancer Golfinopoulos V, Salanti G, Pavlidis N, Ioannidis JP: Survival and disease-progression benefits with treatment regimens for advanced colorectal cancer: a meta-analysis. Lancet Oncol 2007, 8: 898-911.
Advantages of MTM – Ranking of many treatments for the same condition (see later) – Comprehensive use of all available data (indirect evidence) – Comparison of interventions which haven’t been directly compared in any experiment – Improved precision for each comparison
paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani, Fukurawa, Salanti et al Network of experimental comparisons Fluoxetine vs Milnacipran (response to treatment) Meta-analysis: 1.15 (0.72, 1.85) MTM: 0.97 (0.69, 1.32)
Why use Bayesian statistics for meta-analysis? • Natural approach for accumulating data • Repeated updating of meta-analyses fine: posterior should always reflect latest beliefs • People naturally think as Bayesians: they have degrees of belief about the effects of treatment, which change when they see new data • Probability statements about true effects of treatment easier to understand than confidence intervals and p -values
Why use Bayesian statistics for MTM? • Bayesian approach is easier to account for correlations induced by multi-arm trials • Estimation of predictive intervals is straightforward • Estimation of ranking probabilities is straightforward • MTM with two-arm trials only (or ignoring the correlations) Easy with frequentist meta-regression
Fixed effect meta-analysis Trial 1 2 3 4 5 6 7 8 9 10 11 12 Treatment better Control better Effect estimate -1 0 1 random error common (fixed) effect μ yi i
Random effects meta-analysis study-specific effect distribution of effects Trial 1 2 3 4 5 6 7 8 9 10 11 12 Treatment better Control better Effect estimate -1 0 1 random error τμ θi yi i
Random effects meta-regression Explanatory variable, x yi = intersept + slope x Random error Τmeta-regmeta-reg i
• We observe yi in each study (e.g. the log(OR)) • Meta-regression using the treatments as ‘covariates’ • AC, AB, BC studies, chose C as reference Meta-regression • The AC studies have (1,0), the BC studies (0,1) [basic] • AB studies have (1,-1) [functional] yi = C (Treati=A) + BC (Treati=B)
Parametrisation of the network t-PA Angioplasty Acc t-PA Anistreplase Retaplase Streptokinase Choose basic parameters Write all other contrasts as linear functions of the basic parameters to build the design matrix LOR for death in treatments for MI
LOR for death in treatments for MI No. studies Streptokinase t-PA Anistreplase Acc t-PA Angioplasty Reteplase 3 1 1 3 1 1 2 2 2 Use as ‘covariates’ yi= μA t-PA  μB Anistreplasei μC Accelerated t-PAi  μD Angioplastyi  μE Reteplasei -1 1 0 0 0 0 -1 0 1 0 0 0 -1 0 0 1 0 0 -1 0 0 0 1 0 -1 0 0 0 0 1 -1 1 0 0 0 -1 0 0 1 0 0 0 -1 1 0 0 0 -1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Lumley 2002, Stat Med
 X),,,,(Y EDCBA  LOR for death in treatments for MI Matrix of all observations Vector of LogOR yi= μA t-PA  μB Anistreplasei μC Accelerated t-PAi  μD Angioplastyi  μE Reteplasei Design matrix Random effects matrix )V,X(N~Y μ ))τ(diag,(N~ 2 ∆ 0 Variance-covariance matrix (for the observed LOR)
LOR compared to Streptokinase (RE model) - 0.11 (0.06) - 0.43 (0.20) - 0.15 (0.05) -0.00 (0.03) -0.02 (0.03) LOR (SE)Treatment t-PA Anistreplase Accelerated t-PA Angioplasty Reteplase  X),,,,(Y EDCBA  1.01)(0.80,0.90 0.96)(0.44,0.65 0.95)(0.78,0.86 1.06)(0.94,1.00 1.04)(0.92,0.98 95% CIOR
What’s the problem with multi-arm trials? • We need to take into account the correlations between the estimates that come from the same study • A B C yi BC yi AC • The random effects (θi BC, θi AC) that refer to the same trial are correlated as well • You have to built in the correlation matrix for the observed effects, and the correlation matrix for the random effects )V,X(N~Y μ ))τ(diag,(N~ 2 ∆ 0
Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC Hypothetical example Basic parameters: AB and AC
1,1 1,11,1 2,1 2,12,1 3,1 3,13,1 4,1 4,14,1 4,2 4,2 4,2 1 0 0 1 1 1 1 0 0 1                                                                           AB AC y y y y y Meta-regression Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC
1,1 1,11,1 2,1 2,12,1 3,1 3,13,1 4,1 4,14,1 4,2 4,2 4,2 1 0 0 1 1 1 1 0 0 1                                                                           AB AC y y y y y Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC Take into account correlation in observations     1,11,1 2,12,1 3,13,1 4,1 4,1 4,24,1 4,2 4,1 4,2 4,2 0 0 0 00 0 0 0 00 0 0 0 0~ ,0 0 0 0 cov ,0 0 0 0 0 cov ,                                                      v v vN v y y y y v
1,1 1,11,1 2,1 2,12,1 3,1 3,13,1 4,1 4,14,1 4,2 4,2 4,2 1 0 0 1 1 1 1 0 0 1                                                                           AB AC y y y y y Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC Take into account correlation in random effects     2 1,1 2 2,1 2 3,1 2 4,1 4,24,1 2 4,2 4,1 4,2 0 0 0 00 0 0 0 00 0 0 0 0~ ,0 0 0 0 cov ,0 0 0 0 0 cov ,                                                            AB AC BC AB AC N
How to fit such a model? • MLwiN • SAS, R • STATA using metan
Ranking measures from MTM • With many treatments judgments based on pairwise effect sizes are difficult to make • Example: Antidepressants
Ranking measures from MTM • With many treatments judgments based on pairwise effect sizes are difficult to make • Example: Antidepressants • Example: Antiplatelet regimens for serious vascular events
Aspirin vs Placebo Thienopyridines vs Aspirin Thienopyridines vs Placebo 0.5 1 1.5 2 0.32 0.03 <0.01 Aspirin+Dipyridamole vs Aspirin+Thienopyridines Aspirin+Dipyridamole vs Aspirin Aspirin+Dipyridamole vs Placebo Aspirin+Dipyridamole vs Thienopyridines 0 Aspirin+Thienopyridines vs Aspirin Aspirin+Thienopyridines vs Placebo Aspirin+Thienopyridines vs Thienopyridines0.23 0.05 <0.01 0.19 <0.01 <0.01 P-value Comparison Odds Ratio for serious vascular event Favors first treatment Favors second treatment Serious vascular events with antiplatelet regimens
Probabilities instead of effect sizes • Estimate for each treatment the probability to be the best • This is straightforward within a Bayesian framework
% probability A B C D j=1 0.25 0.50 0.25 0.00 j=2 0.50 0.75 0.75 0.00 j=3 0.75 1.00 1.00 0.25 j=4 1.00 1.00 1.00 1.00
% probability A B C D j=1 0.25 0.50 0.25 0.00 j=2 0.25 0.25 0.50 0.00 j=3 0.25 0.25 0.25 0.25 j=4 0.25 0 0 0.75 i the treatment j the rank
Rank of paroxetine Probability 2 4 6 8 10 12 0.00.20.40.6 Rank of sertraline 2 4 6 8 10 12 0.00.20.40.6 Rank of citalopram 2 4 6 8 10 12 0.00.20.40.6 Rank of escitalopram 2 4 6 8 10 12 0.00.20.40.6 Rank of fluoxetine Probability 2 4 6 8 10 12 0.00.20.40.6 Rank of fluvoxamine 2 4 6 8 10 12 0.00.20.40.6 Rank of milnacipran 2 4 6 8 10 12 0.00.20.40.6 Rank of venlafaxine 2 4 6 8 10 12 0.00.20.40.6 Rank of reboxetine Probability 2 4 6 8 10 12 0.00.20.40.6 Rank of bupropion 2 4 6 8 10 12 0.00.20.40.6 Rank of mirtazapine 2 4 6 8 10 12 0.00.20.40.6 Rank of duloxetine 2 4 6 8 10 12 0.00.20.40.6 Ranking for efficacy (solid line) and acceptability (dotted line). Ranking: probability to be the best treatment, to be the second best, the third best and so on, among the 12 comparisons).
% probability A B C D j=1 0.25 0.50 0.25 0.00 j=2 0.50 0.75 0.75 0.00 j=3 0.75 1.00 1.00 0.25 j=4 1.00 1.00 1.00 1.00 The areas under the cumulative curves for the four treatments of the example above are A=0.5 B=0.75 C=0.67 D=0.08 i the treatment j the rank Rank of A CumulativeProbability 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0 Rank of B 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0 Rank of C CumulativeProbability 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0 Rank of D 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0
Preliminary results for ranking 12 antidepressants Rank of paroxetine CumulativeProbability 2 4 6 8 10 12 Rank of sertraline 2 4 6 8 10 12 Rank of reboxetine CumulativeProbability 2 4 6 8 10 12 020406080100 Rank of mirtazapine 2 4 6 8 10 12 35% 77% 1% 92% 020406080100 020406080100020406080100 A comprehensive ranking measure Compared to an imaginary antidepressant which is ‘always the best’, mirtazapine reaches up to 92% of its potential!
• What is inconsistency? • How it manifests itself? Inconsistency
Inconsistency Direct t-PA vs Angioplasty=  0.41 (0.36) 0.02 (0.03) - 0.48 (0.43) Calculate a difference between direct and indirect estimates t-PA Angioplasty Streptokinase LOR (SE) for MI Indirect t-PA vs Angioplasty =  0.46 (0.18) Inconsistency in the loop = 0.05
Inconsistency - Heterogeneity • Heterogeneity: ‘excessive’ discrepancy among study-specific effects • Inconsistency: it is the excessive discrepancy among source-specific effects (direct and indirect)
14 15 16 17 AC=15.5 7 8 9 10 BC=8.2 1 2 3 4 AB=2.3
• In 3 cases out of 44 there was an important discrepancy between direct/indirect effect. • Direction of the discrepancy is inconsistent Glenny et al HTA 2005 Inconsistency Empirical Evidence
Placebo Toothpaste Gel Direct SMD(TvsG) = 0.04 Indirect SMD(TvsG) = – 0.15 IF= 0.11 P-Gel P-Toothpaste I cannot learn about Toothpaste versus Gel through Placebo! Compare Fluoride treatments in preventing dental caries What can cause inconsistency? Inappropriate common comparator
Screening for lung cancer Baker & Kramer, BMC Meth 2002 Chest X-ray Standard Spiral CT A new therapy (possibly unreported in the trials) decreases the mortality but in different rates for the three screening methods Percent receiving new therapy Mortality 30 70 Trial 1: Chest X-ray= Standard Trial 2: Spiral-CT= Standard New Trial: Spiral-CT < Chest X-ray 100 What can cause inconsistency? Confounding by trial characteristics
age Effectiveness Alfacalcidol +Ca Calcitriol + Ca Ca What can cause inconsistency? Confounding by trial characteristics Vitamin D for Osteoporosis-related fractures, Richy et al Calcif Tissue 2005, 76;276 Vitamin D +Ca Different characteristics across comparisons may cause inconsistency
• There is not confounding by trial characteristics that are related to both the comparison being made and the magnitude of treatment difference • The trials in two different comparisons are exchangeable (other than interventions being compared) • Equivalent to the assumption ‘the unobserved treatment is missing at random’ – Is this plausible? – Selection of the comparator is not often random! Assumptions of MTM
• Check the distribution of important characteristics per treatment comparison – Usually unobserved…. – Time (of randomization, of recruitment) might be associated with changes to the background risk that may violate the assumptions of MTM • Get a taste by looking for inconsistency in closed loops Inconsistency Detecting
No. studies T G R V P Fup Baseline Year Water F (yes/no) 69 2.6 11.8 1968 0.2 13 2.3 3.8 1973 0.2 30 2.4 5.9 1973 0.1 3 2.3 2.7 1983 0 3 2.7 NA 1968 0.66 6 2.8 14.7 1969 0 1 2 0.9 1978 0 1 1 NA 1977 0 1 3 7.4 1991 NA 4 2.5 7.6 1981 0.33 Compare the characteristics! Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62: 857-864.
• Check the distribution of important characteristics per treatment comparison – Usually unobserved…. – Time (of randomization, of recruitment) might be associated with changes to the background risk that may violate the assumptions of MTM • Get a taste by looking for inconsistency in closed loops • Fit a model that relaxes consistency – Add an extra ‘random effect’ per loop (Lu & Ades JASA 2005) Inconsistency Detecting
Placebo Toothpaste Varnish Rinse Gel 69 4 1 3 6 31 13 3 1 No treat 9 4 4
-1.0 0.0 0.5 1.0 1.5 2.0 Closed loops NGV NGR NRV PDG PDV PDR DGV DGR DRV PGV PGR PRV GRV AGRV PDGV PDGR PDRV DGRV PGRV PDGRV Evaluation of concordance within closed loops Estimates with 95% confidence intervals R routine in http://www.dhe.med.uoi.gr/software.htm Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62: 857-864.
• Check the distribution of important characteristics per treatment comparison – Usually unobserved…. – Time (of randomization, of recruitment) might be associated with changes to the background risk that may violate the assumptions of MTM • Get a taste by looking for inconsistency in closed loops • Fit a model that relaxes consistency – Add an extra ‘random effect’ per loop (Lu & Ades JASA 2005) Inconsistency Detecting
Inconsistency - Heterogeneity RCTs Meta-analysis of RCTs With homogeneity2 interventions With consistencyMultiple meta-analyses of RCTsbest intervention
References 1. Baker SG, Kramer BS: The transitive fallacy for randomized trials: if A bests B and B bests C in separate trials, is A better than C? BMC Med Res Methodol 2002, 2: 13. 2. Caldwell DM, Ades AE, Higgins JP: Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005, 331: 897-900. 3. Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JP, Churchill R et al.: Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 2009, 373: 746-758. 4. Cooper NJ, Sutton AJ, Lu G, Khunti K: Mixed comparison of stroke prevention treatments in individuals with nonrheumatic atrial fibrillation. Arch Intern Med 2006, 166: 1269-1275. 5. Golfinopoulos V, Salanti G, Pavlidis N, Ioannidis JP: Survival and disease-progression benefits with treatment regimens for advanced colorectal cancer: a meta-analysis. Lancet Oncol 2007, 8: 898-911. 6. Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S: Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second- generation antipsychotics. Am J Psychiatry 2006, 163: 185-194. 7. Jansen JP, Crawford B, Bergman G, Stam W: Bayesian Meta-Analysis of Multiple Treatment Comparisons: An Introduction to Mixed Treatment Comparisons. Value Health 2008. 8. Lu G, Ades AE: Assessing Evidence Inconsistency in Mixed Treatment Comparisons. Journal of American Statistical Association 2006, 101: 447-459. 9. Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004, 23: 3105- 3124. 10. Salanti G, Higgins JP, Ades AE, Ioannidis JP: Evaluation of networks of randomized trials. Stat Methods Med Res 2008, 17: 279-301. 11. Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62: 857-864. 12. Song F, Harvey I, Lilford R: Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol 2008, 61: 455-463. 13. Sutton A, Ades AE, Cooper N, Abrams K: Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 2008, 26: 753-767. 14. Welton NJ, Cooper NJ, Ades AE, Lu G, Sutton AJ: Mixed treatment comparison with multiple outcomes reported inconsistently across trials: Evaluation of antivirals for treatment of influenza A and B. Stat Med 2008, 29: 5620-5639.

More Related Content

PDF
2010 smg training_cardiff_day1_session4_harbord
byrgveroniki
 
PPT
Analysis and Interpretation
byFrancisco J Grajales III
 
PDF
day1(2010 smg training_cardiff)_session2b (1of 2) lewis
byrgveroniki
 
PDF
2010 smg training_cardiff_day1_session3_higgins
byrgveroniki
 
PDF
2010 smg training_cardiff_day2_session2_dias
byrgveroniki
 
PDF
2010 smg training_cardiff_day2_session4_sterne
byrgveroniki
 
PDF
2010 smg training_cardiff_day2_session3_dwan_altman
byrgveroniki
 
PDF
2010 smg training_cardiff_day1_session1(3 of 3)beyene
byrgveroniki
 
2010 smg training_cardiff_day1_session4_harbord
byrgveroniki
 
Analysis and Interpretation
byFrancisco J Grajales III
 
day1(2010 smg training_cardiff)_session2b (1of 2) lewis
byrgveroniki
 
2010 smg training_cardiff_day1_session3_higgins
byrgveroniki
 
2010 smg training_cardiff_day2_session2_dias
byrgveroniki
 
2010 smg training_cardiff_day2_session4_sterne
byrgveroniki
 
2010 smg training_cardiff_day2_session3_dwan_altman
byrgveroniki
 
2010 smg training_cardiff_day1_session1(3 of 3)beyene
byrgveroniki
 

What's hot

PPT
Biostatistics in cancer RCTs
byBhaswat Chakraborty
 
PPTX
Network meta-analysis & models for inconsistency
bycheweb1
 
PPT
Metanalysis Lecture
bydrmomusa
 
PPTX
Bowen & Neill (2013) Adventure Therapy Meta-Analysis Presentation
byDaniel Bowen
 
PPTX
Imran rizvi statistics in meta analysis
byImran Rizvi
 
PDF
Repeated events analyses
byMike LaValley
 
PPTX
Parmetric and non parametric statistical test in clinical trails
byVinod Pagidipalli
 
PDF
Intent-to-Treat (ITT) Analysis in Randomized Clinical Trials
byMike LaValley
 
PPTX
Network meta analysis
byNitisha Thakur
 
PPTX
Meta analysis
bySethu S
 
PPTX
Statistical test
byKumar Mrigesh
 
PPTX
Analyzing the randomised control trial (rct)
byVikash Keshri
 
PPTX
Statistical analysis of clinical data
byDarshil Shah
 
PPTX
Cochrane Collaboration
byNinian Peckitt
 
PPTX
Alternatives to t test
byLONDIWE SHANGE
 
PDF
Network meta-analysis with integrated nested Laplace approximations
byBurak Kürsad Günhan
 
PDF
Meta-analysis when the normality assumptions are violated (2008)
byEvangelos Kontopantelis
 
PPTX
To Cochrane or not: that's the question
byHesham Al-Inany
 
PPT
2010 JSM - Meta Stat Issue Medical Devices
byTerry Liao
 
PPT
Chapter 6 Ranksumtest
bySumit Prajapati
 
Biostatistics in cancer RCTs
byBhaswat Chakraborty
 
Network meta-analysis & models for inconsistency
bycheweb1
 
Metanalysis Lecture
bydrmomusa
 
Bowen & Neill (2013) Adventure Therapy Meta-Analysis Presentation
byDaniel Bowen
 
Imran rizvi statistics in meta analysis
byImran Rizvi
 
Repeated events analyses
byMike LaValley
 
Parmetric and non parametric statistical test in clinical trails
byVinod Pagidipalli
 
Intent-to-Treat (ITT) Analysis in Randomized Clinical Trials
byMike LaValley
 
Network meta analysis
byNitisha Thakur
 
Meta analysis
bySethu S
 
Statistical test
byKumar Mrigesh
 
Analyzing the randomised control trial (rct)
byVikash Keshri
 
Statistical analysis of clinical data
byDarshil Shah
 
Cochrane Collaboration
byNinian Peckitt
 
Alternatives to t test
byLONDIWE SHANGE
 
Network meta-analysis with integrated nested Laplace approximations
byBurak Kürsad Günhan
 
Meta-analysis when the normality assumptions are violated (2008)
byEvangelos Kontopantelis
 
To Cochrane or not: that's the question
byHesham Al-Inany
 
2010 JSM - Meta Stat Issue Medical Devices
byTerry Liao
 
Chapter 6 Ranksumtest
bySumit Prajapati
 

Similar to 2010 smg training_cardiff_day2_session1_salanti

PPTX
Comparative efficacy and acceptability of 21 antidepressant drugs, powerpoint...
byShamim Rahman
 
PDF
Evidence Synthesis for Sparse Evidence Base, Heterogeneous Studies, and Disco...
byInsideScientific
 
PDF
Meta-Analysis in Ayurveda
byAyurdata
 
PPTX
Evidence based psychiatry
byshuchi pande
 
PPTX
Metaanalysis copy
byAmandeep Kaur
 
PPTX
Meta analysis
byVishal Ramteke
 
PPT
Guide for conducting meta analysis in health research
byYogitha P
 
PPTX
Stat Methods in ayurveda
byAyurdata
 
PDF
A gentle introduction to meta-analysis
byAngelo Tinazzi
 
PDF
How to Improve Quality of Services by Integrating Common Factors into Treatme...
byScott Miller
 
PPTX
Meta analysis.pptx
byVishwasATL
 
PPTX
Sonal evidence based orthodontics
bySahasrabudheSonal
 
PPTX
Seminar in Meta-analysis
byأحمد الخريصي
 
PDF
Internal 2012 - individual patient data meta-analysis
byEvangelos Kontopantelis
 
PPTX
WFSBP Guidelines Mania - Prof Grunze
byJP Rajendran
 
PPTX
PHARMACOECONOMIC ANALYSES OF MIRTAZAPINE.pptx
byasmitapandey5196
 
DOCX
linical pharmacogenomics consists of the appli-cation of res.docx
bycroysierkathey
 
DOCX
linical pharmacogenomics consists of the appli-cation of res.docx
bywashingtonrosy
 
PDF
Amsterdam 2012 - one stage meta-analysis
byEvangelos Kontopantelis
 
PDF
RSS 2012 - ipdforest
byEvangelos Kontopantelis
 
Comparative efficacy and acceptability of 21 antidepressant drugs, powerpoint...
byShamim Rahman
 
Evidence Synthesis for Sparse Evidence Base, Heterogeneous Studies, and Disco...
byInsideScientific
 
Meta-Analysis in Ayurveda
byAyurdata
 
Evidence based psychiatry
byshuchi pande
 
Metaanalysis copy
byAmandeep Kaur
 
Meta analysis
byVishal Ramteke
 
Guide for conducting meta analysis in health research
byYogitha P
 
Stat Methods in ayurveda
byAyurdata
 
A gentle introduction to meta-analysis
byAngelo Tinazzi
 
How to Improve Quality of Services by Integrating Common Factors into Treatme...
byScott Miller
 
Meta analysis.pptx
byVishwasATL
 
Sonal evidence based orthodontics
bySahasrabudheSonal
 
Seminar in Meta-analysis
byأحمد الخريصي
 
Internal 2012 - individual patient data meta-analysis
byEvangelos Kontopantelis
 
WFSBP Guidelines Mania - Prof Grunze
byJP Rajendran
 
PHARMACOECONOMIC ANALYSES OF MIRTAZAPINE.pptx
byasmitapandey5196
 
linical pharmacogenomics consists of the appli-cation of res.docx
bycroysierkathey
 
linical pharmacogenomics consists of the appli-cation of res.docx
bywashingtonrosy
 
Amsterdam 2012 - one stage meta-analysis
byEvangelos Kontopantelis
 
RSS 2012 - ipdforest
byEvangelos Kontopantelis
 

Recently uploaded

PDF
EMCEE/ANCHOR VENKAT ( Aim to add value to every stage I step on.)
byEmcee Venkat
 
PDF
DEBT IN THE DUST
byPACETULSA
 
PDF
EMCEE/ANCHOR VENKAT ( Aim to add value to every stage I step on.)
byEmcee Venkat
 
PDF
EMCEE/ANCHOR VENKAT ( Aim to add value to every stage I step on.)
byEmcee Venkat
 
PDF
The Whisperwind Cartographer.pdf CHAPTER 1
bymadhumann4
 
PPTX
Robbing Miona's Wallet and Giving Her an Endless Headache
bypeachdewlight
 
PDF
MAGNET STORY- Coaster Sequence (Version 3)
byMike Pelensky
 
PDF
Thai knock dummy rules: The greatest Thai knock rummy rules
byบุญชัย ศฤงคารศิริ
 
PDF
The Whisperwind Cartographer chp 2 - The Unspoken Language of the Gloved Hand
bymadhumann4
 
PDF
THE FLOW ARCHITECT
byPACETULSA
 
PDF
Call Sheet Media Empowering Storytellers in Film & TV.pdf
bycall sheet media
 
PDF
Why Die My Love 2025 Feels So Real and So Unsettling
bySoap2day
 
PDF
Top 5 Animated Movies of 2025 Full Breakdown
byOregon Institute of Technology
 
PDF
Female Emcee Monisha | Professional Anchor from Chennai for Corporate, Family...
byChennaiEventProfessi1
 
PDF
Top 10 Richest Hollywood Stars of 2025 Revealed
byRingling Collage of Arts and Design Oregon
 
PDF
Presentation - Death Note An Anime of Intrigue_20251209_091034_0000.pdf
byelbekqahhorov25
 
PDF
Inside the Fear and Fury of Hunting Jessica Brok 2025
byAfdah Movies
 
PPTX
Exploring the World Through Cevin Soling’s Lens
byCevin Soling
 
PPTX
Unit 4_Lesson 3_Period 5_English for kids.pptx
byMaiAnhNguyn213259
 
EMCEE/ANCHOR VENKAT ( Aim to add value to every stage I step on.)
byEmcee Venkat
 
DEBT IN THE DUST
byPACETULSA
 
EMCEE/ANCHOR VENKAT ( Aim to add value to every stage I step on.)
byEmcee Venkat
 
EMCEE/ANCHOR VENKAT ( Aim to add value to every stage I step on.)
byEmcee Venkat
 
The Whisperwind Cartographer.pdf CHAPTER 1
bymadhumann4
 
Robbing Miona's Wallet and Giving Her an Endless Headache
bypeachdewlight
 
MAGNET STORY- Coaster Sequence (Version 3)
byMike Pelensky
 
Thai knock dummy rules: The greatest Thai knock rummy rules
byบุญชัย ศฤงคารศิริ
 
The Whisperwind Cartographer chp 2 - The Unspoken Language of the Gloved Hand
bymadhumann4
 
THE FLOW ARCHITECT
byPACETULSA
 
Call Sheet Media Empowering Storytellers in Film & TV.pdf
bycall sheet media
 
Why Die My Love 2025 Feels So Real and So Unsettling
bySoap2day
 
Top 5 Animated Movies of 2025 Full Breakdown
byOregon Institute of Technology
 
Female Emcee Monisha | Professional Anchor from Chennai for Corporate, Family...
byChennaiEventProfessi1
 
Top 10 Richest Hollywood Stars of 2025 Revealed
byRingling Collage of Arts and Design Oregon
 
Presentation - Death Note An Anime of Intrigue_20251209_091034_0000.pdf
byelbekqahhorov25
 
Inside the Fear and Fury of Hunting Jessica Brok 2025
byAfdah Movies
 
Exploring the World Through Cevin Soling’s Lens
byCevin Soling
 
Unit 4_Lesson 3_Period 5_English for kids.pptx
byMaiAnhNguyn213259
 

2010 smg training_cardiff_day2_session1_salanti

  • 1.
    Multiple-Treatments Meta-Analysis Dr Georgia Salanti& Sofia Dias With thanks to Julian Higgins, Tony Ades, Andrea Cipriani, Corrado Barbui University of Ioannina Greece
  • 2.
    Outline – PartI • Concept • Simple indirect comparison • Advantages of the methods • MTM using frequentist meta-regression • Presentation of results • The notion of Inconsistency
  • 3.
    Outline – PartII • Bayesian MTM model • Comparison of models
  • 4.
    Evidence Based Medicine Levelsof evidence For Therapy, Prevention, Aetiology and Harm Randomized Controlled trials (RCTs) Meta-analysis of RCTs Two interventions Centre for Evidence Based Medicine, University of Oxford Cohort studies, Case-control studies • Backbone: meta-analysis • Rigorous statistical models • Clinical practice guidelines – NICE, WHO, The Cochrane Collaboration, HuGENet
  • 5.
    Fluoxetine: 28€ Venlafaxine:111€Sertaline: 76 € “Although Mirtazapine is likely to have a faster onset of action than Sertraline and Paroxetine no significant differences were observed...” “Venlafaxine tends to have a favorable trend in response rates compared with duloxetine” “…statistically significant differences in terms of efficacy …. between Fluoxetine and Venlafaxine, but the clinical meaning of these differences is uncertain…” “…meta-analysis highlighted a trend in favour of Sertraline over other Fluoxetine” 12 new generation antidepressants 19 meta-analyses published in the last two years
  • 6.
  • 7.
    paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapineduloxetine escitalopram milnacipran bupropion paroxetine milnacipran escitalopram fluvoxamine paroxetine 0% sertraline 7% citalopram0% escitalopram 26% fluoxetine 0% fluvoxamine 0% milnacipran 1% venlafaxine 11% reboxetine 0% bupropion 0% mirtazapine 54% duloxetine 0% sertraline duloxetine? Probability to be the best 12 new generation antidepressants 19 meta-analyses published in the last two years
  • 8.
    paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapineduloxetine escitalopram milnacipran bupropion paroxetine milnacipran escitalopram fluvoxamine paroxetine 0% sertraline 7% citalopram0% escitalopram 26% fluoxetine 0% fluvoxamine 0% milnacipran 1% venlafaxine 11% reboxetine 0% bupropion 0% mirtazapine 54% duloxetine 0% sertraline duloxetine? Probability to be the best Current meta-analysis misses data! 12 new generation antidepressants 19 meta-analyses published in the last two years
  • 9.
    A new methodologicalframework Two interventions Randomized Controlled trials (RCTs) Meta-analysis of RCTs Cohort studies, Case-control studies
  • 10.
    Two interventions Multiple-treatments meta-analysis Manydifferent intervention Randomized Controlled trials (RCTs) Meta-analysis of RCTs Cohort studies, Case-control studies A new methodological framework
  • 11.
  • 12.
    sertraline citalopram fluoxetine Lancet 2009 Cipriani,Fukurawa, Salanti et al Network of experimental comparisons
  • 13.
    Indirect comparison A BC • We can obtain an indirect estimate for A vs B from RCTs comparing A vs C and B vs C: MDAB = MDAC – MDBC Var(MDAB) = Var(MDAC) + Var(MDBC)
  • 14.
  • 15.
  • 16.
    Simple exercise: preventedmean caries Comparison MD CIs Placebo vs Toothpaste -0.34 (-0.41, -0.28) Placebo vs Gel -0.19 (-0.30, -0.10) How to compare Gel to Toothpaste? Estimate indirect MD and a 95% CI Toothpaste Gel Placebo
  • 17.
    Flash back tostats… Each estimate has uncertainty as conveyed by the variance, the standard error and the 95% CI Variance=SE2 95% CI (Low CI, High CI): x-1.96·SE to x+1.96·SE : SE=(High CI – Low CI)/3.92
  • 18.
    Pen and paper(and calculator!) exercise! Indirect MDGvsT= MDPvsT – MDPvsG Indirect MDGvsT = -0.34 – (-0.19)= -0.15 Variance Indirect MDGvsT = Variance MDPvsT + Variance MDPvsG Variance MDPvsT = ((high CI –low CI)/3.92)2 Variance MDPvsT= ((-0.28– (-0.41))/3.92)2 =0.0011 Variance MDGvsT= ((-0.10– (-0.30))/3.92)2 =0.0026 Variance Indirect MDGvsT = 0.0011+0.0026=0.0037 SE Indirect MDGvsT = sqrt(0.0037)=0.061 95% CI for Indirect MDGvsT = (-0.15 – 1.96·0.061, -0.15 + 1.96·0.061) 95% CI for Indirect MDGvsT = (-0.27, -0.03)
  • 19.
  • 20.
    Combining direct andindirect evidence • Inverse variance method • Each estimate is ‘weighted’ by the inverse of the variance • Then a common (pooled) result is obtained! IndirectDirect Indirect Indirect Direct Direct MDMD var 1 var 1 var 1 var 1 MDpooled   
  • 21.
    You can dothis with any measure... lnOR, lnRR, RD, mean difference, HR, Peto’s lnOR etc… Indirect MDGvsT = - 0.15 Variance Indirect MDGvsT = 0.0037 Direct MDGvsT = 0.04 Variance Direct MDGvsT = 0.011 Pooled MDGvsT= -0.14 037.0 1 011.0 1 15.0 0037.0 1 04.0 011.0 1 MDpooled    
  • 22.
    sertraline citalopram fluoxetine Lancet 2009 Cipriani,Fukurawa, Salanti et al Network of experimental comparisons LORSF v1 LORCF v2 Indirect estimation LORSC = LORSF - LORCF Var(LORSC) = v1+ v2 LORSC Var(LORSC)
  • 23.
    sertraline citalopram fluoxetine Lancet 2009 Cipriani,Fukurawa, Salanti et al Network of experimental comparisons LORSF v1 LORFC v2 LORSC Var(LORSC) Combine the direct estimate with the indirect estimate using IV methods Get a combined LOR! v4<v3 LORSF v4 Combined LORSC v3 Indirect estimation LORSC = LORSF - LORCF Var(LORSC) = v1+ v2
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
    Advantages of MTM –Ranking of many treatments for the same condition (see later) – Comprehensive use of all available data (indirect evidence) – Comparison of interventions which haven’t been directly compared in any experiment
  • 34.
    Bevacizumab Fluorouracil and leucovorin Fluorouraciland leucovorin+bevacizumab Fluorouracil and leucovorin+irinotecan Fluorouracil and leucovorin+ irinotecan+bevacizumab Fluorouracil and leucovorin+irinotecan +oxaliplatinFluorouracil+leucovorin+oxaliplatin Fluorouracil and leucovorin + oxaliplatin + bevacizumab Irinotecan Irinotecan + oxaliplatin Oxaliplatin Colorectal Cancer Golfinopoulos V, Salanti G, Pavlidis N, Ioannidis JP: Survival and disease-progression benefits with treatment regimens for advanced colorectal cancer: a meta-analysis. Lancet Oncol 2007, 8: 898-911.
  • 35.
    Advantages of MTM –Ranking of many treatments for the same condition (see later) – Comprehensive use of all available data (indirect evidence) – Comparison of interventions which haven’t been directly compared in any experiment – Improved precision for each comparison
  • 36.
    paroxetine sertraline citalopram fluoxetine fluvoxamine milnacipran venlafaxine reboxetine bupropion mirtazapine duloxetine escitalopram Lancet 2009 Cipriani,Fukurawa, Salanti et al Network of experimental comparisons Fluoxetine vs Milnacipran (response to treatment) Meta-analysis: 1.15 (0.72, 1.85) MTM: 0.97 (0.69, 1.32)
  • 37.
    Why use Bayesianstatistics for meta-analysis? • Natural approach for accumulating data • Repeated updating of meta-analyses fine: posterior should always reflect latest beliefs • People naturally think as Bayesians: they have degrees of belief about the effects of treatment, which change when they see new data • Probability statements about true effects of treatment easier to understand than confidence intervals and p -values
  • 38.
    Why use Bayesianstatistics for MTM? • Bayesian approach is easier to account for correlations induced by multi-arm trials • Estimation of predictive intervals is straightforward • Estimation of ranking probabilities is straightforward • MTM with two-arm trials only (or ignoring the correlations) Easy with frequentist meta-regression
  • 39.
    Fixed effect meta-analysis Trial 1 2 3 4 5 6 7 8 9 10 11 12 Treatmentbetter Control better Effect estimate -1 0 1 random error common (fixed) effect μ yi i
  • 40.
    Random effects meta-analysis study-specificeffect distribution of effects Trial 1 2 3 4 5 6 7 8 9 10 11 12 Treatment better Control better Effect estimate -1 0 1 random error τμ θi yi i
  • 41.
    Random effects meta-regression Explanatory variable,x yi = intersept + slope x Random error Τmeta-regmeta-reg i
  • 42.
    • We observeyi in each study (e.g. the log(OR)) • Meta-regression using the treatments as ‘covariates’ • AC, AB, BC studies, chose C as reference Meta-regression • The AC studies have (1,0), the BC studies (0,1) [basic] • AB studies have (1,-1) [functional] yi = C (Treati=A) + BC (Treati=B)
  • 43.
    Parametrisation of thenetwork t-PA Angioplasty Acc t-PA Anistreplase Retaplase Streptokinase Choose basic parameters Write all other contrasts as linear functions of the basic parameters to build the design matrix LOR for death in treatments for MI
  • 44.
    LOR for deathin treatments for MI No. studies Streptokinase t-PA Anistreplase Acc t-PA Angioplasty Reteplase 3 1 1 3 1 1 2 2 2 Use as ‘covariates’ yi= μA t-PA  μB Anistreplasei μC Accelerated t-PAi  μD Angioplastyi  μE Reteplasei -1 1 0 0 0 0 -1 0 1 0 0 0 -1 0 0 1 0 0 -1 0 0 0 1 0 -1 0 0 0 0 1 -1 1 0 0 0 -1 0 0 1 0 0 0 -1 1 0 0 0 -1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Lumley 2002, Stat Med
  • 45.
     X),,,,(Y EDCBA  LORfor death in treatments for MI Matrix of all observations Vector of LogOR yi= μA t-PA  μB Anistreplasei μC Accelerated t-PAi  μD Angioplastyi  μE Reteplasei Design matrix Random effects matrix )V,X(N~Y μ ))τ(diag,(N~ 2 ∆ 0 Variance-covariance matrix (for the observed LOR)
  • 46.
    LOR compared toStreptokinase (RE model) - 0.11 (0.06) - 0.43 (0.20) - 0.15 (0.05) -0.00 (0.03) -0.02 (0.03) LOR (SE)Treatment t-PA Anistreplase Accelerated t-PA Angioplasty Reteplase  X),,,,(Y EDCBA  1.01)(0.80,0.90 0.96)(0.44,0.65 0.95)(0.78,0.86 1.06)(0.94,1.00 1.04)(0.92,0.98 95% CIOR
  • 47.
    What’s the problemwith multi-arm trials? • We need to take into account the correlations between the estimates that come from the same study • A B C yi BC yi AC • The random effects (θi BC, θi AC) that refer to the same trial are correlated as well • You have to built in the correlation matrix for the observed effects, and the correlation matrix for the random effects )V,X(N~Y μ ))τ(diag,(N~ 2 ∆ 0
  • 48.
    Study No. arms# Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC Hypothetical example Basic parameters: AB and AC
  • 49.
    1,1 1,11,1 2,1 2,12,1 3,13,13,1 4,1 4,14,1 4,2 4,2 4,2 1 0 0 1 1 1 1 0 0 1                                                                           AB AC y y y y y Meta-regression Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC
  • 50.
    1,1 1,11,1 2,1 2,12,1 3,13,13,1 4,1 4,14,1 4,2 4,2 4,2 1 0 0 1 1 1 1 0 0 1                                                                           AB AC y y y y y Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC Take into account correlation in observations     1,11,1 2,12,1 3,13,1 4,1 4,1 4,24,1 4,2 4,1 4,2 4,2 0 0 0 00 0 0 0 00 0 0 0 0~ ,0 0 0 0 cov ,0 0 0 0 0 cov ,                                                      v v vN v y y y y v
  • 51.
    1,1 1,11,1 2,1 2,12,1 3,13,13,1 4,1 4,14,1 4,2 4,2 4,2 1 0 0 1 1 1 1 0 0 1                                                                           AB AC y y y y y Study No. arms # Data Contrast i=1 T1=2 1 y1,1, v1,1 AB i=2 T2=2 1 y2,1, v2,1 AC i=3 T3=2 1 y3,1, v3,1 BC i=4 T4=3 2 y4,1, v4,1 y4,2, v4,2 cov(y4,1, y4,2) AB AC Take into account correlation in random effects     2 1,1 2 2,1 2 3,1 2 4,1 4,24,1 2 4,2 4,1 4,2 0 0 0 00 0 0 0 00 0 0 0 0~ ,0 0 0 0 cov ,0 0 0 0 0 cov ,                                                            AB AC BC AB AC N
  • 52.
    How to fitsuch a model? • MLwiN • SAS, R • STATA using metan
  • 53.
    Ranking measures fromMTM • With many treatments judgments based on pairwise effect sizes are difficult to make • Example: Antidepressants
  • 55.
    Ranking measures fromMTM • With many treatments judgments based on pairwise effect sizes are difficult to make • Example: Antidepressants • Example: Antiplatelet regimens for serious vascular events
  • 56.
    Aspirin vs Placebo Thienopyridinesvs Aspirin Thienopyridines vs Placebo 0.5 1 1.5 2 0.32 0.03 <0.01 Aspirin+Dipyridamole vs Aspirin+Thienopyridines Aspirin+Dipyridamole vs Aspirin Aspirin+Dipyridamole vs Placebo Aspirin+Dipyridamole vs Thienopyridines 0 Aspirin+Thienopyridines vs Aspirin Aspirin+Thienopyridines vs Placebo Aspirin+Thienopyridines vs Thienopyridines0.23 0.05 <0.01 0.19 <0.01 <0.01 P-value Comparison Odds Ratio for serious vascular event Favors first treatment Favors second treatment Serious vascular events with antiplatelet regimens
  • 57.
    Probabilities instead ofeffect sizes • Estimate for each treatment the probability to be the best • This is straightforward within a Bayesian framework
  • 58.
    % probability A B CD j=1 0.25 0.50 0.25 0.00 j=2 0.50 0.75 0.75 0.00 j=3 0.75 1.00 1.00 0.25 j=4 1.00 1.00 1.00 1.00
  • 59.
    % probability A B CD j=1 0.25 0.50 0.25 0.00 j=2 0.25 0.25 0.50 0.00 j=3 0.25 0.25 0.25 0.25 j=4 0.25 0 0 0.75 i the treatment j the rank
  • 60.
    Rank of paroxetine Probability 24 6 8 10 12 0.00.20.40.6 Rank of sertraline 2 4 6 8 10 12 0.00.20.40.6 Rank of citalopram 2 4 6 8 10 12 0.00.20.40.6 Rank of escitalopram 2 4 6 8 10 12 0.00.20.40.6 Rank of fluoxetine Probability 2 4 6 8 10 12 0.00.20.40.6 Rank of fluvoxamine 2 4 6 8 10 12 0.00.20.40.6 Rank of milnacipran 2 4 6 8 10 12 0.00.20.40.6 Rank of venlafaxine 2 4 6 8 10 12 0.00.20.40.6 Rank of reboxetine Probability 2 4 6 8 10 12 0.00.20.40.6 Rank of bupropion 2 4 6 8 10 12 0.00.20.40.6 Rank of mirtazapine 2 4 6 8 10 12 0.00.20.40.6 Rank of duloxetine 2 4 6 8 10 12 0.00.20.40.6 Ranking for efficacy (solid line) and acceptability (dotted line). Ranking: probability to be the best treatment, to be the second best, the third best and so on, among the 12 comparisons).
  • 61.
    % probability A B CD j=1 0.25 0.50 0.25 0.00 j=2 0.50 0.75 0.75 0.00 j=3 0.75 1.00 1.00 0.25 j=4 1.00 1.00 1.00 1.00 The areas under the cumulative curves for the four treatments of the example above are A=0.5 B=0.75 C=0.67 D=0.08 i the treatment j the rank Rank of A CumulativeProbability 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0 Rank of B 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0 Rank of C CumulativeProbability 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0 Rank of D 1.0 1.5 2.0 2.5 3.0 3.5 4.0 0.00.20.40.60.81.0
  • 62.
    Preliminary results forranking 12 antidepressants Rank of paroxetine CumulativeProbability 2 4 6 8 10 12 Rank of sertraline 2 4 6 8 10 12 Rank of reboxetine CumulativeProbability 2 4 6 8 10 12 020406080100 Rank of mirtazapine 2 4 6 8 10 12 35% 77% 1% 92% 020406080100 020406080100020406080100 A comprehensive ranking measure Compared to an imaginary antidepressant which is ‘always the best’, mirtazapine reaches up to 92% of its potential!
  • 64.
    • What isinconsistency? • How it manifests itself? Inconsistency
  • 65.
    Inconsistency Direct t-PA vsAngioplasty=  0.41 (0.36) 0.02 (0.03) - 0.48 (0.43) Calculate a difference between direct and indirect estimates t-PA Angioplasty Streptokinase LOR (SE) for MI Indirect t-PA vs Angioplasty =  0.46 (0.18) Inconsistency in the loop = 0.05
  • 66.
    Inconsistency - Heterogeneity •Heterogeneity: ‘excessive’ discrepancy among study-specific effects • Inconsistency: it is the excessive discrepancy among source-specific effects (direct and indirect)
  • 67.
    14 15 1617 AC=15.5 7 8 9 10 BC=8.2 1 2 3 4 AB=2.3
  • 68.
    • In 3cases out of 44 there was an important discrepancy between direct/indirect effect. • Direction of the discrepancy is inconsistent Glenny et al HTA 2005 Inconsistency Empirical Evidence
  • 69.
    Placebo Toothpaste Gel Direct SMD(TvsG) =0.04 Indirect SMD(TvsG) = – 0.15 IF= 0.11 P-Gel P-Toothpaste I cannot learn about Toothpaste versus Gel through Placebo! Compare Fluoride treatments in preventing dental caries What can cause inconsistency? Inappropriate common comparator
  • 70.
    Screening for lungcancer Baker & Kramer, BMC Meth 2002 Chest X-ray Standard Spiral CT A new therapy (possibly unreported in the trials) decreases the mortality but in different rates for the three screening methods Percent receiving new therapy Mortality 30 70 Trial 1: Chest X-ray= Standard Trial 2: Spiral-CT= Standard New Trial: Spiral-CT < Chest X-ray 100 What can cause inconsistency? Confounding by trial characteristics
  • 71.
    age Effectiveness Alfacalcidol +Ca Calcitriol +Ca Ca What can cause inconsistency? Confounding by trial characteristics Vitamin D for Osteoporosis-related fractures, Richy et al Calcif Tissue 2005, 76;276 Vitamin D +Ca Different characteristics across comparisons may cause inconsistency
  • 72.
    • There isnot confounding by trial characteristics that are related to both the comparison being made and the magnitude of treatment difference • The trials in two different comparisons are exchangeable (other than interventions being compared) • Equivalent to the assumption ‘the unobserved treatment is missing at random’ – Is this plausible? – Selection of the comparator is not often random! Assumptions of MTM
  • 73.
    • Check thedistribution of important characteristics per treatment comparison – Usually unobserved…. – Time (of randomization, of recruitment) might be associated with changes to the background risk that may violate the assumptions of MTM • Get a taste by looking for inconsistency in closed loops Inconsistency Detecting
  • 74.
    No. studies TG R V P Fup Baseline Year Water F (yes/no) 69 2.6 11.8 1968 0.2 13 2.3 3.8 1973 0.2 30 2.4 5.9 1973 0.1 3 2.3 2.7 1983 0 3 2.7 NA 1968 0.66 6 2.8 14.7 1969 0 1 2 0.9 1978 0 1 1 NA 1977 0 1 3 7.4 1991 NA 4 2.5 7.6 1981 0.33 Compare the characteristics! Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62: 857-864.
  • 75.
    • Check thedistribution of important characteristics per treatment comparison – Usually unobserved…. – Time (of randomization, of recruitment) might be associated with changes to the background risk that may violate the assumptions of MTM • Get a taste by looking for inconsistency in closed loops • Fit a model that relaxes consistency – Add an extra ‘random effect’ per loop (Lu & Ades JASA 2005) Inconsistency Detecting
  • 76.
  • 77.
    -1.0 0.0 0.51.0 1.5 2.0 Closed loops NGV NGR NRV PDG PDV PDR DGV DGR DRV PGV PGR PRV GRV AGRV PDGV PDGR PDRV DGRV PGRV PDGRV Evaluation of concordance within closed loops Estimates with 95% confidence intervals R routine in http://www.dhe.med.uoi.gr/software.htm Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62: 857-864.
  • 78.
    • Check thedistribution of important characteristics per treatment comparison – Usually unobserved…. – Time (of randomization, of recruitment) might be associated with changes to the background risk that may violate the assumptions of MTM • Get a taste by looking for inconsistency in closed loops • Fit a model that relaxes consistency – Add an extra ‘random effect’ per loop (Lu & Ades JASA 2005) Inconsistency Detecting
  • 79.
    Inconsistency - Heterogeneity RCTs Meta-analysisof RCTs With homogeneity2 interventions With consistencyMultiple meta-analyses of RCTsbest intervention
  • 80.
    References 1. Baker SG,Kramer BS: The transitive fallacy for randomized trials: if A bests B and B bests C in separate trials, is A better than C? BMC Med Res Methodol 2002, 2: 13. 2. Caldwell DM, Ades AE, Higgins JP: Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005, 331: 897-900. 3. Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JP, Churchill R et al.: Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 2009, 373: 746-758. 4. Cooper NJ, Sutton AJ, Lu G, Khunti K: Mixed comparison of stroke prevention treatments in individuals with nonrheumatic atrial fibrillation. Arch Intern Med 2006, 166: 1269-1275. 5. Golfinopoulos V, Salanti G, Pavlidis N, Ioannidis JP: Survival and disease-progression benefits with treatment regimens for advanced colorectal cancer: a meta-analysis. Lancet Oncol 2007, 8: 898-911. 6. Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S: Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second- generation antipsychotics. Am J Psychiatry 2006, 163: 185-194. 7. Jansen JP, Crawford B, Bergman G, Stam W: Bayesian Meta-Analysis of Multiple Treatment Comparisons: An Introduction to Mixed Treatment Comparisons. Value Health 2008. 8. Lu G, Ades AE: Assessing Evidence Inconsistency in Mixed Treatment Comparisons. Journal of American Statistical Association 2006, 101: 447-459. 9. Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004, 23: 3105- 3124. 10. Salanti G, Higgins JP, Ades AE, Ioannidis JP: Evaluation of networks of randomized trials. Stat Methods Med Res 2008, 17: 279-301. 11. Salanti G, Marinho V, Higgins JP: A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J Clin Epidemiol 2009, 62: 857-864. 12. Song F, Harvey I, Lilford R: Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol 2008, 61: 455-463. 13. Sutton A, Ades AE, Cooper N, Abrams K: Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 2008, 26: 753-767. 14. Welton NJ, Cooper NJ, Ades AE, Lu G, Sutton AJ: Mixed treatment comparison with multiple outcomes reported inconsistently across trials: Evaluation of antivirals for treatment of influenza A and B. Stat Med 2008, 29: 5620-5639.