Superiority, non-inferiority, equivalence studies - what is the difference?

43,199 views

Published on

Published in: Health & Medicine, Business
2 Comments
11 Likes
Statistics
Notes
No Downloads
Views
Total views
43,199
On SlideShare
0
From Embeds
0
Number of Embeds
48
Actions
Shares
0
Downloads
621
Comments
2
Likes
11
Embeds 0
No embeds

No notes for slide
  • For example: The null hypothis becomes the alternative hypothesisType I error becomes type II error and vice versaITT is suddenly not as good as per protocol analysesThere is some backwardness that can be very confusing – so I tried to come up with a simple analogy to capture the basic concepts of superiority-non-inferiority –and equivalence and I ended up with soccer:
  • Superiority, non-inferiority, equivalence studies - what is the difference?

    1. 1. Superiority-Equivalence-Non-Inferiority Trials: What does it all mean? Gerald Gartlehner
    2. 2. What decision-makers want to know Is the new treatment better than the established one ? If not, is it equally effective and preferable for some other reasonÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    3. 3. Overview  Definitions and concepts  Challenges of non-inferiority trials  What does this mean for systematic reviews?  Language considerations for comparative effectiveness reviewsÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    4. 4. Proving Efficacy 1. Showing superiority of one (new) treatment over another (placebo or active) 2. Showing equivalence or non- inferiority of a new intervention relative to an already existing efficacious treatmentÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    5. 5. The interpretation of non-inferiority and equivalence can be confusingÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    6. 6. Ranking position: 72Ranking position: 34
    7. 7. Gerald GartlehnerÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.atDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    8. 8. Superiority game: the winner takes it all….Österreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    9. 9. Equivalence game – equally good or clearly better  Equivalence margin: tied or less than 1 goal differenceÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    10. 10. Non-inferiority game– at least not substantially worse….  Non-inferiority margin: can‘t lose with more than 1 goal difference Gerald GartlehnerÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.atDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    11. 11. Definitions Superiority trial Objective: To determine a clinically relevant difference between two interventions Equivalence trial Objective: To determine whether a (new) intervention is neither worse nor better than another (established) intervention Non-inferiority trial Objective: To determine whether a (new) intervention is not inferior to another (established) interventionÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    12. 12. Equivalence - Non-Inferiority The naïve approach:  If a head-to-head trial shows no statistically significant difference, two interventions are “equivalent”  Problem – underpowered studies or high variance will create “equivalent” treatments The statistical approach:  Define a margin of non-inferiority or equivalence  If 95% confidence interval of the difference DOES NOT cross the margin, the new intervention is non-inferior or equivalentÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    13. 13. Arthroscopy vs. sham arthoscopy in patients with knee osteoarthritis favors sham favors arthroscopy MID 20 10 0 10 20 Knee Specific Pain Scale: difference in points Mosley et al. New England J Med, 2002;347:81-88Österreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    14. 14. Non-inferiority margin (d) The limit of acceptable inferiority: Minimal important difference Clinical judgement Statistical considerationsÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    15. 15. Determining the non-inferiority margin statisticallyStandard vs. placebo2 points (1.5 to 2.5)New vs. standard 2 1 0 Maximimum non-inferiority margin: dmax = 1.5 Fractional preservation of treatment effect: f = 0.5 d = dmax x (1 – f) d = 1.5 x ( 1 – 0.5) = 0.75
    16. 16. Peculiar issues of non-inferiority trials: the backwardnessNull hypothesis and alternative hypothesis are reversedType I and type II errors are reversedPer-protocol analyses can be more important than ITT analysis (ITT analyses are biased towards finding no difference)P-value is one-sided (0.025)
    17. 17. Assay Sensitivity and Constancy Assumption The ability of a trial to distinguish effective from ineffective treatments (depends on the effect size the trial wants to detect).Österreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    18. 18. Assay SensitivityAssumptions in non-inferiority trials: The efficacy of the active control was preserved in the non-inferiority study (i.e. that it had assay sensitivity). If it was not, equivalence or non-inferiority conclusions are meaningless (The non-inferior drug could have no effect at all).
    19. 19. Constancy Assumption Active comparator must be well established and have predictable and consistent treatment effects Participants must be similar to those in trial establishing efficacy Outcomes must be similar to those in trials establishing efficacy
    20. 20. Biocreep Effective treatment Still clinically relevant? 2 1 0Österreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    21. 21. Key Points for critical appraisalWas a non-inferiority margin defined based on clinical considerations and statistical reasoning?Was it established a priori?Was the study powered based on the non-inferiority margin?Was an ITT and a per-protocol analysis conducted?Was the trial design (e.g. eligibility criteria) consistent with placebo controlled trials of the established treatment?
    22. 22. What does this mean for systematic reviews ? For meta-analyses – data can be used just as from any superiority trial For qualitative assessments – language considerationsÖsterreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    23. 23. Language considerations “..there was no statistically significant difference between A and B..” “..studies failed to show a difference..” Can mean: 1) The evidence shows equivalence 2) The evidence is inconclusive (because confidence intervals are wide-lack of precision) AHRQ guidance: Assessing equivalence and non-inferiority [draft]Österreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    24. 24. Language considerations Better: “..treatments A and B had similar mortality rates..” Expressing non-inferiority: “…treatment A is at least as effective as treatment B for [Outcome or study objective]…” AHRQ guidance: Assessing equivalence and non-inferiority [draft]Österreichische Cochrane Zweigstelle (ÖCZ) ∙ www.cochrane.at Gerald GartlehnerDepartment für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
    25. 25. Do we need to establish equivalence margins for CERs ?

    ×