Superiority Trials Versus Non-Inferiority Trials to Demonstrate Effectiveness (3rd Anti-Infectives Summit)
Superiority Trials VersusNon-Inferiority Trials to Demonstrate Effectiveness Kevin A. Clauson, PharmD 3rd Anti-Infectives Summit
Why I Should Not Be Allowed to Fly http://xkcd.com/651/
Objectives• Explore superiority, equivalence, and non-inferiority (NI) trial designs• Compare critical differences between superiority and NI trials and stumbling blocks involving issues such as the non-inferiority margin• Examine potential issues with non-inferiority studies: study design, scientific, statistical, and regulatory
If I asked you to stand up and define randomization, assaysensitivity, and null hypothesis, are you confident you could do so?1. Yes2. No
RandomizationAllocation method where all subjects have equal chance of study group assignment
Null versus AlternativeHypotheses• H0 – no difference in treatments• H1 – there is a difference in treatmentsErrors• Type I error (false )• Type II error (false )
FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001. Assay Sensitivity Property of a clinical trial that candistinguish an effective treatment from a less effective or ineffective treatment
Superiority TrialsPurpose To detect a difference between two drugsGoals Establish new drug is statistically superior to active control (and/or placebo) [Easier] Establish new drug is clinically superior to active control (and/or placebo) [Harder]
Equivalence Trials Purpose: To confirm the absence of a meaningfuldifference between treatments
Equivalence Trials Equivalence is inferred whenENTIRE confidence interval falls exclusively withinequivalence margins(between –Δ and Δ)
What is the PURPOSE of a non-inferiority trial?20% 1. To show the new drug is as good as comparator20% 2. To show the new drug can be better than comparator20% 3. To show the new drug is not unacceptably worse than comparator20% 4. To show the new drug is clinically equivalent to comparator20% 5. To show the new drug performs no better or no worse than comparator
Non-inferiority (NI) TrialsPurpose: To demonstrate that a new drug is not worse than an active comparator by more than a pre-specified amount*NOT: That the two drugs are equivalentNOT: That the new drug isn’t inferior to the active comparator *non-inferiority margin, delta (Δ),
from Certain Clinical Trials, July 2010. GAO. FDAs Consideration of Evidence What Factors Are Prompting Increased Scrutiny of NI Trials?1. Regulatory (volume) 2. Scientific 3. Political
Regulatory Factors (Volume)Source: GAO. FDAs Consideration of Evidence from Certain Clinical Trials, July 2010.
Scientific FactorsTwo most common reasons FDA failed toapprove sponsors NDAs with NI evidence: 1. Poor selection of active control 2. Unclear determination of non-inferiority margin
Clinicianunderstanding of non-inferioritypurpose, margin, and issues
What is the biggest factor in navigating NI research?1. Regulatory2. Scientific3. Political4. They are tied too closely together to separate
Balance Needs in NI Study 1. Determine historical evidence of sensitivity to drug effects exists (constancy assumption) 2. Design a trial 3. Set a non-inferiority margin FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001.
Historical Evidence of SensitivitySimilarly designed trials in the past regularlydistinguished effective treatments from less effective or ineffective treatmentsBarring this, demonstration of efficacy froma showing of non-inferiority is not possible FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001.
Designing an NI Trial Non-inferiority trial shouldmimic design of superiority trial Primary variables, severity ofcondition, concomitant illnesses, method of diagnosis, etc. MJA 2009;190(6):326-330.
Active comparator (drug)should ideally be widely used with established efficacy viasuperiority trial and identical indication & conditions Biometric Journal 2009;51(1):185-92.
Structure of NI Trial Design Ideally incorporates a placebo AND active comparator Validate/establish internal validity
Data AnalysisUse of BOTH intent-to-treat (ITT) and per protocol (PP) methods of data analysis should be employedGuidance used to suggest PP analysisfor NI due to dilution of effect by ITT Assumptions since proven faulty Stat Med 2005;24(1):1-10.
Remember for NI Margin…Defined effect size for (active) control from previous trials ―Comprehensive synthesis‖ ofevidence justifying effect size of active control (and thus the proposed NI margin) necessary during protocoldevelopment and submission to FDA PLoS ONE 2010;5(10):e13550.
Remember for NI Margin… Adequate support must beprovided to FDA (21 CFR 314.126) FDA strongly prefers clinicalendpoint over surrogate endpoint when possible PLoS ONE 2010;5(10):e13550.
MJA 2009;190(6):326-330.NI Margin must be determined by combination ofclinical considerations and statistical methods
Inference for Non-InferiorityDelta Limits and Confidence Intervals (95%) Non-inferiority shown Non-inferiority shown Non-inferiority not shown Non-inferiority shown - 0 NI Margin Treatment Difference
H0=null hypothesis. Ha = alternative hypothesis; Δ =difference in event rates between new and standard treatments; S=Δ insuperiority trials. E=Δ in equivalence trials; NI =Δ in non-inferiority trials*Testing for non-inferiority is in one direction only — even if superiority exists (dashed arrow), it is not the hypothesis being tested. MJA 2009;190(6):326-330.
What is the most commonly usedNI margin in anti-infective studies?1. 0.5%2. 1%3. 5%4. 10%5. 20%6. 25%
How did publishednon-inferiority trials do at exam time?
Good Enough? Review of 8 NI trials from JAMA & NEJM 6 of 8 trials gave detail on justifications of NI margin 6 of 8 concluded NI However, analysis by Kaul & Diamond only concluded NI in 4 of 8 trials Ann Intern Med 2006;145(1):62-69.
Wangge et al. review showed less thanhalf (45.7%) of 232 NI trials reported method used to generate NI margin Less than 5.0% reported on similarity of current trial with comparator trialsNo overall difference pre & post release of CONSORT (2006); method of NImargin actually reported less frequently PLoS ONE 2010;5(10):e13550.
There were 53 trials (22.9% of total N) of anti-infective drugsMost (77.8%) anti-infectivetrials used an NI margin of percentage difference between 10 to 20%. PLoS ONE 2010;5(10):e13550.
Blinding Superiority vsNon-inferiority PLoS ONE 2010;5(10):e13550.
SuperiorityInvestigator with bias of new drug’s superiority cannot directly manipulate results to support belief Non-inferiority Investigator in NI trial can bias results byassigning similar ratings regardless of patient response/arm PLoS ONE 2010;5(10):e13550.
What is Biocreep?20% 1. Inability to demonstrate bioequivalence20% 2. Slow crawl towards finding of non- inferiority with ITT analysis20% 3. Temporal reduction of efficacy of drugs using NI trials as evidence20% 4. Process of insertion of biologics into drug pipeline20% 5. The name of the villain from the Hellraiser movies
Food and Drug AdministrationGuidance Origin (FDA), Center for Drug Evaluation and Research (CDER), Division of Anti- Infective and Ophthalmology Products and the Division of Special Pathogen and Transplant Products, Office of Antimicrobial Products Anti-Infective Drugs Advisory Committee (sponsored by FDA)
However, there is not a FDA consensus for all infectious disease indications
Non-inferiority, Bugs, and Drugs Acute bacterial sinusitis (ABS) Acute bacterial exacerbation of chronic bronchitis (ABECB) Acute bacterial otitis media (ABOM)CDER. Antibacterial Drug Products: Use of Noninferiority Trials to Support Approval , November 2010.
Predicated upon reviews of data for ABS, ABECB, and ABOM along with Anti-Infective Drugs Advisory Committee findings FDA unable to determine recommended NI margin for those three areas Issue clearer for nosocomial/more serious infectious disease indicationsFDA Guidance on Anti-infectives
If this had been an elevator prep (not pitch)• Assay sensitivity and historical evidence is crucial for determination of NI feasibility• Sponsor NI margin selection is contingent upon multiple factors in eyes of FDA• Let guidance by FDA, other bodies, and CONSORT help inform protocol development on front end