2. Why I Should Not Be Allowed to Fly
http://xkcd.com/651/
3. Objectives
• Explore superiority, equivalence, and
non-inferiority (NI) trial designs
• Compare critical differences between
superiority and NI trials and stumbling
blocks involving issues such as the
non-inferiority margin
• Examine potential issues with non-inferiority
studies: study design, scientific, statistical,
and regulatory
4. If I asked you to stand up and
define randomization, assay
sensitivity, and null hypothesis, are
you confident you could do so?
1. Yes
2. No
8. Null versus Alternative
Hypotheses
• H0 – no difference in
treatments
• H1 – there is a
difference in treatments
Errors
• Type I error (false )
• Type II error (false )
9. FDA. E 10 Choice of Control Group and
Related Issues in Clinical Trials, 2001.
Assay Sensitivity
Property of a clinical trial that can
distinguish an effective treatment from a
less effective or ineffective treatment
10. Superiority Trials
Purpose
To detect a difference between two drugs
Goals
Establish new drug is statistically superior
to active control (and/or placebo) [Easier]
Establish new drug is clinically superior to
active control (and/or placebo) [Harder]
11. Equivalence
Trials
Purpose:
To confirm
the absence of a
meaningful
difference between
treatments
12. Equivalence
Trials
Equivalence is
inferred when
ENTIRE confidence
interval falls
exclusively within
equivalence margins
(between –Δ and Δ)
13. What is the PURPOSE of a
non-inferiority trial?
20%
1. To show the new drug is as good as
comparator
20% 2. To show the new drug can be better
than comparator
20% 3. To show the new drug is not
unacceptably worse than comparator
20% 4. To show the new drug is clinically
equivalent to comparator
20% 5. To show the new drug performs no
better or no worse than comparator
14. Non-inferiority (NI) Trials
Purpose: To demonstrate that a new
drug is not worse than an
active comparator by more
than a pre-specified amount*
NOT: That the two drugs are equivalent
NOT: That the new drug isn’t inferior to the
active comparator
*non-inferiority margin, delta (Δ),
15. from Certain Clinical Trials, July 2010.
GAO. FDAs Consideration of Evidence
What Factors Are Prompting
Increased Scrutiny of NI Trials?
1. Regulatory (volume) 2. Scientific 3. Political
17. Scientific Factors
Two most common reasons FDA failed to
approve sponsors NDAs with NI evidence:
1. Poor selection of active control
2. Unclear determination of non-inferiority
margin
20. What is the biggest factor in
navigating NI research?
1. Regulatory
2. Scientific
3. Political
4. They are tied too
closely together to
separate
21. Balance Needs in NI Study
1. Determine historical
evidence of sensitivity to
drug effects exists
(constancy assumption)
2. Design a trial
3. Set a non-inferiority margin
FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001.
22. Historical Evidence of Sensitivity
Similarly designed trials in the past regularly
distinguished effective treatments from less
effective or ineffective treatments
Barring this, demonstration of efficacy from
a showing of non-inferiority is not possible
FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001.
23. Designing an NI Trial
Non-inferiority trial should
mimic design of superiority trial
Primary variables, severity of
condition, concomitant illnesses,
method of diagnosis, etc.
MJA 2009;190(6):326-330.
24. Active comparator (drug)
should ideally be widely used
with established efficacy via
superiority trial and identical
indication & conditions
Biometric Journal 2009;51(1):185-92.
25. Structure of NI
Trial Design
Ideally
incorporates a
placebo AND active
comparator
Validate/establish
internal validity
26. Data Analysis
Use of BOTH intent-to-treat (ITT) and
per protocol (PP) methods of data
analysis should be employed
Guidance used to suggest PP analysis
for NI due to dilution of effect by ITT
Assumptions since proven faulty
Stat Med 2005;24(1):1-10.
27. Remember for NI Margin…
Defined effect size for (active) control
from previous trials
―Comprehensive synthesis‖ of
evidence justifying effect size of active
control (and thus the proposed NI
margin) necessary during protocol
development and submission to FDA
PLoS ONE 2010;5(10):e13550.
28. Remember for NI Margin…
Adequate support must be
provided to FDA (21 CFR 314.126)
FDA strongly prefers clinical
endpoint over surrogate endpoint
when possible
PLoS ONE 2010;5(10):e13550.
30. Inference for Non-Inferiority
Delta Limits and Confidence Intervals (95%)
Non-inferiority shown
Non-inferiority shown
Non-inferiority not shown
Non-inferiority shown
- 0
NI Margin
Treatment Difference
31. H0=null hypothesis. Ha = alternative hypothesis; Δ =difference in event rates between new and standard treatments; S=Δ in
superiority trials. E=Δ in equivalence trials; NI =Δ in non-inferiority trials
*Testing for non-inferiority is in one direction only — even if superiority exists (dashed arrow), it is not the hypothesis being tested.
MJA 2009;190(6):326-330.
32. What is the most commonly used
NI margin in anti-infective studies?
1. 0.5%
2. 1%
3. 5%
4. 10%
5. 20%
6. 25%
34. Good Enough?
Review of 8 NI trials
from JAMA & NEJM
6 of 8 trials gave detail on
justifications of NI
margin
6 of 8 concluded NI
However, analysis by
Kaul & Diamond only
concluded NI in 4 of 8
trials
Ann Intern Med 2006;145(1):62-69.
35. Wangge et al. review showed less than
half (45.7%) of 232 NI trials reported
method used to generate NI margin
Less than 5.0% reported on similarity
of current trial with comparator trials
No overall difference pre & post release
of CONSORT (2006); method of NI
margin actually reported less frequently
PLoS ONE 2010;5(10):e13550.
36. There were 53 trials
(22.9% of total N) of
anti-infective drugs
Most (77.8%) anti-infective
trials used an NI margin of
percentage difference
between 10 to 20%.
PLoS ONE 2010;5(10):e13550.
38. Superiority
Investigator with bias of
new drug’s superiority
cannot directly
manipulate results to
support belief
Non-inferiority
Investigator in NI trial
can bias results by
assigning similar ratings
regardless of patient
response/arm
PLoS ONE 2010;5(10):e13550.
41. What is Biocreep?
20%
1. Inability to demonstrate
bioequivalence
20% 2. Slow crawl towards finding of non-
inferiority with ITT analysis
20% 3. Temporal reduction of efficacy of
drugs using NI trials as evidence
20% 4. Process of insertion of biologics into
drug pipeline
20% 5. The name of the villain from the
Hellraiser movies
43. Food and Drug Administration
Guidance Origin (FDA), Center for Drug
Evaluation and Research
(CDER), Division of Anti-
Infective and Ophthalmology
Products and the Division of
Special Pathogen and
Transplant Products, Office of
Antimicrobial Products
Anti-Infective Drugs Advisory
Committee (sponsored by FDA)
44. However, there is not a FDA consensus
for all infectious disease indications
45. Non-inferiority,
Bugs, and Drugs
Acute bacterial
sinusitis (ABS)
Acute bacterial
exacerbation of
chronic bronchitis
(ABECB)
Acute bacterial
otitis media
(ABOM)
CDER. Antibacterial Drug Products: Use of Noninferiority
Trials to Support Approval , November 2010.
47. Predicated upon reviews of data for ABS,
ABECB, and ABOM along with Anti-Infective
Drugs Advisory Committee findings
FDA unable to determine recommended
NI margin for those three areas
Issue clearer for nosocomial/more serious
infectious disease indications
FDA Guidance on Anti-infectives
49. If this had been an elevator prep
(not pitch)
• Assay sensitivity and historical evidence is
crucial for determination of NI feasibility
• Sponsor NI margin selection is contingent
upon multiple factors in eyes of FDA
• Let guidance by FDA, other bodies, and
CONSORT help inform protocol
development on front end