Presentation at the 3rd Anti-Infectives Summit by IQPC in Philadelphia, PA on Superiority vs Non-Inferiority Trial Design

1. Superiority
Trials Versus
Non-Inferiority
Trials to
Demonstrate
Effectiveness
Kevin A. Clauson, PharmD
3rd Anti-Infectives Summit

2. Why I Should Not Be Allowed to Fly
http://xkcd.com/651/

3. Objectives
• Explore superiority, equivalence, and
non-inferiority (NI) trial designs
• Compare critical differences between
superiority and NI trials and stumbling
blocks involving issues such as the
non-inferiority margin
• Examine potential issues with non-inferiority
studies: study design, scientific, statistical,
and regulatory

4. If I asked you to stand up and
define randomization, assay
sensitivity, and null hypothesis, are
you confident you could do so?
1. Yes
2. No

5. Randomization
Allocation method where all subjects have
equal chance of study group assignment

6. Controls
Active
Placebo

7. Blinding Reduce risk of
observation
bias

8. Null versus Alternative
Hypotheses
• H0 – no difference in
treatments
• H1 – there is a
difference in treatments
Errors
• Type I error (false )
• Type II error (false )

9. FDA. E 10 Choice of Control Group and
Related Issues in Clinical Trials, 2001.
Assay Sensitivity
Property of a clinical trial that can
distinguish an effective treatment from a
less effective or ineffective treatment

10. Superiority Trials
Purpose
 To detect a difference between two drugs
Goals
 Establish new drug is statistically superior
to active control (and/or placebo) [Easier]
 Establish new drug is clinically superior to
active control (and/or placebo) [Harder]

11. Equivalence
Trials
Purpose:
To confirm
the absence of a
meaningful
difference between
treatments

12. Equivalence
Trials
Equivalence is
inferred when
ENTIRE confidence
interval falls
exclusively within
equivalence margins
(between –Δ and Δ)

13. What is the PURPOSE of a
non-inferiority trial?
20%
1. To show the new drug is as good as
comparator
20% 2. To show the new drug can be better
than comparator
20% 3. To show the new drug is not
unacceptably worse than comparator
20% 4. To show the new drug is clinically
equivalent to comparator
20% 5. To show the new drug performs no
better or no worse than comparator

14. Non-inferiority (NI) Trials
Purpose: To demonstrate that a new
drug is not worse than an
active comparator by more
than a pre-specified amount*
NOT: That the two drugs are equivalent
NOT: That the new drug isn’t inferior to the
active comparator
*non-inferiority margin, delta (Δ), 

15. from Certain Clinical Trials, July 2010.
GAO. FDAs Consideration of Evidence
What Factors Are Prompting
Increased Scrutiny of NI Trials?
1. Regulatory (volume) 2. Scientific 3. Political

16. Regulatory Factors (Volume)
Source: GAO. FDAs Consideration of Evidence from Certain Clinical Trials, July 2010.

17. Scientific Factors
Two most common reasons FDA failed to
approve sponsors NDAs with NI evidence:
1. Poor selection of active control
2. Unclear determination of non-inferiority
margin

18. Political Factors

19. Clinician
understanding of
non-inferiority
purpose, margin,
and issues

20. What is the biggest factor in
navigating NI research?
1. Regulatory
2. Scientific
3. Political
4. They are tied too
closely together to
separate

21. Balance Needs in NI Study
1. Determine historical
evidence of sensitivity to
drug effects exists
(constancy assumption)
2. Design a trial
3. Set a non-inferiority margin
FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001.

22. Historical Evidence of Sensitivity
Similarly designed trials in the past regularly
distinguished effective treatments from less
effective or ineffective treatments
Barring this, demonstration of efficacy from
a showing of non-inferiority is not possible
FDA. E 10 Choice of Control Group and Related Issues in Clinical Trials, 2001.

23. Designing an NI Trial
Non-inferiority trial should
mimic design of superiority trial
Primary variables, severity of
condition, concomitant illnesses,
method of diagnosis, etc.
MJA 2009;190(6):326-330.

24. Active comparator (drug)
should ideally be widely used
with established efficacy via
superiority trial and identical
indication & conditions
Biometric Journal 2009;51(1):185-92.

25. Structure of NI
Trial Design
Ideally
incorporates a
placebo AND active
comparator
Validate/establish
internal validity

26. Data Analysis
Use of BOTH intent-to-treat (ITT) and
per protocol (PP) methods of data
analysis should be employed
Guidance used to suggest PP analysis
for NI due to dilution of effect by ITT
Assumptions since proven faulty
Stat Med 2005;24(1):1-10.

27. Remember for NI Margin…
Defined effect size for (active) control
from previous trials
―Comprehensive synthesis‖ of
evidence justifying effect size of active
control (and thus the proposed NI
margin) necessary during protocol
development and submission to FDA
PLoS ONE 2010;5(10):e13550.

28. Remember for NI Margin…
Adequate support must be
provided to FDA (21 CFR 314.126)
FDA strongly prefers clinical
endpoint over surrogate endpoint
when possible
PLoS ONE 2010;5(10):e13550.

29. MJA 2009;190(6):326-330.
NI Margin must be determined by combination of
clinical considerations and statistical methods

30. Inference for Non-Inferiority
Delta Limits and Confidence Intervals (95%)
Non-inferiority shown
Non-inferiority shown
Non-inferiority not shown
Non-inferiority shown
- 0
NI Margin
Treatment Difference

31. H0=null hypothesis. Ha = alternative hypothesis; Δ =difference in event rates between new and standard treatments; S=Δ in
superiority trials. E=Δ in equivalence trials; NI =Δ in non-inferiority trials
*Testing for non-inferiority is in one direction only — even if superiority exists (dashed arrow), it is not the hypothesis being tested.
MJA 2009;190(6):326-330.

32. What is the most commonly used
NI margin in anti-infective studies?
1. 0.5%
2. 1%
3. 5%
4. 10%
5. 20%
6. 25%

33. How did published
non-inferiority trials
do at exam time?

34. Good Enough?
 Review of 8 NI trials
from JAMA & NEJM
 6 of 8 trials gave detail on
justifications of NI
margin
 6 of 8 concluded NI
 However, analysis by
Kaul & Diamond only
concluded NI in 4 of 8
trials
Ann Intern Med 2006;145(1):62-69.

35. Wangge et al. review showed less than
half (45.7%) of 232 NI trials reported
method used to generate NI margin
Less than 5.0% reported on similarity
of current trial with comparator trials
No overall difference pre & post release
of CONSORT (2006); method of NI
margin actually reported less frequently
PLoS ONE 2010;5(10):e13550.

36. There were 53 trials
(22.9% of total N) of
anti-infective drugs
Most (77.8%) anti-infective
trials used an NI margin of
percentage difference
between 10 to 20%.
PLoS ONE 2010;5(10):e13550.

37. Blinding
Superiority
vs
Non-inferiority
PLoS ONE 2010;5(10):e13550.

38. Superiority
Investigator with bias of
new drug’s superiority
cannot directly
manipulate results to
support belief
Non-inferiority
Investigator in NI trial
can bias results by
assigning similar ratings
regardless of patient
response/arm
PLoS ONE 2010;5(10):e13550.

40. What is Biocreep?

41. What is Biocreep?
20%
1. Inability to demonstrate
bioequivalence
20% 2. Slow crawl towards finding of non-
inferiority with ITT analysis
20% 3. Temporal reduction of efficacy of
drugs using NI trials as evidence
20% 4. Process of insertion of biologics into
drug pipeline
20% 5. The name of the villain from the
Hellraiser movies

42. FDA Guidance on
Clinical Drug Trials

43. Food and Drug Administration
Guidance Origin (FDA), Center for Drug
Evaluation and Research
(CDER), Division of Anti-
Infective and Ophthalmology
Products and the Division of
Special Pathogen and
Transplant Products, Office of
Antimicrobial Products
Anti-Infective Drugs Advisory
Committee (sponsored by FDA)

44. However, there is not a FDA consensus
for all infectious disease indications

45. Non-inferiority,
Bugs, and Drugs
 Acute bacterial
sinusitis (ABS)
 Acute bacterial
exacerbation of
chronic bronchitis
(ABECB)
 Acute bacterial
otitis media
(ABOM)
CDER. Antibacterial Drug Products: Use of Noninferiority
Trials to Support Approval , November 2010.

46. FDA Guidance on Anti-infectives

47. Predicated upon reviews of data for ABS,
ABECB, and ABOM along with Anti-Infective
Drugs Advisory Committee findings
FDA unable to determine recommended
NI margin for those three areas
Issue clearer for nosocomial/more serious
infectious disease indications
FDA Guidance on Anti-infectives

48. ICH9
ICH10
CHMP/EMEA
CONSORT
FDA
GAO

49. If this had been an elevator prep
(not pitch)
• Assay sensitivity and historical evidence is
crucial for determination of NI feasibility
• Sponsor NI margin selection is contingent
upon multiple factors in eyes of FDA
• Let guidance by FDA, other bodies, and
CONSORT help inform protocol
development on front end

50. Research Is Like Eating Marshmallows

51. Images
• http://xkcd.com/651/
• http://www.null-hypothesis.co.uk
•
• http://www.gao.gov/new.items/d10798.pdf
• http://www.nature.com/nrd/journal/v5/n10/pdf/nrd2170.pdf
• http://images.wellcome.ac.uk/indexplus/obf_images/7c/e7/dcef0f0fef89d4824b39357b57b5.jpg
• http://upload.wikimedia.org/wikipedia/commons/b/b0/CDC-10046-MRSA.jpg
Unreferenced pictures
are licensed images

52. Burning Questions
clauson@nova.edu www.unhub.com/kevinclauson