This is a brief overview of Noninferiority Trials. Look at WHAT they are and WHY we use them Cover the basic info needed to ANALYZE the quality of these reports and draw APPROPRIATE conclusions.
A trial where researchers want to show that the experimental treatment is not WORSE than another treatment by more than a CLINICALLY NEGLIGIBLE AMOUNT.
Like a typical RCT (aka superiority trial), these studies provide comparison info for making evidence-based decisions. So why not just use a head-to-head trial that we’re more familiar with? Consider the following: You have 2 medications in the same class. They performed so similarly against placebo that no one is willing to invest time/money in a head-to-head trial that will likely result in no clear superiority. When we expect the treatments to be the same, a Noninferiority trial can provide a justification OTHER THAN GREATER EFFICACY for choosing one medication over another (ex: decreased cost, better side effect profile, reduced pill burden). Additionally, if a clear superiority or inferiority exists, a properly executed noninferiority trial will spot it.
Essentially, Superiority & Equivalence trials are two distinctly different kinds of trials, with Noninferiority trials as a sort-of hybrid between the two. OBJECTIVE Superiority: Show one treatment is better than the other Equivalence: Show that both treatments are the same (think of bioequivalence trials – AB rating) Noninferiority: Show that the difference between the two treatments is so small, that they might as well be the same (think of therapeutic interchange) HYPOTHESES Superiority: Assume the treatments are the same & hope to show they are actually different Equivalence & Noninferiority: Assume the treatments are different & hope to show they are the same LIMIT Superiority: set an upper limit (P). Results are statistically significant when they are less than that upper limit P Equivalence: Set both an upper AND lower limit. Results are significant when they are contained entirely between the upper & lower limits. Noninferiority: BOTH a range AND an upper limit (∆) ∆=“noninferiority margin”
How likely am I to encounter a noninferiority trial? A study in September’s Pharmacotherapy journal reviewed the frequency of these studies. They found a clear trend of increasing frequency.
Why are they so popular? Placebo control inappropriate: For instance, it would be unethical to replace an active medication with a placebo when investigating a drug for use in acute asthma attack Can be used for physical interventions (ex: surgical procedures) as long as there is an accepted standard of treatment to compare it to. Risk-benefit analyses: Consider our previous example of the two treatments. These are helpful in making formulary decisions. Comparing a drug to itself: allows a manufacturer to set min/max dosing, determine the best titration strategies, and fine tune their formulations Inferior/Noninferior/Superior claims are all appropriate: Superiority trials can ONLY establish whether a superiority exists, or does not exist.
Biggest disadvantage is that these trials have very specific design & reporting requirements. Based on the studies currently available, these requirements may not be fully understood. If there is no reference treatment, if it is controversial, or inconsistent against placebo (ex: antidepressants, Alzheimer meds) then this trial is inappropriate Larger sample size can make a noninferiority trial more difficult & more expensive.
Start with the assumption that the treatments are different. Note the wording of the alternate hypothesis. It is not enough to state that one treatment is not inferior to another. Claims of noninferiority HAVE to reference the margin.
In designing these studies, we start with the standard procedures (ex: randomization, blinding & control) but the control has to meet some specific criteria It must be a well-established therapy There must be at least 1 superiority trial establishing its advantage over placebo. That superiority trial becomes the blueprint for the remaining design decisions (ex: inclusion/exclusion criteria, outcome measures & trial conduct) Like any trial, a limit to measure our results against is set. Standard RCTs state the P value. Noninferiority studies state the upper limit (AKA noninferiority margin) Unique to Noninferiority trials, is that the JUSTIFICATION for that choice MUST be given. It has to be both statistically AND clinically valid.
In setting the margin, we want the smallest value that would be clinically meaningful. The key question to consider – What would be clinically relevant to THIS situation? A clinician might be willing to accept a 5% difference in treatment for the prevention of acne, but not a 5% difference in the prevention of death-due-to-stroke. Ideally, we want a value smaller than the difference between the reference treatment and its placebo. Example: We look at the superiority trial establishing the reference treatment as better than placebo & see that the reference treatment was 12% better than placebo. If we set the margin at 7%, we are accepting that our experimental treatment may be up to 5% worse than the comparator, but at least 7% better than placebo. Once a margin is determined, we use that choice, the size of the CI and the desired study Power to calculate sample size.
Analysis in these trials can be tricky. ITT can shift the bias towards noninferiority. Per-Protocol only accounts for the patients who completed all the study parameters appropriately. It can also introduce bias – and that bias is less predictable. Noninferiority trials should USE and REPORT THE RESULTS from Both Methods. Those results can then be treated equally for a balanced conclusion, or the more pessimistic results can be used for a conservative interpretation.
Results are stated as CI. Both single-sided & two-sided intervals are valid. In any study, reporting the results without comparing them to the prestated limit is inappropriate. In superiority trials, results (typically a single value) are compared with the upper limit P. With THESE studies, the results are typically a range of values that are fully encompassed by the CI. We compare the entire CI with both the established range AND the upper limit.
The established range (AKA Zone of Inferiority) is tinted for clarity. The noninferiority margin is at point ∆. When the entire CI is less than zero (case A), researchers may appropriately claim superiority. When the upper limit of the CI is less than the noninferiority margin (case B & C), researchers can claim noninferiority. Even though part of the CI for case B falls within the “treatment is better” territory, it would be inappropriate for researchers to claim “possible superiority” as it could lead to misunderstanding. When the upper limit of the CI is greater than ∆, the results are, at best, inconclusive. However, if the entire CI is above the margin, researchers should report it as inferior. A claim of “inconclusive” would be inaccurate.
Multiple reviews of noninferiority trials have been done. Reviewers found examples of repeated design and reporting flaws. Recognizing these flaws makes us more effective at evaluating the evidence at hand
Improper or misleading terminology: The terms noninferiority and equivalence are often misused. Authors treat them synonymously. Or they choose ambiguous expressions like “not substantially lower than” or “comparable to,” leading to misinterpretation. It could be simple confusion, however, drug trial reports are now commonly used for marketing purposes. . Manufacturers don’t want to say “Our drug was no more than 5% worse than the one we compared it to.” In providing a positive spin for advertising purposes, the results are often misleading. Inappropriate noninferiority margin: The margin must be determined using both statistical rationale AND clinical judgment. Reviewers found multiple instances in which clinical judgment was discounted = margins that were too large for clinical relevance. A strategy to evaluate appropriateness: compare effect size (absolute difference in treatments ÷ by standard deviation). Researchers may NOT use this approach to set the margin, as it must be stated prior to trial initiation – NOT calculated after the fact. Confused error types: Because the hypotheses of noninferiority & superiority trials are swapped, readers may be confused when authors refer to a Type I or Type II error without giving its precise meaning.
Incomplete results analysis: Researchers commonly report EITHER Intention-To-Treat OR Per-Protocol analysis. Individually, each may lead to biased conclusions. Bias is reduced when both methods are used and ALL RESULTS are reported. Noninferiority claims from a superiority trial : Noninferiority trials can provide evidence for a claim of superiority. BUT, when superiority trials result in a finding of no superiority, we CANNOT infer equivalence or noninferiority. Reviewers found more than one case where superiority trials were wrongly reported as noninferiority or equivalence trials. Be suspicious when an study does not include a justification for the noninferiority margin, when the margin appears to have been calculated at study completion, or when the margin was not referenced in calculating sample size.
Noninferiority trials are gaining in popularity. Increased publication = greater role in influencing the FDA & other decision makers. Need to have a clear understanding of these trials to appropirately recommend or select treatment for the patients in our care.
The quality of a noninferiority study is directly dependent on its adherence to the design and reporting requirements.
Noninferiority Trials Presentation
Noninferiority Trials An Overview Hollie Sturgeon, PharmD
What is a noninferiority trial? <ul><li>A clinical trial wherein the objective is to establish that the experimental treatment is not clinically worse than the active comparison treatment by more than a small, predetermined margin. </li></ul>
<ul><li>Treatment A </li></ul><ul><li>$35/Month </li></ul><ul><li>Treatment B </li></ul><ul><li>$50/Month </li></ul>Similar to classically designed head-to-head superiority trials, noninferiority trials provide comparison data for evaluating treatment alternatives. CONCEPT: If Treatment A is similar enough in efficacy to Treatment B that the difference between them is clinically negligible, then my patient can use the less expensive alternative.
How are noninferiority trials different from other trials? Superiority Equivalence Noninferiority Objective To determine if one treatment is superior to another To determine if an experimental treatment and an active reference treatment are similar in effect within a prestated range To determine that an experimental treatment is not clinically worse than an active reference intervention by more than a small, preset margin Null Hypothesis There is no difference between the two interventions. One treatment is superior to the other The experimental treatment is equivalent to the comparator within the prestated margin Alternate Hypothesis One treatment is superior to the other One treatment is superior to the other The experimental treatment is no worse than the comparator with respect to the prestated margin Limit P (-∆ to +∆) (0 to +∆)
How common are noninferiority trials? <ul><li>During the 10-year period between 1999 & 2009, 582 noninferiority trials were published. </li></ul><ul><li>Between 1989 and 1999, there was one. </li></ul><ul><li>Of the 43 New Drug Applications approved by the FDA between 2002 and 2009, 2/3 cited evidence obtained from noninferiority trials </li></ul>“ Our study identified a clear trend of increasing frequency of publication of noninferiority trials” 
Advantages of noninferiority trials <ul><li>Useful when placebo control is inappropriate. </li></ul><ul><li>Not limited to pharmaceutical therapy </li></ul><ul><li>Can be used for risk-benefit analyses </li></ul><ul><li>Appropriate for comparing a specific intervention to itself (dose vs. dose or formulation vs. formulation) </li></ul><ul><li>Provides evidence for inferiority, noninferiority OR superiority claims </li></ul>
Disadvantages of noninferiority trials <ul><li>Must meet specific design and analysis parameters to be useful. </li></ul><ul><ul><ul><li>These requirements appear to be poorly understood by investigators and their readers. </li></ul></ul></ul><ul><li>Not recommended when the reference treatment is not well established, or is inconsistent when compared with placebo </li></ul><ul><li>An appropriate sample size for noninferiority trials is usually larger than that required for superiority trials </li></ul>
Noninferiority Trial Design <ul><li>To correctly interpret the results of a noninferiority trial, we must first understand its specific methodology and design requirements. </li></ul>
<ul><li>The hypothesis is the opposite of classically designed superiority studies. </li></ul><ul><li>Null hypothesis: One treatment is superior to the other. </li></ul><ul><li>Alternate hypothesis: The experimental treatment is no worse than the comparator with respect to the prestated noninferiority margin. </li></ul>Noninferiority Trial Design
<ul><li>Typical trial procedures such as randomization and blinding apply; however, stricter limitations are placed on the active control. </li></ul><ul><ul><ul><li>Comparator must be a well-established intervention with at least one superiority trial establishing its clinical advantage over placebo. </li></ul></ul></ul><ul><li>Overall study design is structured as closely as possible to that superiority trial (inclusion/exclusion criteria, outcome measures, trial conduct, etc.) </li></ul><ul><li>A noninferiority margin must be determined prior to trial initiation. </li></ul><ul><ul><ul><li>Justification for the margin is provided, stating both statistical and clinical validation. </li></ul></ul></ul>Noninferiority Trial Design
<ul><li>The noninferiority margin should be no larger than the “smallest value representing a clinically meaningful difference” between the two interventions. </li></ul><ul><ul><ul><li>Ideally a value smaller than the minimum difference between the active comparator and a placebo. </li></ul></ul></ul><ul><ul><ul><li>For example, if the referenced superiority trial showed the comparator to be 12% better than placebo, a researcher might set the noninferiority margin at 7% </li></ul></ul></ul><ul><ul><ul><li>In other words, 5% worse than the comparator, but still 7% better than placebo </li></ul></ul></ul><ul><li>Noninferiority margin, Confidence Interval and desired Power all influence sample size. </li></ul>Noninferiority Trial Design
<ul><li>Intention-To-Treat (ITT) analysis can shift bias towards a finding of noninferiority when dropouts & protocol violations occur </li></ul><ul><li>The bias introduced by Per-Protocol analysis is less predictable, especially when the rates & reasons for patient loss differ between treatment groups. </li></ul><ul><li>Noninferiority trials should employ & report both methods of analysis. </li></ul><ul><ul><ul><li>Balanced conclusions may be derived when both methods are treated with equal importance. </li></ul></ul></ul><ul><ul><ul><li>A more conservative approach would be to base conclusions on the more pessimistic of the analyses. </li></ul></ul></ul>Noninferiority Trial Design
Noninferiority Trial Interpretation <ul><li>The tinted area represents the zone of inferiority. </li></ul><ul><li>When the entire CI is less than zero, the treatment is clearly superior </li></ul><ul><li>When the upper limit of the CI is less than ∆, the treatment is noninferior </li></ul><ul><li>When the upper limit of the CI is greater than ∆, the result is inconclusive </li></ul><ul><li>When the entire CI is greater than ∆, the treatment is clearly inferior </li></ul>Superior Noninferior Noninferior Inconclusive Inconclusive Inferior
Common Errors <ul><li>Reviewers have found examples of repeated design and reporting flaws. Becoming familiar with the more typical faults will make it easier to evaluate quality studies & draw appropriate conclusion. </li></ul>
<ul><li>Improper or misleading terminology </li></ul><ul><ul><ul><li>Quality reporting uses proper vocabulary, such as “Treatment A is not inferior to (or is equivalent to) Treatment B with regard to the margin predetermined as ____.” </li></ul></ul></ul><ul><li>Inappropriate noninferiority margin </li></ul><ul><ul><ul><li>Must be defined prior to trial onset. </li></ul></ul></ul><ul><ul><ul><li>Justification must include be statistically & clinically relevant </li></ul></ul></ul><ul><li>Confused error types </li></ul><ul><ul><ul><li>Type I error = an inferior treatment is accepted as noninferior </li></ul></ul></ul><ul><ul><ul><li>Type II error = a noninferior treatment is mistakenly rejected </li></ul></ul></ul>Common errors to watch for
<ul><li>Incomplete analysis of results </li></ul><ul><ul><ul><li>Quality articles include both Intention-To-Treat and Per-Protocol analyses, enabling the reader to correctly interpret the evidence and form appropriate conclusions. </li></ul></ul></ul><ul><li>Noninferiority claims from a superiority trial </li></ul><ul><ul><ul><li>When articles omit pertinent information about the noninferiority margin, or the sample size is calculated without reference to that margin, clinicians should treat the subsequent information with some skepticism </li></ul></ul></ul>Common errors to watch for
<ul><li>The publication of noninferiority trials is gaining in frequency. Evidence from these trials influence current treatment policies, and play a role in the decision making process for new drug acceptance. Healthcare providers involved in the recommendation or selection of evidenced-based interventions should have a clear understanding of how to evaluate the quality of these studies and interpret their results. </li></ul><ul><li> </li></ul>Summary
<ul><li>The quality of noninferiority studies depends on their adherence to basic design requisites and reporting obligations. </li></ul><ul><li>Standard trial procedures (randomization, blinding, etc) </li></ul><ul><li>Active comparator that is well-established backed by at least one superiority trial </li></ul><ul><li>Overall study design mirrors reference superiority trial </li></ul><ul><li>A prestated noninferiority margin that is statistically and clinically justified </li></ul><ul><li>Sample size based on noninferiority margin, Confidence Interval size and desired study Power </li></ul><ul><li>Both Intention-To-Treat and Per-Protocol analyses used and reported </li></ul><ul><li>Results reported as Confidence Intervals </li></ul><ul><li>Conclusions drawn from comparison of Confidence Intervals with the noninferiority margin </li></ul><ul><li>Claims of noninferiority correspond to the results provided and are written in precise, standardized vocabulary </li></ul>Summary
<ul><li>Henanff AL, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials. JAMA. 2006;295(10):1147-51 </li></ul><ul><li>Dasgupta A, Lawson KA, Wilson JP. Evaluating equivalence and noninferiority trials. Am J Health-Syst Pharm. 2010;67:1337-43 </li></ul><ul><li>Gotzsche PC. Lessons from and cautions about noninferiority and equivalence randomized trials. JAMA. 2006;295(10):1172-74 </li></ul><ul><li>Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJW. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152-60 </li></ul><ul><li>Suda KJ, Hurley AM, McKibbin T, Motl Moroney SE. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011;31(9):833-839 </li></ul><ul><li>Greene WL, Concato J, Feinstein AR. Claims of equivalence in medical research: are they supported by the evidence? Ann Intern Med. 2000;132:715-722 </li></ul>References