This simulation study examines how frailty, or heterogeneity in HIV infection risk within a study population, may impact the apparent declining efficacy seen in some randomized HIV intervention trials over time. The study uses mathematical modeling to simulate different trial scenarios varying factors like frailty level, intervention efficacy waning, and population risk distribution. The goal is to quantify how frailty alone could cause efficacy measures like the risk ratio to approach 1 at later time points even if the intervention effectiveness remains constant.
Similar to Measuring the Potential Impact of Frailty on the Apparent Declining Efficacy in Randomized Trials of HIV Interventions: A Simulation Study (20)
Measuring the Potential Impact of Frailty on the Apparent Declining Efficacy in Randomized Trials of HIV Interventions: A Simulation Study
1. “Measuring the Potential Impact of Frailty on the Apparent
Declining Efficacy in Randomized Trials of HIV Interventions: A
Simulation Study”
Felicia P. Hardnett
Mathematical Statistician
Quantitative Sciences and Data Management Branch
2. Motivation
Recent advancements in HIV prevention have
given researchers hope that effective HIV
interventions might soon become widely available
Additional advancements in clinical trials
methodology have also occurred to measure the
efficacy of these interventions more accurately
3. Problem
The results of recent HIV intervention trials have
been somewhat disappointing and difficult to
explain
The efficacy of the interventions appears to
decline over time
4. Recently Published Trials
Two recently published trials (1 vaccine trial and
1 microbicide trial) concluded that intervention
effectiveness decreased over time1,2.
The investigators attributed this to:
Waning vaccine efficacy (vaccine trial)
Decreasing adherence (microbicide trial)
1Abdool Karim Q, Abdool Karim SS, Frohlich JA, Grobler AC, Baxter C, Mansoor LE, et al. Effectiveness and safety of tenofovir
gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science 2010; 329:1168–1174.
2Michael N. RV 144 update: vaccination with ALVAC and AIDSVAX to prevent hiv-1 infection in thai adults. 17th conference on
retroviruses & opportunistic infections, (2010).
http://app2.capitalreach.com/esp1204/servlet/tc?c¼10164&cn¼retro&e¼12354&m¼1&s¼20431&&espmt¼2&mp3file
¼12354&m4bfile¼12354.
5. Alternative Explanation
In addition to these phenomena, the authors of a
recently published opinion piece assert that
frailty (due to heterogeneity in infection risk) is
another possible explanation1.
This explanation is rarely cited in the literature as
a possible explanation for declining efficacy.
1O’Hagan JJ, Hernan MA, Walensky RP, Lipstitch M. Apparent declining efficacy in randomized trials: examples
of the Thai RV144 HIV vaccine and South African CAPRISA 004 microbicide trials. AIDS 2012, 26:123-126.
6. The Potential Impact of Frailty
Even if the efficacy of an intervention remained
constant, frailty could give the appearance that its
declining
This could cause researchers to reject an
effective intervention
7. Purpose
To explore the potential impact of frailty on the
results of randomized trials of HIV interventions
9. What is Frailty?
Heterogeneity in infection risk within a study
population
Causes change in the composition of the study
population over time
Causes the measure of effect (risk ratio) to
approach 1 over time
10. Illustration of a hypothetical disease process
within a population
Population at risk
# persons who never
develop disease
Time
• As people become infected, the population at risk decreases over time and
eventually plateaus
• The rate of decline depends on disease incidence
• The curve plateaus at the number of persons who will never develop the
disease (low/no risk people)
11. Population at risk Illustration of Frailty as presented in the paper
High risk
Low risk
Time
The opinion piece asserts:
• High risk individuals will be infected early on and will be removed from
the population at risk first.
• This will leave lower risk individuals in the risk population resulting in lower
disease incidence at later time points.
12. Graphical representation of disease incidence
N0
High risk
Population at risk (N)
n1
Low risk
n2
# persons who never
develop disease
Time
t0 t1 t2 t3
Incidence=Number who become infected (n)
From t0 to t1
Number intially at risk (N0)
13. Graphical representation of disease incidence
N0
Population at risk (N)
n1
n2
# persons who never
develop disease
Time
t0 t1 t2 t3
• Fewer cases diagnosed at a later time point because the high risk
people are gone.
• Incidence, therefore decreases.
14. Intervention Scenario
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• If the intervention is effective, it will prolong the time before high-risk individuals in
the treatment arm will become infected.
• Incidence decline in the placebo group will be larger because those at high risk will
be quickly removed from the population at risk.
15. Intervention Scenario
Rate ratio= incidence (treatment arm)
incidence(placebo)
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• As a result, the time-specific rate ratio will increase from a value of less than one
to a value of one or greater.
• This process is termed “frailty”,“survivor bias”, “survivor cohort effect”,
“crossing of hazards” or “depletion of susceptibles”.
16. Intervention Scenario
Rate ratio= incidence (treatment arm)
incidence(placebo)
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• As frailty increases, the curve becomes more steep early on and less steep towards
the end.
• RR approaches 1 sooner.
17. Intervention Scenario
Rate ratio= incidence (treatment arm)
incidence(placebo)
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• As frailty increases, the curve becomes more steep early on and less steep towards
the end.
• RR approaches 1 sooner.
18. Possible Impact on Rate
Measures
This risk ratio comparing the incidence in placebo
and treatment group becomes increasingly
attenuated as follow-up time increases.
This occurs even if risk factors were balanced
between study arms at baseline and if effect of
intervention is constant over time.
19. Is Frailty Really Important?
Based on the information presented in the paper:
With competing factors such as waning vaccine
efficacy and decreasing adherence, it’s not clear
how important frailty is in explaining declining
efficacy in the two trials.
21. Current Study Approach
We designed several study scenarios using
study-related, intervention-related and
population-related parameters.
We held study-related factors constant (e.g.,
sample size, follow-up time, intervention
effectiveness).
22. Current Study Approach
We varied population and intervention-related
parameters (e.g. waning and frailty)
We estimated the risk ratio at each time point for
each scenario and quantified the change that is
attributable to frailty.
23. Modeling Description
Definition of Parameters
Model Assumptions
Scenario Design
Results
Conclusions
24. Modeling Description
Definition of Parameters
Model Assumptions
Scenario Design
Results
Conclusions
25. Model Parameters
Study Intervention Population
Fixed • Sample Size Intervention Distribution of
• Follow-up Efficacy Population
Period across Risk
• Number of Groups
Risk Groups
Varied Waning Probability of
-- Disease
25
26. Risk Groups
The study population is divided into mutually-
exclusive groups ranging from very high risk to very
low risk.
35. Modeling Description
Definition of Parameters
Model Assumptions
Scenario Design
Results
Conclusions
36. Model Assumptions
Sufficient sample size
The treatment arms have equal sample sizes.
Disease risk is balanced between both treatment
arms at the beginning of the study.
Non-differential loss to follow up.
37. Model Assumptions
The intervention is effective at reducing the
probability of disease and presents no adverse
effects (i.e., increasing in the probability of
infection) at any point in time.
Intervention efficacy is constant across all risk
groups.
Intervention waning/non-adherence is constant
across all risk groups.
38. Modeling Description
Definition of Parameters
Model Assumptions
Scenario Design
Results
Conclusions
39. Features of Study Scenarios that
remain fixed
Equal sample size in each treatment arm
Ten-year follow-up time
Five HIV risk groups ranging from very high risk
to very low risk
Intervention effectiveness - 50%
40. Features of Study Scenarios that
remain fixed
Distribution of study population across risk groups
0.6
0.5
0.4
0.3
0.2
0.1
0
Very High High Moderate Low Very Low
41. Features of Study Scenarios that
are varied
Waning- the rate at which the intervention loses
its effectiveness
Frailty- heterogeneity in disease risk across the
5 risk groups
57. Modeling Description
Definition of Parameters
Model Assumptions
Scenario Design
Results
Conclusions
58. Conclusions
With the exception of the most extreme cases,
frailty (heterogeneity in disease risk) doesn’t
appear to have much of an impact on outcome
measures in randomized trials of HIV
interventions
The impact of frailty appears substantial in
scenarios when HIV infection is a virtual
certainty in the highest risk group and negligible
in the lowest risk group
59. Conclusions
This study condition is unlikely to occur in most
trials where higher risk individuals are commonly
recruited.
Therefore, frailty is less likely to explain a
substantial portion of the declining efficacy in
many HIV intervention trials
This is a graphical representation of a hypothetical disease process within a population. It is hypothetical because it doesn’t specify the size of the population at risk or the actual length of time of observaton. This depiction also doesn’t account for loss-to-follow up or censoring. It assumes that all persons remain in the population at risk until removed by disease. The intent is to illustrate the fact that the population at risk of disease declines over time and eventually tapers off and plateaus at the number of persons who will never develop the disease. The rate of decline depends on the incidence of disease within the population.
In the opinion piece, the authors suggest a form of selection bias as a possible explanation for declining intervention efficacy in randomized trials. High risk individuals are removed from the population early on ultimately leaving only low risk people in the population who have a much lower incidence of disease at later time points.
This is a graphical representation of the change in incidence over time as suggested by the authors. Disease incidence is a measure of the proportion of a population who become infected during a specific time period (denoted by the blue lines). The numerator is the number of persons who become infected, or n. The denominator is the number of persons initially at risk, or N0.
This is a graphical representation of the change in incidence over time as suggested by the authors. Disease incidence is a measure of the proportion of a population who become infected during a specific time period (denoted by the blue lines). The numerator is the number of persons who become infected, or n. The denominator is the number of persons initially at risk, or N0.
Moving forward to an actual intervention scenario, assuming that the intervention is effective, the high risk group in the treatment arm will take longer to become infected because of the protection conferred by the intervention. However, they will still be among the first to become infected leaving the low risk group in the population at later time points.The incidence decline in the placebo group will be larger because the high risk subgroup will not benefit from the intervention.Over time, this incidence difference will gradually resolve.
As a result, the time-specific rate ratio will increase from a value of less than one to a value of one or greater. This process is referred to as “frailty”, “survivor bias”, “survivor cohort effect”, “crossing of hazards” or “depletion of susceptibles.”
As a result, the time-specific rate ratio will increase from a value of less than one to a value of one or greater. This process is referred to as “frailty”, “survivor bias”, “survivor cohort effect”, “crossing of hazards” or “depletion of susceptibles.”
As a result, the time-specific rate ratio will increase from a value of less than one to a value of one or greater. This process is referred to as “frailty”, “survivor bias”, “survivor cohort effect”, “crossing of hazards” or “depletion of susceptibles.”