Cluster randomized trials are frequently used in health service evaluation. It is common practice to use an analysis model with a random effect to combine between cluster information about treatment effects. It is increasingly being acknowledged that intervention effects might vary across clusters, or the variation between clusters might differ across the randomized arms. It has been proposed in both parallel cluster trials, stepped-wedge and other crossover designs that this heterogeneity can be allowed for by incorporating additional random effect(s) into the model. Here we show that the choice of model parameterization needs careful consideration as some parameterizations for additional heterogeneity induce unnecessary assumptions. We suggest more appropriate parameterizations, discuss their relative advantages and demonstrate the implications of these model choices using practical examples of a parallel cluster trial and a simulated stepped-wedge trial.

1. Modeling differential clustering and treatment effect
heterogeneity in parallel and stepped wedge cluster trials
Karla Hemming
University of Birmingham
Monica Taljaard (Ottawa) and Andrew Forbes (Monash)
March 4, 2016
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 1 / 26

2. Treatment effect heterogeneity - in iRCTs
Sub-group analysis common - desire to identify in whom, for whom
and where the intervention works.
Whilst it is recommended that this be pre-specified and by the use of
interaction tests, this guidance is poorly adhered to. 1
Treatment effect heterogeneity can be incorporated using either fixed
or random interaction terms.
1
Dealing with heterogeneity of treatment effects: is the literature up to the
challenge? Nicole B Gabler, Naihua Duan, Diana Liao, Joann G Elmore, Theodore G
Ganiats, Richard L Kravitz Trials. 2009; 10: 43.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 2 / 26

3. Treatment effect heterogeneity - random or fixed effects?
Fixed effect interaction terms straightforward - preferable when
number of subgroups small.
Random effects approaches preferable when number of groups large
(i.e. site).
Different ways of parameterizing these models. 1 2
These different ways have gone largely unnoticed.
1
Applied Mixed Models in Medicine by Helen Brown and Robin Prescott
2
Modeling site effects in the design and analysis of multisite trials Daniel J. Feaster,
Susan Mikulich-Gilbertson, Ahnalee M. Brincks Am J Drug Alcohol Abuse.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 3 / 26

4. Cluster randomized trials (CRTs) - basic set up
Cluster randomized trials are frequently used in health service
evaluation.
It is common practice to use an analysis model with a random effect
to combine between cluster information about treatment effects.
This is typically done by using a linear or generalized linear mixed
model with a random effect for cluster. 1
1
Donner, Allan and Klar, Neil; Design and analysis of cluster randomization trials in
health research; 2000
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 4 / 26

5. Differential variation between arms - in parallel CRTs
Can’t estimate treatment by cluster heterogeneity in CRTs
In parallel cluster trials - whilst it may be anticipated that the treatment
effect might vary across clusters, because clusters are either fully exposed
or unexposed to the intervention, the trial design does not allow
estimation of this.
Different treatments might be expected to induce homogeneity - or
even heterogeneity. 1
This will manifest itself as different ICCs in intervention and control
arms.
Also important in trials with differential clustering between arms -
perhaps where a group therapy is compared with an individual
therapy. 2
1
Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated
cross-sectional binary measurements. Stat Med. 2001
2
Roberts, Chris and Roberts, Stephen A; Design and analysis of clinical trials with
clustering effects due to treatment; 2005; Clinical Trials
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 5 / 26

6. Treatment effect heterogeneity - in cluster cross-over trials
Can estimate treatment by cluster heterogeneity in cross-over CRTs
In cross-over cluster trials (i.e. stepped wedge) - because clusters are
crossed with treatment - the design does allow estimation of cluster by
treatment effects.
In cross-over designs and stepped wedge designs, because each cluster
receives the intervention and control, possible to estimate cluster by
treatment interactions.
Again different ways of parameterizing these models. 1 2
These different ways AGAIN have gone largely unnoticed.
1
Hughes JP, Granston TS, Heagerty PJ. Current issues in the design and analysis of
stepped wedge trials. Contemp Clin Trials. 2015
2
Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Sample size
calculation for a stepped wedge trial. Trials. 2015
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 6 / 26

7. When should such extra heterogeneity be allowed for?
At the design stage.
At the analysis stage.
Model choice is therefore important from the outset.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 7 / 26

8. Objectives
Here we show that careful choice is needed over model
parameterization of heterogeneity.
Outline different parameterizations.
Demonstrate that some parameterizations induce unnecessary
assumptions.
Show early results from a simulation study investigating statistical
optimality of the different methods.
Preliminary recommendations for appropriate model
parameterizations.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 8 / 26

9. Assumptions
Two arm trial:
Continuous outcome.
Testing for superiority.
Cluster randomization:
Assume reasonable number clusters (more than 40).
Analysis using individual level data, using mixed models with a random
effect for cluster.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 9 / 26

10. CRT: Basic model (single random effect)
Let us consider a two arm parallel CRT. The conventional analysis model
for this simple setup is 1:
yij = µ + xij θ + αj + eij (1)
αj ∼ N[0, τ2
]
eij ∼ N[0, σ2
w ]
Under this model the correlation between two observations in the same
cluster, the intra-cluster correlation (ICC), will be:
ρ =
τ2
τ2 + σ2
w
i : individual; j : cluster
1
Donner, Allan and Klar, Neil; Design and analysis of cluster randomization trials in
health research; 2000
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 10 / 26

11. Model extension A: two independent random effects
Two separate random effects, one for treatment and one for control, are
incorporated 1:
yij = µ + xij θ + xij α(T)j + (1 − xij )α(C)j + eij (2)
α(T)j ∼ N[0, τ2
T ]
α(C)j ∼ N[0, τ2
C ]
eij ∼ N[0, σ2
w ]
The ICC in the control and intervention clusters will be:
ρC =
τ2
C
τ2
C + σ2
w
ρT =
τ2
T
τ2
T + σ2
w
Assumptions
No restrictions made on relative magnitude of two ICCs
1
Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated
cross-sectional binary measurements. Stat Med. 2001
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 11 / 26

12. Model extension B: a (SIMPLE) random interaction
Parametrization that includes a random interaction between the treatment
covariate and cluster 1:
yij = µ + xij θ + α(M)j + xij α(I)j + eij (3)
α(M)j ∼ N[0, τ2
M]
α(I)j ∼ N[0, τ2
I ]
eij ∼ N[0, σ2
w ]
The ICC in the control and treatment clusters will be:
ρC =
τ2
M
τ2
M + σ2
w
ρT =
τ2
M + τ2
I
τ2
I + τ2
M + σ2
w
Assumptions
ICC in treatment clusters greater than or equal to ICC in control clusters
1
Roberts, Chris and Roberts, Stephen A; Design and analysis of clinical trials with
clustering effects due to treatment; 2005; Clinical Trials
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 12 / 26

13. The stepped wedge cluster randomized trial (SW-CRT) -
basic model
We extend the basic model (1) for parallel CRTs to the SW-CRT, by
incorporating fixed effects for each step 1:
yijs = µ + xijsθ + αj + ts + eijs (4)
αj ∼ N[0, τ2
]
eijs ∼ N[0, σ2
w ]
Note
This notation also applies without any loss of generality to cross-over trials
and before and after parallel studies, with repeated cross-sectional
measurements.
i : individual; j : cluster; s : step
1
Hussey and Hughes; Design and analysis of stepped wedge cluster randomized trials;
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 13 / 26

14. Model extension A: two independent random effects
Two separate random effects, one for treatment and one for control 1:
yijs = µ + xijsθ + xijsα(T)j + (1 − xijs)α(C)j + ts + eijs (5)
α(T)j ∼ N[0, τ2
T ]
α(C)j ∼ N[0, τ2
C ]
eijs ∼ N[0, σ2
w ]
The ICC in the control and intervention periods will be:
ρCC =
τ2
C
τ2
C + σ2
w
ρTT =
τ2
T
τ2
T + σ2
w
ρCT = 0
Assumptions
No restrictions made on relative magnitude of two ICCs. But, ρCT
assumed zero.
1
Ukoumunne OC, Thompson SG. Analysis of cluster randomized trials with repeated
cross-sectional binary measurements. Stat Med. 2001 - FOR REPEATED
CROSS-SECTION DESIGN
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 14 / 26

15. Model extension B: an independent random interaction
Model treatment heterogeneity using a SIMPLE interaction term 1 2:
yijs = µ + xijsθ + α(M)j + xijsα(I)j + ts + eijs (6)
eijs ∼ N[0, σ2
w ]
and that
α(M)
α(I)
∼ N
0
0
,
τ2
M 0
0 τ2
I
The ICC in the control and treatment clusters will be:
ρCC =
τ2
M
τ2
M + σ2
w
ρTT =
τ2
M + τ2
I
τ2
I + τ2
M + σ2
w
Assumptions
ICC in treatment clusters greater than or equal to ICC in control clusters;
ρCT is non-zero.
1
Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ. Trials. 2015
2
Hughes JP, Granston TS, Heagerty PJ. Contemp Clin Trials. 2015
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 15 / 26

16. Model extension C: a non-independent random interaction
Model treatment effect heterogeneity using an interaction term and
allowing for a covariance term (like in RCT literature):
yijs = µ + xijsθ + α(M)j + xijsα(I)j + ts + eijs (7)
eijs ∼ N[0, σ2
w ]
and that
αM
αI
∼ N
0
0
,
τ2
M σMI
σMI τ2
I
The ICC in the control and treatment clusters will be:
ρCC =
τ2
M
τ2
M + σ2
w
ρTT =
τ2
M + τ2
I + 2σMI
τ2
I + τ2
M + σ2
w + 2σMI
Assumptions
No restrictions made on relative magnitude of two ICCs; ρCT is non-zero.
1
Modeling site effects in the design and analysis of multisite trials Daniel J. Feaster,
Susan Mikulich-Gilbertson, Ahnalee M. Brincks Am J Drug Alcohol Abuse.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 16 / 26

17. Model fitting
Stata
We have fitted these models in Stata 14 using the xtmixed function, using
REML methods. ICCs were calculated using variance ratios.
Care needed
Some care is needed to make sure you are fitting the model you think you
are fitting.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 17 / 26

18. Example: A parallel cluster randomised trial - with more
variability in outcomes in intervention clusters
Parallel cluster trial conducted in 53 schools (clusters).
Behavioral intervention to prevent obesity.
Outcome is BMI measured at the end of the trial.
Total of 689 observations in the intervention arm and 778
observations in the control arm.
Average cluster size of 24.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 18 / 26

19. Example: usefully advocates careful choice needed over
parameterizations
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 19 / 26

20. Simulation study for parallel CRT
Simulation study parameters
Continuous outcome; two arms; large trial (100 centers each of size 50);
SES=0.1; Cluster randomization; Compare model A and model B only;
Generate data from model A; 1000 simulations
Scenarios
Scenario 1: No treatment by site heterogeneity.
Scenario 2: Variation between control clusters smaller than that of
intervention clusters (concordant with model B).
Scenario 3: Variation between control clusters greater than that of
intervention clusters (discordant with model B).
Findings:
No model comes out as preferable for bias of treatment effects
No model comes out as preferable for coverage of treatment effects
Model B gives biased estimate of ICCs under scenario 3
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 20 / 26

21. Simulation study for cross-over CRT
Simulation study - greater heterogeneity in control clusters.
Continuous outcome; two arms; large trial (100 centers each of size 50);
SES=0.1; Cluster cross-over randomization; Compare model A, B and C;
Generate data from model C; 1000 simulations
Model A:
No evidence of bias.
Coverage slightly too high.
Model B:
No evidence of bias.
Coverage too low.
Model C:
No evidence of bias
Coverage slightly too low.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 21 / 26

22. Simulation study - with greater heterogeneity between
control clusters than intervention clusters (scenario 3)
ICC for control clusters 0.01; for intervention clusters 0.001
Model C coverage and Model A coverage > Model B coverage (0.953 vs
0.936)
ICC for control clusters 0.05; for intervention clusters 0.001
Model C coverage and Model A coverage > Model B coverage (0.957 vs
0.926)
ICC for control clusters 0.17; for intervention clusters 0.001
Model C coverage and Model A coverage > Model B coverage (0.964 vs
0.881)
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 22 / 26

23. Conclusion and discussion
Need for appreciation of assumptions implicit when fitting random
effects treatment heterogeneity models.
Choice of parameterization important - some parameterizations make
fewer assumptions and this would seem the sensible model choice.
Early simulations suggest model paramaterisation might have more
important implications for cross-over trials but not be important for
parallel trials.
Simulations suggest little evidence for bias for treatment effect; but
potential for under coverage (standard error too small) from some
models.
Some models give poor estimate of ICC.
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 23 / 26

24. Thank you!
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 24 / 26

25. Model extension E: alternative
Treatment heterogeneity modeled by what we call a true interaction term
1:
yijs = µ + xijsθ + αj + xijsα(T)j + (1 − xijs)α(C)j + ts + eijs (8)
eijs ∼ N[0, σ2
w ]
and that:


αj
α(C)j
α(T)j

 ∼ N




0
0
0

 ,


τ2
α 0 0
0 τ2
αT 0
0 0 τ2
αc




Assumed that τ2
αT = τ2
αC in 2.
1
Turner RM, White IR, Croudace T; PIP Study Group. Analysis of cluster
randomized cross-over trial data: a comparison of methods. Stat Med. 2007 Jan
30;26(2):274-89. PubMed PMID: 16538700.
2
Applied Mixed Models in Medicine by Helen Brown and Robin Prescott
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 25 / 26

26. Example: A SW cluster randomised trial - with more
variability in outcomes in intervention clusters
SWl cluster trial conducted in X clusters
Across X steps
Outcome is XX
Total of XXX observations
Average cluster size of XX per cluster per period
Karla Hemming (University of Birmingham) Treatment effect heterogeneity in cluster trials March 4, 2016 26 / 26