This presentation is aimed at presenting the issues associated with subgroup analyses in clinical trials: the different types of subgroup analyses and the statistical issues associated with the conduct of subgroup analyses.

1. © NICE 2018. All rights reserved. Subject to notice of rights.
Subgroup analyses
F. Maignen

2. Why subgroups? The importance
of (some) covariates…
Baseline covariate: qualitative factor or quantitative
variable measured or observed before a subject
starts taking the study medication (usually before
randomisation) and expected to influence the primary
variable to be analysed.
Examples:
Previous therapies
Age
Severity of disease
Given phenotype of a gene (BRCA2 mutation)
Ethnicity, etc.
2

3. You can take into consideration the
influence of a covariate in your
analysis either …
Stratification: performing a statistical procedure
separately in groups (strata) with implied pooling
across groups to reduce the effect of group factor.
OR
Subgrouping: estimate of the treatment effect in a
subset of a clinical trial population. Patients excluded
from a particular subgroup are described as the
complement subgroup.
3

4. Baseline covariate … analysis …
4
Subgrouping
Unstratified (black line)
Stratified
(green line)

5. Baseline covariate … what could
go wrong?
Assuming that a particular covariate plays an
important role in the efficacy of a given product …
what could go wrong during the conduct of your trial?
5

6. What can you do to avoid this
situation and achieve …
6
Stratified randomisation … include your covariate in
the randomisation factors.

7. Baseline covariates …
You should:
- Adjust your statistical analysis for your baseline
covariate
OR you can:
- Perform a subgroup analysis
AND/OR if this covariate is likely to greatly
influence the efficacy of your new technology.
- Perform a stratified randomisation (i.e. add your
baseline covariate in your stratification factors) to
avoid any imbalance (wrt this covariate) between
the arms of your study.
7

8. The risk of subgroup analyses …
FALSE CLAIMS!!!!!!!!!!!!!!!!!!!!
8

9. The risk of subgroup analyses …
For a hypothesis test, I have (usually) a 5% risk of
rejecting a null hypothesis which is actually true (risk
alpha) … What would happen if I was performing
simultaneous tests (say 6) in the same trial?
9

10. Multiplicity
1 – (1 – 0.05)6 = 26.5%
The more you perform statistical
tests, the higher the risk alpha is
inflated i.e. the higher the likelihood
of wrongly rejecting a null
hypothesis …
10

11. Circumstances in which you will
increase your alpha error rate …
include:
• Multiple endpoints (primary, then
secondary) or comparisons (e.g.
doses)
• Interim analyses
• Subgroup analyses …
It is important to control the error rate
i.e. to maintain the OVERALL error rate
at 5% …
11

12. Bonferroni correction
Use alpha/n (n = number of hypotheses tested) i.e.
use 0.05/6 (0.008) instead of 0.05 …
BUT WHAT COULD BE THE CONSEQUENCE OF
THIS CORRECTION ON YOUR HYPOTHESIS
TESTING???
12

13. What we’ve seen so far …
• Some baseline covariates can influence the
efficacy of a product.
• These covariates may or may not have been
identified before the conduct of a trial.
• You can stratify or adjust your clinical trial analysis
according to this (these) covariate(s) = 1 estimate
for the treatment effect.
• You can perform a stratified randomisation
according to a limited number of covariates (to
avoid imbalances in your trial).
• You probably want to make sure that the efficacy
is homogeneous across different subgroups of
patients in which the efficacy is NOT expected to
differ (or may differ).
• You may want to estimate the efficacy of your new
product in a particular subgroup of patients.
13

14. What we’ve seen so far …
• Stratification can have different meanings
(stratified analysis OR stratified randomisation).
• Subgrouping and stratification mean different
things.
• Subgroup analyses are attached to additional
(statistical and financial) costs (i.e. adjustment for
multiple comparisons).
• Post-hoc subgroup analyses are
BAAAAAAAAAAAD.
14

15. Does this medicine …
15

16. work (better or differently) …
1/ in men than in women?
2/ in adults and elderly (say people over 60)?
3/ in persons who own a female hamster (like I just
found in a subgroup analysis)?
4/ only in patients carrying a specific mutation (e.g.
patients carrying the BRAC2 mutation) compared to
the general population of patients?
WHAT ARE YOU LOOKING FOR IN EACH OF
THESE SUBGROUPS?!?!?!?
16

17. The different types of subgroup
analyses
- Exploratory analyses: differential responses from
early trials or from clinical trials that failed to
establish treatment efficacy in its intended
population. Hypothesis generating at best.
- Supportive analyses: consistency of treatment
effect across subgroups that has established
treatment efficacy in its intended overall
population.
- Inferential analyses: aimed at establishing
treatment efficacy in a pre-defined targeted
subgroup and/or overall population.
17

18. Connect the dots …
Covariate
Men vs women
Adults vs elderly
Owns a pet hamster
BRCA 2 mutation
18
Interpretation
Homogeneity
Heterogeneity
Inferential
Spurious / exploratory

19. Connect the dots …
Covariate
Men vs women
Adults vs elderly
Hamster
BRCA 2 mutation
19
Interpretation
Homogeneity
Heterog. (adjust.) or safety
Inferential
Spurious / exploratory

20. Supportive subgroup analyses
The statistical analysis will include an interaction
term (interaction treatment*subgroup), the efficacy is
assumed to be homogeneous across strata
(subgroups) if the interaction term is not significant.
Supportive subgroup analyses must be interpreted
with caution:
- Lack of statistical power to show differences
- Differences can show up by complete chance
- Multiple comparisons …
- Credibility, replication and confirmation of the
result +++
20

21. Inferential subgroup analyses
Completely different objective: the result of the
subgroup analysis will result in a specific efficacy
claim in the population pre-defined by the subgroup.
The trial and the analysis must be designed a priori
to establish (greater) efficacy in this subgroup
(compared to the rest of the population – the tested
product may OR may not be efficacious in the
complement subgroup).
21

22. Inferential subgroup analyses
22

23. Inferential subgroup analyses
The issues which will come up in SA might relate to:
- Power of the test
- Relative treatment effect and relative size of the
subgroup
- Adjustments for multiple comparisons (and
allocation of alpha)
- Method chosen for the analysis.
23

24. Biomarker stratified design
24
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
BM- BM+ BM- BM+
control Product
Prognosis biomarker
control BM-
control BM+
Product BM-
Product BM+

25. Biomarker stratified design
25
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
control Product control Product
BM- BM+
Predictive biomarker
BM- control
BM- Product
BM+ control
BM+ Product

26. Biomarker stratified design
26
0
0.1
0.2
0.3
0.4
0.5
0.6
control Product control Product
BM- BM+
Prognosis and predictive biomarker
BM- control
BM- Product
BM+ control
BM+ Product

27. Example of a biomarker
H0: Null hypothesis for the overall population
H+: Null hypothesis for the subgroup
3 possibilities:
- Reject H0 and fail to reject H+: broad indication in
the entire population of the trial
- Reject H+ and fail to reject H0: restricted
indication in the subgroup of patients who express
the biomarker
- But … what happens if H0 and H+ are both
rejected?
27

28. Example of a biomarker
If H0 and H+ are both rejected …
Make sure that the effect in the overall population is
not driven by an effect in the subgroup (i.e. that the
product is effective in the BM- subgroup of patients)
and that the effect is different in the BM+/BM-
populations (otherwise he BM is pointless) …
If not:
“Enhanced” indication both in the overall population
and pre-specified subgroup of patients (who express
the biomarker).
28

29. Conclusions
Two types of subgroup analyses:
- Pre-planned (adequate when properly designed)
or post-hoc (are baaaad unless purely exploratory)
What is the role of the baseline covariate?
Is the trial designed to address the key question of
the analysis (design or the trial and conduct of the
analysis):
- Baseline covariate adjustment
- Stratified randomisation
- Subgroup analysis
Objective of the subgroup analysis (exploratory,
supportive or inferential) +++ MAKE IT PRE-
PLANNED. 29

30. Conclusions
Statistical issues might relate to:
- Power of the test,
- Relevance of the subgroup / biomarker,
- Relative effect and size of the subgroup,
- Adjustments for multiple comparisons.
30