This document discusses new clinical trial designs, including their benefits and risks. Complex innovative trial designs (CIDs) like master protocols and adaptive designs can more efficiently develop new therapies. However, CIDs also carry risks like erroneous decisions from interim analyses and threats to internal validity from breaking blinding or randomization. These risks can be mitigated through measures like independent monitoring committees and adherence to study protocols. Additionally, certain CID types like basket trials generally provide only low levels of evidence due to issues like small sample sizes and lack of randomization between study arms. Overall CIDs show promise but require careful planning and oversight to minimize risks to trial integrity and conclusions.

1. 1
New clinical trial
designs: the good,
the bad and the
ugly.
Francois
Consilium July 2021

2. 2
Background

3. 3
Randomised comparative (blinded)
clinical trials? Myths debunked …
Personal analysis of the FDA clinicaltrials.gov database in Feb
2018:
• approx. 180,000 clinical trials registered
• including 144,000 interventional clinical trials,
• 94,000 completed studies
• Mostly for medicines
• RCT is not the norm …

4. 4
Randomised?

5. 5
Comparative?
1/3 of the trials are not
comparative …

6. 6
Masking?

7. 7
Complex innovative trial designs:
why?
Comparative randomised parallel groups clinical trials (RCT) should remain the gold
standard in the development of new therapies: unbiased, good internal validity.
Traditionally involve one intervention (and an appropriate active or inactive comparator)
and one disease plus a limited set of covariates.
However RCTs pose some general problems:
• Lack of flexibility (generally one primary objective) and poor external validity.
• Can be unnecessarily long.
• It is difficult to obtain evidence of efficacy in subpopulations or subgroups of patients:
subgroup analyses must be pre-specified and are often exploratory.
• Inferential subgroup analyses require large sample sizes (and are generally restricted
to 1 subgroup).
• Classic RCTs are not always suited to address some complex scientific questions.

8. 8
Complex innovative trial designs: mostly
because of new molecular targets …
New therapeutic approaches and the diseases with multiple targets:
• Prognostic biomarker (disease prognosis): A biomarker used to identify likelihood of a
clinical event, disease recurrence or progression in patients who have the disease or
medical condition of interest.
• Predictive biomarker (predictive of the response to a treatment): A biomarker used to
identify individuals who are more likely than similar individuals without the biomarker
to experience a favourable or unfavourable effect from exposure to a medical product
or an environmental agent.
• Therapies targeting biomarkers (e.g. pembrolizumab, entrectinib, larotrectinib) which
can be expressed in different organs, with different levels of efficacy (e.g. FOCUS4
trial).
• The clinical prognostic or predictive values of the biomarker are not always fully
characterised that the start of a clinical trial.
• Diseases with different potential targets or treatments (NSCLC, SARS-Cov-2
infection).

9. 9
Biomarkers: RCT design implications
0
0.1
0.2
0.3
0.4
0.5
0.6
control Product control Product
BM- BM+
Prognosis and predictive
biomarker
BM- control
BM- Product
BM+ control
BM+ Product
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
control Product control Product
BM- BM+
Predictive biomarker
BM- control
BM- Product
BM+ control
BM+ Product
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
BM- BM+ BM- BM+
control Product
Prognosis biomarker
control BM-
control BM+
Product BM-
Product BM+

10. 10
Biomarkers: 2 basic designs
Biomarker stratified design: the biomarker is used as an explanatory covariate to
randomise and stratified the analysis of the trial (comparative vs a population of
patients whose disease does not express the biomarker).
Biomarker enriched design: the eligible patients are screened before enrolment in
the trial. Only patients whose disease expresses the biomarker are enrolled in the
clinical trial (comparison of experimental product vs SoC).
Simple scenario: In the case of 1 biomarker,
1 disease (1 histology type in case of cancers) and
1 treatment.

11. 11
Biomarkers: need for more complex
designs i.e. master protocols
These simple approaches are compounded by the fact that biomarkers can be
expressed in different tumour types, 1 histology type of cancer can express
different biomarkers which are actionable. The expression of a biomarker does
not guarantee that a product targeting this biomarker will be effective in this
disease (tumour).
Mostly driven by the advent of new anticancer agents (e.g. TK inhibitors).
?
?

12. 12
Complex innovative trial designs:
why?
• Efficiency (“trial factories”).
• Avoid running studies when unnecessary because the treatment is ineffective or
there is early evidence of efficacy.
• Modify the allocation ratio to enable patients to receive most effective
treatments (ALIC4E).
• Add treatment arms as new investigational products become available without
having to re-run a trial.
• Remove treatment arms (for futility, safety) (pick-the-winner / drop-the-loser).
• Stop the trial at an early stage for success or lack of efficacy.
• Dose-selection within a clinical efficacy study.
• Use an optimum number of patients while achieving the same goal with the same
operating characteristics.
• Platform trials can accommodate evolution of the standard of care.

13. 13
Complex innovative trial designs:
why?
• Logistics:
• Avoid running separate studies when therapies are developed in the same
indication (e.g. prostate cancer, SARS-Cov-2 infection). This also involves
administrative and regulatory procedures (ethics committees and regulatory
approval).
• Particularly important in orphan diseases or in studies conducted in small
populations …
• and in the context of Public Health emergencies for example COVID
pandemic (e.g. RECOVERY is a large simple platform).
• Public health value of complex innovative trial designs:
• Recently demonstrated in the COVID-19 (RECOVERY, REMAP-CAP, etc …).
• Other examples include STAMPEDE, I-SPY 2, etc …
• More transparent go/no go decision process between phases 2 and 3 (seamless).

14. 14
Complex innovative trial designs:
important issues to consider …
When designing, running, analysing or interpreting the results of a CID trial it is
important to:
• Maintain the operating characteristics of the trial (namely type 1 / type 2
errors to pre-specified levels), in particular: avoid any inflation of type 1 and
the risk of erroneous decision (say that an intervention is effective when it
is not).
• Maintain the internal validity of the study (e.g. randomisation, blinding,
biases).
• Maintain the integrity of the study (e.g. missing or erroneous data,
differential missingness) especially during the follow-up period.

15. 15
Different types of
complex innovative trial
designs

16. 16
Complex innovative trial designs:
definitions
Encompass a heterogeneous group of study designs: they generally include two
types of study designs: adaptive trials and master protocols (designed to assess the
role of biomarkers). These also include trials using a Bayesian approach (monitoring
and analysis) (FDA, 2020).
• Master protocols are trial designs that test multiple drugs and/or multiple
subpopulations in parallel under a single protocol, without a need to develop
new protocols for every trial. The term master protocol is often used to
describe the design of such trials, with variable terms such as umbrella, basket,
or platform describing specific designs.
• An adaptive design is defined as a clinical trial design that allows for
prospectively planned modifications to one or more aspects of the design based
on accumulating data from subjects in the trial. They allow the trial to adjust to
information that was not available when the trial began. A lot of adaptative
designs rely on the conduct of pre-planned interim analyses.
• Trials using a Bayesian approach (monitoring or analysis).

17. 17
Master protocols: basket trials
Drug 1
Control
Drug 2
Control
Histology
Biomarkers Intervention

18. 18
Master protocols: umbrella trials
Histology Biomarkers
Drug 1
Control
Drug 2 vs control
Drug 3 vs control
Intervention

19. 19
Complex innovative trial designs:
biomarkers examples
From Blagden et al. 2020.

20. 20
Adaptive designs: main types

21. 21
Adaptive designs: main types

22. 22
Frequentist vs
Bayesian:
Bayes theorem
and definitions
• Bayes theorem of conditional probabilities.
• P(A|B) = P(A and B)/P(B) = P(B|A)/P(A)*P(B) (Bayes
theorem)
• P(B|A)/P(A) is the likelihood.
• P(B) = prior probability.
• Where P(A|B) = posterior probability.
• Prior: left to the choice of the sponsor & can be
informative.

23. 23
Bayesian analyses of clinical trials
Different analyses for different purposes:
• Monitoring and adaptation: sample size adjustment, change in
allocation ratio, interim analyses and stopping rules.
• Analysis: hypothesis testing (acceptance or rejection of null
hypothesis).

24. 24
Clinical trials and Bayesian adaptations /
analyses …
• Bayesian analyses use or borrow information from previous experiences or knowledge (e.g. other trials) to inform prior distributions to
compute posterior distributions (e.g. posterior probability that a hypothesis is true or false). Most of them use “classical designs” (incl.
primary hypothesis and sample size).
• Should reflect positions of ignorance, scepticism and enthousiasm.
• They are not necessarily prespecified, unique, known or important.
• There is no “correct” prior.
Advantages
• Clinical trials adaptation: Allow clinical trials to be much more efficient (reduced sample sizes, stop clinical trials early, allocate patients
efficiently).
• Predictive modelling (using predictive distributions) for trial adaptations.
• Monitoring & sequential analyses (no inflation of type 1): controversial.
• Beautiful and funny (and beautiful).

25. 25
Adaptive trials
over time
Number of adaptive clinical trials
registered in clinicaltrials.gov (per phase)
between 2001 and 2013 (source: Hatfield
et al. 2016).

26. 26
Adaptive designs: clinicaltrials.gov
Source: Bothwell LE et al. 2018

27. 27
Complex innovative trial designs:
risks
From a HTA perspective, CID trials will raise different issues wrt to the estimate
of the relative treatment effect:
• Trials which use a classic RCT architecture but which will either use an
“unconventional” statistical approach to monitor then adapt the design or to
analyse the results of the clinical trial: adaptive trials, Bayesian monitoring or
analyses, master protocols (umbrella, platform), etc.
• Trials which deviate from the classic RCT architecture: baskets (uncontrolled)
which may require additional external comparison using generally observational
methods.

28. 28
Complex innovative trial designs:
mitigate the risks
Whereas these trials are adequate to show efficacy (for regulatory purposes),
these studies can raise some issues in the view of reimbursement (HTA) decisions.
Control the risk of erroneous decisions (interim and Bayesian analyses):
Maintain the operating characteristics of the trial esp. control the inflation of type
1 error or check priors (when informative) especially if obtained from earlier
studies (against H0): simulation studies and sensitivity analyses.
Be careful of subgroup analyses (inferential vs exploratory): increased power esp.
with Bayesian analyses.
Possible bias of the treatment effect? unclear.
Non-binding futility rules (!) … counterintuitive but very popular (they are
designed to preserve the type 1 error).

29. 29
Complex innovative trial designs:
mitigate the risks
Ensure the internal validity and integrity of the trial:
Interim analyses break the equipoise which can pose a risk to the internal validity
and integrity. The uncertainty will depend on the adaptations of the trial based on
the results of interim analyses, whether:
• No adaptation (e.g. the company uses the result to apply for MA), the trial
continues OR
• The company introduces a trial adaptation which breaks the standard
comparative parallel groups design OR if the trial is stopped.
Importance of a physically independent adaptation committee with procedures in
place: information from interim analyses may be obtained by the sponsor and
given to investigators (or unblinded sample size reestimation) may pose a risk to
the integrity and internal validity.

30. 30
Complex innovative trial designs:
mitigate the risks
Therefore, it is important to:
• Control the communication concerning interim analyses (sponsor, investigators
and patients).
• Maintain follow-up: to decrease the uncertainty on the efficacy due to the lack
of follow-up if the trial is interrupted for early efficacy (MAPs).
• Adhere to the study protocol (avoid any protocol deviations and lack of
compliance): try to minimise bias linked to the unblinding (e.g. objective clinical
endpoints), minimise differential drop-out.
• Maintain controlled follow-up to obtain estimates of relative effectiveness (on
relevant endpoints or over a relevant period): complicated interpretation if
patients switch to another arm. Delayed start designs do not allow estimates of
relative effectiveness (against relevant SoC).
• Monitor safety: also increases uncertainty on safety.

31. 31
Trials using Bayesian analyses:
mitigate the risks …
• A lot of issues incl. computational issues.
• Interpretation: Sensitivity analyses. Tocilizumab study (CORIMUNO).

32. 32
Basket trials: low level of evidence
• Different populations.
• Parallel conduct of single arm studies (non-randomised)
(owing to different standards of care in the diseases
covered by the trial), even if design could integrate
comparators.
• (quasi-) impossibility to use historical controls (the
biomarker status of the controls is unlikely to be
documented).
• Only possible common endpoint: overall response rate with
tenuous relationship with overall survival.

33. 33
Basket trials: pooling
Risks associated with pooling:
False positive: if the treatment effect is driven by the effect in a given indication
(esp. in case of unequal sample sizes across the arms of the trial).
False negative: if the treatment effect is diluted by the indications in which the
product is ineffective.
Heterogeneity of the effect.
Small sample sizes in some arms of the studies.
Pruning of indications and “random high bias”.
We need to be very wary of the biomarker approach: vemurafenib was supposed
to be effective in BRAF mutated tumours but proved ineffective in BRAF mutated
colorectal cancer.

34. 34
Basket trials: pooling the
results

35. 35
Complex innovative trial designs: other issues
1. Trial planning and design: complexity of the design, need to closely involve a statistician. Necessary early
dialogue with regulators and HTA (when appropriate).
2. Funding: grants and funding accommodate flexible designs (flexible resources) with great difficulty.
3. Logistic issues: These trials often require important infrastructures and experienced teams (e.g.
monitoring).
4. Analytical and monitoring complexity which require specific expertise.
5. Patients informed consent.
6. Reporting: CONSORT ACE statement (Dimairo M et al. 2020).

36. 36
Conclusions

37. 37
Conclusions: CIDs
• GOOD … provide an efficient way of developing new therapies.
• Their Public Health value was demonstrated esp. in the context of the SARS-Cov-2
pandemic (e.g. RECOVERY).
• CID trials encompass different types of studies: adaptive and master protocols.
• BAD: Each of these study designs have their own advantages and risks:
• Risk of erroneous decision
• Risk to the internal validity
• Risk to the trial integrity
• These risks can be mitigated to a certain extent.
• UGLY: These trials are generally associated with increased uncertainty (low level of
scientific evidence).
• Good and bad trial designs are often driven by conditional / early and lenient
regulatory approvals.

38. 38
Thanks!