pragmatic trial designs

1. Pragmatic Trial
Presenter
Dr Nipa Mendapara
Junior Resident
Department of pharmacology
AIIMS New Delhi

2. CONTENT
• Introduction
• Challenges and limitations of Randomized Controlled Trials
• Defining pragmatic trial
• Explanatory Vs a pragmatic trial
• The PRECIS-2 tool
• Pragmatic features of a trial with example
• Summary
2

3. Introduction
• Pragmatic Clinical Trial was first introduced by Schwartz and Lellouch in
1967. These clinical trials are one of the sources that generate real-world
data.
• At present, most regulators across the world use results from randomized
clinical trials (RCT) for approving drugs. RCT provides proof of the efficacy
of the drug in a controlled environment.
• However, results are usually not enough to prove the effectiveness of the
drugs in real-world situations. Real-world evidence (RWE) generated from
routinely collected data may help to improve generalisability of the results
but have challenges. To fulfil the needs of other stakeholders like patients,
physicians, and policymakers, another approach is required.
3

4. Introduction
• Traditional explanatory randomized controlled trial (RCT) utilizing surrogate
endpoints to establish efficacy and safety in a highly selected population under
optimal conditions has been the gold standard for researchers and regulators
alike for many decades.
• These trials, which generate high-quality robust data with high intrinsic validity
upon which to base conclusions about causal relationships, answer the
explanatory question “is the intervention efficacious and safe in tightly
controlled, artificial conditions?”
• However, they ignore the more pragmatic question “is this an effective and safe
option for my patient?” The pragmatic trial design was developed to answer the
latter, which is a key question that can inform potentially life-altering decisions
required by payers, clinicians, and even patients themselves.
4

5. Randomized Controlled Trials
5

6. Definition of Pragmatic Clinical Trial (PCT)
• PCT can be defined as trials “designed for the primary purpose of informing
decision-makers regarding the comparative balance of benefits, burdens, and
risks of a biomedical or behavioural health intervention at the individual or
population level.” (Califf and Sugarman 2015)
• The significant elements of the PCT are:
(1) designed for various decision makers viz. clinicians, regulators, administrators,
and policy-makers
(2) enroll a representative patient population for whom the decision is relevant;
(3) measure a broad range of outcomes that impact clinical practice decisions by
providing comparative benefits of intervention
6

7. CLINICAL STUDIES
7
Explanatory
(Traditional)
Clinical trial
Pragmatic
(Exploratory)
Clinical trial
Observational
study
Randomization
Comparability
between intervention
groups= decreased
selection bias
Real world
evidence(RWE)
Effectiveness in regular
practice = better
evidence for clinicians
and policy- makers

8. Explanatory Vs a Pragmatic trial
Explanatory
: tests
efficacy
Homogeneous
patients
Tightly defined
intervention
Clearly defined
control group,
often placebo
Objective/surrog
ate outcomes
Short-term
follow-up time
(e.g., 6 weeks
Pragmatic :
tests
effectiveness
Real-life patients
Flexible
intervention
with changes
Active comparator
instead of placebo
Clinically
important
outcomes
Longer-term
follow-up times
(e.g., 6 months)
8

9. Designing trials that are fit for purpose
• The original PRECIS (PRagmatic Explanatory Continuum Indicator
Summary) tool was developed in 2005–08 by 25 international trialists
and methodologists to help trialists work through their design
decisions so that they might avoid designing a trial that did not match
their own intentions for the trial.
9

10. The PRECIS-2 tool: designing trials that are fit for purpose
Gartlehner et al proposed a set of
seven domains to evaluate the
explanatory or pragmatic nature of a
trial
Thorpe et al (2009), a few years later,
introduced the PRagmatic Explanatory
Continuum Indicator Summary
(PRECIS) tool.
PRECIS-2(2015)—a validated,
improved version of the tool has been
developed (nine domain)—together
with guidance for how to use it
• Although they acknowledged that efficacy
(explanatory) and effectiveness (pragmatic)
exist in a continuum, they used a binary
system (yes/no) in the evaluation of these
domains.
• PRECIS was created to enable investigators to
design trials acknowledging the
explanatory/pragmatic continuum in 10
domains
• As PRECIS tool weaknesses : no rating scale,
problems with some domains, needing better
guidance, and not validated
10

11. Measuring pragmatism in trials
Score of 1  VERY
EXPLANATORY
Score of 2  RATHER
EXPLANATORY
Score of 3  EQUALLY
PRAGMATIC AND
EXPLANATORY
Score of 4  RATHER
PRAGMATIC
Score of 5  very
pragmatic
11
PRAGMATIC EXPLANATORY CONTINUUM INDICATOR (PRECIS-2) TOOL
Scoring each domain can be done using a 5-point Likert scale

12. • Trials that are predominantly explanatory in their design generate
spoke and wheel diagrams that are close to the hub, whereas those
with a more pragmatic approach produce diagrams that are closer to
the rim.
• In reality, few trials are purely explanatory or purely pragmatic, and
for the most part, a well-designed PRT that maintains high levels of
external and internal validity will seek to strike an optimal balance
between the two study types, thereby producing a diagram that
would be somewhere in the middle of the wheel
12

13. Explanatory vs Pragmatic
Pragmatic study
Explanatory study
13

14. 1. Eligibility criteria—Who is selected to participate in the
trial?
Most extreme pragmatic approach
Inclusion-
• Intervention would likely be used for children,
elderly people, people with a range of
comorbidities
• People with a probable diagnosis rather than
a laboratory confirmed diagnosis
• Narrow eligibility criteria if the particular
narrow criteria are similar to those routinely
used for those participants in their usual care
setting
Exclusion-
• Aged over 80 or with multiple morbidities
would not receive the intervention in usual
care (harm > good)
Approach that reduce the PRECIS-2 score
Excluding people
• Not known or shown to be highly adherent to
the intervention
• Using tests or measures that are not used or
available in usual care
• Not expected to be highly responsive to the
intervention
• Children, people over the age of 65, and
pregnant women
• Whose adherence and follow-up may pose
difficulties
• Dependent on help for activities of daily living
14

15. 1. Eligibility criteria
15

16. 2. Recruitment—How are participants recruited into the
trial?
Most extreme pragmatic approach
• Only people who attend a clinic with the
condition of interest are recruited after they
present on their own behalf without any
overt recruitment effort
• Recruit from more than one clinic
Approaches that reduces PRECIS-2 score
• Searching medical record systems for
eligible participants and then mailing
invitation letters
• Media advertising campaigns
• Incentives
16

17. 2. Recruitment
17

18. 3. Setting—Where is the trial being done?
Most pragmatic approach
• Trial in an identical setting to which you
intend the results to be applied
• Even settings that seem rather restrictive
could still be a highly pragmatic design choice
if this setting is the usual care setting where
patients are treated for the particular health
condition
Approach that reduces the PRECIS-2 score
• Selecting participating centers from among
only specialist or academic centres
• Running the trial in a single center
18

19. 3. Setting
19

20. 4. Organisation—What expertise and resources are
needed to deliver the intervention?
Most pragmatic approach
• Slot the intervention into the usual
organisation of care for the condition of
interest, making use of no more than the
existing healthcare staff and resources in that
setting
• Sometimes a change to how care is organised
is itself the intervention being evaluated i.e.
Zwarenstein et al’s trial of directly observed
treatment (DOTS) versus self-supervised
treatment for tuberculosis
Approach that reduces the PRECIS-2 score
• Increasing the number of healthcare or other
professionals
• Providing significant levels of additional
training to increase the expertise of
healthcare professionals
• Requiring healthcare professionals to have
some minimum level of experience
• Requiring healthcare professionals to have a
specialty certification & Increasing the
resources to deliver the intervention
20

21. 4. Organisation
21

22. 5. Flexibility (delivery)—How should the intervention be
delivered?
Most pragmatic approach
• Methodology of how to deliver an
intervention is not rigidly prescriptive in the
protocol
• Ex. details of how to perform a surgical
procedure could be left entirely to the
surgeon, or how to deliver an educational
programme is left to the discretion of the
educator
Approach that reduces the PRECIS-2 score
• Highly specified, protocol driven intervention
• Monitor the compliance of those delivering
the intervention with the protocol and
measures
• Timing of intervention delivery is tightly
defined
• Providers undertake additional interventions
• Restrictions are placed on the number and
types of co-interventions
• Specific direction for applying permitted co-
interventionsmanaging complications or side
effects of the intervention
22

23. 5. Flexibility (delivery)
23

24. 6. Flexibility (adherence)—What measures are in place to
ensure participants adhere to the intervention?
Most pragmatic approach
• Allow for full flexibility in how end user
recipients engage with the intervention
• In usual care, health professionals encourage
patients to take medication or follow therapy
as best they can, and such encouragement
would not count against a pragmatic design
• No special measures to enforce engagement
or compliance
Approach that reduces the PRECIS-2 score
• Having a trial pre-screening stage where
patients are evaluated for adherence with the
intervention
• Withdrawing patients from the trial if their
adherence with the intervention drops below
some specified level
• Having measures in place to monitor patient
adherence with the protocol and measures to
address poor adherence
24

25. 6. Flexibility (adherence)
25

26. 7. Follow-up—How closely are participants followed up?
Most pragmatic approach
• No follow-up contact at all with recipients and
to obtain outcome data by other means or
have minimal additional data collection
• Trials may follow recipients well beyond the
intervention period
Approach that reduces the PRECIS-2 score
• Follow-up visits that are more frequent
• Unscheduled follow-up visits are triggered by
a primary outcome event or by an intervening
event
• Patients are contacted if they fail to keep trial
appointments. This would not reduce the
score if this was also done in usual care
• More extensive data are collected
• Visits are longer than usual care and involve
additional or different staff
26

27. 7. Follow-up
27

28. 8. Primary outcome—How relevant is it to participants?
Most pragmatic approach
• To select an outcome that is of obvious
importance from the patient’s perspective &
relevant to commissioners of care
• Measure outcome in a way that is the same
or similar to the way they are measured in
usual care
Approach that reduces the PRECIS-2 score
• Choosing a surrogate outcome (such as blood
test)
• Using a composite primary outcome in which
some of the elements are less important to
patients or participants than others
• Choosing an outcome that is important but
mainly to providers
• Measuring an outcome that is important but
at a time that is earlier than would be normal
in usual care
28

29. 8. Primary outcome
29

30. 9. Primary analysis—To what extent are all data
included?
Most pragmatic approach
• Most trials are a superiority design so the
most pragmatic approach with regard to the
analysis would be to make no special
allowance in the analysis for non-adherence,
practice variability, etc.
• Analysis would typically be an intention-to-
treat analysis using all available data.
• For some trials taking an explanatory
approach (dose finding trials are an example),
it may be appropriate to have primary
analysis restricted in the ways mentioned
Approach that reduces the PRECIS-2 score
Excluding -
• Non-compliant recipients (per protocol
analysis)
• Data from non-adherent providers
• Data from trial sites or providers who recruit
below expected volume
• Analyse recipients to treatment received
instead of treatment randomised (as treated
analysis)
30

31. 9. Primary analysis
31

32. 32
Summary

33. References
• Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M.
The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015
May 8;350:h2147.
• Anstrom KJ et al. Design and rationale of a multi-center, pragmatic, open-
label randomized trial of antimicrobial therapy - the study of clinical efficacy
of antimicrobial therapy strategy using pragmatic design in Idiopathic
Pulmonary Fibrosis (CleanUP-IPF) clinical trial. Respir Res. 2020 Mar
12;21(1):68.
• Califf RM, Sugarman J. Exploring the ethical and regulatory issues in
pragmatic clinical trials. Clin Trials. 2015 Oct;12(5):436-41.
33

34. 34
Thank you